Neural networks have proven effective learning machines for a variety of challenging AI tasks, as well as surprisingly good models of brain areas that underly real human intelligence. However, most successful neural networks are trained in a supervised fashion on labelled datasets, requiring the costly collection of large numbers of annotations. Unsupervised approaches to learning in neural networks are thus of substantial interest for furthering artificial intelligence, both because they would enable the training of networks without the need for annotation, and because they would be better models of the kind of general-purpose learning deployed by humans. In this talk, I will describe a spectrum of recent approaches to unsupervised learning, based on ideas from cognitive science and neuroscience. First, I will discuss breakthroughs in neurally-inspired unsupervised learning of deep visual embeddings that achieve that achieve performance levels on challenging visual categorization tasks that are competitive with those of direct supervision of modern convnets. Second, I'll discuss our work building perception systems that make accurate long-range predictions of physical futures in realistic environments, and show how these support richer self-supervised visual learning. Finally, I'll talk about the use of intrinsic motivation and curiosity to create interactive agents that self-curricularize, producing novel visual behaviors and learning powerful sensory representations.