There is a rapid and substantial development in perceptual abilities, in general, and vision, in particular, in the first year of life after birth (0–12 months). At the same time that these changes in perception are occurring, it is now known that young infants are also engaging in sophisticated types of learning and memory suggesting early competency in non-perceptual abilities and availability of neural systems located higher in the cortical hierarchy than perception. Thus, there is a developmental co-occurrence between early visual development and an infant's sophisticated learning and memory abilities. Along with extensive research in adults indicating a role of memory in vision, this developmental co-occurrence suggests that an infant's learning and memory abilities could play a role in supporting and shaping perceptual development. However, the dominant model of perceptual development is exclusively feed-forward/bottom-up and does not include the feedback neuro-anatomical connections that would allow sophisticated learning and memory systems to directly shape perception in a top-down fashion. Thus, the first step toward understanding whether and how learning and memory is involved in perceptual development is to either accept this exclusively feed-forward/bottom-up model or to adopt a top-down model that incorporates feedback from learning and memory systems. Focusing on visual development, the differential commitments of these two types of models of perceptual development are outlined. The current evidence for top-down models is presented and a series of future directions are articulated.