Join our global community of over 40, 000 students studying in diverse programs across six continents. We have a world-class faculty and scholars who strive for excellence in addressing the challenges of our diverse world. As an institution without boundaries, we draw inspiration from the rich cultural institutions and professional opportunities in our cities. Discover more about our programs and join our community of over 40, 000 students studying worldwide. Our exceptional students, faculty, and scholars are dedicated to making a difference in the world by confronting its diverse challenges. As an institution that breaks down barriers, we derive inspiration from our cities and the renowned cultural institutions and professional opportunities they offer. Click here to explore our extensive range of opportunities. Recent advancements in AI systems, such as GPT-4, have enabled them to learn and use human language. However, the vast amount of language input they learn from exceeds what children receive during language acquisition. The best AI systems train on trillions of words, while children only receive millions annually. Consequently, researchers have been skeptical about the relevance of AI advancements to human learning and development. To address this skepticism, a team of researchers from New York University conducted an experiment. They trained a multimodal AI system using video recordings captured through the eyes and ears of a single child, from the age of six months to their second birthday. The goal was to determine if the AI model could learn words and concepts from a child's everyday experiences.

The results, published in the journal Science, revealed that the model could successfully learn a substantial number of words and concepts based on limited input. Remarkably, the video recordings only covered about 1% of the child's waking hours, yet it was sufficient for language learning. "We demonstrate, for the first time, that a neural network trained on this developmentally realistic input from a single child can learn to associate words with their visual representations, " said Wai Keen Vong, a research scientist at NYU's Center for Data Science. Brenden Lake, an assistant professor at NYU, added, "By using AI models to study the language-learning process of children, we can shed light on the factors that contribute to word acquisition, such as language-specific biases, innate knowledge, and associative learning. " The experiment revealed that the AI model's learning capacity exceeded common expectations. The NYU researchers analyzed more than 60 hours of footage, containing approximately a quarter of a million word instances linked to video frames. The researchers trained a multimodal neural network, combining a vision encoder and a language encoder through contrastive learning. This method enabled the model to associate visual and linguistic cues, gradually learning which words corresponded with specific visuals. The model's performance was evaluated using tests similar to those used for infants, and it demonstrated the ability to learn a significant number of words and concepts. It also displayed the capacity to generalize its knowledge to visually diverse instances, similar to how children demonstrate generalization in laboratory tests. The study suggests that AI models, trained on naturalistic data similar to that received by children, can effectively learn word associations through the use of generic learning mechanisms found in neural networks. Further support for the research was provided by the U. S. Department of Defense's Defense Advanced Research Projects Agency and the National Science Foundation. Ethical considerations were taken into account, including parental consent and approval from NYU's Institutional Review Board, regarding the participation of the child and the study methodology.