This year, social media platforms were suddenly flooded with artistic images created by the Midjourney software, which generates artworks based solely on a text prompt. Meanwhile, text-generating software like ChatGPT can write resumes, compose poetry, and even assist doctors in communicating with their patients. These types of generative artificial intelligence have generated headlines ranging from the fearful ("How Could AI Destroy Humanity?") to the exhilarated ("Why AI Will Save the World"). However, amidst society's grappling with generative AI as a mainstream phenomenon, it is already reshaping the way scientists conduct research and develop technology. Acknowledging this reality, Caltech has introduced a unique course on the cutting edge of generative AI technology called EE/CS 148: Large Language and Vision Models. This spring, the course provided students with a glimpse behind the scenes of generative AI, teaching them how it truly functions and how to harness its capabilities for their specific needs. "Our aim was to introduce this technology to Caltech students, " says Georgia Gkioxari, an assistant professor at Caltech in computing and mathematical sciences and electrical engineering, as well as a William Hurt Scholar. "We wanted to teach our students how to think beyond the current applications and explore how we can extend these advancements to address the various scientific applications undertaken here at Caltech. " Gkioxari, alongside Professor of Electrical Engineering Pietro Perona, who had traditionally taught an introductory computer vision course, led the class this spring. Perona decided to revamp the computer vision offering when Gkioxari joined Caltech's faculty in January, bringing with her industry experience gained during a six-year tenure with Facebook/Meta AI. "I felt like I was falling behind and not keeping up with all these new techniques [in generative AI], " Perona explains.

"When Georgia arrived, I thought, 'Okay, she's intelligent, highly energetic, and more knowledgeable than I am. If I collaborate with her in teaching a class, I will learn something. '" What is Generative AI?Gkioxari emphasizes that AI is an umbrella term and somewhat colloquial. Both she and Perona, along with their colleagues, view these technologies as forms of machine learning, where software learns from vast amounts of data rather than possessing true artificial intelligence or "thinking" capabilities. "The ultimate aim is to achieve genuine artificial intelligence, " she states. "However, within the context of our teaching and current community efforts, we refer to it as machine learning or data-driven statistics. " Perona agrees, adding, "AI will occur when our machines can draw inferences and reason beyond the training data. At present, we observe very little of this happening. " While specialized programs like ChatGPT or DALL-E2 focus on text and images, the underlying technology of generative AI remains the same: they are all based on deep neural networks. "There is utility in practical application, as well as in cultivating a mindset, " observes Suzanne Stathatos. "This class accomplishes both. Students start with a rudimentary understanding and conclude with the ability to read a machine learning paper and implement it independently. "-->