Artificial intelligence (AI) has made progress in generating jokes, but a recent study reveals its lack of understanding of what makes them funny. The study involved testing AI models and humans on tasks related to the New Yorker magazine's Cartoon Caption Contest entries. These tasks included matching jokes to cartoons, identifying winning captions, and explaining the humor. Humans outperformed AI in all tasks, suggesting that AI's comprehension of humor still has room for improvement. While AI can generate a large number of jokes, it struggles to comprehend their underlying comedic elements. Researchers used New Yorker Cartoon Caption Contest entries to assess AI models and humans in tasks such as matching jokes to cartoons, identifying winning captions, and explaining the humor. Humans consistently outperformed AI, despite advancements in AI technology such as ChatGPT. Although machines are making progress in understanding humor, they are not quite there yet. The lead author of the study, Jack Hessel, highlighted the use of tests to evaluate AI models' understanding.

Hessel acknowledged that while it is a defensible position to claim that machines cannot truly understand humor as understanding is a human trait, he still found the performance of machines on these tasks impressive. The study titled "Do Androids Laugh at Electric Sheep?Humor 'Understanding' Benchmarks from The New Yorker Caption Contest" received a best-paper award at a recent meeting of the Association for Computational Linguistics. The researchers analyzed over 700 New Yorker caption contests from 14 years, testing two types of AI models - those based on image analysis and those based on analysis of human summaries of cartoons. The relationship between the cartoons and captions in the New Yorker contests required more sophisticated understanding compared to datasets with direct image and caption relationships. The tasks involved matching a cartoon with a caption, ranking caption quality, and generating explanations of how a high-quality caption related to the cartoon. Human-generated explanations were favored over AI-generated explanations by a ratio of 2-to-1. Although AI falls short of understanding humor, the authors suggest that it could be a valuable collaborative tool for humorists in brainstorming ideas. The study involved contributions from various researchers and humorists and was funded by the Defense Advanced Research Projects Agency, AI2, and a Google Focused Research Award.