AI-generated images and texts are increasingly indistinguishable from human-created versions. AI-generated paintings and faces are often mistaken for human-made, and AI-generated humor matches human jokes in engagement. However, there's a common bias against AI art; when people know an artwork is AI-generated, they rate it as lower quality. Generative language models (LLMs), like OpenAI’s GPT-3 and Meta’s Llama 2, are advancing towards human-level text creation. These models generate texts that mimic human writing, with some indistinguishable from human texts. Nevertheless, it’s argued that AI lacks the creativity to produce high-quality poetry, as poetry requires creativity and meaning beyond AI's current capacities. Despite these challenges, AI has made notable progress in generating poetry, often producing high-quality works. Yet, non-experts struggle to distinguish AI-generated poems from human-written ones, as they may use different criteria. Research shows non-expert readers could not reliably differentiate AI poetry from human poetry. Poems generated without human input were often rated as human-authored. A bias persists where people evaluate AI-created poems lower than human counterparts if informed about the AI origin, underlining a disconnect between perception and actual quality. Two experiments explored AI-generated poetry's recognizability and quality. In Study 1, 1, 634 participants assessed poems by ten renowned poets and AI-written poems in their styles.

Participants struggled to correctly identify AI poems, often mistaking them for human-created, indicating a reliance on flawed heuristics. Study 1's results showed AI poems were frequently misidentified as human, highlighting participants' biases and misconceptions. A model analysis suggested that certain poem features, like rhyming and line counts, influenced participants' judgments, revealing flawed heuristics since AI poems often meet these structural criteria better than human-written ones. Study 2 involved 696 participants evaluating poetry quality on several dimensions. AI-generated poems consistently received higher ratings than human-created ones, except when participants were informed about AI authorship, which led to lower evaluations. Despite recognizing AI-generated poetry's quality, participants showed a bias favoring human authorship when informed. The analysis showed that perceptions of AI poems are influenced by expectations and aesthetic preferences. The experiments indicate that AI-generated poetry achieves high levels of quality and authenticity to those of human-generated poetry, challenging prior notions about AI's limitations. While expertise or prior experience with poetry did not enhance participants' ability to discern between AI and human poems, the studies suggest that misconceptions about AI capabilities persist. The experiments underline the significant advancements in generative AI models and highlight an ongoing discrepancy between technological capabilities and public perception. Participants rated AI-generated poems more highly on aesthetic dimensions, showcasing a shift in AI craftsmanship. As AI-generated texts become more common, expectations and perceptions are likely to evolve, potentially aligning more closely with AI's true capabilities.