It’s Not a (Beep! Beep!) Poet (and it doesn’t know it)

Writer Tom Averill examines what poetry reveals about the potential and the limitations of AI text bots.

This image was rendered by AI illustration program Dalle*2 with the entirety of this article entered as the visual prompt text. Prompt and adaptation by Savva Pettengill.

Story by Thomas Averill

Artificial intelligence—its uses and abuses, its advantages and threats, its potential to make us more efficient yet also more vulnerable, our power to manipulate it, and its power to manipulate us—has been dominating the news, especially in the world of writing. ChatGPT is creating everything from legal briefs to obituaries, real estate descriptions to sonnets. So it seemed natural to ask: “Given AI’s leap in generating creative text, can it write creditable poetry?”

As a professor of creative writing and Kansas literature at Washburn University, I taught people not only to write poetry but also to appreciate the best literature of our state. Since my retirement, I’ve taught classes for the Osher Institute (over-55 adult learning housed at the University of Kansas), the most recent course on prairie writing. We studied 30 prairie-themed poems, and I gave my students a challenge: Pick your five favorite poems, pick five words or phrases from each one, then write your own prairie poem. Many rose to the challenge. Participant Alice Eberhart-Wright wrote this:

April Devotional
Bluestem, meadowlark.
Walk, listen. Stop!
Taste first wild strawberries.
Sing each season:
Hymns of coyote sorrow; thrill of summer storms. Feast under blue skies pillowed with clouds.
Fill your nostrils with smoke at dusk—
Your eyes, radiating with seasonal burns,
Give thanks for hills of redbuds tiptoeing from Sleeping Flint Hills. Stark black branches,
Pink-inked pens.

I especially like her tiptoeing redbuds, their blossoms “pink-inked pens.” That’s real poetry: surprising and satisfying.

How might ChatGPT do, given a similar challenge? To find that out, I recruited David Lee King, digital services director at Topeka & Shawnee County Public Library. We asked AI to create poetry grounded in the Kansas experience. Could the program write a creditable poem? Could it simulate the highest art of language, poetry? We wanted six poems, including a sonnet, a sestina, several prairie poems (some of them using a model; we inputted the 30 poems I collected for the Osher Prairie Writing course).

The results were not immediately satisfying. As King told me, “The output of ChatGPT is as good as the input, or the prompts, we provide.” So, we gave as much input, and as many prompts, as possible. For one free-verse poem with the theme of “prairie seasons,” King made 34 prompts, including such instructions as “emphasize weather and seasons”; “limit length to 30 to 50 lines”; “do not rhyme” (ChatGPT loves rhyme and never could comprehend free verse—no meter, no rhyme!).

“This is still in the experimental stages,” King later said, “and requires a lot of interaction with a generator who has to think of and give creative commands. ChatGPT was often a reluctant student, unable to respond to commands about rhyming and poem length.” But King remained a patient and persistent teacher.

Once the poetry was generated, King and I met with Traci Brimhall, current Kansas poet laureate, who teaches at Kansas State University, and Leslie VonHolten, who writes about the prairie and prairie poetry. Before reporting our commentary, I want to share a couple of stanzas from two of the ChatGPT poems, to give a feel for what we reacted to. The first is the poem that King had to urge through 34 drafts, and the second focuses specifically on Topeka:

Seasons on the Kansas Prairie (by ChatGPT)
Kansas prairie, vast and wide,
Where earth and sky seem to collide,
A place where seasons come and go,
And weather changes, a natural flow.
In winter, the cold winds bite,
And snowflakes dance in the pale light,
A white blanket covering the ground,
A silence that fills the world around.

The above image was rendered by Dalle*2 AI illustration program with text from the ChatGPT poem “Seasons on the Kansas Prairie” as the visual prompt. Illustration prompted and adapted by Savva Pettengill.

Heartland Hues: Ode to Topeka, Kansas (by ChatGPT)
Topeka, heart of Kansas,
where the prairie winds whisper
through sun-kissed wheat fields,
and the Kansas River flows.
This city has a soul,
woven with hard work,
determination, and pride.

What to make of such poetry? According to King, ChatGPT works by choosing “the next most likely word.” No wonder, then, that the work is full of cliches like “come and go,” and “white blanket” for snow. As VonHolten observed, “The poems have vocabulary, expected words, but not with the intention of making the reader think.” Brimhall agreed: “It’s just noise. Because it doesn’t matter. It doesn’t add anything to our experience as human beings. It has no being, no personal experience, no emotion. Being a person is hard, and needs our attention, so we want poetry to help us be human and attentive—not just noise.” Compare the stanzas above to poet Steven Hind’s evocation of season and prairie: Prairie,/ you say, and hear the grass/ speaking through the thorny wind/ season after season.

I agree with Brimhall and VonHolten. ChatGPT poetry has no voice, no individuality, no mental or intellectual geography—something that we create over years of work as writers. “The poetry reminded me of old patriotic songs,” said King, “but patriotic about Kansas, as full as they are of standard, even expected and predictable thoughts.”

The above image was rendered by MindJourney AI illustration program with text from this article input as the visual prompt. “ChatGPT was often a reluctant student, unable to respond to commands about rhyming and poem length.” Illustration prompted and adapted by Savva Pettengill.

We all agreed that the poems resemble poems. But they do not read as though they were written by a poet, “by an individual,” we all said in one way or another. When a good poet writes a patriotic poem, or any poem for that matter, it will go beyond clichés.

“A good poet pays deep attention to language, to images, to subtleties in thinking and expression,” Von Holten says. “ChatGPT doesn’t have that.” Brimhall agrees: “Poets are learning all the time, are getting better at craft and art, and after years are often still struggling to get it right. Artificial intelligence isn’t struggling, so how much chance does AI have of accomplishing real craft and art?” King notes that “ChatGPT does best with standard forms and structures, with things known, fact-based and maybe ‘logical’—but not beyond.”

From my own knowledge as a writer, each piece we write comes from years of our unique experience and singular engagement with language. ChatGPT has no accumulated emotional experience or depth. Think of the lived experience with place and language in this poem by Kansas legend William Stafford:

The Little Girl by the Fence at School
Grass that was moving found all shades of brown, moved them along, flowed autumn away galloping southward where summer had gone. And that was the morning someone’s heart stopped and all became still. A girl said, “Forever?”
And the grass. “Yes. Forever.” While the sky —
The sky—the sky—the sky.

Poetry is a good tool for comparing human and AI writing skills because poetry is difficult. Even when intelligent humans try for poetry, they often have trouble going beyond. The bot-generated poems remind VonHolten of poems her ancestors “might be moved to write after seeing a beautiful sunset or having a remarkable experience. Admirable expressions of a moment, formal in rhythm and rhyme. They are written in poetic form, but are not quite what we expect from poetry.

For me, the best versions of the AI poetry we created still read like some of the newspaper verse published in rural Kansas newspapers at the turn of the 20th century, what one Kansas poet labeled, “flapdoodle, jingoistic verse.” They are simplistic, literal, as though written by Amelia Bedelia, the children’s book character who makes mistakes because she can’t understand more than one meaning of a word, planting light bulbs instead of flower bulbs, stripping sheets by taking shears to them.

Amelia Bedelia in Kansas (by ChatGPT) // Oh, Kansas, dear Kansas, a land so grand, Where Amelia Bedelia lent her helping hand. She set foot on your soil, so full of glee, Ready to embark on a literal spree. // She went to the prairie, where wildflowers bloom, Mistaking a haystack for a cozy bedroom. She fluffed up the hay, tucked it in just right, Thinking she had turned it into a bed for the night. // She journeyed to Wichita, a bustling town, Mistaking a cow for a taxi, hoofing around. She hopped on its back, thinking it was a ride, Clueless to the laughter from folks on the side. // In Topeka, she marveled at the Sunflower State, Mistaking the sunflowers for yellow cupcakes so great. She iced them with care, sprinkles everywhere, Delighting the birds who came for a sugary affair. // Amelia explored Dodge City, the Wild West so bold, Mistaking a wanted poster for an ad to be sold. She painted her face, dressed up like a crook, But the sheriff just chuckled and gave her a look. // In Lawrence, she wandered through fields of wheat, Mistaking the grains for socks she could treat. She knitted them together, made pairs by the ton, And soon everyone had wheat socks on the run. // Kansas, dear Kansas, Amelia had her way, In her own unique style, she brightened each day. Though her literal interpretations caused some strife, Her zest for life made her a cherished part of your life.

We all agreed that the ChatGPT poetry lacks elements and emotions that poets are known for: invention, surprise, freedom, insight, and love. In poetry, we take a journey with the poet; in ChatGPT we go nowhere new. King reinforces this notion: “Chat has no individuated language: it has words, and it is digesting and spitting out only words, but without personality.”

Since all of us went into the project of AI poetry with open minds, we weren’t initially eager to be critical. But we had no choice. Perhaps it was a relief to each of us that ChatGPT won’t be replacing poets soon. Brimhall does use elements of AI with her poetry students. In one exercise, they take their own poems, use Google translator to see it in another language, maybe two, then they ask Google to translate it back to English, thus “creating insights into possible changes in the language.” King has used ChatGPT to write rough drafts, everything from his own biographical statement to blog entries. “Of course, they need a lot of revision,” he adds.

AI as Rough Draft
I took the idea of using AI as a rough draft seriously. Here is the initial version from ChatGPT, “Whispers of the Prairie.”
Whispers of the Prairie (by ChatGPT)
On the Kansas prairie,
the land stretches out
in vastness and flatness.
Underneath the wide sky,
the grasses sway and rustle
with the rhythm of the wind.
In springtime,
the prairie is lush with life.
Wildflowers bloom in brilliant hues,
and the grasses grow tall and green.
Prairie chickens dance and call,
and newborn calves take their first steps.
Summer brings a different scene.
The air is thick with heat,
and the grasses turn golden and dry.
Thunderstorms roll across the horizon,
lighting up the sky with jagged bolts.
Prairie dogs scurry in and out of their burrows,
and coyotes howl in the distance.
In fall, the prairie transforms again.
The grasses fade to shades of brown,
and the leaves of the cottonwoods turn yellow.
Geese fly overhead in V-shaped formations,
and monarch butterflies migrate south.
The prairie is alive with the sound of crickets,
and the smell of harvested fields.
Winter brings a hush to the prairie.
The grasses lie dormant beneath a blanket of snow, and the wind bites with icy teeth.
Cattle huddle together for warmth,
and the only sounds are the crunch of boots on snow and the distant call of a solitary bird.
Through all the seasons,
the prairie endures.
Its beauty and resilience a testament
to the power of the land and the sky.

Using the ChatGPT version as a rough draft, I revised and created my own version with a new title.
Prairie Seasons (by Tom Averill via ChatGPT)
Kansas prairie stretches
vast, level, under wide sky,
Grass wrestles wind.
Spring wildflowers dance with
Prairie chickens, and newborn
calves take first steps.
Summer air thickens until
black clouds bolt the horizon.
Coyotes shrill thunder.
Fall grasses fade to brown,
cottonwood leaves to yellow.
Geese V themselves south.
Winter grasses rest under snow.
Icy wind bites huddled cattle.
Boots crunch crust.
A solitary bird is enough.
The prairie, land sky,
season after season.

I began with the notion that we might fear ChatGPT and artificial intelligence because we worry about being manipulated and tricked, or about finding ourselves in a future where information is separated from human interpretation, where we can’t tell the difference between falsehood and truth. Perhaps we are already moving in that direction, and AI is a sop for our alarm. As Thad McIlroy wrote in the Publishers Weekly article (6/2/23): “There’s much chatter about the (very real) threat of Amazon being flooded—truly flooded—with books generated largely via a GPT. It’s already happening.” Certainly, we’re overwhelmed by the potential of AI to change our lives. But, from what we saw in the ChatGPT Kansas poems, we don’t yet need to despair that the highest art of language, and its practitioners—poetry and poets—will soon be replaced. As human beings, we have a lived experience and a need to share it. AI will never have life, or a need to share. It is generated; we are generative.

Shakespeare, AI and Nose-Pickers My nephew Will Averill, writer, poet, and playwright among other things, asked ChatGPT to generate a Shakespearean sonnet imploring his son to keep his fingers from his nose. The result:

Oh dear child, my sweet and cherished one,
I beg of thee to heed my humble plea,
And let thy little fingers find no fun
In digging deep for treasure that’s unclean.
Thy nostrils, made for breathing in fresh air,
Shall not be used for mining of thy nose,
For such uncouth behavior isn’t fair
To those who round thee, fair or foul it shows.
Oh, think of all the germs that may lurk there,
A host of tiny beasts that doth abound,
Just waiting to attack and do their share
Of making thee feel unwell and unsound.
So listen to thy mother’s gentle voice,
And pick thy boogers not, but make a choice.

He shared it with friends without telling them it was AI-generated, and was told that the sonnet lacked zest, color, and “Will” behind it. Given his experience with ChatGPT, I asked Will for some observations. “ChatGPT writes poetry like an eighth grader,” Will said, “knowing something of language and form, but lacking emotion and depth. What makes art great is the struggle against our limitations. ChatGPT is not struggling. It is unaware that it has limitations.” On the act of writing, Will observed that “ChatGPT lacks the capacity for self-destruction, the knowledge that we write to keep from destroying ourselves. … We sense the struggle of the writer to make sense, to be original, to be both chaotic and comprehensible. As readers, we benefit from being part of that process. ChatGPT does not begin to participate in that process. So, there’s no real relationship that can develop between writer and audience, each improved by the interaction.”

This article is based on a version of the story that appears in the fall 2023 edition of Topeka Magazine. Note: It is the policy of our publication not to use AI-generated text or AI-generated art unless, as in this article, the AI-generated text or art is the focus of a story.