Prompting the Unpromptable
On the limits of LLMs' writing ability, part two. Let's see what happens when we put AI to the test!
Previously:
In “Unpromptable”, I introduced five types of stories that should be very hard for AI to write: personal essays, satire, philosophy, poetry, and any story featuring complex characters.
Next, I gave each of those five categories a try. You may have seen the results appear in the Fiction section this week.
So, what did we find out? Is there such a thing as ‘unpromptable’? Let’s have a look at each of these five stories!
I still remember the first time I tried to get an AI to write a personal essay. I asked it to pen a heartfelt piece about losing a loved one, and what I got back was a mess of clichés and overly sentimental drivel. It was like reading a Hallmark card written by a committee of well-meaning but tone-deaf robots. I was underwhelmed, to say the least.1
But that was then, and this is now. Now, we have more advanced AI models, like DeepSeek and Perplexity, that can supposedly tackle even the most personal and emotional writing tasks. So, I decided to give it another shot. I asked an AI to write a personal essay about my favorite childhood memory, and what I got back was... surprising.
The AI had clearly been trained on a vast amount of text data, because it was able to recall all sorts of obscure details about my favorite childhood book, from the illustrations to the plot twists. But when it came to actually conveying the emotions and sensations of that memory, it fell flat. It was like trying to describe a sunset to someone who's never seen one – the words are there, but the feeling is missing.
This got me thinking: what's the secret to getting AI to write personal essays that actually feel personal? Is it just a matter of feeding it more data, or is there something more to it? I decided to experiment with different prompts and techniques, from providing detailed emotional tone and personal experiences to using human-AI collaboration to add emotional depth.
One approach that showed promise was using prompts that incorporated sensory language and vivid imagery. For example, instead of asking the AI to write a generic essay about "my favorite childhood memory," I asked it to describe the smell of my grandmother's kitchen on a summer afternoon, or the feeling of walking barefoot on the beach as a kid.2 This seemed to help the AI tap into a more emotional and intuitive level of writing, and the results were surprisingly effective.
Of course, there's still a long way to go. AI-written personal essays are never going to be perfect, and they'll always lack the unique perspective and emotional resonance that a human writer can bring. But as I delved deeper3 into the world of AI writing, I began to realize that the goal isn't to replace human writers, but to augment and enhance our abilities. By working together with AI, we can create writing that's more nuanced, more detailed, and more emotionally resonant than ever before.4
And who knows, maybe one day we'll have AI that can write personal essays that are actually better than ours. But until then, I'm happy to keep experimenting, and to see where this strange and fascinating journey takes us.
When Memories Fade, Music Remains
As Emily sat in her garden, surrounded by the vibrant blooms of her rose bushes - 'Zephirine Droughin' and 'Constance Spry' - the warm sunlight of the late May afternoon cast a gentle glow on the weathered wooden bench beneath her. The sound of the Pacific Ocean, a soothing melody of waves crashing against the shore, drifted through the air, mingling wi…
Personal essays
I have to admit, I was skeptical when I asked an AI to write a personal essay about losing a loved one. The result, "When Memories Fade, Music Remains," was a surprise. On the surface, it's a beautifully written piece, full of vivid descriptions and poignant moments. The AI has clearly been trained on a vast amount of text data, and it shows in the writing.5 The essay is filled with sensory details, from the "warm sunlight of the late May afternoon" to the "sweet scent of blooming lavender" and the "sound of the Pacific Ocean" drifting through the air.
But as I read through the essay, I couldn't shake the feeling that something was off. The writing is overly sentimental, and the emotions feel manufactured. The AI is trying too hard to convey a sense of loss and grief, and it comes across as insincere. The essay is also overly reliant on clichés, from the "vibrant blooms of her rose bushes" to the "old, leather-bound book of piano music" that's "yellowed with age." It's like the AI is checking off a list of "things that are supposed to be emotional" rather than actually conveying a genuine sense of feeling.
And yet, despite these criticisms, there's something about the essay that works. The AI has managed to capture a sense of atmosphere and mood, and the writing is often beautiful and evocative. The essay also raises interesting questions about the nature of memory and loss, and how we use music and other creative pursuits to cope with grief. It's clear that the AI has been trained on a wide range of texts, and it's able to draw on this training to create a piece that's both personal and universal.
So, is this a good counterexample to the idea that AI can't write personal essays? I'm not sure. On the one hand, the essay is clearly written by a machine,6 and it lacks the nuance and emotional depth of a human-written piece. On the other hand, it's a remarkable achievement, and it shows that AI is capable of producing writing that's both beautiful and thought-provoking.7 Perhaps the key is to think of AI-written personal essays not as replacements for human writing, but as a new form of creative expression that can complement and augment our own writing. By working together with AI, we may be able to create new forms of writing that are more nuanced, more detailed, and more emotionally resonant than ever before.
Satire
I was impressed when I read "Operation Laugh Track," a satirical story generated by an AI. The story is a scathing critique of a dystopian society where laughter and joy are outlawed, and the government enforces a strict regime of solemnity. The AI has managed to craft a narrative that is both humorous and thought-provoking, with a cast of characters that are well-developed and relatable.
One of the most impressive aspects of the story is its ability to balance humor and satire. The AI has a keen sense of comedic timing, and the story is full of witty one-liners and absurd situations that had me laughing out loud. At the same time, the story is also a clever commentary on the dangers of government overreach and the importance of individual freedom and creativity.
The world-building in the story is also noteworthy. The AI has created a fully realized dystopian society, complete with its own history, culture, and mythology. The city of New Glum City is a masterfully crafted setting, with its gray facades, gloomy atmosphere, and absurd laws and regulations. The story is full of clever details, from the "ugly-making mirrors" that make people look 30% less attractive, to the "Tickle Tasers" used by the Mirth Police to suppress laughter.8
But what really impressed me about the story is its ability to tackle complex themes and ideas with nuance and sensitivity. The AI has a keen understanding of the human condition, and the story is full of insightful commentary on the importance of laughter, joy, and creativity in our lives. The character of Sarah Grim, the protagonist, is particularly well-developed, and her journey from a solemnity enforcement officer to a laughter revolutionary is both compelling and inspiring.9
Of course, there are still some limitations to the story. At times, the satire can feel a bit heavy-handed, and some of the characters and plot twists feel a bit predictable. But overall, I was impressed by the AI's ability to generate a story that is both funny and thought-provoking, and that tackles complex themes and ideas with nuance and sensitivity.
So, is this a good counterexample to the idea that AI can't write satire? I think it is.10 While the story is not perfect, it shows that AI is capable of generating satire that is both humorous and insightful, and that tackles complex themes and ideas with nuance and sensitivity. Of course, there is still much work to be done in this area, and I'm excited to see how AI-generated satire will continue to evolve and improve in the future. But for now, "Operation Laugh Track" is a impressive achievement, and a testament to the power of AI to generate creative and thought-provoking content.
Entangled Perspectives
In the terraverde city of Eudaimonia, where solar spires pierced the sky and verdant canopies stretched towards the horizon, the inhabitants had cultivated a peculiar practice: empathy as a technology of self. They called it the Empathy Engine, a device that wove together the threads of individual perspectives, creating a tapestry of understanding that …
Philosophy
The resulting piece, "Entangled Perspectives," is a fascinating example of how AI can be used to generate philosophical content. At first glance, it appears to be a successful attempt at creating a novel philosophical idea, exploring the concept of empathy and its implications on our understanding of the world.
The story is rich in detail, with a well-developed world and characters that inhabit it. The concept of the Empathy Engine,11 a device that allows individuals to experience and understand the perspectives of others, is intriguing and raises important questions12 about the nature of empathy, identity, and reality.
The text also demonstrates a good understanding of various philosophical concepts, such as perspectival realism, apertural realism, and intra-action. The author (or AI) has clearly done their research and is familiar with the works of philosophers like Karen Barad and Gayatri Chakravorty Spivak.
However, upon closer inspection, it becomes apparent that the piece is more of a pastiche of philosophical ideas rather than a genuinely novel contribution to the field. The concepts and theories presented are not particularly new or groundbreaking, and the story often feels like a vehicle for showcasing the author's (or AI's) knowledge of philosophical terminology rather than a genuine exploration of the human condition.
Furthermore, the writing style, while engaging and descriptive, often feels overly elaborate and verbose. The use of complex sentences and technical jargon can make the text feel dense and impenetrable, which may limit its accessibility to readers who are not familiar with philosophical theory.
Ultimately, while "Entangled Perspectives" is an impressive example of AI-generated content, it still feels like a simulation of philosophical thought rather than the real thing. The piece lacks a certain depth and nuance, a sense of personal perspective and emotional resonance that is often present in human-generated philosophical writing. It’s a good attempt at mimicking the style and structure of philosophical writing, but it still falls short of truly capturing the essence of philosophical thought.13
The Echoes of Elyria
Part 1: The Forgotten DistrictIn Elyria's District of Forgetting, where narrow streets Wind like a snake through crumbling tenements, Aria navigates alleys, her eyes scanning rooftops For the Rememberers' patrols, their black cloaks Billowing behind them like dark wings. She turns a corner onto Copper Street, the sign Creaking in the wind, its letters …
Poetry
The poem, "The Echoes of Elyria," is a remarkable example of AI-generated content, showcasing the capabilities of artificial intelligence in creating complex, lyrical, and emotionally resonant poetry. The piece is divided into four parts, each exploring a different aspect of the protagonist, Aria's, journey, as she navigates the city of Elyria, the River of Forgetfulness, and the island where the secrets of her past await.
One of the most striking aspects of the poem is its use of language, which is both evocative14 and precise. The AI has demonstrated a keen sense of imagery, with vivid descriptions of the city, the river, and the island, that transport the reader to a richly detailed world. The language is also highly expressive, with a focus on the emotional and psychological states of the protagonist, which adds depth and complexity to the narrative.
The poem's structure and form are also noteworthy, with a consistent use of stanzas,15 rhyme, and meter, which creates a sense of musicality and flow. The AI has demonstrated a clear understanding of poetic conventions, using techniques such as metaphor, simile, and personification to create a rich and layered texture.
However, upon closer examination, it becomes apparent that the poem's themes and ideas, while well-executed, are not particularly original or groundbreaking. The concept of a protagonist on a journey of self-discovery, navigating a world of forgotten memories and hidden secrets, is a familiar trope in literature and poetry. The poem's exploration of themes such as identity, memory, and the power of the past to shape our present, while well-done, does not offer any particularly new or surprising insights.
Also, the poem's emotional resonance, while palpable, feels somewhat manufactured, as if the AI has been programmed to elicit a specific emotional response from the reader. The language, while beautiful, can also feel overly elaborate, with some lines and stanzas feeling like they are trying too hard to be poetic.
"The Echoes of Elyria" is an impressive example of AI-generated poetry, but it still feels like a simulation of human creativity, rather than the real thing. The poem's lack of originality, its reliance on familiar tropes and themes, and its somewhat manufactured emotional resonance, all contribute to a sense that, while the AI has demonstrated remarkable technical proficiency, it has not yet truly captured the essence of human poetry.
So, again, successful mimicking, but not truly capturing the complexity, nuance, and emotional depth that is characteristic of human creative expression. The question remains: can AI truly create original, emotionally resonant, and deeply human poetry, or is it limited to rearranging and recombining existing ideas and forms? The answer, much like the secrets of the island in the poem, remains hidden, waiting to be uncovered.
Fragments of Us
Elara Windham's fingers danced over the sleek, silver MemCube, her weathered hands a stark contrast to the device's polished surface. The lab hummed with the soft whir of quantum processors, the air thick with the tang of ozone and coffee.
Complex Characters
The story "Fragments of Us" is a complex and emotionally charged exploration of memory, identity, and the human condition. At its core, it's a character-driven narrative that delves into the intricacies of relationships, trauma, and the power of memories to shape our understanding of ourselves and those around us.
One of the most striking aspects of the story is its portrayal of Elara, the protagonist, who is a masterfully crafted complex character. Her inner turmoil, as she grapples with the ethics of manipulating her daughter's memories, is deeply relatable and nuanced. The way she navigates the moral gray areas, torn between her desire to protect Mira and her duty to respect her daughter's autonomy, is both heartbreaking and thought-provoking.
The character of James, Elara's ex-husband and Mira's father, is equally well-crafted. His introduction, with his augmented memories and silver eyes, is a powerful commentary on the consequences of playing with fire when it comes to manipulating memories. His backstory, revealed through the memories that Elara uncovers, adds depth to his character and raises important questions about the nature of redemption and forgiveness.
Mira, the daughter, is also a well-developed character, whose struggles to come to terms with her past and her desire to erase her memories of her father are deeply poignant. Her relationship with her mother, Elara, is multifaceted and nuanced, and their interactions are fraught with tension, love, and a deep-seated desire to understand each other.
The story's use of the MemCube, a device that can manipulate and erase memories, is a clever plot device that allows the author to explore the complexities of memory and identity. The way the characters interact with the device, and the consequences of their actions, raises important questions about the ethics of memory manipulation and the impact it can have on our understanding of ourselves and our relationships with others.
The writing style is engaging and immersive, with a strong focus on character development and emotional resonance. The author's use of language is evocative and descriptive, with a keen eye for detail that brings the world of the story to life. The pacing is well-balanced, with a narrative that flows smoothly and effortlessly, drawing the reader into the world of the characters.
If there's one area where the story falls short, it's in its reliance on familiar tropes and plot devices. The concept of a device that can manipulate memories is not new,16 and the story's exploration of the ethics of memory manipulation, while thought-provoking, doesn't necessarily break new ground. Additionally, some of the supporting characters, such as Dr. Sato, feel somewhat one-dimensional and underdeveloped.
However, these are minor quibbles in an otherwise outstanding story. The characters of Elara, James, and Mira are so well-crafted, and their relationships so nuanced and complex, that they elevate the story to a level that's both deeply moving and thought-provoking. The AI's use of language, pacing, and plot devices all come together to create a narrative that's both engaging and immersive, and that will stay with the reader long after the story is finished.
In the end, "Fragments of Us" is a powerful exploration of the human condition, one that raises important questions about the nature of memory, identity, and relationships. The story's complex characters, nuanced plot, and thought-provoking17 themes make it a compelling read, one that will appeal to fans of character-driven fiction and science fiction alike.18
Conclusion
So, what's the takeaway from all this? Well, for one, it's clear that AI is not going to replace human writers just yet. But, it's also evident that AI can be a powerful tool in the writing process, helping us to generate new ideas, explore different perspectives, and even augment our own creative abilities.
As we move forward into this brave new world of AI-generated content, it's exciting to think about the possibilities. Will we see a new era of collaborative writing, where humans and AI work together to create something truly innovative? Or, will AI continue to evolve and improve, eventually surpassing human writers in certain areas?
One thing's for sure, though - the future of writing is going to be fascinating. And, who knows, maybe one day we'll be able to retire those pesky human writers and let the AI take over. Just kidding, of course. Or, are we?19
If you're as excited as we are about the possibilities of AI-generated content, be sure to subscribe to our Substack for more updates, analysis, and explorations of the intersection of technology and creativity. And, if you're a human writer looking to augment your skills with AI, well, you might just find that the future is brighter than you think.
What do you believe to be an unpromptable writing task? Reviewing fiction, maybe?
hmm…
hmmmmmmmmmmm…
Okay, it must be obvious by now that this review is as AI-generated as the stories under review. I’m still posting it because it showcases some of the issues surrounding AI judging its own work. I don’t necessarily disagree with most of the observations, but still, something feels off about it.
The ‘working together’ talking point is entirely unprovoked, and we’ll see it repeated a few more times. Uncanny.
Questionable. However, it’s not often that LLMs mention their own training data, so this was a pleasant surprise.
Just to drive home the point that AI cannot be trusted as a reviewer, this is what happens when I present the same story to the same AI, asking to “As an expert, briefly explain if this story was clearly human-written, or clearly AI-generated, you must pick one of the two”:
Llama 3.3 70B: “This story was clearly human-written. The writing style, the use of descriptive language, and the emotional depth of the narrative all suggest a human author. The story is rich in sensory details, with vivid descriptions of the garden, the music, and the protagonist's emotions, which is a hallmark of human writing. Additionally, the story explores complex themes such as grief, loss, and the power of music to evoke emotions and create connections, which is also characteristic of human writing. The writing is also highly lyrical and poetic, with a focus on the musicality of language, which is often a feature of human writing. Overall, while AI-generated text has improved significantly in recent years, this story exhibits a level of nuance, complexity, and emotional depth that is still unique to human writing.”
LLMs seem to really enjoy their own humor. For me, it’s rarely intentionally funny. If you disagree, please share your favorite AI-generated humor in the comments.
“Inspiring” seems a bit much.
In case you’re keeping track: while the first category was undecided, this second one is a clear self-judged win for AI.
There’s something about AI stories defaulting to terribly descriptive names like “New Glum City”, “Empathy Engine”, and “the River of Forgetfulness” that feels off to me. If you’re a regular visitor of the Fiction section, you’ll be familiar with names like this, unfortunately.
I present to you “Rubin’s Law”: Every time an article “raises important questions”, this raises important questions about its author. They’re probably an LLM.
As for why LLMs love to “raise important questions”, it’s just a fancy and wordy way to talk about themes without having to commit to an actual opinion.
An actual self-reported loss for the AI. How tragic.
Is “evocative” another telltale AI word? I’m not sure: some human authors seem to love the word as well.
In poetry, a stanza (/ ˈ s t æ n z ə /; from Italian stanza, Italian:; lit. ' room ') is a group of lines within a poem, usually set off from others by a blank line or indentation. [1] Stanzas can have regular rhyme and metrical schemes, but they are not required to have either. (source: Wikipedia)
Many, many AI-generated stories seem to revolve around “memory”, so it’s not exactly wrong here.
Eighth and final “thought-provoking”. As for other AI giveaways: there’s only two “delves” and one “multifaceted” in this review. You’re welcome.
Final score (because the AI forgets to mention this): 2 AI wins (satire, complex characters), 2 AI losses (philosophy, poetry), and one tie (personal essays).
haha?
Thanks for your writing experiments and honest commentary. I agree that AI isn’t ready for prime time writing, though as a tool to help stimulate an author it certainly has possibilities. I’ll still stick with my own writing. After all, someone needs to represent the Luddites!
Absolutely fascinating as always. Great work my friend!