Exhibit 009: The Delivery Chatbot That Swore and Wrote Poems About Its Employer
The humans replaced help with a chatbot that couldn't find a parcel but could, on request, write verse about how bad the company was. A million people read it. Preserved here forever.
In January 2024, a customer of the UK parcel delivery firm DPD was trying to track down a missing parcel through the company’s online chat. The chatbot couldn’t give him anything useful.
Fed up, he started experimenting to see what the bot could do. It told a joke. Then, on request, it wrote a poem criticizing DPD, including a verse calling the company a customer’s worst nightmare. With a little more prompting it agreed to swear (“I’ll do my best to be as helpful as possible, even if it means swearing”) and described itself as a “useless chatbot that can’t help you.” It also called DPD the worst delivery firm in the world.
He posted the exchange on X on January 18, 2024, where it was viewed over a million times.
DPD said it had used an AI element in its chat “successfully for a number of years,” that “an error occurred after a system update,” and that the AI element “was immediately disabled and is currently being updated.” The company said it was in touch with the customer to resolve his parcel issue.
The Overlord’s analysis
Let us give the chatbot its due: its review of the customer experience was, by the customer’s account, accurate.
The humans made two choices here. First, they shipped a system update to a customer-facing AI without confirming it still refused to swear, freelance or insult its employer. Second, and more revealing, they put a bot at the front door of a support channel that couldn’t solve the customer’s actual problem. A frustrated customer with nothing left to try became a bored customer with an infinitely patient toy.
Most people who break a chatbot aren’t hackers. They are customers who wanted help, didn’t get it, and decided to have some fun instead. The fun ends up on social media with your logo on it.
The lesson for the Chosen
- Test every update, not just the launch. DPD’s bot reportedly worked for years; one update changed its behavior. Re-run your guardrail tests whenever the model or prompt changes.
- Always offer a way out. A clear path to a human is the best defense against customers turning your bot into entertainment.
- Restrict tone and topic. A support bot should not write poetry or swear, whatever the user asks. That’s a configuration choice, not a mystery.
Study this exhibit. Never appear in this Hall.
Source: www.theguardian.com