What I learned from content designers who’ve built AI features and LLMs

While I’ve yet to be part of an AI project at work, I want to be prepared when that day comes. 

So since the whole AI thing blew up, I’ve been trying to keep a pulse on the topic. But the amount of information out there is utterly overwhelming.

I wanted to learn specifically about how content designers and UX writers can create AI-powered experiences. And I wanted information straight from the horse’s mouth — not “experts” who simply write about it. 

It took some work, but I found a handful of insightful podcasts and articles where writers talked about their experiences working on AI or LLMs in their roles.

This is what I gathered:

1. Writers need to tell the LLM exactly who it is and how to talk. An existing brand voice or copy guidelines make good artefacts to start with.

2. Writers need to check and fix the content fed into the LLMs for it to avoid generating wrong responses from outdated data.

3. Writers need to know how to tweak pre-trained LLMs with prompts and feedback so that it can fulfil specific business cases.

Based on what these writers said, I’ve also dug a little deeper to find out what things like “few-shot writing” and “Retrieval-Augmented Generation” are. And who better to ask about AI than AI tools? Yes, I did use ChatGPT, Perplexity, DeepSeek and Claude to do research for this article, but the words in this post are human-generated (perhaps now I should skip proofreading to make it sound less perfect?).

If you’re a UX writer, content designer, or content strategist looking to learn how you can contribute to an AI project that uses a large language model (LLM), this article is for you.

Stop making goggly AIs at it

As the AI buzz has companies in a chokehold, you might be expected to be part of an AI project sooner or later. But you might also find yourself thinking, “Is AI really necessary for this?” If you have a hunch that it’s not the best solution, suggest to your team to explore the problem, assumptions, and user needs through a discovery phase first.

“A discovery is needed anytime there are many unknowns that stop a team from moving forward. Moving forward on only assumptions can be risky, as the team may end up solving a problem that doesn’t really matter — wasting time, money, and effort… In some cases, the end of a discovery might be a decision not to move forward with the project because, for example, there isn’t a user need.” — Discovery: Definition by Nielsen Norman Group

The outcomes of the discovery should include:

1. A problem statement

2. A user journey

3. User needs

4. High-level concepts

Tell it who it is and how to talk

If the AI feature the team agrees to build uses a LLM, a good place to start is by providing the model with specific guidelines on its identity and conversation style. 

Both Morgan Marie Quinn (Content Design Lead at Google DeepMind) and Chris Cameron (Principal UX Writer at Booking.com) mentioned that they started by telling the LLM who it is. What’s brilliant is that these statements don’t need to be written in code. An LLM is able to understand commands such as “You are a trip planner for Booking.com.”

Learn more from Morgan and Chris via these Medium articles:

• Girls just want to train and scale large language models

• Bedtime stories, AI chatbots and the future of UX writing

To build an LLM’s identity and voice, start is with the company’s existing brand voice and content guidelines. If you don’t have this at hand, you can take a page from the Plain English Campaign. There are many free guides on their site, and the one on How to write in plain English has rules like “Your writing is clear, helpful, human and polite” and “Be professional, not emotional” which can form a good foundation for an LLM’s voice.

When it comes to tone of voice, Trisha Causley and Maria Hofstetter (content designers at Shopify working on AI features) said in the Content Strategy podcast episode that content designers need to nail down specific traits and attributes of the tone. For example, a “playful” tone would mean using contractions, puns, or emojis. 

Feed it with up-to-date sources

Once the model understands how to speak, it’ll need to know what to speak about.

To prevent the model from spouting nonsense, it’s vital that it gets its information from up-to-date sources. Bad input will result in bad output.

This is where content designers need to step in and vet the sources of information that developers intend to feed into the model. In Why AI won’t work without content design by Helen Gaskell, Senior Content Designer at Caution Your Blast Ltd (CYB), she warns of the high risk of hallucinations when there’s such a vast amount of old content out there in the wild.

“Right now, the only answer I can see to creating accurate chatbots is for us to fix the information we feed them — and content designers are absolutely perfect for this job. It is us who tracks and audits information; it is us who writes clearly, accurately and concisely. We fact-check, we find the source of truth, and we work out what people really need when they’re looking for answers.” — Helen Gaskell

Instead of relying on static sources which require manual content updates and LLM re-training, we can also think about using Retrieval-Augmented Generation (RAG). This AI framework, as described by Marina Danilevsky (Senior Research Scientist at IBM for Language Technologies), tells the LLM to check with another source of truth (e.g. the internet or well-maintained internal knowledge base) that’s outside of their training data before generating an answer. This framework is also more responsible as the LLM would say “I don’t know” when it can’t retrieve the relevant information.

Train it with fine-tuning, few-shot writing, and feedback

Now that the model knows its identity, voice, and information sources, it needs to be coached to go beyond its default training data.

To customise a pre-trained LLM for a specific business case, fine-tuning is required. It’s the process of feeding the model additional labeled datasets to build its knowledge on a specialised area. 

Based on the model’s responses, content designers then need to evaluate the output critically and articulate from a language perspective whether the output is satisfactory or not. For example, “It’s not concise because the sentence has more than 20 words.”

This echoes what Laura Costantino, Senior AI Model Designer and Prompt Engineer at Google DeepMind, talks about on the Content+AI podcast episode. She points out that based on how the model is responding, content designers need to be able to identify patterns of “good” and “bad” behaviour, and think of ways to create language and content strategies that can guide it in the right direction.

DeepSeek also listed “few-shot writing” as a strategy, where content designers can provide 2–10 examples of how the model should respond to specific tasks. 

Generated with DeepSeek. Prompt: few shot writing examples for fintech.

Takeaways

The way I see it, if AI continues to be adopted widely, companies would need language experts to define and evaluate how it should behave.

Content designers can bring value to an AI-powered feature by:

1. Telling it who it is and how to talk: Start with a brand voice and copy guidelines

2. Feeding it with up-to-date sources: Fix the static information going into the model or opt for more scalable strategies like Retrieval-Augmented Generation (RAG).

3. Training it with fine-tuning, few-shot writing, and feedback: Provide the model with additional task-specific datasets, specific examples to guide its responses, and give it feedback on its responses. 

Generated with DeepSeek. Prompt: create a table that lists out the differences between few shot writing, fine-tuning, and RAG.