ChatGPT and Content Strategy: Will it work?

ChatGPT is all the rage these days. Launched in November 2022, it is a language model developed by OpenAI that uses deep learning to generate text. It’s been touted to generate high-quality content in a fraction of time compared to traditional methods. 

According to OpenAI CEO Sam Altman, it gained 1 million users in five days. Compare that to Facebook, which crossed the same mark in 10 months. 

Naturally, we got curious and wanted to find out if it could be useful for our content strategy work.

PebbleRoad has a team of content strategists who help clients to create, deliver and govern useful and usable content. Some of our tasks include: 

• Primary and secondary research to understand business goals, user needs, current behaviours and more.
• Information architecture and content modelling to improve findability, consistency, reusability, searchability and content management.
• Style guides to define the voice and tone of the organisation and ensure that content owners can create quality content at scale.
• Content design to answer user needs most effectively (e.g. through calculators or tabs instead of dense paragraphs).
• Copyediting to simplify complex language and processes.

Putting ChatGPT to task

Would ChatGPT be able to replicate the work we do or at least help us do our work more quickly? We ran a quick experiment by asking it to do some of the tasks we normally do:

• Brainstorm questions for a UX research interview
• Propose a content model for advisory circulars
• Propose voice and tone guidelines based on sample text and scenarios
• Propose content style guidelines for tabs and toggles
• Rewrite an existing page

The results

We found that ChatGPT was great for:

• ‍Brainstorming, first drafts and rephrasing: The tool helped us to brainstorm interview questions efficiently and suggested questions we had not thought of! It could also propose possible fields, field types, instructions and character limits for our sample content model.‍
• Drafting voice and tone guidelines: The tool was able to propose some useful voice and tone guidelines based on sample text AND a scenario. More specific guidelines should still be crafted in consultation with internal stakeholders and tested with users, though.‍
• Simplifying technical terms in plain English: ChatGPT excels at explaining complex terms and ideas. This can cut editing time significantly and empower subject matter experts to simplify content at the source (instead of going through rounds of edits and discussions).

But from our experience, it seemed to struggle with:

• Editing longer copy: Other AI tools such as Writer might be better at applying and managing multiple rules from your content style guide. Especially if you are applying those rules to longer pieces of information. 
• Content design: As it is still a text-based tool, ChatGPT might not be able to propose the best content patterns for consumption. For example, it struggled to propose more unique and complex patterns such as calculators, tabs and toggles
• Content style guidelines: As of the time of writing (February 2023), ChatGPT seemed to conflate design, development and content guidelines. Though some of the guidelines might be relevant, teams should definitely refine  their guidelines based on experience and best practices.

Interacting with ChatGPT

Our interactions have taught us that we need to ask the right questions, along with the appropriate inputs to get more out of ChatGPT.

Caveat emptor

An important thing to bear in mind is that while trillions of words have been fed into ChatGPT, its training data only goes up to 2021. (Accurate as of February 2023) This means it isn’t aware of more recents events, new discoveries or research breakthroughs. For instance, it doesn’t have the latest on the war in Ukraine. ChatGPT Plus, the paid version, however might offer new features and improvements.

Also, as its training data is drawn solely from the Internet, which can contain inaccuracies or misinformation, it’s super important to verify what it churns out by checking multiple sources!

As computer scientist Jonathan May puts it: “... it doesn’t have facts, per se. It just knows what word should come next… ChatGPT doesn’t try to write sentences that are true. But it does try to write sentences that are plausible.”

How to get the best out of ChatGPT

(As advised by ChatGPT itself)

• Be clear and specific: Ask direct, well-defined questions to get the most accurate and helpful answers. P/S: Try this prompt that gets ChatGPT to write its own prompts!
• Give context: Give ChatGPT as much information as possible about the topic you're asking about so that it can give you the most relevant response.  P/S: you can even ask it to write in the role of a specific person e.g. a tech reviewer or a food reviewer. 
• Use proper language and grammar: ChatGPT has been trained on high-quality, written text, so using proper language and grammar will help it understand and respond to your questions better.
• Be patient: Remember that ChatGPT is a machine and may need time to process your request. It may also ask for clarification or additional information if needed.
• Provide feedback: If ChatGPT gives an incorrect or unsatisfactory response, provide feedback so that it can learn and improve its responses. It’s a “baby”, albeit a super intelligent one!  

More experiments to come

Artificial Intelligence is here to stay, and it's improving exponentially every day. Learning to get the best and most out of AI is a valuable skill we need to hone right now.

We'll certainly be running more #PRexperiments on AI and other emerging tech to see how they can help us deliver our best work for clients.

If you would like to explore using AI in your organisation, do connect with us by emailing info@pebbleroad.com.