This blog is an excerpt from the ebook: “A primer for data readiness for generative AI.” Download the complete version here.

Whether your team uses a foundation model as a general-purpose productivity aide or supplements it with your organization’s unique data, two critical keys to success include writing good prompts and catching hallucinations. If your team does end up training a foundation model with proprietary data, make sure to protect and govern that proprietary data.

Prompt design and prompt engineering

A prompt is the text input a user supplies to an AI model to receive an output. Prompt design and prompt engineering are the new disciplines centered around 1) crafting prompts that elicit helpful responses and 2) systematizing and governing the use of prompts, respectively.

Crafting good prompts is fundamentally about clearly communicating to the model in a way that minimizes ambiguity and provides as much useful context as possible. This is typically done through the prompt interface itself, though accessing a generative AI through an API often provides direct access to parameters as well.

The following principles originate from the Udemy course The Complete Prompt Engineering for AI Bootcamp by Mike Taylor and James Phoenix – Check it out if you want to learn more about prompt design and prompt engineering! The authors know their stuff.

1. Give directions – Add adjectives, context and other descriptors and guidance about your intended output. This includes including the user’s persona in the prompt – you can tell the model what to emphasize and what kind of detail to offer.
   
   Models such as ChatGPT will often ask you to do so if you provide an especially sparse prompt. The naive approach to writing a prompt is also known as zero shot prompting, in which a request is made of a model without enumerating any explicit examples, guidance or broader context. This forces the model to generalize from preexisting knowledge, with a high likelihood of producing spurious results. 

2. Provide examples – The quality and consistency of responses can be vastly improved by providing both positive and negative examples for the model to anchor itself.

3. Format the response – Show or enumerate to the model the format of the response you want. If you need information presented as a numbered list or bullet points, it doesn’t hurt to say so explicitly. This consideration is especially important for generating code; if you need data presented in JSON or tables, a foundation model should be able to produce those, as well.

4. Divide labor – Not everything has to be accomplished with a single prompt. You can take advantage of your session’s short-term memory with a succession of prompts that add additional details, direction, formatting, etc. to continuously refine the output. You can add context at any point during the process using embeddings from a vector database. Finally, you can import the output from one model into another, more specialized one to further refine your results.
   
   
5. Evaluate outputs and iterate – Prompting should be treated as an iterative process. The “engineering” side of prompt engineering is a matter of systematically testing, evaluating and iterating through different configurations of prompts.

The following is an example of a naive prompt:

Can I have a list of marketing headlines for blogs?

A more helpful prompt provides as much context as possible:

Can I have a list of ten marketing headlines for blog?
Description: Explain the benefits of automated data integration
Audience: analysts, data engineers, data scientists
Seed words: data integration, data engineering, analytics, idempotence, change data capture, automation, efficiency, reliability, ETL, ELT
Examples: What is the architecture of automated data integration?, the ultimate guide to data integration

Poor prompt design won’t just return misleading or meaningless results. Poor – and malicious – prompting can lead a model to expose confidential information in its response. It can also generate responses that describe or endorse antisocial behavior. Microsoft’s Tay chatbot famously became a bigoted monster within hours of deployment after interactions with malicious users. Even worse, hacks known as prompt injections can be used to misdirect a model and produce any response the hacker wants, including social engineering such as phishing attempts.

The ability of generative AI to produce content on a large scale while mimicking well-known styles and patterns also poses a danger in terms of intellectual property theft. Yet another malicious application of generative AI is to intentionally produce content that is false and violating in some way, such as combining deepfakes with explicit content.

Minimizing hallucinations

Earlier, we discussed the importance of evaluating outputs. A well-known pitfall of generative AI is that it can produce spurious results that aren’t factual or connected to reality in any way (we’ve all seen the AI-generated pictures of people with an improbable number of fingers). The stakes can be high; there has already been at least one documented instance of a lawyer citing hallucinated court cases. There are several ways to minimize the frequency and impact of hallucinations.

The first is to simply fact-check results and reject bad ones. Recall that generative AI models learn through reinforcement; one approach may be to keep humans in the loop to extensively test the answers provided by a model until it meets some threshold for accuracy and relevance before any system built on top of a generative AI model is released to the public or used to support any serious decisions.

The second approach is to ensure a high-quality upstream training set by carefully curating and governing the data your organization provides to its foundation model. You don’t have to put all of your organization’s data into a vector database to feed your foundation model. Instead, you should carefully consider the problem you are trying to solve using generative AI and select for content that is unbiased, accurate, relevant and internally consistent. During the COVID pandemic, diagnostic AIs failed to catch COVID cases in large part due to poor quality data.

A third approach is to supplement the foundation model with a knowledge graph that explicitly encodes real, semantic relationships between concepts. This enforces explainability.

You should pick a foundation model that was trained on a similar base of data as your own data. There are many publicly available models that specialize in different media (text, images, audio, etc.), domains (geospatial analysis, music, etc.) and use cases (sentiment analysis, data visualization, etc.).

Protect your generative AI

There are several general principles to safeguarding your organization’s use of generative AI. One principle is fundamentally an upstream data governance and security issue, and mainly concerns anonymizing, encrypting or altogether excluding sensitive data from the training set. This is a prerequisite for all lawful and ethical uses of data, per GDPR and other legal and regulatory requirements. 

Knowledge graphs provide an opportunity to explicitly encode safeguards into a generative AI model. A knowledge graph can provide a model with access control policies and flag data that is too sensitive to safely expose. 

Another principle is to maintain control over the prompts themselves. Rejecting or sanitizing prompts that include certain phrases or concepts before they can reach the model is one way to prevent obvious misuse.

The mirroring principle is to screen the output of a model, ensuring that end users are never exposed to outputs that contain sensitive data, contain abusive or antisocial messages, etc. In both cases, the screening can be performed through human review, a rule-based system or even another AI trained specifically to classify and evaluate content.

Finally, generative AIs are ultimately trained and improved over time with human supervision in the loop. Allowing end users of a generative AI system to report bad results will improve your model’s efficacy and safety.

[CTA_MODULE]