Part 3 of The AI Engineering Playbook, we dive into the craft of communication. A guide to prompt engineering, from simple "few shot" examples to advanced techniques like "Chain of Thought" and managing your prompts like production code.
In the last part of this series, we discussed the high level architecture of an AI system. Now, we zoom in to the most crucial point of interaction, the prompt. Prompt engineering is far more than just "asking the AI a question", it is a legitimate engineering discipline that combines instruction, data, and examples to control a model's behaviour.
Mastering this craft is the key to building AI features that are reliable, predictable, and robust.
1. The Basics: From Zero Shot to Few Shot
The most fundamental concept in prompting is the difference between a "zero shot" and a "few shot" prompt.
Zero Shot: You ask the model to perform a task directly, with no prior examples.
Few Shot: You provide the model with one or more examples of the task being completed correctly before you ask it your question.
A perfect example of this came up when I was helping with making a Twitch bot. A zero shot prompt, like asking it to respond to comments, would often fail to grasp a user's sarcasm. However, by providing a few shot example of a sarcastic comment and a witty, sarcastic reply, the bot learned the pattern. It began to understand the nuance and could even be sarcastic back, making it feel much more alive and intelligent. For any task that requires nuance, few shot is the way to go.
2. Advanced Techniques: Forcing the Model to "Think"
For complex reasoning or mathematical tasks, you cannot just ask for the answer. You need to guide the model's reasoning process. The most powerful technique for this is Chain of Thought (CoT) prompting.
The idea is simple: you instruct the model to "think step by step" or "show its work." This forces it to slow down and follow a logical process instead of jumping to a likely, but incorrect, conclusion.
I used this exact technique in my CMS Admin Co pilot. I needed it to calculate complex race scores from a set of rules. Initially, it would often hallucinate the final numbers. The solution was a form of Chain of Thought. I had to explicitly instruct it to use correct mathematical principles like BODMAS and to break down its calculations. By forcing it to follow a logical sequence, its accuracy went from unreliable to production ready.
3. Output Control: Getting the Format You Need
One of the biggest challenges when using an LLM in an application is that it loves to be chatty. It will add conversational text like "Sure, here is the JSON you requested..." which will break your code.
The key is to be extremely direct and leave no room for ambiguity. I am constantly telling the LLM what to do in my prompts. I use firm, prescriptive language to force the output into the exact format I need. My prompts are filled with instructions like:
"Only output a valid JSON object and nothing else."
"Only use the provided functions."
"Do not include any conversational text in your response."
You cannot be polite or subtle. You must engineer the prompt to remove any possibility of the model deviating from the required format.
4. Prompt Management: Treating Prompts as Code
As your applications grow, simply having these prompts as long strings in your application code becomes unmanageable. Prompts are a critical part of your application's logic and should be treated as such.
In my own workflow, I always manage prompting via code, with all prompts stored in template files and checked into Git. This provides version history and allows for peer review. I also keep them modular, so small, reusable prompt fragments (like the rule about only returning JSON) can be referenced and combined as needed. This "containerised" approach makes the entire system much easier to maintain, test, and scale.
Conclusion
Prompt engineering is the new frontier of software development. It is a skill that requires precision, logic, and creativity. By mastering these techniques, you can move beyond creating simple AI novelties and begin engineering truly robust and reliable AI powered features.
In Part 4, we will dive deep into the most powerful architectural pattern for building with custom knowledge: Retrieval Augmented Generation (RAG).