Part 2 of The AI Engineering Playbook, we move from model parameters to the big picture. A guide to the crucial architectural decisions you must make before writing code: choosing the right model, designing your system, and budgeting for success.
In Part 1 of this series, we covered the core language of AI, the fundamental parameters like tokens and temperature that control the model's output. Now, we zoom out from the individual API call to the big picture. Before you write a single line of code for your AI feature, you need to make a series of high level architectural decisions.
Making these choices thoughtfully is what separates a successful, viable product from an expensive, frustrating science experiment.
1. Model Selection: A Strategic Choice, Not a Popularity Contest
The first question is always, which model should I use? The debate between a powerful commercial API versus a flexible open source model is a crucial one.
My personal approach is always driven by the specific job to be done. It is not about what is most hyped, but what is most suitable. For my freelance CMS project, the goal was a cost effective, reliable help bot. I already had a Google suite for the client, so using Google's AI API was a natural, low friction choice. In contrast, for my personal coding assistants, I often prefer Anthropic's Claude due to its large context window and strong reasoning capabilities. While this blog uses Google Nano Banana for image creation due to its ability to create natural looking images. The choice is always pragmatic.
The third path, running an open source model on your own hardware, offers total control and privacy but comes with significant maintenance overhead. It is a trade off you have to make consciously.
2. System Design: Synchronous vs. Asynchronous
Not all AI tasks are created equal. A critical design decision is whether your system will process requests synchronously (the user waits for an answer) or asynchronously (the task runs in the background).
I have faced this decision directly in my recent projects, and they serve as a perfect case study.
The Synchronous Case: The "Admin Co pilot" in my CMS is a classic synchronous task. An admin asks a question like "calculate this scoring formula" and expects an answer immediately. The user experience demands a real time, blocking response.
The Asynchronous Case: In contrast, my video analysis SaaS is a perfect asynchronous task. A user uploads a multi hour video for processing. It would be a terrible user experience to make them stare at a loading spinner for 20 minutes. Instead, the process is kicked off in the background, and the user can be notified by email or a dashboard update when their clips are ready.
Choosing the right pattern is fundamental to designing a good user experience.
3. Budgeting: The Hard Reality of "Tokenomics"
The most brilliant AI feature is useless if it is not financially viable. The economics of token usage, or "Tokenomics," is a reality that every AI developer must confront.
I learned this the hard way on a previous project, which I detailed in my "Journey Through Three Backend Rewrites" post. My second attempt at a backend used an always on, containerised solution in a VPC, and the AWS costs were shockingly high for a solo project that was not generating revenue.
That experience taught me to be ruthless about cost from day one. My personal best practices now include:
Constant Monitoring: I always keep a close eye on my token usage via cloud dashboards and set up billing alerts. There are no surprise bills if you are paying attention.
Vectorise to Minimise Context: I always vectorise my data for any RAG based system. This is not just for accuracy; it is a powerful cost saving measure. A vector search allows me to send a small, highly relevant piece of context to the model instead of a large, expensive one.
Conclusion: Plan Before You Code
Choosing your model, deciding on a synchronous or asynchronous architecture, and planning your budget are the three pillars of a successful AI project. These are the decisions that prevent costly rewrites and ensure the feature you build is not just clever, but practical, sustainable, and provides a great experience for your users.
In Part 3, we will dive into the craft of communicating with the model, a deep dive into the art and science of Prompt Engineering.