AI Engineering Playbook Part 7: A Guide to LLM Evaluation & Observability

So far in this series, we have architected and optimised our AI application. It is fast, cost effective, and follows best practices. But now we face the most important question, is it any good? And how do we fix it when it is not?

This is the "day two" problem. Building a reliable AI system is an ongoing process of evaluation, testing, debugging, and learning. This post covers my practical approach to these challenges.

1. Evaluation: How Good is the Answer?

Evaluating the quality of a non deterministic output is a major challenge. An emerging idea is to use an "LLM as a Judge," where one AI grades another's response. While I see the potential in using an LLM as a judge, I also believe it is only as good as the prompting used to control it.

For my projects, I have not needed to use it. Instead, I rely on a more deterministic and pragmatic approach. A combination of unit tests, checking for expected base responses, and validating the cosine match of the retrieved context to ensure its relevance. This gives me a high degree of confidence in the quality of the inputs, which is the most critical factor.

2. Testing: Building a Robust Safety Net

You cannot test an LLM's output like a pure function, but you can absolutely test the system around it.

• For my Q&A bots, it is easier to test. I can create a suite of tests with specific questions and assert that the generated answers contain the key information that should have been retrieved from the context.

• For the Admin Co pilot in my CMS, where it performs calculations, the answers had to be correct. This allowed me to write much stricter tests for the final outcome. However, this still required testing with a large library of example prompts to ensure it could handle the variety of ways a human might ask a question.

3. Debugging: The Philosophy of "No Bad Answers"

This has become my core philosophy for debugging AI systems: the AI has never given a bad answer, it is the answer you allow it to give. An unexpected response is not a random error from the model, it is a bug in my own system. The response is not refined enough because the input or the context you have fed it was not good enough.

My debugging process is always to check my own work first:

1. Was the correct context retrieved by the RAG pipeline?

2. Was the final prompt constructed correctly?

If the answer to both is yes, then the prompt itself needs refinement. To understand why the model went off track, you can use tools like LangSmith to get a detailed trace. Or, a powerful technique is to simply ask the model to give a detailed output and the reasoning behind it. This "meta debugging" is an incredible way to see the model's "thought process" and learn how to refine your instructions.

4. The Feedback Loop: Learning From Your Users

The ultimate test of your system is how it performs with real users. That is why building a feedback loop is essential.

For my projects, I have a planned feedback loop for when the bot gives a wrong answer, like a simple thumbs up or down. But I also wanted to be more proactive. For the CMS helper bot, I built a feature where if it detects user annoyance from certain phrases, excessive capital letters, or aggressive language in their query, it will automatically interrupt its own process. It will then offer to contact an admin and create a message for them via the in site messaging system.

This turns a potentially negative user experience into a positive and helpful support interaction, providing valuable feedback while also helping the user.

Conclusion

Building a trustworthy AI system is an ongoing engineering discipline. By combining automated testing, pragmatic evaluation, a disciplined debugging philosophy, and a robust user feedback loop, we can move beyond building clever demos and start delivering professional, production ready AI applications.

In Part 8, we will cover the crucial topic of keeping our systems safe: a look into AI security.