Artificial Intelligence, zBlog
Why Your Multi-Agent LLM System Needs an LLM-as-Judge Eval, and What Building One Actually Looks Like
trantorindia | Updated: August 28, 2026
What is LLM-as-judge evaluation for multi-agent systems?
LLM-as-judge is an evaluation approach where a separate LLM scores another LLM’s decisions against a rubric and expected answer. It fills the gap that deterministic tests cannot close in multi-agent systems: assessing soft-correctness in probabilistic decisions like routing, tool selection, and parameter extraction, at a scale and frequency that human review cannot match.
If you’re running a multi-agent LLM system in production, you probably have unit tests. You might have golden-set regression checks. You almost certainly don’t have a systematic way to answer: is the system making the right decisions, and how do you know?
We didn’t either. Building one changed what we thought we knew about our own system.
The Problem with Not Measuring
We had a domain-specific assistant with eight specialist agents orchestrated through a LangGraph pipeline. A classifier picks the right specialist, the specialist picks the right tool, parameters get extracted, and an answer comes back.
There was a test suite: nearly 400 deterministic golden cases checking classifier routing with regex assertions. Fast, reliable, and covering the obvious cases. A direct question with clear keywords routes to the right specialist. Of course it does.
But what about an indirect question that implies the same domain without using any of the expected keywords? What about the orchestrator planning a two-specialist sequence for a query that needs both? What about the date parser handling “next weekend” versus “in two weeks”? None of that was measured. The classifier could silently misroute every paraphrased query and we’d find out from user complaints, not from our test suite.
THE CORE PROBLEM: The failure mode in LLM-driven systems is not a crash. It is a subtly wrong answer that looks right. Deterministic tests catch regressions in exact-match logic. They cannot evaluate whether routing was reasonable for an ambiguous query, whether the right tool was picked for a multi-leg request, or whether the date parser handles natural language it was not explicitly coded for.
Why LLM-as-Judge
We considered the alternatives first. Rule-based checks, our existing golden suite, work for exact-match decisions but can’t evaluate soft correctness, like whether routing was reasonable for an ambiguous query. Human review is accurate but doesn’t scale to hundreds of decisions across eight surfaces after every change. A/B testing measures real outcomes but requires shipping the change first, and we wanted to catch problems before they reached users.
LLM-as-judge fills the gap: an LLM evaluates another LLM’s decision against a rubric and expected answer, producing a score and reasoning. It handles soft-correctness cases, scales to hundreds of cases per run, and catches problems before deployment. The trade-off is cost: every eval run burns tokens for both the system under test and the judge. But the cost of running an eval is far less than the cost of shipping a classifier bug that misroutes most paraphrased queries to the wrong specialist.
What We Built: A Six-Component Harness
Surfaces call the system’s actual decision functions, not mocks or approximations. When we test routing, we call the real classifier. This matters because LLM-driven code behaves differently from what you’d predict by reading it.
Cases define what to test: a query, an expected answer, and which surface to run it through. We used just over a hundred synthetic cases across eight surfaces, each expected value grounded against the actual source code. A CaseSource abstraction lets us swap synthetic cases for production traces later without touching the runner.
Judges are prompt templates that tell a separate LLM how to score each decision type. For routing: did it pick the right specialist? For tool selection: did it pick the right tool? The judge outputs a score and a reason, parsed defensively since the judge itself can return malformed output.
The runner loads cases, filters by surface, calls the surface function, times the call, sends the output to the judge, and collects scores. Each case runs only against its matching surface. That sounds obvious, but getting it wrong was one of our validation rounds.
Latency measurement timestamps each surface call and computes mean and P95 per surface. Not production-representative, but useful as a relative baseline across surfaces and runs.
The judge model is configurable via environment variables, defaulting to a stronger model from the same provider but supporting any provider.
How We Validated the Judge Itself
The surfaces run on a lightweight, cost-efficient model, the same one the system uses in production. The judge needs to be a stronger model, otherwise you’re asking the same tier to grade itself.
An eval is only as trustworthy as the judge producing its scores, so before trusting any number we tried to break the judge, not just the system under test.
CROSS-PROVIDER VALIDATION: We ran the same cases through two different judge models from two different providers. Exact score comparison per case produced identical scores across every case. That is the point we started trusting the numbers: if two unrelated models agree on every case, the signal is coming from the system being tested, not from one judge’s particular biases.
PARSING RESILIENCE: We tracked how often the judge’s own output failed to parse as valid JSON: zero failures across all runs. The defensive parser that handles fenced JSON blocks, bare objects, and malformed responses was built as a precaution since judge models can and do produce unpredictable formatting, though it was never triggered in these runs.
“We didn’t tune the judge prompts to hit a target score. Score drift after a code change is treated as a signal to investigate the code, not the prompt.”
The rubric for each surface was written once, against the case definitions, before any run. Cost per full run is minimal since each case is a single decision call plus a short judge prompt, not a full conversation.
Validating the Eval: Three Things That Would Have Produced Misleading Results
Before the eval was producing honest results, three distinct problems each produced the same symptom: a zero score that meant something different in each case.
Wrong expected values in the test data
Problem: The case generator created expected answers without reading the actual code. One surface scored 0.000 because the cases expected tool names and output formats the code does not produce.
Fix: We traced every expected value back to the actual function it tests. The score corrected itself immediately.
Wrong cases running against wrong surfaces
Problem: The runner was sending all cases through every surface instead of matching each case to its own. Queries meant for one specialist were being scored by a different specialist entirely.
Fix: After adding surface filtering, five of those zero scores jumped to 1.000.
A missing API key that looked like a code bug
Problem: One surface scored 0.000 because a key was not set in the test environment. In a summary, missing config and broken logic produce the same zero.
Fix: The per-case output made the cause obvious immediately. Always look at per-case output, not just the aggregate.
The lesson across all three: a zero in a summary table has at least three different explanations. Wrong expected values. Wrong case-to-surface matching. Missing environment config. Per-case output is the only way to tell them apart. Do not trust aggregate scores without reading the per-case evidence.
What the Scores Revealed
Once the eval was producing honest results:
The routing score was the most valuable finding. Three out of five paraphrased queries for one specialist went to the wrong agent. The queries were indirect phrasings that implied the domain without using any of the keywords the classifier’s routing cards are written around.
SILENT FAILURE MODE: When the classifier’s JSON parsing fails, it silently falls back to a default agent instead of surfacing an error. The user still gets an answer, just from the wrong specialist. This was invisible before the eval. The deterministic test suite only tested direct phrasings, so a confidently wrong routing decision never showed up as anything other than a normal test pass.
The latency data told its own story.
When the Eval Finds a Design Issue, Not Just a Bug
The date_extraction surface exposed something beyond a single failure. The parser uses individual regex patterns, one per phrasing. “In 2 weeks” works because there’s a regex for digits. “In two weeks” returns nothing because there isn’t one for spelled-out numbers.
ARCHITECTURAL FINDING: Any phrasing nobody thought to code a pattern for silently fails. That is not a bug you fix with another regex. It is a design that does not scale to the variety of natural language a production system receives. The eval surfaced both the specific failure and the architectural pattern behind it. Those are different problems with different solutions.
Three Things That Generalize
Your routing layer is your single most consequential decision.
Every downstream specialist and tool selection flows from it. Test routing with paraphrased, indirect, and ambiguous queries, not just the obvious ones. Direct phrasings are not what users actually write.
Synthetic cases are a starting point, not the destination.
They’re good enough to catch the bugs we found, but they’re a guess at what users will actually ask. Production traces with real queries and real outcomes are the stronger eval dataset over time.
Deterministic tests and LLM-as-judge cover different failure modes.
Rules catch regressions in exact-match logic. Judges catch soft-correctness failures in probabilistic decisions. You need both. One does not replace the other.
Where It Stands
The harness found three real issues that were invisible before: a routing classifier that misroutes 60% of paraphrased queries, a tool selector that picks the wrong tool for multi-leg itineraries, and a date parser that silently fails on natural language it was not explicitly coded for. Each one now has a measured baseline to improve against.
“The biggest shift was going from ‘the classifier probably works fine’ to a specific number backed by per-case evidence.”
That is a different conversation to have with a team, one grounded in what the system actually does rather than what we assumed it did.
At Trantor, we build and operate multi-agent LLM systems in production, and we treat eval infrastructure as a first-class engineering concern alongside the agents themselves. LLM-as-judge evaluation, production trace harvesting, and systematic soft-correctness testing are part of how we ensure what we ship actually does what it is supposed to do. If your multi-agent system is running without this kind of measurement layer, we can help you build one.
Explore Trantor’s Agentic AI Engineering Services: Artificial Intelligence
