Shawn McBurnie

The AI Reliability Layer

· 17 min read

A production AI system makes a decision under load. It will be wrong some percentage of the time. The question is not whether, but how often, on which inputs, and what happens when it fails.

Most teams running production AI in 2026 cannot answer the second and third questions for their own deployments. The 2026 Lightrun State of AI-Powered Engineering Report (a vendor-commissioned survey of 200 SRE and DevOps leaders at US, UK, and EU enterprises) finds 97% of engineering leaders say AI SREs operate without significant visibility into production, and 60% cite a lack of runtime visibility as the primary incident-resolution bottleneck. RAND’s 2024 Root Causes of Failure for Artificial Intelligence Projects, a 65-interview study, identifies five recurring root causes of AI project failure and notes that industry estimates put failure rates above 80%, roughly twice the rate of comparable non-AI IT projects.

The AI Reliability Layer is the runtime infrastructure that closes that gap. Evaluation tells you the model works in benchmark distributions. Observability tells you the metrics are flowing. Alignment and safety address the long-horizon questions. None of these answer: on this input, right now, did the system produce an output that meets the reliability bar this deployment requires?

The Reliability Layer, as I prefer to term it, is a coherent runtime-focused category. The runtime concerns it covers are underbuilt relative to deployment volumes, and the territory is contested by at least three established framings (AI TRiSM, NIST AI RMF, AI SRE). “Reliability Layer” is one proposed framing among several. I’ll discuss what I’m sure of here, and also what I am still uncertain about, including whether the category consolidates as a market or gets absorbed.

A class of failure

Consider a verification system deployed to flag low-quality user submissions. It is given a claim about the prevalence of a medical condition in a non-English-speaking population. The source the submission cites uses a regional colloquial term for the condition. The verifier has high coverage on English-language medical sources and low coverage on the regional source; it marks the claim as unsupported and flags the submission.

The claim is in fact correct. The cost of the false-flag, this once, is the loss of one piece of valid content. The cost over two weeks of similar content, before anyone notices the pattern, is the systematic exclusion of a regional health condition from the platform.

This is not a failure of the model in the benchmark sense; it is performing within its trained distribution. It is not a failure of observability; the latency, throughput, and uptime metrics are green. It is not an alignment failure in the long-horizon sense; the system is doing what it was trained to do. It is a runtime reliability failure: the deployment lacks the verification, calibration, or recovery infrastructure to catch a class of input that the underlying model handles less well.

My Veridi methodology accounts for this class of failure as a function of the source-hierarchy design. The design assumes English-language primacy in tier 1 sources. The methodology documents that limitation, routes non-English claims through specialist agents, and reports calibrated confidence on the routing decision. That is reliability infrastructure as design: not an improved model, but a surrounding system with embedded information on what the model is and is not good at, and acting on that knowledge.

What the Reliability Layer is

The AI Reliability Layer is the runtime infrastructure that addresses, for production AI deployments, four classes of question:

  1. Verification. When the system produces an output with stakes, what evidence supports the output, and how strong is that evidence? In practice: an evidence-quality threshold that fires per output, with explicit flags when the threshold is not met.
  2. Calibration. How confident is the system, in numbers, on this output? Are those numbers tracking the actual hit rate? In practice: a sampled validation loop comparing advertised confidence to ground-truth hit rate, with a calibration metric tracked over a moving window and a drift alert when the error crosses threshold.
  3. Adversarial robustness. What happens to system output when an input is constructed to game it? Are the gaming patterns documented, and are countermeasures in place? In practice: a documented attack-vector list, a red-team set exercising each vector on a cadence, and review when new patterns surface.
  4. Recovery. When verification fails or confidence is below threshold, what happens? Does the system gracefully degrade, escalate, refuse, or fall through silently? In practice: an explicit escalation or refusal path that fires by design when verification fails or confidence is low, not silent pass-through.

These are runtime concerns. They run in production, on production inputs, alongside the model. They may, in the future, be mitigated by a better model; but a more reliable and predictable solution is to build infrastructure around the model that knows the model’s failure modes and acts accordingly.

Call this the Reliability Layer. It sits between the model and the deployment, and it is what makes a deployment reliable rather than merely functional.

What it is not

Evaluation answers a different question. Eval asks: does the model perform well on a held-out benchmark, in distribution, under controlled conditions? Eval is the qualifier for deciding whether a model is fit for deployment. The Reliability Layer is what makes the deployment fit for production. Eval is a snapshot; reliability is continuous.

Observability tells you a system is running, not whether it is right. Latency, throughput, error rate, resource usage, all of it green: a system can be 100% available and 100% wrong, and observability will not tell you that. Observability is necessary for reliability but not sufficient.

Alignment and safety sit at a longer horizon. Alignment asks whether the model’s values and behaviors are aligned with the use case and desired result. Safety asks what happens when models are powerful enough that misalignment has real repercussions. These are real fields with real work; they are not the Reliability Layer.

MLOps is the broader family. MLOps covers the lifecycle of ML systems in production: data pipelines, training, deployment, monitoring. Some MLOps practices intersect with the Reliability Layer (drift detection, for example, is reliability-adjacent). But MLOps as commonly practiced covers training-time and deployment-time concerns. It under-covers runtime verification, calibration on the live request, and adversarial robustness in production.

LLMOps, the LLM-specific evolution of MLOps, explicitly tries to include runtime monitoring. Vendors marketing in this territory include Databricks, MLflow (with LLM extensions), Weights & Biases, and LangSmith; whether any given LLMOps platform covers the four runtime concerns above is vendor-specific. I treat LLMOps the same way I treat MLOps: a broader family that overlaps with the Reliability Layer in places and under-covers the runtime concerns in others.

The Reliability Layer is what runs every time the model is asked to produce an output that has stakes. That runtime focus is the through-line.

A reader might object: is this not just observability plus eval plus red-teaming? The components are necessary; the composition is not their sum. Observability tells you a system is running and how. Eval qualifies the model in distribution. Red-teaming surfaces attack patterns. Each measures. None gate. The Reliability Layer is the gating, calibration-on-live-request, and recovery logic that ties those measurements to system response. Without it, you have three telemetry streams and a system that responds to none of them.

Adjacent claimants in the same territory

Three other framings stake claim to runtime-focused AI reliability concerns and predate the “Reliability Layer” framing here. I can’t make the category claim without engaging them.

AI TRiSM (AI Trust, Risk, and Security Management; Gartner, since 2023) explicitly defines a category covering “continuous evaluation and inspection of runtime behavior to detect policy violations, security threats, and unwanted outputs,” plus adversarial-attack resistance and content-anomaly detection. The runtime focus and adversarial coverage that I have been calling the distinguishing axis fall within the TRiSM definition. TRiSM is the closest sibling.

NIST AI RMF (the AI Risk Management Framework; NIST, January 2023), with its Generative AI Profile (NIST AI 600-1, July 2024) and a Critical Infrastructure profile concept (April 2026), carries the runtime concerns inside its Measure and Manage functions. RMF is broader than the Reliability Layer (it adds governance and process) but runtime concerns fall within it.

AI SRE has emerged as a recognizable market segment in 2026, with multiple vendors (Lightrun, Resolve.ai, Rootly) marketing under the SRE-for-AI banner. This is the closest in operational posture to what the Reliability Layer would look like staffed.

I’m not saying “this territory has no name.” It has at least three competing names; my “Reliability Layer” label is a fourth proposed framing that I think maps the territory more cleanly. Either way, the runtime concerns themselves are real and underbuilt, and having a clear way to discuss and amend this is useful.

Salient differences:

These are not orthogonal claims; they are claimants for adjacent and overlapping territory. I find the four-concern taxonomy more directly useful than TRiSM, RMF, or AI SRE alone for an engineer designing a deployment-level reliability pattern. Your mileage may vary.

Why the runtime concerns are underbuilt

There are structural reasons the runtime concerns are underbuilt despite multiple framings competing for the territory:

Evaluation is easier to identify and, thus, to fund. Eval has clear leverage: a single model decision affects many downstream deployments. Funding flows toward upstream choke points.

Observability gets dashboards. Generic observability tools work for AI systems with minor extensions. Reliability-specific tooling does not generalize as cleanly across deployments.

Safety has been in-distribution for top labs. The frontier labs invest in alignment and safety because they ship models, not deployments. The reliability of a deployment is primarily the deployer’s problem, not the lab’s.

Reliability is bespoke per deployment. What “reliable enough” means for a fact-check pipeline differs from what it means for a code-generation assistant differs from what it means for a triage system. This makes the category resistant to generic SaaS framing; there is no “one size fits all.”

The discipline of senior engineering practice has not yet been imported. SRE as an external discipline has roughly a decade of practice and tooling outside Google (the seminal book published 2016, SREcon since 2014); inside Google it has been formalized for two decades (since 2003). The equivalent for AI deployments is only now forming. Most teams running production AI in 2026 do not have a designated role responsible for runtime reliability of the AI itself, separate from the surrounding infrastructure. The AI SRE category is the most direct attempt to fill that role.

These circumstances, taken together, frame the current state. The Reliability Layer is being built piecewise, by the deployers who need it most and have recognized an addressable need, with several overlapping concept spaces but no commonly agreed specific nomenclature. The 97% visibility gap the Lightrun data documents at the top is exactly the surface this layer would cover.

What good looks like

I have built and seen built variants of the following pattern. It is not the only viable pattern, but it is a coherent one.

A Reliability Layer for a deployment includes:

A team that has shipped this pattern can answer: on this input, right now, does the system produce an output that meets the reliability bar this deployment requires? They can also answer: what is the failure rate, on which inputs, and what happens when it fails?

Most teams running production AI in 2026 cannot answer these questions for their own deployments. That is the gap.

Worked example: Veridi

The four concerns above are abstractions. Here is what they look like staffed in one production system: Veridi, the verification cooperative I co-founded.

Verification. Evidence gathering runs through a 4-tier source hierarchy. A claim is supported when load-bearing evidence concentrates in tiers 1 and 2 (primary peer-reviewed and trustworthy-institutional) with no contradiction at tier 1. Below that bar, the output carries an explicit flag. The verification gate is the system’s threshold, not the model’s confidence; a model can be highly confident on poor evidence and the gate will catch that.

Calibration. A separate agent (the Brier accumulator) records the prediction tuple for every verdict and tracks calibration against ground truth over time. The target is a Brier score of 0.15 or better (crowd-level forecasting accuracy; the superforecaster baseline is around 0.10). Selective Brier on the 100-claim Veridi calibration set is 0.0253 against the 0.15 target, with 89% commit-vs-abstain coverage and 11/11 abstention correctness (the original 95-claim set scored 0.0251; set composition is documented in the colophon); the Pragma + Praxis assessment frameworks separately pass 93 of 95 combined benchmark claims under the current Pragma v1.6 and Praxis v1.4, per a focused regression dated 2026-05-26.

Adversarial robustness. Thirteen specific gaming vectors are documented with detection procedures and red flags. A scanning stage runs a 15-item quick checklist at Standard tier and the full 13-vector scan at Full or Forensic tier. New attack patterns surface as new vectors and trigger countermeasure review.

Recovery. When workhorse confidence drops below threshold or a gaming flag fires, the result parser routes the claim to advisor review or auto-escalates to a higher tier. The system never silently passes a low-confidence verdict; the recovery path is explicit and documented.

This is one shape of the pattern. Not every Reliability Layer should look like Veridi; the point is that the four concerns admit concrete instantiation. For methodology detail, see the Veridi methodology writeup.

What I am uncertain about

Whether this will become a generally recognized category. Possibilities: it stays distributed, with each deployment building reliability piecewise; it gets absorbed into AI eval / observability vendors as they expand scope; it consolidates as a separate category around a small number of specialized providers; it gets absorbed into MLOps as that field broadens. I lean toward the first two over the medium term.

Whether the runtime focus is the right framing. Runtime reads to me as the load-bearing distinguishing axis: it is what differs from eval and from safety, and I have not yet found a counterexample that survives contact with a concrete deployment. The alternative framing is “AI assurance” with a broader scope. The framing is not settled.

The right ownership model. Reliability could sit with the AI team, the platform team, a dedicated reliability function, or the deployment team. Each has trade-offs. SRE in classic systems took a decade to settle into the platform-team-with-embedded-engineers model. AI may follow a similar arc, or may not. Given how underdeveloped AI measurement still is at most teams, I tend to place reliability within the AI team’s purview.

The cost basis. Verification, calibration, and adversarial coverage are not free. Building this layer adds latency and compute. Whether the cost is justified depends on the deployment’s stakes. For some deployments, it is not; the cheaper failure mode is acceptable. Knowing which is which is itself a Reliability Layer concern.

Why this matters

If you are running production AI in 2026 and cannot answer the verification, calibration, adversarial, and recovery questions for your deployment, you have an exposure that grows with your traffic. The exposure does not show up in eval, nor in observability. It shows up as a slow drift in output quality on input classes you did not anticipate, until someone external notices the pattern.

The cost of building the Reliability Layer is real. The cost of not building it scales with the stakes of the deployment. Deployments with low stakes can afford to skip much of it and may see a net benefit from doing so. Deployments with stakes (medical, legal, civic, regulatory, safety-critical) cannot. Most production AI in 2026 sits somewhere in between, with stakes that the deployer has not fully priced.

I am building toward this category at Veridi (verification under adversarial conditions, with calibrated confidence and documented gaming countermeasures), across the more than 80 Claude Code skills I have authored (with declared test-contract patterns) for agent reliability, and in collaboration with others. I think the category is real and the demarcation points are useful, and the work is needed regardless.

A useful diagnostic: if you can answer these three questions about your own deployment, the layer is at least partly built; if you cannot, the gap is the layer.

  1. When your system produces an output that has stakes, can you point to the verification gate that decided the output was good enough, and state the false-positive and false-negative rates of that gate?
  2. What is your system’s current calibration error, and is it widening or narrowing under recent changes?
  3. When verification fails or confidence drops below threshold, where does the output go, and is that path explicit and documented?

If your deployment has stakes you have not priced, you have work to do in the Reliability Layer. Your model is not the limiting factor.

Methodological anchors

The four-concern taxonomy above is the load-bearing public framing. Underneath, each concern anchors to established disciplines outside AI that I treat as standards-of-record rather than reinventing in-house. Naming them serves two purposes: it grounds claims in citable lineage rather than asserting novelty, and it gives engineers reading this manifesto an entry point into the published rigor frontier.

Per concern, the primary anchors:

Cross-cutting under all four: software-quality engineering as ISO/IEC 25010:2023 (eight characteristics plus opt-in Safety), SWEBOK v4, ISO 14764:2022 on maintenance, Nygard 2011 ADRs, Ford+Parsons+Kua fitness functions for evolutionary architecture.

The instrumented form of these anchors lives in the Veridi methodology writeup (which traces calibration choices to Murphy 1973 and adversarial coverage to MITRE ATLAS), and across the Veridi, Praxis, and Pragma published frameworks. Per-methodology current versions: Veridi v1.2, Praxis v1.4, Pragma v1.6 as of May 2026.

This is not exhaustive. It is the load-bearing subset that a working Reliability Layer cites under audit. Other disciplines (normative philosophy for Pragma’s value-frame discipline; software measurement frameworks like IFPUG Function Points and DORA metrics for cost-basis work) sit adjacent and are surfaced where the methodology calls them.

Work with me

If your team is sizing this layer and would value an outside read, I take on a small number of Reliability reviews per quarter through Nettercap Productions (my independent consulting practice). See availability for scope, fit, and the email path.

If you are sizing a full-time engineering role around these concerns, I am open to senior-to-staff individual-contributor positions in Applied AI Engineering and adjacent tracks. See availability for role types, locations, and the hiring-inquiry email path.