Technical Reference · Oct 2026

AI Engineering Stack

Four eras. What broke. Where the money goes.

Oct 2026

§1

The Token Price Fallacy

The paradox
Token prices fell 280× in two years. Enterprise AI compute spend grew 320%. Cheaper tokens produced more calls, not fewer.

Where spend goes in an agentic task

Call 1 — first inference · ~40%
Calls 2–N — refinement loops · ~60% ◄ the real target

McKinsey July 2026 — one agentic task triggers 10–20 LLM calls. $0.10–0.50/request → $150K–750K/month at 10K users.

93%
of organisations exceeded their AI budget in 2026
McKinsey, May 2026
7.2×
median monthly LLM spend growth year-over-year
Digital Applied, Apr 2026
44%
of D&A leaders have financial guardrails in place today
Gartner, Mar 2026 (n=353)
40%
of agentic AI projects will be cancelled by 2027
Gartner Hype Cycle, 2026
Case in point
Uber exhausted their 2026 AI coding budget in 4 months — capped at $1,500/user/month.
Microsoft cancelled most internal Claude Code licenses, June 30, 2026.
§2

The Four-Era Evolution

Era 1
Prompt Engineering
2022–23
"What to say to the model"

Year one. Each user writes their own prompts. Works for demos and narrow tasks with stable scope.

What broke

Quality per user. Business rules in prompts go stale. Model updates silently degrade every prompt. 20–30% accuracy ceiling.

Cost failure

Every retry invisible. No budget cap. No session limit. Total cost of ownership hidden in human review time, not compute spend.

Era 2
Context Engineering
2024–25
"What the model needs to know" — methodology, not a layer

Centralises semantic context. Removes per-user variability. AI-available ≠ AI-consumable.

What broke

Model knows what to know, not what to do. No stop rules. No optimization goal. Klarna: perfect context, wrong goal — $60M to rehire.

Cost failure

40% NL accuracy without semantic layer → 60% more failure-triggered retries. Multi-model setups add routing complexity CE doesn't address.

Era 3
Intent Engineering
2026
"What the model must achieve — within what limits" — methodology, not a layer

Huryn 8-component model: Strategy · Objectives · Stop Rules · Autonomy Boundaries. Encodes organizational goals, not just data.

What broke

Intent buried in code — not auditable or versioned. Model-coupled: swap Opus for Haiku = full re-engineering. Assumptions hidden in natural language.

Cost failure

No Stop Rules = agent refines past "good enough". 60% of agentic spend is in refinement loops with no exit condition.

Era 4
Specification Engineering
Emerging
"What can be proved — at what cost — by whom — at what version"

Versioned, model-agnostic specs. Clauses carry unique IDs. Example prompts are unit tests. Spec IS the regulatory artifact.

Still maturing

No dominant toolchain. No spec completeness standard. Most tools in Beta or Preview. Practitioner adoption early.

Cost breakthrough

Model-agnostic spec = swap Opus → Haiku freely under budget pressure. Only layer that reduces cost without reducing governance.

The failure chain

PE → CE
Each user supplied their own context → quality was unpredictable, every call was unique, no caching possible, hidden cost in human review time.
CE → IE
Context was governed, behavior was not → agents wandered past "good enough" with no exit condition, consuming 60% of budget in refinement loops.
IE → SE
Intent was encoded in code — couldn't be audited, ported, or proved. Model-coupled intent = can't swap to a cheaper model = cost locked in.
§3

Tech Landscape by Layer

Layer Era 1 · PE · 2022–23 Era 2 · CE · 2024–25 Era 3 · IE · 2026
Orchestration
Frameworks
  • LangChain — prompt chains
  • DSPy — Stanford, programmatic prompting
  • Guidance — Microsoft, structured generation
  • PromptFlow — Azure
  • LlamaIndex — context management
  • Haystack — RAG pipelines
  • Semantic Kernel — Microsoft, context + tools
  • Chroma / Pinecone / pgvector — vector / RAG stores
  • MCP protocol — Anthropic → industry standard for context delivery
  • Atlan / Alation — data catalogs
  • LangGraph — stateful agent loops
  • CrewAI — role-based multi-agent
  • AutoGen / AG2 — self-brands as "open-source AgentOS"
  • Mastra — TypeScript-native
  • OpenAI Agents SDK — Responses API + loops + handoffs + guardrails
  • Anthropic Agent SDK — model + harness + tools + environment (4-component)
  • Google A2A protocol — cross-framework agent interoperability
Cloud
Platforms
  • Azure OpenAI
  • OpenAI API
  • AWS Bedrock (basic)
  • GCP Vertex AI Studio
  • Databricks Genie Ontology — GA Apr '26
  • Snowflake Cortex Analyst + Horizon Context — GA Jun '26
  • MS Fabric Semantic Models
  • Unity Catalog — data lineage
  • Apache Ossie — YAML semantic portability, Jan '26
  • dbt MetricFlow — 83% NL accuracy benchmark
  • Databricks Mosaic AI
  • AWS Bedrock AgentCore — multi-agent runtime
  • Google ADK — Python GA May '26 · Go Jun '26 · TypeScript GA Aug '26 · Java 1.0 Mar '26
  • Copilot Studio — multi-agent GA Mar '26
  • MS Fabric ops agents — Preview
Observability
  • Basic API logs
  • OpenAI usage dashboard
  • dbt tests
  • Data quality checks
  • Atlan / Alation catalogs
  • Langfuse
  • AgentOps
  • Phoenix — Arize
  • Weights & Biases

Era 4 · SE · Governance / OS Layer

SE lives here — the policy, spec, and audit layer above the harness.

Orchestration (Spec Tooling)
Where specs are authored and enforced
  • MS Agent Governance Toolkit (MIT, Apr '26) — Agent OS: YAML/OPA Rego/Cedar, sub-ms; Agent Mesh: crypto ID, trust 0–1000, 5 tiers; EU AI Act + HIPAA + SOC2 + OWASP×10
  • Open Agent Spec (ACM CAIS '26) — portable across LangGraph, CrewAI runtimes
  • AWS Kiro — spec-driven: requirements → design → tasks → code
  • GitHub Spec Kit (open-sourced Sep '25)
  • OpenAI Model Spec — unique clause IDs + prompts as unit tests
  • Anthropic Claude Constitution (84-page, Jan '26)
Cloud Platforms (AgentOS)
Policy enforcement + cost governance at runtime
  • Databricks Unity AI Gateway (Beta, Jun '26) — 30%+ smart routing; spend caps at 4 levels: user / use-case / workspace / account; SQL policies as UC functions applied to MCPs
  • AWS Bedrock AgentCore Policy (Cedar, GA) — deterministic authorization; temporal session-history auth
  • AWS Agent Registry (Preview, Apr '26) — central agent inventory
  • MS Fabric + Purview + Entra ID — governance + identity layer
  • Copilot Studio Agent 365 — shared policies + lifecycle oversight
Observability (Era 4)
Audit-grade lineage, not just logs
  • AgentOps — full agent lifecycle tracking
  • Compliance evidence packages
  • Audit trail traceable to spec clause
  • BCBS 239 / DORA data lineage
Cedar convergence signal: Both AWS AgentCore and MS Agent Governance Toolkit independently chose Cedar for deterministic per-action authorization. Two major platforms converging on the same policy language is an industry standard forming.
§4

AgentOS vs AgentHarness

Category terms, not single products. The market is converging on this vocabulary.

AgentHarness
The layer that runs agents
  • Wraps, controls, tests, monitors, evaluates the agent loop
  • LangGraph · CrewAI · AutoGen/AG2 · Mastra · OpenHarness
  • Observability: AgentOps · Langfuse · Phoenix (Arize)
  • Solves: agent loop management, tool calling, multi-agent handoffs
Does not solve: cost governance, regulatory audit, model portability
AgentOS
The layer that governs agents — sits above the harness
  • Policy engine — determines what agents can do, at what cost, in what order
  • MS Agent Governance Toolkit "Agent OS" component (YAML/OPA Rego/Cedar)
  • Databricks Unity AI Gateway — spend caps + smart routing
  • AWS AgentCore Policy — Cedar, temporal session auth
  • AG2 self-brands as "open-source AgentOS"
  • Solves: cost guardrails, action authorization, compliance evidence
Relationship
AgentHarness runs agents. AgentOS governs them. A spend cap in Databricks Unity AI Gateway is AgentOS. The LangGraph loop that executes is AgentHarness. Both are required. Neither replaces the other.
§5

Six Cost Control Levers — Verified Numbers

# Lever What it fixes Numbers
1 Context quality
CE layer
Databricks Genie Ontology · Snowflake Cortex · dbt MetricFlow 		Poor semantic context → misunderstood query → failure-triggered retry loops. CE is a cost lever, not just a quality lever. Every wasted token is a CE failure. 83% vs ~40% NL accuracy with/without semantic layer
→ ~50% fewer retry loops
dbt MetricFlow, 2025
2 Prompt caching
Native: Anthropic, OpenAI, Google APIs 		Long system prompts re-sent on every call. Cache the stable parts — pay only for the delta.
Caveat: misconfigured caching can increase the bill — cache write premium + misses. Measure before committing.
 ~90% off cached tokens — Anthropic
~50% off — OpenAI
~75% off — Gemini (>32K tokens)
Enterprise: 40–80% input cost cut at volume
3 Model routing
RouteLLM (OSS) · Databricks Unity AI Gateway · AWS Bedrock IPR · LiteLLM 		Every task hits the frontier model — simple tasks overpay. Route by complexity: cheap for simple, frontier for hard.
Caveat: one benchmark showed router cost 35× a fixed cheap model. Route only at volume.
 31–44% real agentic savings — RouteLLM (ICLR 2025)
72% reduction, <3% quality drop — AgentRouter (ICML '26)
30%+ per task — Databricks internal
~50% in production — Uber, Stripe, Ramp
4 Loop guardrails
AgentOS: Databricks Unity AI Gateway · AWS AgentCore · MS Agent Governance Toolkit 		Agents refine indefinitely — no session ceiling, no exit signal. Per-session spend caps + circuit breakers + retry limits + tool-call depth limits. G1000 orgs face 30% rise in underestimated AI infra costs by 2027 (IDC, 2026)

Only 44% of orgs have financial guardrails today (Gartner, 2026)
5 Intent encoding / Stop Rules
IE: LangGraph · CrewAI · AG2 · Mosaic AI 		No encoded "done" criteria → agent refines past the useful point. Stop Rules + Autonomy Boundaries = runtime loop termination.

Stop Rules are a cost feature, not just a safety feature. A spend cap fires at a dollar threshold. A Stop Rule fires when the domain goal is met. Those aren't the same thing. 		60% of agentic spend is in loops

Agent that knows when to stop spends 60–70% less than one that doesn't.

Klarna: $60M to rehire after agent optimised cost-per-token, not customer lifetime value
6 Spec portability
SE: OpenAI Model Spec · Anthropic Constitution · AWS Kiro · Open Agent Spec 		Intent is model-coupled → can't swap to a cheaper model without full re-engineering. Model-agnostic spec = choose cheapest model that still honours intent. Only lever that reduces cost without reducing governance. All other levers trade quality or governance for savings. SE doesn't — it lets you choose the cheapest model that honours the spec.

Databricks combined techniques: up to 90% savings (routing + caching + model swap)
§6

The Argument

Accountable AI, by design.
Every platform ships cost controls. Databricks has spend caps. AWS has Cedar. Microsoft has trust scoring. None of them know your business logic.
A spend cap that fires at the wrong point doesn't reduce cost — it breaks output.
CE makes context precise → fewer retries → lower token waste.
IE makes behavior bounded → fewer refinement loops → lower call volume.
SE makes intent portable → cheaper models viable → lower unit cost.
AgentOS enforces it at runtime → no runaway sessions → lower infrastructure spend.
That's the methodology gap the platforms can't sell you. The platform ships the controls. We ship the judgment about when to use them.
"The platform ships AI. We ship accountability."