MLOps · Chapter 07

LLMOps and Agentic AI Systems

Design, evaluate, and operate production LLM applications and agentic workflows with reliable context, tools, controls, and cost-aware observability.

7 lecture hoursSyllabus: 7.1–7.5Focus: LLM systems in production

Chapter outcomes

  1. Describe production architectures for LLM applications.
  2. Manage prompts, models, and evaluations systematically.
  3. Design retrieval- and tool-augmented AI systems in production.
  4. Explain agentic workflows and tool use patterns with control points.
  5. Operate LLM systems with observability, cost, and reliability practices.
7.1

Production Architecture of LLM Applications

LLM quality and safety emerge from the full application stack, not the model alone.

LLM application architecture

  • Client/UI layer: user input, session state, UX constraints, feedback capture.
  • Orchestration layer: prompt assembly, routing, workflows/chains, policy enforcement.
  • Context layer: retrieval (RAG), memory, business data, tools/APIs.
  • Model layer: hosted or self-hosted LLMs, model selection/routing, parameters.
  • Controls & ops: validation, moderation, logging/tracing, caching, rate limits.
User/App Orchestrator Retrieval/Memory LLM(s) Tools/APIs Guards/Obs
Figure 7.1 — Production LLM apps combine models, context, tools, and controls.
7.2

Prompt, Model and Evaluation Management

Treat prompts, models, and evals as versioned, testable system artifacts.

Managing prompts and models

Prompt management

  • Version system prompts, templates, variables, and formatting rules
  • Separate instructions, context, and user input clearly
  • Test against regressions and injection/edge cases

Model management

  • Track model/provider/version and parameters
  • Route by task, cost, latency, or quality needs
  • Plan fallbacks and provider/model migration

Evaluation management

  • Offline: benchmark sets, rubrics, LLM-as-judge (with calibration), retrieval metrics
  • Online: user feedback, task success, escalation/safety rates, A/B tests
  • Regression suites: critical workflows and known failure cases
Version prompt + model + retrieval config + eval dataset together for meaningful comparisons.
7.3

Retrieval- and Tool-Augmented AI Systems in Production

Ground responses and enable actions with external knowledge and capabilities.

RAG and tool augmentation

Retrieval-Augmented Generation (RAG)

  • Pipeline: ingest/chunk → embed/index → retrieve/rerank → prompt with context → generate
  • Benefits: fresher/domain knowledge, reduced hallucination risk when grounded
  • Risks: poor chunking/retrieval, stale index, context misuse, citation errors

Tool augmentation

  • LLM calls APIs/functions (search, DB, calculators, workflows)
  • Requires schema validation, permissions, timeouts, and safe execution boundaries
Production controls: validate retrieved content and tool inputs/outputs, limit tool permissions, log actions, and handle partial failures. Separate retrieval quality evaluation from generation quality.
7.4

Agentic Workflows and Tool Use

Agents plan and act across steps, increasing capability and operational risk.

Agentic systems: patterns and safeguards

Agentic workflow. An LLM-driven process that can plan, decide among steps, call tools, and iterate based on intermediate results to achieve a goal.

  • Patterns: ReAct/tool-use loops, planner-executor, multi-agent delegation, human-in-the-loop checkpoints.
  • State: task memory, intermediate artifacts, tool results, budgets/limits.

Safeguards

  • Explicit step/tool budgets, timeouts, and termination conditions
  • Least-privilege tool access and approval for high-impact actions
  • Structured outputs and validation between steps
  • Comprehensive tracing of decisions and tool calls
Risk: Autonomous loops can amplify errors, cost, or unsafe actions without strict boundaries and observability.
7.5

LLM Observability, Cost and Reliability

Operate LLM systems with visibility into quality, behavior, spend, and failures.

Operating LLM systems in production

Observability

Trace prompts, context, tool calls, model responses, latency, errors; capture user feedback and safety events (with privacy controls).

Reliability

Fallback models/routes, retries/timeouts, caching, graceful degradation, circuit breakers for tool/provider outages.

Cost control

Token/context budgeting, model selection/routing, caching, prompt compression, batching/streaming, rate limits.

Quality & safety

Online evals, guardrails, policy filters, hallucination/format checks, human escalation for high-risk cases.

Key metrics: task success, groundedness/quality scores, safety incident rate, p95 latency, cost per request/task, tool failure rates.

Checkpoint

Scenario

An agentic support assistant can search docs and issue refunds via tools. What production controls are required before enabling it?

Answer: Scoped/least-privilege tool permissions, input/output validation, step and spend limits, approval/human-in-the-loop for refunds, full tracing and audit logs, evals for task/safety failures, fallback behavior, and monitoring for cost, latency, and unsafe actions.

Summary

1

Architecture

LLM apps are systems of prompts, context, models, tools, and controls.

2

Managed artifacts

Version and evaluate prompts, models, and configurations together.

3

Ground and act safely

RAG and tools add capability but require validation and boundaries.

4

Operate continuously

Observability, cost control, and reliability are core LLMOps concerns.

References

  1. Chip Huyen. AI Engineering. O’Reilly, 2025.
  2. LangChain/LlamaIndex and vector DB docs for RAG/tool-use patterns (concepts).
  3. OWASP Top 10 for LLM Applications and NIST AI RMF for risk/safety considerations.
End of chapter

Questions?

Next: Chapter 08 — Scalable and Responsible AI Systems