Design, evaluate, and operate production LLM applications and agentic workflows with reliable context, tools, controls, and cost-aware observability.
LLM quality and safety emerge from the full application stack, not the model alone.
Treat prompts, models, and evals as versioned, testable system artifacts.
Ground responses and enable actions with external knowledge and capabilities.
Agents plan and act across steps, increasing capability and operational risk.
Agentic workflow. An LLM-driven process that can plan, decide among steps, call tools, and iterate based on intermediate results to achieve a goal.
Operate LLM systems with visibility into quality, behavior, spend, and failures.
Trace prompts, context, tool calls, model responses, latency, errors; capture user feedback and safety events (with privacy controls).
Fallback models/routes, retries/timeouts, caching, graceful degradation, circuit breakers for tool/provider outages.
Token/context budgeting, model selection/routing, caching, prompt compression, batching/streaming, rate limits.
Online evals, guardrails, policy filters, hallucination/format checks, human escalation for high-risk cases.
An agentic support assistant can search docs and issue refunds via tools. What production controls are required before enabling it?
LLM apps are systems of prompts, context, models, tools, and controls.
Version and evaluate prompts, models, and configurations together.
RAG and tools add capability but require validation and boundaries.
Observability, cost control, and reliability are core LLMOps concerns.
Next: Chapter 08 — Scalable and Responsible AI Systems