Level 3: Managed¶
Last updated: 2026-07-27
List of references
At Level 3, the application is operated as a production service. Teams use observability, incident response, governance, and performance controls to maintain reliable and compliant behavior as traffic, models, and data change.
Source: Log Analytics workspace architecture.
Production operating model¶
| Area | Managed practice |
|---|---|
| Monitoring | Centralize application, model, dependency, and security telemetry in Azure Monitor and Log Analytics. |
| Alerts | Alert on elevated errors, latency, safety findings, cost anomalies, and quality regressions. |
| Governance | Enforce approved regions, network configuration, tagging, and identity controls with Azure Policy. |
| Performance | Track latency, throughput, capacity, token use, caching effectiveness, and model endpoint health. |
| Incident response | Define owner, severity, mitigation, rollback, and communication paths for model-related incidents. |
Monitoring signals¶
- Service health: availability, error rates, throttling, and dependency health.
- Experience: completion latency, abandoned requests, user feedback, and fallback rate.
- Model behavior: groundedness, relevance, safety, and prompt-injection findings.
- Cost: token usage, requests by model, supporting compute, and budget variance.
- Change: release version, prompt version, grounding index version, and evaluation set version on each request where appropriate.
Important
Logging should not become a data-exposure path. Minimize captured prompt and response data, apply access controls and retention policies, and mask or exclude sensitive data where required.
Advance to Level 4: Optimized when the managed baseline is stable enough to support continuous, evidence-based optimization.