Agent365 Observability and Agent Health¶
Atlanta, USA
Last updated: 2026-08-04
References
Observability objectives¶
Observability must cover reliability, quality, security, and governance in a single operating view, not separate dashboards with no correlation.
Source: How Microsoft 365 Copilot works.
Telemetry domains¶
| Domain | Example signals |
|---|---|
| Availability | Success rate, latency, timeout rate |
| Quality | Citation quality, fallback rate, retry rate |
| Safety | Policy blocks, restricted-content matches |
| Security | Risky session attempts, anomalous tool calls |
| Governance | Unapproved configuration drift, expired exceptions |
Health model¶
- Define service-level indicators (SLIs) by agent tier.
- Set service-level objectives (SLOs) for user-critical journeys.
- Attach alert thresholds to operational impact, not raw volume.
- Route incidents by owner and risk tier.
Health scorecard example¶
| Metric | Green | Amber | Red |
|---|---|---|---|
| End-to-end success rate | >= 99% | 97% to 98.9% | < 97% |
| P95 (95th percentile) latency | <= 3s | 3s to 6s | > 6s |
| Policy violation rate | <= 0.1% | 0.1% to 0.5% | > 0.5% |
| Ownerless active agents | 0 | 1 to 2 | > 2 |
Validation tests before production¶
- Prompt-response success under peak load.
- Tool invocation traceability by request identifier.
- Policy block and escalation workflow visibility.
- Evidence retention for regulated audit scenarios.
Business example¶
A high-volume support agent appears healthy by uptime but fails quality checks. Observability reveals high fallback and tool timeout rates during peak periods. The team scales dependencies, updates timeout strategy, and improves response quality while preserving security and governance controls.