Learning tracks by role
AI learning track: Engineers taking AI to production
For engineers and architects responsible for operating AI systems beyond a working demo.
After this track: You can define release checks, trace failures and bound the cost and authority of a production agent.
Follow these 10 steps in order, or return to the one you need for your current work. Reading times are estimates; listening times cover the full episode.
-
How do you evaluate an AI agent?
Start with release checks that cover tool choices and task completion as well as the final answer.
-
What's the difference between AI observability, evaluation, and benchmarking?
Separate traces, quality judgments and comparisons before deciding what your monitoring stack must measure.
-
What are AI agent guardrails, and how do you set them?
Turn known failure modes into runtime boundaries before exposing the agent to production traffic.
-
How do you cut the cost of running an AI agent?
Use traces to find expensive calls and retries once quality checks can catch regressions from cost changes.
-
Latency
Define the response-time constraint alongside cost before optimizing a workflow users still have to wait for.
-
Why do multi-agent systems fail, and how do you make them reliable?
Inspect coordination and error propagation before splitting a working agent across more participants.
-
What are the types of AI agent memory?
Decide what should persist across runs after identifying the state that failures can corrupt or make stale.
-
How do you give an AI agent an identity and permissions?
Give each acting component an identity and bounded permissions before connecting it to privileged services.
-
Architecting Reliable Agentic AI | Cisco’s Giovanna Carofiglio on the AGNTCY Collective
Compare your architecture with a discussion of agent interoperability and reliability before expanding across systems.
-
The AI Agent Trust Gap: Bridging Risk to Reliability | Elastic’s Philipp Krenn
Close with the trust and reliability tradeoffs you will need to explain when deciding whether to ship.
Choose another role or continue through the five-level pathway.