Phase 4 — Production & Depth · Lesson 41 · 20 XP
Reliability: retries, fallbacks, queues
Providers have outages and rate limits — a retry with backoff (Lesson 9) handles brief blips, but during a provider-wide outage no amount of retrying the same provider helps. A fallback switches to a second model provider when the primary errors out or times out repeatedly, keeping the product working instead of fully down.
A queue decouples accepting a request from processing it, smoothing out bursty load instead of letting spikes overwhelm your service directly. An operation is idempotent if running it twice has the same effect as running it once — that property is what makes it safe to retry in the first place, since a non-idempotent retry (e.g. "charge the card again") can cause real damage.
Exercise
Add a fallback to a second model provider in your Lesson 26 or 40 project, triggered when the primary provider errors or times out. Simulate a failure (e.g. point at a bad URL or invalid key) and confirm the fallback actually activates.
Check yourself
1. Why is a retry alone not enough during a provider-wide outage, and what does a fallback add?
2. What does "idempotent" mean for an operation, and why does it matter when deciding whether it's safe to retry?
Project: tool-using agent with tracing
Answer the check-yourself questions to unlock this