You have a task that requires complex reasoning 10% of the time and simple extraction 90% of the time. How do you architect a "Router" to save costs?

When an upstream provider returns a 429: Too Many Requests, how do you implement a "Circuit Breaker" pattern so your entire application doesn't crash?

If your inference latency is high because the model is too big for one GPU, do you scale horizontally or vertically? What if the latency is high because you have too many concurrent users?

If your goal is to process 1,000,000 documents as fast as possible (offline), how does your deployment strategy differ from a real-time chatbot (online)?