If your goal is to process 1,000,000 documents as fast as possible (offline), how does your deployment strategy differ from a real-time chatbot (online)?

How do you monitor for "Concept Drift" in an LLM application? If the model's output starts getting shorter over time, is that a deployment failure or a data failure?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?

Standard logs store text. Why might you want to store the embeddings of your production inputs and outputs in a vector database for monitoring?