Standard logs store text. Why might you want to store the embeddings of your production inputs and outputs in a vector database for monitoring?

Can you use "Spot" or "Preemptible" GPU instances for real-time inference? What happens to the user's request if the cloud provider reclaims the GPU mid-generation?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?

If you have 100 different customers, each with a custom-tuned LoRA adapter, do you need 100 different GPU clusters? How would you serve them efficiently on one cluster?