Can you use "Spot" or "Preemptible" GPU instances for real-time inference? What happens to the user's request if the cloud provider reclaims the GPU mid-generation?

In a high-concurrency environment, how does PagedAttention prevent the GPU from running out of memory (OOM) when multiple users are chatting simultaneously?

If your goal is to process 1,000,000 documents as fast as possible (offline), how does your deployment strategy differ from a real-time chatbot (online)?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?