In a high-concurrency environment, how does PagedAttention prevent the GPU from running out of memory (OOM) when multiple users are chatting simultaneously?

How do you track which specific feature or user in your app is driving the most "Token Spend"?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?

When switching from one model to another (let's say Llama 3 to Llama 3.1), how do you perform a Blue/Green swap? How do you handle the state of ongoing "streaming" conversations during the switch?