How do you integrate prompt changes into a CI/CD pipeline? Should a "Prompt Change" trigger a full deployment or just a configuration update?

When switching from one model to another (let's say Llama 3 to Llama 3.1), how do you perform a Blue/Green swap? How do you handle the state of ongoing "streaming" conversations during the switch?

In a high-concurrency environment, how does PagedAttention prevent the GPU from running out of memory (OOM) when multiple users are chatting simultaneously?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?