How do you integrate prompt changes into a CI/CD pipeline? Should a "Prompt Change" trigger a full deployment or just a configuration update?

When switching from one model to another (let's say Llama 3 to Llama 3.1), how do you perform a Blue/Green swap? How do you handle the state of ongoing "streaming" conversations during the switch?

How do you track which specific feature or user in your app is driving the most "Token Spend"?

When would you choose to run a model locally on a user's device (using WebLLM or ONNX) instead of the cloud? Focus on privacy and cost.