How would you implement a "Token Quota" system to prevent a single user or a bug in your code from spending $1,000 on API calls in an hour?

When is it more cost-effective to use a "Pay-per-token" API (like OpenAI) versus hosting your own model on a dedicated cloud instance (like an AWS g5 instance)?

How do you integrate prompt changes into a CI/CD pipeline? Should a "Prompt Change" trigger a full deployment or just a configuration update?

If your inference latency is high because the model is too big for one GPU, do you scale horizontally or vertically? What if the latency is high because you have too many concurrent users?