When would you choose to run a model locally on a user's device (using WebLLM or ONNX) instead of the cloud? Focus on privacy and cost.

When is it more cost-effective to use a "Pay-per-token" API (like OpenAI) versus hosting your own model on a dedicated cloud instance (like an AWS g5 instance)?

In a serverless GPU environment, what is a "Cold Start"? How does the size of your model weights (e.g., a 70B model) impact the time it takes for a new instance to start serving traffic?

How would you implement a "Token Quota" system to prevent a single user or a bug in your code from spending $1,000 on API calls in an hour?