Explain the concept of using a "Stronger" model (like GPT-4o or Claude 3.5 Sonnet) to grade a "Weaker" model’s output. What are the risks of "Self-Preference Bias" in this setup?

Instead of checking for exact words, how would you use BERTScore or Cosine Similarity of embeddings to evaluate if an LLM's summary is accurate?

Why is it harder to A/B test an LLM prompt than a UI button color? How do you account for the "non-deterministic" nature during the test?

Guardrails add an extra check. How do you evaluate if the safety benefit of a guardrail outweighs the 200ms latency penalty it adds?