What is the difference between testing your model on a static CSV file (Offline) vs. monitoring real user "Thumbs Up/Down" feedback (Online)?

How do you evaluate a RAG system’s performance when the answer is not present in the retrieved documents? (Does it correctly say "I don't know"?)

Explain the concept of using a "Stronger" model (like GPT-4o or Claude 3.5 Sonnet) to grade a "Weaker" model’s output. What are the risks of "Self-Preference Bias" in this setup?

How do you measure if the LLM actually answered the user’s question, even if the facts it provided were technically true?