Why is it harder to A/B test an LLM prompt than a UI button color? How do you account for the "non-deterministic" nature during the test?

How do you programmatically check if an LLM is making things up that aren't in the provided search results?

How do you evaluate a RAG system’s performance when the answer is not present in the retrieved documents? (Does it correctly say "I don't know"?)

Why is a standard unit test (asserting that output == "expected") often a bad way to test an LLM? How do you handle a model that gives three different, but correct, answers to the same prompt?