If you are using an LLM to grade another LLM, why is it critical to provide a "multi-point rubric" rather than just asking "Is this answer good?"

Question

Accepted Answer

Without a rubric, an LLM judge is inconsistent. A rubric breaks the score into specific dimensions (e.g., "Accuracy: 1-5," "Clarity: 1-5," "Formatting: 1-5") and defines exactly what a "1" vs. a "5" looks like for each.

If you are using an LLM to grade another LLM, why is it critical to provide a "multi-point rubric" rather than just asking "Is this answer good?"

Practice Your Response

Similar Questions in Reliability & Evaluation