127
my llm judge gives 4.6 out of 5 to answers i deliberately broke llm-as-judge
Judge prompt asks for a 1 to 5 rating on helpfulness and accuracy given the question, the retrieved context and the answer. As a sanity check I fed it answers with fabricated citations, answers truncated mid sentence,…