Fact-Verification Scores Improve Mostly Because the Evidence Does, Not the Answers

The Finding

An arXiv preprint (2609.27064) examines what drives gains in joint fact-verification scores, which grade a model's answer and its submitted evidence together. On FEVEROUS, where strict score means the share of claims with both a correct answer and a complete annotated evidence group, the authors swapped DCUF evidence for UnifEE evidence across four trained DeBERTa checkpoints and 7,890 claims. Strict score rose 9.61 percentage points, versus 1.96 points for answer accuracy, with a paired 95% interval of [8.77, 10.43] conditional on those checkpoints. Changing only the evidence passed to the scorer accounted for 7.92 or 9.08 points depending on whether answers came from DCUF or UnifEE evidence. The authors then generated 470,400 responses from two 8B models (Qwen and Llama) on FEVER, FEVEROUS, and SciFact under two answer formats and two context budgets; raising context from 256 to 2,048 tokens lifted the fixed-answer evidence gain on FEVEROUS by 3.84 and 3.10 points respectively. Those effects fell short of the study's prespecified cross-dataset criterion, and some intervals extended past a two-point small-effect bound. The authors note aggregate accuracy and evidence-coverage rates can miss the claim-level pattern. [1]