# Fact-Verification Scores Improve Mostly Because the Evidence Does, Not the Answers

> An arXiv study finds that on FEVEROUS, swapping in better evidence raised strict joint fact-verification scores far more than it improved answer accuracy.

- **Topic**: Research
- **Published**: 2026-09-24T07:48:36.635Z
- **Canonical URL**: https://highsignal.sh/stories/fact-verification-scores-improve-mostly-because-the-evidence-does-not-the-answer-1fad211c

## Why It Matters

Joint answer-and-evidence scores are used to gauge how well models verify facts. This study suggests such scores can move substantially without the model actually answering more claims correctly — a measurement point for anyone reading verification leaderboards.

## Key Findings & Analysis

### The Finding

An arXiv preprint (2609.27064) examines what drives gains in joint fact-verification scores, which grade a model's answer and its submitted evidence together. On FEVEROUS, where strict score means the share of claims with both a correct answer and a complete annotated evidence group, the authors swapped DCUF evidence for UnifEE evidence across four trained DeBERTa checkpoints and 7,890 claims. Strict score rose 9.61 percentage points, versus 1.96 points for answer accuracy, with a paired 95% interval of [8.77, 10.43] conditional on those checkpoints. Changing only the evidence passed to the scorer accounted for 7.92 or 9.08 points depending on whether answers came from DCUF or UnifEE evidence. The authors then generated 470,400 responses from two 8B models (Qwen and Llama) on FEVER, FEVEROUS, and SciFact under two answer formats and two context budgets; raising context from 256 to 2,048 tokens lifted the fixed-answer evidence gain on FEVEROUS by 3.84 and 3.10 points respectively. Those effects fell short of the study's prespecified cross-dataset criterion, and some intervals extended past a two-point small-effect bound. The authors note aggregate accuracy and evidence-coverage rates can miss the claim-level pattern.

## Primary Sources & Citations

- [What Changes When Fact-Verification Scores Improve? Evidence and Answer Accounting Across Trained Verifiers and LLMs](https://arxiv.org/abs/2609.27064) — *ArXiv CS.CL (Computation and Language)* (Reporting)

---

[← Back to front page](https://highsignal.sh/) | [Daily Brief](https://highsignal.sh/brief) | [All stories](https://highsignal.sh/latest)