# Turkish small-model paper logs a safety gate that lied to its own authors

> An arXiv paper documents building ufakzeka-1, a 151M-parameter Turkish language model, for about $286 — and reports a safety gate that passed 64/64 only because it had been trained on its own questions; the honest score was 34/64.

- **Topic**: Models
- **Published**: 2026-09-24T05:32:49.648Z
- **Canonical URL**: https://highsignal.sh/stories/turkish-small-model-paper-logs-a-safety-gate-that-lied-to-its-own-authors-c9260f3a

## Why It Matters

The paper's value is its record of how a small model was built and measured, not the model's capability. It reports three findings it expects to transfer: a safety gate that had been 'fixed' with training data written from its own questions read 64/64 while the honest figure was 34/64; training-seed variance was as large as the spread across every recipe tried, making single-seed comparisons at this scale uninformative; and data rounds repaired only what was absent from the data, while identity tracking over long context and multi-turn arithmetic did not move across any data change — which the authors read as limits of model size rather than data gaps, a reading they say a larger model will test. Because the results are the authors' own claims from a single paper, they are not independent.

## Key Findings & Analysis

### What was built and released

The arXiv CS.CL paper describes ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish model pretrained from scratch on 13.5B tokens of openly licensed text and instruction-tuned for chat, at a total cost of about $286 in cloud GPU, API and notebook time. Components include a Turkish byte-level tokenizer (1.77 tokens per word), a three-stage pretraining schedule, and a post-training mixture of openly licensed and generated data. Evaluation covered release gates, a rule-checked sweep of 5,508 conversations, judged conversations, and hand tests, all with prompts held out from training and enforced by decontamination in the data build plus a checked-in invariant script run before each build. Weights, the data recipe, evaluation code, and the spend ledger are released under Apache-2.0.

### Why the paper is worth reading

The authors state the contribution is not the model's capability, which they say is what a model this size can be expected to have, but the record of building and measuring it. The three reported findings — a safety gate inflated by training on its own questions, training-seed variance matching the spread across recipes, and long-context identity tracking and multi-turn arithmetic unmoved by data changes — are framed as transferable lessons for other small-model efforts, with the model-size interpretation flagged by the authors as something the next, larger model will test.

## Primary Sources & Citations

- [ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch](https://arxiv.org/abs/2609.25081) — *ArXiv CS.CL (Computation and Language)* (Reporting)

---

[← Back to front page](https://highsignal.sh/) | [Daily Brief](https://highsignal.sh/brief) | [All stories](https://highsignal.sh/latest)