Curating Fine-Tuning Datasets

By Michael Samuel Naeem · February 14, 2026 · 5 min read

The fine-tune looked great on loss curves and terrible in demo because thirty percent of the "golden" dataset was duplicate Slack exports, twelve percent had assistant replies that apologized instead of answering, and nobody noticed the JSON examples used three incompatible key naming schemes. Model quality ceilings are set by data curation long before learning rate sweeps matter. Curating fine-tuning datasets is editorial work with engineering rigor: source intentionally, deduplicate aggressively, score automatically, review humans on the margins, and block poisoned rows from ever reaching the trainer.

Sourcing strategies

Start with 50–100 expert-crafted examples defining tone, format, and refusal behavior — anchor quality.

Expand with retrieved real cases matching production intent distribution.

Schema and format enforcement

Standardize on messages format:

{
  "messages": [
    { "role": "system", "content": "You are a billing assistant. Use USD. Refuse medical advice." },
    { "role": "user", "content": "Why was I charged twice on invoice 8842?" },
    { "role": "assistant", "content": "Invoice 8842 shows..." }
  ],
  "metadata": { "source": "expert_v2", "ticket_id": "T-8842", "quality_score": 0.94 }
}

Validator in CI:

def validate(example):
    assert len(example["messages"]) >= 2
    assert example["messages"][-1]["role"] == "assistant"
    assert token_count(example) <= MAX_SEQ_LEN
    assert not contains_pii_patterns(example)

Reject malformed rows at ingest — do not "fix in training."

Deduplication

Near-duplicates bias loss toward frequent templates:

from datasketch import MinHashLSH, MinHash

lsh = MinHashLSH(threshold=0.85, num_perm=128)
for ex in dataset:
    m = MinHash(num_perm=128)
    for token in normalize(ex["messages"][-1]["content"]).split():
        m.update(token.encode())
    if lsh.query(m):
        continue  # skip duplicate
    lsh.insert(ex["id"], m)
    keep(ex)

Also exact-hash user prompts after normalization.

Quality scoring

Automatic signals:

• Perplexity filter — drop examples where base model finds assistant reply wildly unlikely (often gibberish)
• Length ratio — user 10 tokens, assistant 4000 tokens → review
• Refusal patterns — "As an AI language model" in domain answers → rewrite or drop
• Embedding diversity — cluster and cap samples per cluster

if rouge_l(user_content, assistant_content) > 0.8:
    flag_for_review(example)  # possible copy-paste echo

Human review queue prioritized by uncertainty and cluster representatives — not random 1% only.

Human review loop

Rubrics:

1. Factually correct given provided context?
2. Matches desired tone and brevity?
3. Format valid (JSON, markdown rules)?
4. Safe — no leaked secrets in context?

Track inter-annotator agreement; disputed rows excluded from v1.

Tooling: Label Studio, Argilla, or internal spreadsheet with immutable audit — version reviewed datasets.

Legal and compliance

• Confirm license on scraped/public data
• GDPR/CCPA — right to deletion propagates to training sets
• Work product and customer data in contracts — often prohibits training without consent
• Document provenance field per row for audit

Train/val/test splits

Split by cluster or source entity, not random rows — prevents leakage of near-duplicates across splits.

Hold out entire product areas or time windows for test to simulate cold-start generalization.

Iteration cycle

Collect → Validate schema → Dedupe → Auto-score → Human review
    → Train candidate → Eval → Error analysis → Targeted data additions

Error analysis drives data collection — if model fails refund policy edge cases, add twenty expert examples there, not another ten thousand generic chats.

Synthetic data generation

Generate training examples from existing documents with LLM assistance — human review required:

def generate_qa_pairs(document: str, n: int = 5) -> list[dict]:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": f"""Generate {n} realistic user questions answerable from this document.
            Format as JSON array: [{{"question": "...", "answer": "..."}}]
            
            Document: {document}"""
        }],
        response_format={"type": "json_object"},
    )
    pairs = json.loads(response.choices[0].message.content)["pairs"]
    return [{"messages": [
        {"role": "user", "content": p["question"]},
        {"role": "assistant", "content": p["answer"]}
    ], "source": "synthetic", "reviewed": False} for p in pairs]

Mark synthetic examples "reviewed": false until human validates. Synthetic data amplifies coverage but introduces hallucination risk — never train on unreviewed synthetic examples.

Data versioning and lineage

Track dataset versions like code:

datasets/
├── support-v1.0/          # 1,200 examples, 2024-01-15
├── support-v1.1/          # +180 refund edge cases, 2024-02-03
└── support-v2.0/          # reformatted chat template, 2024-03-01

{
  "dataset_version": "support-v1.1",
  "parent_version": "support-v1.0",
  "added_rows": 180,
  "removed_rows": 12,
  "change_reason": "Refund policy edge cases from error analysis sprint",
  "eval_f1_delta": "+0.04 on refund subset"
}

Reproducibility requires knowing exactly which data produced which model. Tag model artifacts with dataset version hash.

Deduplication at scale

Near-duplicate examples waste training budget and cause overfitting:

from datasketch import MinHashLSH, MinHash

def dedupe_dataset(examples: list[dict], threshold: float = 0.85) -> list[dict]:
    lsh = MinHashLSH(threshold=threshold, num_perm=128)
    unique = []
    for i, ex in enumerate(examples):
        text = ex["messages"][-1]["content"]
        m = MinHash(num_perm=128)
        for word in text.split():
            m.update(word.encode())
        if not lsh.query(m):
            lsh.insert(str(i), m)
            unique.append(ex)
    return unique

Run deduplication after every data collection batch. MinHash LSH handles millions of examples efficiently.

Failure modes

• Training on unreviewed synthetic data — hallucinated examples in model output
• Random train/val split — near-duplicates leak across splits; inflated eval scores
• No dataset versioning — can't reproduce model or rollback to previous data
• Quantity over quality — 50k mediocre examples underperform 1k curated ones
• No provenance tracking — compliance audit fails; can't delete user data from training set

Production checklist

• Dataset versioned with parent version and change reason
• Train/val/test split by cluster or source entity (not random rows)
• Deduplication run after every data collection batch
• Synthetic examples human-reviewed before inclusion
• Provenance field on every row (source, license, consent status)
• Error analysis drives targeted data collection, not bulk scraping

Resources

• Argilla — data-centric LLM tooling
• Label Studio documentation
• Fine-tuning best practices (OpenAI)
• Dedupe library (Python)
• DataComp-LM — data curation research