A team wants to generate most of its next training set with an existing model. What signals would tell you that the data is becoming low-diversity or causing model collapse, and how would you preserve useful synthetic data without recursively amplifying errors?

How to think about it

I would treat synthetic data—examples produced by a model—as a controlled supplement, not a new ground truth. Warning signs include rising exact or semantic duplication, disappearing rare cases, narrower labels and outputs, repeated errors, and declining performance on an untouched real-data holdout; I would preserve value with real-data anchors, strict provenance, independent validation, and no synthetic-on-synthetic recursion.

Why this happens

A language model does not contain a clean database of facts. It learns a probability distribution: which words, answers, labels, and styles tend to appear together.

Suppose a real support dataset contains a rare but important case: a customer whose refund was denied after a chargeback. It appears in only 1 out of 10,000 tickets. A model trained on the real set may know the case imperfectly, but it at least has a chance of representing it.

Now ask that model to generate 1 million new answers. It will preferentially produce common, high-probability cases: ordinary refunds, password resets, shipping delays. The rare case may disappear from the generated sample simply because it is rare. A second model trained on that sample sees the absence as evidence that the case does not matter.

That is the core of model collapse, meaning progressive loss of useful distributional information when models are trained on recursively generated data. The missing information is often in the tails: unusual users, minority dialects, uncommon failure modes, long documents, and low-frequency facts. Errors can follow the same path. A plausible but wrong answer becomes a target, then the next model repeats it more confidently.

Low diversity is not proof of collapse. A narrow task can legitimately have repetitive data. The important question is whether the generated set is becoming less representative of the real cases the model must handle.

Signals I would monitor

First, make provenance queryable. Provenance means recording where each example came from: the original record, generator model and version, prompt, decoding settings, validation result, and generation depth. Generation depth zero is human or external data; depth one is generated directly from it; depth two is generated using a synthetic example. Depth two should trigger immediate suspicion.

Signal	What to compare	Why it matters
Exact duplicates	Duplicate rate by source and model version	Reveals memorisation and wasted training capacity
Near duplicates	Similarity clusters using an embedding model, plus text checks	Finds paraphrases that exact matching misses
Lexical variety	Distinct one-, two-, and three-token sequences	Detects formulaic language, though not real semantic variety
Coverage	Counts for intents, languages, lengths, entities, and rare slices	Shows which parts of the real distribution are vanishing
Quality	Human audits, deterministic checks, and independent validators	Catches fluent errors and corrupted labels
Downstream behaviour	Metrics on a fixed, untouched real-data holdout	Measures whether the training set still teaches the intended task

An embedding is a numeric representation of meaning. Embedding similarity can find two answers that use different words but say almost the same thing. It is useful, but there is no universal cutoff such as “cosine similarity above 0.9 is a duplicate.” The right threshold depends on the embedding model and the task, so I would calibrate it on reviewed pairs.

I would also track conditional diversity: variation within a meaningful slice, not just across the whole dataset. Ten thousand different answers about password resets do not compensate for losing all diversity in medical, legal, or multilingual cases.

A simple lexical diagnostic is distinct-2, the number of unique adjacent token pairs divided by the total number of adjacent token pairs:

def distinct_2(texts):
    pairs = []
    for text in texts:
        tokens = text.lower().split()
        pairs.extend(zip(tokens, tokens[1:]))
    return len(set(pairs)) / len(pairs) if pairs else 0.0

This is a warning light, not a semantic truth detector. Randomly increasing temperature can raise distinct-2 while producing creative nonsense. Diversity must be checked against coverage and correctness.

A concrete scenario

Imagine a support team with 200,000 human-written tickets and verified responses. It generates 800,000 synthetic responses, making an 80 percent synthetic training set. The generator is prompted with real tickets and the current support policy.

After one generation round, the dashboard shows:

• Exact duplicate responses rise from 3 percent to 16 percent.
• Distinct-2 falls from 0.41 to 0.22.
• The 95th percentile nearest-neighbour similarity, measured with the same embedding model, rises from 0.88 to 0.96.
• The share of examples for 12 rare intents falls from 5.2 percent to 1.1 percent.
• On a fixed real-data holdout, macro F1 for those rare intents falls from 0.74 to 0.61, while common-intent performance stays around 0.90.
• Human review finds confidently wrong policy answers in 11 percent of generated samples, up from 4 percent in the previous batch.

Those numbers are illustrative, not universal thresholds. The important pattern is the agreement between independent signals. The data is becoming repetitive, rare cases are disappearing, and real-world tail performance is declining.

The most revealing symptom may be a lower training loss. The new model can become better at predicting the synthetic set while becoming worse at the actual support workload. That is not a paradox. It has learned the generator’s habits.

How I would preserve useful synthetic data

I would use five controls.

Keep an independent real-data anchor. Preserve the original human or externally sourced data, including difficult and rare examples. Do not overwrite it with generated replacements. Keep a separate holdout that is never used for prompting, filtering, or training. Otherwise the evaluation quietly becomes part of the feedback loop.

There is no magic rule that 20 percent real data is always enough. In the example, 200,000 real tickets may be a reasonable starting anchor, but I would test several mixtures—real-only, 20 percent synthetic, 50 percent synthetic, and 80 percent synthetic—using the same evaluation slices. If tail quality drops sharply at 80 percent, the answer is not to argue with the graph.

Prevent recursive generation. Generate from real inputs or trusted structured records, and use a fixed generator snapshot while constructing the dataset. Store generation depth and reject records whose inputs are already synthetic unless there is a specific, tested reason to use them. Training model B on model A’s outputs and then asking model B to generate model C’s training data is how a small bias acquires tenure.

Validate with checks that do not share all the generator’s blind spots. Use deterministic rules where possible: schema validation, executable tests for code, database constraints, policy checks, or retrieval against an authoritative source. Add human review for a stratified sample, with extra attention to rare slices and high-impact decisions. An independent model can help, but using the same model family as both writer and judge is weak evidence; shared mistakes can receive unanimous approval.

Deduplicate and allocate by slices. Remove exact duplicates and review near-duplicate clusters. Set quotas or minimum coverage targets for languages, intents, lengths, entities, and known hard cases. Keep some naturally frequent cases because their frequency is real, but do not let them consume the entire synthetic budget.

Train and evaluate with provenance-aware experiments. Compare models trained on different source mixtures. Report quality separately for real, synthetic, rare, multilingual, long-context, and adversarial slices. Track calibration too: calibration measures whether a stated confidence of 80 percent is correct about 80 percent of the time. A model that sounds more certain while its rare-case accuracy falls is not improving.

The senior-level nuance

Synthetic data is not inherently harmful. It can be excellent for format expansion, controlled paraphrases, data augmentation, and tasks with a reliable oracle. For example, generated code can be tested by compiling and running it; generated arithmetic can be checked exactly. In those cases, validation can break the error-amplification loop.

The risk is higher for open-ended factual answers, subjective preferences, and safety-sensitive decisions, where a fluent answer may have no cheap correctness test. Also, “more diverse” does not necessarily mean “more representative.” Ten stylistic rewrites of the same mistaken claim are still one mistake wearing different hats.

I would therefore challenge the premise that most of the next set should be synthetic. If the team cannot retain independent real data, cannot identify generation depth, and cannot validate outputs, it does not have a synthetic-data pipeline. It has a confidence amplifier.

What they’ll ask next

“Would you just lower the sampling temperature?”
No. Temperature changes randomness and surface variety. It may reduce repetition, but it does not restore missing real-world cases or correct factual errors. I would fix coverage, provenance, and validation first.

“How much real data must remain?”
There is no portable percentage. I would use a fixed real holdout and ablation experiments to find where performance on important slices begins to degrade. The minimum is set by the task, the tail risk, and the strength of the validation oracle.

“Can another model judge the synthetic examples?”
Yes, as one signal. Not as the only gate. Pair it with deterministic checks, human audits, source-level deduplication, and real-data evaluation. Two models can share the same blind spot very efficiently.

One line to say in the room

“I would accept synthetic data only when it improves a fixed real-world holdout without shrinking tail coverage, and I would make provenance and generation depth strong enough that synthetic errors cannot quietly become ground truth.”