Synthetic data
AI-made data for tests, evals and privacy gaps — almost nobody sells it.
Every AI lab trains on synthetic data, but only 3 of 62 audited tools sell generation to customers as a feature. For a small business the real uses are narrow — test data, eval sets, privacy-safe exports, padding a thin training set — and a free open-source generator covers most of them. Knowing the element exists mainly protects you from vendors who use the phrase as a spec-sheet decoration.
You need this when
- You need realistic test or eval data and cannot put real customer records on a staging system.
- You must hand data to a contractor, demo or BI tool without exporting real names, IBANs or notes.
- You have 40 labelled examples of a rare case and need 400 to fine-tune or benchmark honestly.
You can skip it when
- Your real dataset is big enough and you are allowed to use it — use it; synthetic adds no new knowledge.
- You want to fix a biased or thin dataset — instead fix collection at the source; the generator copies the bias.
- A vendor pitches it as an add-on you would never configure — instead use the free MOSTLY AI SDK or a Flash-Lite prompt for about €0.12 per 1,000 records.
The long version — open when you want the depth
What it is — in one coffee-break
Synthetic data is data a model made up on purpose. You give a generator a description, or a sample of your real records, and it produces new records that behave like the originals without being any of them. Every serious AI lab uses synthetic data internally — that is not what this element measures. The test for Sy is harsher: does the vendor ship generation to you, pointed at your own data? That is why the box is nearly empty — three of 62 audited tools score at all. For most of the market this is an ingredient in the kitchen, not an item on the menu.
When you actually need it (and when you don't)
For a small business the honest list is three items long. One: test and eval data — 200 realistic customer emails to test a routing workflow, because you cannot park real ones on a staging server. Two: privacy-sensitive gaps — handing data to a contractor, demo or BI tool without real names, IBANs or medical notes. Three: a thin set — 40 labelled examples of a rare complaint type where you need 400.
Everything else is where money goes to die. Synthetic data cannot create knowledge that was not in your originals, and it faithfully reproduces whatever bias sat in those 40 examples. If your real data is big enough and you may use it, use it. And never train a model in a loop on its own output: recursive training on generated data degrades quality generation after generation — model collapse. The mitigation is boring and it works — accumulate, do not replace. Keep real data in the mix instead of swapping it out.
How to recognize good vs bad implementations
A good implementation reports on itself: a fidelity and privacy report, constraints you can set, referential integrity across linked tables, seeding from your real schema, and a plain statement of which columns it failed on. A bad one is a prompt box emitting plausible CSV rows. Three questions expose the difference in a demo:
"Show me the privacy report for this generated table — how many rows are near-copies of a real record, and what is your distance-to-nearest-record metric?" "Generate 5,000 rows across three linked tables while I watch, then show me the foreign keys still resolve and the date ordering still holds." "Which fields did the generator fail on, and what does it do with a column it does not understand?"
Then the compliance question most buyers skip: synthetic output is not automatically anonymous. The EDPB's draft anonymisation guidelines of 7 July 2026 say so directly: a synthetic set still fails if individuals can be singled out, linked to other sources, or inferred from it — see this practical read of the guidelines. "It's synthetic, so GDPR doesn't apply" is the wrong answer and a useful tell.
What this costs
Cheap at the small end, quote-only at the large end. Tonic's Fabricate has a free tier with $5 a month of credits and a Plus tier at $29 a month with $25, metered near $0.17 per standard generation turn and $0.37 for a complex multi-table one. Their de-identification products carry no public price at all — assume a five-figure annual conversation and a procurement cycle to match.
The free path is genuinely good — for most operators the right answer, not the fallback. For text, prompt a cheap hosted model: at Gemini 2.5 Flash-Lite rates of $0.10 per million input tokens and $0.40 per million output, 1,000 synthetic support tickets of ~300 tokens each costs about $0.14 — roughly €0.12. For tabular data, the MOSTLY AI Synthetic Data SDK is Apache-2.0, runs locally on CPU with Python 3.11+, and makes millions of rows without a GPU. Cost: your laptop and an afternoon. Paid platforms earn their money on governance, audit trails and integrity across dozens of tables — buy one when you have that problem.
Where to see it scored
Only 3 of the 62 audited tools score here at all, and that scarcity is the finding. Runway (6.5) leads, though not with business data: its robotics line ships data augmentation turning existing robot trajectories into new environments, lighting and object layouts. Gemini / Google (5.5) scores the platform route — generation and evaluation tooling you assemble yourself, powerful but not a finished product. Decagon (4) is lowest and narrowest: it autogenerates simulated customer conversations — accents, interruptions, angry callers — to test support agents before go-live. Useful, but it is test data for Decagon's own agents, not a generator you can aim elsewhere. Read the empty column as a buying signal: when a vendor pitches synthetic data, ask which of those three shapes it actually is.
Flashcards
Check yourself
1. What qualifies a tool to score on the Sy element?
2. Which is NOT a realistic small-business use of synthetic data?
3. What causes model collapse?
4. Under the EDPB's July 2026 draft guidelines, synthetic data is...
5. Roughly what does 1,000 LLM-generated support tickets cost at Flash-Lite rates?
Cheat sheet
- Test: does the vendor ship generation to customers, not just use it internally?
- 3 of 62 audited tools score here. Empty column = mostly marketing language.
- Real SMB uses: test data, eval sets, privacy-safe exports, thin-set padding.
- Never loop-train on your own model's output. Accumulate real data, don't replace it.
- EDPB, 7 Jul 2026: synthetic data is NOT automatically anonymous under GDPR.
- Free path: MOSTLY AI SDK (Apache-2.0, CPU) or ~$0.14 per 1,000 LLM-made records.
Who actually does this well
| Best on this element | Score | Why it scored that |
|---|---|---|
| Roark | 8.5 | Manufacturing realistic callers is the core mechanic: personas carry language, accent, gender, base emotion and speaking style, environments add background noise, and templates fan |
| Bluejay | 8 | Generating synthetic conversation data is the core mechanic: personas with varied languages, accents, emotion and background noise, bulk-created from CSV, plus auto-generated scena |
| Hamming AI | 8 | Scenario generation from a prompt plus documentation is the feature Hamming claims to have pioneered, and it runs at volume: hundreds of characters with voices, accents and noise p |
| Maxim AI | 8 | Synthetic datasets generate from scratch or from a reference dataset with configurable inputs, expected outputs, scenarios, personas and expected steps, and simulation itself is sy |
| Promptfoo | 8 | Adversarial test generation is real synthetic data as a product: `redteam generate` writes thousands of context-aware attacks per target, and `redteam poison` builds poisoned-docum |
| Cekura | 7.5 | Thousands of synthetic conversations per release, generated from the agent description, across personas with varied accents, noise and conversational styles, plus a standing librar |
| Cognigy (NiCE Cognigy) | 7.5 | Simulator's digital-twin synthetic-customer generation is a genuine, well-documented, customer-facing capability, though scoped to conversation testing rather than general-purpose |
| Coval | 7.5 | The Test Set Generator drafts test cases from a plain description plus attached prompts or docs and generates typed attributes such as a ticket number of format X-###. The persona |
And the other end of the same column:
| Weakest | Score | Why it scored that |
|---|---|---|
| Langfuse | 3.5 | No synthetic-data product. The Assistant can create a dataset on request and the roadmap mentions using AI to generate code-based evaluators, but there is nothing comparable to a g |
| Meta Llama ecosystem | 3.5 | Glimmer's model card names synthetic data generation as an intended use and Meta used synthetic data in its own train-time mitigations, but there is no synthetic-data tooling or se |
| Tines | 3 | Tines 3B proposes tests as you build and generates dummy data when it needs it, asking before it touches live data or apps. That is the entire synthetic-data story: useful, and nar |
| Helicone | 2.5 | No synthetic-data generation of any kind. Datasets come from real production requests or a CSV import. Scored low rather than N/A because competitors in this category ship generati |
Scored on 26 of 163 audited tools. Every score links to the full audit and its reasoning.