Test Data Generation

Explanation

Learning outcomes

After completing Test Data Generation, you should be able to define it precisely, explain where it sits in a generative AI system, identify the units that cross its boundaries, implement or examine a small representative artifact and evaluate both useful behavior and failure behavior. You should be able to distinguish a conceptual claim from an empirical result and a demonstration from a production guarantee.

At the Beginner level, the expected scope is generative AI foundations, programming prompts, model APIs, coding assistants, application workflows, code generation, testing, debugging and documentation. Competence means making assumptions visible. Record the model and revision, data source and license, prompt or training configuration, software environment, hardware when relevant, random seed, evaluation cases and reviewer instructions. Without that record, a result can be interesting but it is not reliably reproducible.

Write a system contract before choosing a fashionable model or framework. State the intended user, input distribution, allowed data, required output, unacceptable result, latency and cost boundary, privacy rule, human review point and success measure. For Test Data Generation, the primary measurable concerns include compilation rate, test pass rate, mutation score, defect recall, false-positive rate, proof obligations discharged and execution cost. These concerns should guide implementation rather than being added after a demo looks convincing.

Core idea and scope

Test Data Generation uses executable checks, analysis or formal constraints to determine whether generated software satisfies a stated contract. The operational unit is requirement, program state, test or property, execution or proof result, counterexample and regression record. Draw those units and the arrows between them. Label which inputs are trusted, which are untrusted, where probabilistic behavior enters, where data is retained and which component has authority to cause an external effect.

Generative AI Programmer models estimate or sample outputs from patterns learned in data. Fluent output is not the same as verified knowledge, intent or understanding. A model can produce a useful answer and still invent a source, omit an exception or follow a malicious instruction embedded in retrieved content. Design the surrounding system so that important claims, permissions and consequences do not depend on fluency alone.

Scope determines whether a conclusion is valid. A text-generation result does not automatically transfer to image generation; an English evaluation does not establish multilingual behavior; one hosted model revision does not represent all models; a clean benchmark does not reproduce live traffic. State what has and has not been tested. For current services, preserve version identifiers because provider behavior and defaults can change.

The most important failure family for this topic is tests that mirror faulty code, flaky environments, weak oracles, unsound analysis, missed side effects and generated proofs that are not machine-checked. Convert that phrase into concrete cases. Include ordinary examples, boundary conditions, malformed inputs, adversarial inputs, absent context, conflicting instructions and distribution shifts. A good lesson studies the failure surface rather than presenting only a successful output.

System model and data flow

Model the end-to-end path: user or upstream system, input validation, preprocessing, tokenizer or encoder, model execution, decoding or sampling, output validation, tool or retrieval boundary, user interface, logging and feedback. Not every topic uses every stage, but the diagram exposes hidden dependencies and authority.

For Test Data Generation, trace one example through requirement, program state, test or property, execution or proof result, counterexample and regression record. At every transition, write the schema, size or token budget, possible error, retry policy and ownership. Identify nondeterminism from sampling, concurrency, approximate search, hardware kernels or external services. Decide which nondeterminism is acceptable and which outputs require reproducible fixtures.

Treat context as a scarce and contested resource. More context can raise cost, dilute relevant evidence and create a larger attack surface. Establish selection rules and source priorities. Separate developer instructions, user data, retrieved documents and tool outputs. Untrusted content must remain data even when it contains text that resembles instructions.

Track provenance. Derived chunks, embeddings, synthetic samples, adapted checkpoints and cached responses need links to their inputs, transformation code and versions. Provenance enables deletion, incident analysis, license review and reproducibility. A filename such as final-v2 is not lineage.

Data, models and configuration

Data quality is task-specific. Inspect coverage, duplication, language, time range, label consistency, sensitive information, licenses and representation of affected groups. Split before any transformation that can leak near-duplicates. For retrieval, keep related documents from leaking across evaluation splits. For training, document filtering and deduplication decisions because they change both capability and risk.

Choose the simplest model that meets the contract. Compare at least one modest baseline. Larger models may improve some tasks while increasing latency, cost, privacy exposure and operational dependence. Open and hosted models have different transparency, maintenance and security tradeoffs; neither category is automatically safer or better.

Configuration is part of the artifact. Record prompt templates, message roles, decoding settings, embedding model, chunking rules, index parameters, reranker, tool schemas, fine-tuning hyperparameters, quantization format and safety filters as applicable. Validate configuration at startup and reject unknown fields. Silent defaults make experiments and incidents difficult to explain.

Pin versions or immutable hashes where available. Record mutable aliases only as convenience labels. If a provider does not expose an immutable revision, timestamp the run and retain representative outputs and contract tests. Re-run the evaluation suite after any model, prompt, data, index, dependency or policy change.

Implementation workflow

Use this Beginner workflow: define a small programming task and acceptance tests, provide only relevant context, request a reviewable change, inspect the diff, run the formatter, compiler and tests, then revise one variable at a time. Begin with a vertical slice small enough to understand. A small verified pipeline is more informative than a large framework graph whose data and errors cannot be traced.

Define typed boundaries. Structured output needs a schema and validation; a tool call needs argument validation and authorization; a retrieved source needs an identifier and trust label; a training example needs a versioned schema; an evaluation result needs a case identifier and scorer version. Reject invalid values explicitly instead of coercing them into plausible-looking results.

Separate model-facing and user-facing representations. Models may need compact instructions and machine-readable tool descriptions, while users need meaningful status, evidence, uncertainty and recovery. Do not expose raw hidden prompts or secrets, and do not execute model-generated code, URLs, queries or commands without policy enforcement outside the model.

Make timeouts, cancellation and partial failure first-class. Streaming can fail after some content is visible. Retrieval can return no trustworthy evidence. A tool can succeed while the final explanation fails. Preserve idempotency for repeated state-changing operations and expose a safe recovery path.

Programming and repository discipline

Generated code is an untrusted proposal, not a finished change. Inspect the patch before execution. Confirm that it touches only intended files, preserves public interfaces unless the task requires a migration, avoids embedded credentials and does not weaken tests, authentication, authorization, validation or error handling. Reject unrelated cleanup because it expands review scope and hides causality.

Use the native toolchain as the primary oracle. Parse or compile the code, format and lint it, run focused tests, then run the appropriate broader suite. Add type checking, static analysis, dependency and secret scanning, database migration checks, accessibility tests or performance measurements where the stack requires them. A model saying that code should work is never equivalent to these checks passing.

Repository context should be selected, not dumped. Begin with repository instructions, the failing or requested behavior, nearby symbols, interfaces, callers, tests and dependency manifests. Retrieve additional files when an observed dependency requires them. Preserve generated sources, vendored code and lockfiles according to project policy. Long context can still omit the one contract that matters while consuming attention with irrelevant files.

Make executable acceptance criteria independent of the candidate implementation. If the same model writes the function and tests from the same mistaken assumption, both may agree while the requirement is violated. Add boundary cases from the specification, a regression for the original defect and at least one reviewer-authored or property-based check. Mutation testing can reveal assertions that execute without detecting meaningful faults.

Run generated commands and code inside a restricted, disposable environment. Mount only the needed repository, inject short-lived credentials only when unavoidable, deny unnecessary network access and cap CPU, memory, time and output. Require human approval for writes outside the worktree, package publication, deployment, billing, destructive database operations and contact with external people or systems.

Worked Generative AI Programmer example

Before running the example, predict inputs, outputs, invariants and likely failures. Then connect each line to Test Data Generation; do not treat the snippet as a complete production system.

A compact artifact for **Test Data Generation**:

```text
from dataclasses import dataclass
import ast

@dataclass(frozen=True)
class ChangeRequest:
requirement: str
allowed_files: frozenset[str]
required_tests: tuple[str, ...]

def preflight_python_patch(filename: str, source: str, request: ChangeRequest) -> None:
if filename not in request.allowed_files:
raise PermissionError(f"unexpected file: {filename}")
ast.parse(source, filename=filename) # Syntax is necessary, not sufficient.
if "eval(" in source or "exec(" in source:
raise ValueError("dangerous dynamic execution requires explicit review")

request = ChangeRequest(
requirement="Reject negative invoice totals without changing valid invoices",
allowed_files=frozenset({"src/invoice.py", "tests/test_invoice.py"}),
required_tests=("pytest -q tests/test_invoice.py", "pytest -q"),
)
# Apply the candidate in an isolated checkout, then run the recorded commands.
```

The example is intentionally small. Pin every external model, dataset and library version before treating its output as evidence.

Extend the example with at least three normal cases, two edge cases and two adversarial cases. Save raw results, not only screenshots. Add assertions for schemas and deterministic policy decisions. Where model output is probabilistic, evaluate a set or distribution rather than demanding one exact sentence.

For a source-code task, save the original commit, prompt, retrieved context identifiers, proposed patch, tool commands and outputs. Review the diff for scope and security before executing it in an isolated checkout. Record compiler diagnostics, test failures and any manual edits so the final result does not falsely attribute human corrections to the model.

The example deliberately separates a small trustworthy mechanism from claims about overall quality. Add a model call only after the surrounding validation, evaluation and logging behavior is understandable. Never place real secrets, private user data or unlicensed content in a classroom fixture.

Evaluation and measurement

Evaluation starts with the decision the system must support. Translate that decision into representative cases, a scoring rule and a release threshold. For Test Data Generation, useful measurements include compilation rate, test pass rate, mutation score, defect recall, false-positive rate, proof obligations discharged and execution cost. Define every metric mathematically or operationally and identify whether higher or lower is better.

Use multiple evidence types. Deterministic checks are strong for syntax, schemas, citations and policy invariants. Model graders can scale nuanced review but inherit model bias and prompt sensitivity. Human reviewers can assess usefulness and harm but need training, blinding and agreement checks. Combine them intentionally rather than presenting one score as ground truth.

Preserve slices. Aggregate accuracy can hide failure for a language, document type, topic, demographic group, tool, context length or adversarial condition. Set critical safety gates separately from average quality. Report numerator, denominator, exclusions and uncertainty, not only percentages.

Avoid evaluation leakage. Do not tune repeatedly on the final test set. Search for exact and near-duplicate benchmark items in training or prompt material when possible. Maintain a development set, a protected regression set and periodically refreshed challenge cases from real failures. A benchmark score is evidence about that benchmark under the recorded setup, not universal intelligence.

Safety, security and responsible use

The principal risk areas are invented APIs, insecure code, destructive commands, exposed secrets, licensing uncertainty, weak tests and accepting fluent explanations without execution. Perform a lightweight threat and impact analysis even for a tutorial. Name protected assets, possible attackers or accidental failures, affected people, trust boundaries and controls. Rank risks by credible impact and exposure rather than dramatic examples alone.

Prompt instructions are not a security boundary. Enforce authentication, authorization, network policy, sandboxing, output encoding, query parameterization, spending limits and human approval in conventional code. Treat retrieved pages, uploaded files, tool results and prior messages as untrusted. Validate destinations and arguments at the moment an action is attempted.

Minimize data. Do not send private material to a model merely because a provider makes it convenient. Define retention, deletion, access and incident procedures. Redact logs carefully while preserving enough identifiers to debug. Synthetic data can reproduce sensitive patterns and must be evaluated rather than assumed anonymous.

Communicate limitations in the user experience. Identify generated content when context requires it, show citations that actually support claims, enable correction and escalation, and avoid presenting calibrated-sounding confidence without evidence. High-impact domains require qualified review and should not delegate final authority to an unconstrained generator.

Performance, cost and operations

Measure latency as a distribution. For interactive generation, distinguish queue time, retrieval time, time to first token, generation rate, tool time and total completion time. Record input and output token counts, media size, cache state, concurrency and hardware or provider region. A single warm request is not a capacity result.

Cost includes training or API charges, accelerators, storage, indexes, observability, evaluation, human review and incident response. Optimize after identifying the dominant term. Shorter prompts, better retrieval, caching, batching, smaller models, quantization and speculative methods can help, but each may change quality, privacy or failure behavior.

Bound resources. Limit input and output size, tool steps, retries, retrieved documents, wall time and spend. Apply backpressure under overload. Cache only when identity, permissions, freshness and sensitive data are part of the key and policy. A fast incorrect or cross-user response is not an optimization.

Operate every change as a versioned release. Evaluate offline, deploy to a small canary, monitor quality and safety as well as infrastructure, compare against a stable control and retain immediate rollback. Define who can approve a new model or prompt and who responds when monitoring detects harm.

Independent verification

Run the artifact on at least ten normal, edge and adversarial inputs. Save the exact model identifier, prompt, settings and raw outputs. Score them with a written rubric, then ask a second reviewer to inspect factual support, safety and usability without seeing your initial scores.

Use write the contract independently, retain a trusted baseline and verify generated tests or proofs by execution, mutation or a separate checker. The independent path should fail differently from the implementation path. For example, manually inspect retrieval evidence instead of trusting only an automated faithfulness judge, or use a second framework implementation for a tensor calculation. Agreement between correlated tools is weaker than evidence from genuinely independent methods.

Record failures and limitations. A schema validator cannot establish factual truth; a benchmark cannot represent every user; a red team cannot prove the absence of attacks; a successful replication on identical hardware does not establish portability. Verification is a documented reduction of uncertainty, not a claim of perfection.

Common failure modes

1. **Vague task definition.** A demo is declared successful because the output looks fluent. Repair it with explicit cases, rubrics and thresholds tied to a user decision.
2. **Unversioned dependencies.** A model alias, dataset, index or prompt changes and the earlier result cannot be reproduced. Store versions, hashes and representative outputs.
3. **Happy-path evaluation.** Only polished examples are shown. Add malformed, ambiguous, adversarial, multilingual, long-context and unavailable-dependency cases.
4. **Metric substitution.** A convenient automated score replaces human usefulness or safety. Validate metrics against the intended decision and preserve disagreement.
5. **Authority leakage.** Generated text is allowed to trigger tools or transactions without validation. Put authorization and least-privilege controls outside the model.
6. **Hidden cost.** Quality is improved by an impractical increase in tokens, latency or review labor. Report the complete quality-cost frontier.
7. **Overgeneralization.** Results from one model, seed or benchmark become broad claims. Repeat across justified conditions and narrow the conclusion.
8. **Missing recovery.** Timeouts, provider errors or unsafe outputs leave users stranded. Design cancellation, retry, fallback, escalation and rollback.

For Test Data Generation, begin diagnosis with tests that mirror faulty code, flaky environments, weak oracles, unsound analysis, missed side effects and generated proofs that are not machine-checked. Reproduce the smallest failing case, capture the complete configuration and trace, identify the earliest violated invariant, change one factor and retain the case as a regression test.

Practice questions

1. Define Test Data Generation without using marketing language.
2. Where does it sit in an end-to-end generative AI system?
3. What is the operational unit: requirement, program state, test or property, execution or proof result, counterexample and regression record?
4. Which assumptions about users, data and models must be documented?
5. What simple baseline would make the main claim meaningful?
6. Which configuration values must be versioned for reproducibility?
7. Design three normal, two edge and two adversarial cases.
8. Explain one way data leakage or contamination could invalidate evaluation.
9. Choose two metrics from compilation rate, test pass rate, mutation score, defect recall, false-positive rate, proof obligations discharged and execution cost and define them precisely.
10. Which result requires human review, and what rubric should reviewers use?
11. How would you test the failure family tests that mirror faulty code, flaky environments, weak oracles, unsound analysis, missed side effects and generated proofs that are not machine-checked?
12. Identify every trust boundary and label untrusted content.
13. Which controls must be enforced outside the model?
14. How would latency, cost and quality change with a smaller model?
15. What evidence would justify deployment, and what evidence would stop it?
16. Design an independent verification path that can reveal a different error.
17. What limitations belong in a model card, system card or research report?
18. Formulate one falsifiable follow-up hypothesis and an ablation to test it.

Hands-on project

Build a small version-controlled program with a generated change, automated tests, static checks, documented prompts and independent code review focused on Test Data Generation. Start with a one-page system contract and diagram. Assemble at least twenty representative cases with documented provenance and usage rights. Implement a simple baseline and one deliberate improvement. Keep model, data, prompts, indexes, code and environment versioned.

Add automated schema and policy checks, a human-review rubric and a failure taxonomy. Measure compilation rate, test pass rate, mutation score, defect recall, false-positive rate, proof obligations discharged and execution cost; report distributions and important slices. Include at least five adversarial or failure cases related to tests that mirror faulty code, flaky environments, weak oracles, unsound analysis, missed side effects and generated proofs that are not machine-checked. Record latency, token or compute use and estimated cost under a repeatable workload.

Produce a release or research report containing the question, method, configuration, results, uncertainty, safety review, rejected alternatives and limitations. Include exact reproduction commands that contain no secrets. Ask another person to run the package without verbal help, record differences and revise the documentation. The project is complete only when a reviewer can inspect the evidence chain from input and versioned configuration to output, score and conclusion.

Summary

Test Data Generation is useful when it is treated as a measurable component of a larger sociotechnical system. The essential discipline is to connect requirement, program state, test or property, execution or proof result, counterexample and regression record to an explicit contract, representative evidence and bounded authority. The central measurements are compilation rate, test pass rate, mutation score, defect recall, false-positive rate, proof obligations discharged and execution cost, while the main failure family is tests that mirror faulty code, flaky environments, weak oracles, unsound analysis, missed side effects and generated proofs that are not machine-checked.

Apply the Beginner workflow: define a small programming task and acceptance tests, provide only relevant context, request a reviewable change, inspect the diff, run the formatter, compiler and tests, then revise one variable at a time. Preserve versions and provenance, compare a baseline, evaluate normal and adversarial cases, protect data and tools with non-model controls, measure cost and latency, communicate uncertainty and retain failures as regression evidence. Those habits make the lesson durable even as models and libraries change.