ReproAssertView on GitHub
Open source · Python / pytest · alpha

The test
before the fix.

Turn a public GitHub issue and an exact commit into a minimal test candidate, then verify its failure inside a locked-down Docker boundary—before anyone changes production code.

Run it locallyInspect the verified self-fixture

Self-fixture verified. Current maximum claim: repeatable_base_failure. Historical benchmark: 0 / 20. No semantic-validity claim yet.

public issue #1 · strict profile v1verified 2026-07-09

# Public self-fixture. Generate no fix.

$ reproassert issue \
  https://github.com/Atomics-hub/reproassert/issues/1 \
  --commit 7b03e8f7f4b7... \
  --generator-command ./examples/deterministic_generator.py
claimrepeatable_base_failure
collectionpassed
base reruns3 / 3 same fingerprint
patch  evidence/live-demo/candidate.patch
report evidence/live-demo/reproassert-report.json
replay 3 / 3 same fingerprint
Self-owned fixture · fresh replay matched · not benchmark evidence
InputIssue URL + exact SHA
OutputTest patch + JSON evidence
BoundaryDocker only, network off
Benchmark0 / 20 scored

Evidence before confidence

A claim ladder with a hard stop.

ReproAssert records what happened; it does not promote a consistent failure into semantic truth. Higher claims stay visibly locked until they earn different evidence.

  1. 01
    rejected

    Rejected

    Policy, collection, setup, or evidence failed.

    implemented
  2. 02
    collected

    Collected

    The exact candidate test exists and pytest can collect it.

    implemented
  3. 03
    repeatable_base_failure

    Repeatable base failure

    Same issue-marked failure on every bounded base rerun.

    current max
  4. 04
    differential_reproduction

    Differential reproduction

    Requires repeated buggy and fixed revision evidence.

    not produced
  5. 05
    maintainer_validated

    Maintainer validated

    Requires independent human evidence outside the CLI.

    not produced

Hostile by default

The repository is data until the sandbox says otherwise.

Issue prose is never copied into a command. Repository code and candidate tests execute only inside the strict verifier. A trusted generator adapter stays outside that boundary and receives only an explicit environment allowlist.

Read the threat model
NetworkDisabled during verification
FilesystemRead-only root and workspace
IdentityNon-root UID/GID 65532
PrivilegesAll capabilities dropped
Resources1 CPU · 1 GiB · 128 PIDs
FallbackNo native host execution

Residual risk is explicit. Docker shares a kernel on Linux, test output can be adversarial, and a user-selected generator adapter is trusted host code. A repeated failure is bounded evidence—not proof of issue semantics or complete safety.

Public benchmark ledger · v0.1

Twenty frozen cases. Zero scored results.

0/ 20
01pending
02pending
03pending
04pending
05pending
06pending
07pending
08pending
09pending
10pending
11pending
12pending
13pending
14pending
15pending
16pending
17pending
18pending
19pending
20pending

This is preregistration, not performance. The manifest is frozen across 10 repositories. Every assigned case, including abstentions and infrastructure failures, stays in the denominator.

Inspect the ledger
Continue only if the evidence trends toward all gates
≥ 6 / 20semantically valid
< 10 minmedian runtime
≈ $1median cost per success or measured path
1 + 3one validation, three willing to reuse

Thin by design

One narrow, inspectable loop.

Provider-neutral generation feeds a deterministic controller. The controller owns every execution argument and every artifact path.

  1. 01

    Pin

    Canonical issue + exact SHA

  2. 02

    Bound

    Safe archive + limited context

  3. 03

    Screen

    One test + static policy

  4. 04

    Verify

    Collect + repeat in Docker

  5. 05

    Record

    Patch + replayable report

A
candidate.patch

One new pytest file. No production edits.

B
reproassert-report.json

SHA, image, limits, exit codes, logs, fingerprints, hashes.

C
reproassert replay <report>

Fresh fetch and controller-owned rerun from bounded data.

Useful locally. Open source.

Start with the boundary, not a cloud account.

The alpha supports canonical public GitHub issues and Python/pytest. It does not install repository dependencies, access private repositories, or fall back to executing on your host.

Python 3.10+uv or venvDocker
Read the install guide
local setupno hosted account
$ git clone https://github.com/Atomics-hub/reproassert.git
$ cd reproassert
$ uv sync

$ uv run reproassert sandbox build
$ uv run reproassert doctor
Docker CLIready
Sandbox imagerequired
Native fallbackdisabled

Business hypothesis · not an offer

Hosted operations may be valuable. The free core stays useful.

Illustrative path only
51teams×$199/ month=$10,149MRR

Zero willingness-to-pay, conversion, retention, or margin evidence has been measured. Private-repo runners and billing wait for technical and maintainer gates.

RA / 01

Don’t trust the fix.
Reproduce the failure.

Inspect the code, the frozen benchmark, and the exact security boundary. Then run the open-source slice locally.

Open ReproAssert on GitHub