frc-scheduler-server: first-party-rescan
What happened, why, and what came out of it. Written by the harness from its own records; a person decides what to do with it. The same story opens the run's folder as README.md; the example's write-up is on its example page.
repository frc-scheduler-serverlanguage pythonmode rescanhead 2ad04243861bmodel qwen38-27bstarted 2026-09-15T20:02:08+00:00harness 0.1.0run 50ec3b2446fe
frc-scheduler-server. This is the application's own code, frc-scheduler-server, at commit 2ad04243861b. No known vulnerability applies to it, so there is nothing to prove; the tests below characterize what the code does today. Accept the 4 candidate tests if they look right, act on the findings below.
Who, what, why, where, when
| Who | The AI Test Harness 0.1.0 did the work without a person in the loop. A person decides the result; nothing in this run has been merged. |
| What | A scheduled scan of frc-scheduler-server at 2ad04243861b, no change under review. 7 of 64 packages carry known vulnerabilities. Tests for the application's own code at 2ad04243861b. |
| Why | frc-scheduler-server: the application's own changed code has no tests from anyone else; the harness tests what the change added or changed. |
| Where | Every test ran in a sealed sandbox (podman, network none, image docker.io/library/python:3.12-slim); the model was qwen38-27b in fixed mode; the run's own files are in this folder. |
| When | Started 2026-09-15 20:02:08 UTC. The last stage to write was self-verification at 2026-10-13 18:34:44 UTC. |
Decisions the harness made
- frc-scheduler-server: cut 3 generated test(s) before execution: 3 fails on baseline after repair.
- frc-scheduler-server: discarded 1 generation attempt(s): 1 no parseable test file after repairs.
- frc-scheduler-server: triage decided per test: 4 accept as candidate.
- frc-scheduler-server: finding, redundant at confidence 0.5: frc-scheduler-server: test_canonical_filename_encodes_shape killed none of 25 sampled mutants in this run. Route: escalate: below confidence threshold.
- frc-scheduler-server: finding, redundant at confidence 0.5: frc-scheduler-server: test_canonical_path_uses_base_dir killed none of 25 sampled mutants in this run. Route: escalate: below confidence threshold.
- frc-scheduler-server: finding, redundant at confidence 0.5: frc-scheduler-server: test_load_canonical_missing_returns_none killed none of 25 sampled mutants in this run. Route: escalate: below confidence threshold.
- frc-scheduler-server: finding, redundant at confidence 0.5: frc-scheduler-server: test_parse_csv_reads_rows killed none of 25 sampled mutants in this run. Route: escalate: below confidence threshold.
- frc-scheduler-server: mutation score 0.04 is below the blueprint's 0.6; the packet says so, and the reviewer sees which tests killed nothing.
Actions it took
- frc-scheduler-server: wrote a review packet with 4 accepted test(s) as a patch, ready for a pull request.
- frc-scheduler-server: drafted VEX statements for Product Security.
- frc-scheduler-server: signed the test-result statement (PASSED).
- frc-scheduler-server: no pull request was opened in this run; the packet holds what one would carry.
Actions it did not take, by design
- It did not merge anything. Tests reach a repository only through a pull request that a person merges.
- It did not change production code and did not propose a fix. Fixes come from a separate fix pipeline; this harness only tests.
- It did not publish a VEX statement. The drafts are proposals for Product Security.
- It did not delete any existing test. Retirements are proposed with evidence for a person to act on.
How it happened, stage by stage
What each stage read, did and wrote, with the file behind every number.
Self-verificationstage 0
Read
the harness's own code and the failure register
Did
- 28 register checks, all passed; sandbox probe skipped; model probe skipped.
Intakestage 1
Read
the repository at base and head, its dependency manifest
Did
- Resolved 63 packages at head and 63 at base with ; looked every version up in OSV: 7 with advisories at head; 0 changed; work list of 64 rows.
Analysisstage 2
Read
the work list; package source, advisories, the application's call sites
Did
- 1 package(s) analyzed: frc-scheduler-server score 35, budget full, 0 advisories, 0 breaking changes.
Generationstage 3
Read
the fact bundle per package; the model; the sealed sandbox for baseline runs
Did
- Model qwen38-27b, mode fixed.
- frc-scheduler-server: 4 tests kept in 1 file(s), 6 model calls; discarded: negative: no parseable test file after repairs.
Validation executionstage 4
Read
the candidate tests; the environments at head, base or the fixed candidate
Did
- frc-scheduler-server: sandbox runs new, new-rerun on podman (image sha256:d6b046b27fe57002af7); 4 tests, 4 pass on head, 0 flaky, 0 fix-pinning confirmed; 350 lines of the package covered.
- Mutation: 211 sites on executed lines, 25 sampled, 1 killed, score 0.04.
- Relevance: 4 generated and 47 application tests examined; 0 obsolete, 4 redundant proposed, horizon this change.
Triagestage 5
Read
every verdict and finding from stage 4
Did
- frc-scheduler-server: 4 findings, 4 tests to accept, 0 to discard or regenerate, 4 escalated to a person (threshold 0.7).
Packetstage 6
Read
triage, the tests, the advisories
Did
- frc-scheduler-server: 4 accepted tests in the patch; VEX drafts {}.
Attestationstage attest
Read
the manifest, the tests, the results; UDLM's chain code
Did
- frc-scheduler-server: 4 provenance records; statement PASSED, key sha256:4f2defc2e375, verified at build; UDLM records sealed, schema problems 0.
Assessmentstage assess
Read
every record the run wrote, against the blueprint's goals
Did
- 9 goals met, 1 not met, 2 not applicable, 1 measured without a target.
Test pull requeststage propose
Read
the packet and the records
Feedbackstage 7
Read
the pull request as a person decided it
frc-scheduler-server 2ad04243861b
In plain terms. This is the application's own code, frc-scheduler-server, at commit 2ad04243861b. No known vulnerability applies to it, so there is nothing to prove; the tests below characterize what the code does today. Accept the 4 candidate tests if they look right, act on the findings below.
Tests, one row each
| Test | Category | Old | New | Verdict | Action |
|---|
test_canonical_filename_encodes_shape | unit | na | pass | candidate: passes on head | accept as candidate |
test_canonical_path_uses_base_dir | unit | na | pass | candidate: passes on head | accept as candidate |
test_load_canonical_missing_returns_none | unit | na | pass | candidate: passes on head | accept as candidate |
test_parse_csv_reads_rows | unit | na | pass | candidate: passes on head | accept as candidate |
Findings
| Class | Confidence | Finding | Route |
|---|
| redundant | 0.5 | frc-scheduler-server: test_canonical_filename_encodes_shape killed none of 25 sampled mutants in this run | escalate: below confidence threshold |
| redundant | 0.5 | frc-scheduler-server: test_canonical_path_uses_base_dir killed none of 25 sampled mutants in this run | escalate: below confidence threshold |
| redundant | 0.5 | frc-scheduler-server: test_load_canonical_missing_returns_none killed none of 25 sampled mutants in this run | escalate: below confidence threshold |
| redundant | 0.5 | frc-scheduler-server: test_parse_csv_reads_rows killed none of 25 sampled mutants in this run | escalate: below confidence threshold |
Mutation: which test killed what
211 sites, 25 sampled, 1 killed, score 0.04.
| Test | Mutants killed | Killed only by this test |
|---|
tests.test_frc_scheduler_server_unit | 1 | 1 |
Retirement proposals
| Test | Reason | Evidence |
|---|
test_canonical_filename_encodes_shape | redundant-no-kills | test_canonical_filename_encodes_shape killed none of 25 sampled mutants in this run |
test_canonical_path_uses_base_dir | redundant-no-kills | test_canonical_path_uses_base_dir killed none of 25 sampled mutants in this run |
test_load_canonical_missing_returns_none | redundant-no-kills | test_load_canonical_missing_returns_none killed none of 25 sampled mutants in this run |
test_parse_csv_reads_rows | redundant-no-kills | test_parse_csv_reads_rows killed none of 25 sampled mutants in this run |
Discarded during generation
- negative: no parseable test file after repairs
The records, in plain terms
7 UDLM records, each a fact from this run in the estate's own language. Intent is what the harness proposed, Requested is what a person was asked to decide, Realized is what landed and who made it so. All are sealed with a content hash.
| Record | State | What it says | Written |
|---|
| Job | Requested | The harness was asked to run on frc-scheduler-server from None to 2ad04243861b in rescan mode. | 2026-09-15 20:02:08 UTC |
| Job | Realized | The run finished; its outputs are the records this statement seals. | 2026-09-15 20:08:18 UTC |
| SoftwarePackage | Discovered | frc-scheduler-server 2ad04243861b (generic) is the version this change installs. | 2026-09-15 20:08:18 UTC |
| TestEvidence | Intent | test_canonical_filename_encodes_shape is a unit test in tests/test_frc_scheduler_server_unit.py; it passed on head in the sealed sandbox. | 2026-09-15 20:08:18 UTC |
| TestEvidence | Intent | test_canonical_path_uses_base_dir is a unit test in tests/test_frc_scheduler_server_unit.py; it passed on head in the sealed sandbox. | 2026-09-15 20:08:18 UTC |
| TestEvidence | Intent | test_load_canonical_missing_returns_none is a unit test in tests/test_frc_scheduler_server_unit.py; it passed on head in the sealed sandbox. | 2026-09-15 20:08:18 UTC |
| TestEvidence | Intent | test_parse_csv_reads_rows is a unit test in tests/test_frc_scheduler_server_unit.py; it passed on head in the sealed sandbox. | 2026-09-15 20:08:18 UTC |
The run against the blueprint's goals
| Id | Goal | Measured | Verdict | From |
|---|
| G1 | Nothing merges without a person | harness produced patches and drafts only; no git push, no VEX published | met | packet.md |
| G2 | Every kept test was compiled and run in the sandbox before a reviewer sees it | 4 tests ran on head, again for flakes, and on base; 0 flaky | met | results.json |
| G3 | A test that passes on both versions is not presented as CVE evidence | 0 fix-pinning confirmed; pass-on-both CVE tests are classified test-bug and excluded from VEX 'fixed' | met | triage.json |
| G4 | Stage 0 passed before evidence was produced | 28 of 28 register checks; sandbox probe skipped; model probe skipped | met | selfcheck.json |
| G5 | Generated tests ran with no network, no secrets, read-only root, disposable container | {'capabilities': 'all dropped', 'disposable': True, 'limits': {'cpus': '2', 'memory': '2g', 'pids': 512, 'timeout_s': 90 | met | sandbox.json |
| G6 | Every accepted test carries a provenance record and a signed attestation | 4 records; envelopes verified locally: [True]; signer is a development key | met (development signer) | statement.dsse.json |
| G7 | Every advisory on the work item has a CVE-targeted test attempt and a draft VEX statement | no advisory on the work item (the application's own code, or packages with no known vulnerability) | not applicable | vex.openvex.json |
| G8 | The run surfaces a finding the PR diff does not show | frc-scheduler-server: ['frc-scheduler-server: test_canonical_filename_encodes_shape killed none of 25 sa', 'frc-scheduler-server: test_canonical_path_uses_base_dir killed none of 25 sample', 'frc | met | triage.json |
| G9 | Packet within 2 hours of trigger for depth 0 and 1 | intake 2026-09-15T20:02:45+00:00 -> packet 2026-09-15T20:29:42+00:00 | see measurement | run.json summary.json |
| G10 | Every number in the packet points at a file in the work directory | packet sections cite artifact paths; attestation subjects are the patch and the manifest digests | met | statement.json |
| G11 | Test strength measured by mutation and coverage delta | frc-scheduler-server: score 0.04 on 25 sampled mutants; coverage delta: no baseline overlay suite yet | not met (target 0.6 on sampled mutants) | mutation.json |
| G12 | The pipeline's own gaps are stated, never inherited silently | analysis predates the pipeline facts; rerun analysis | not applicable | |
| G13 | Every behaviour the change reaches ends in exactly one state, and a gap carries its reason | 40 behaviour(s) reached: 0 validated, 0 for your environment, 40 stated gap(s) (1 executed-not-checked, 19 no-test-generated, 20 strength-not-sampled); 189 reference(s) set aside with a reason | met | accounting-summary.json accounting.json |
What is in this folder, and who it is for
What is in this folder, and who it is for
Every file exists for a reader or a tool named here. The plain-English files come first; the machine files stay because they are the proof and the replay.
Everyone
README.md: The story of this run: who, what, why, where, when, the outcome, the decisions and the actions.triage/accounting-summary.json: The accounting rolled up: how many behaviours were reached, validated, handed to the team, or left as stated gaps, and why.assess/assess.md: The run against the blueprint's thirteen goals, each with the measurement and the file it came from.
Reviewers (the developer or team that owns the change)
intake/worklist.json: The work list: every package at base and head, what changed, what is reachable, what carries advisories.analyze/frc-scheduler-server/facts.json: The fact bundle for one package: versions, API diff, call sites, advisories, the risk score and the budget it earned.generate/frc-scheduler-server/tests/: The candidate test files as generated, before any decision.execute/frc-scheduler-server/results.json: The verdict per test: outcome on head, on the re-run and on the other version, coverage, and how each was judged.execute/frc-scheduler-server/mutation/mutation.json: How strong the accepted tests are: which mutants of the package's source they killed, and the score.execute/frc-scheduler-server/relevance.json: Which existing tests this change makes obsolete or redundant, proposed with evidence; nothing is deleted.triage/frc-scheduler-server/triage.json: Every verdict and finding classified, with a confidence and a route; below the threshold a person decides.triage/frc-scheduler-server/accounting.json: The accounting: every behaviour the change reaches in exactly one state, validated, for your environment or a stated gap with its reason, with the evidence per behaviour.packet/frc-scheduler-server/packet.md: The review packet: the verdict in plain terms, one row per test, the findings, the VEX drafts, the retirements, the recommended action.packet/frc-scheduler-server/tests.patch: The accepted tests as a patch in the layout they will live in.packet/frc-scheduler-server/pull-request.md: The text and the file list the test pull request carries, whether or not one was opened.
Product Security
Supply Chain Security and auditors
run.json: What was run, with which tool versions, on which repository by name.selfcheck/selfcheck.json: Stage 0: every failure-register check and the sandbox and model probes, before any evidence was produced.intake/sbom.new.cdx.json: The software bill of materials at head, CycloneDX.generate/frc-scheduler-server/model-calls/: Every prompt and every response, numbered, with the call's settings; the full transcript of what the model saw and said.attest/frc-scheduler-server/MANIFEST.json: The provenance record per accepted test: model, prompt and response digests, sandbox image digest, verdicts.attest/frc-scheduler-server/statement.json: The in-toto test-result statement; its subjects are the patch, the record and each evidence record's head.attest/frc-scheduler-server/statement.dsse.json: The statement signed as a DSSE envelope.attest/frc-scheduler-server/signer.pub.pem: The public key that verifies the envelope.attest/frc-scheduler-server/attest.json: Which key signed, whether it verified locally, and what the run could not back.attest/frc-scheduler-server/udlm/: The same facts as sealed UDLM records: the vulnerability, the package, the run, each accepted test, the draft VEX.
The person who ran the harness
Tools (kept for replay and proof; not meant to be read)