frc-scheduler-server: pr-fix-known-vulns-run6

What happened, why, and what came out of it. Written by the harness from its own records; a person decides what to do with it. The same story opens the run's folder as README.md; the example's write-up is on its example page.

repository frc-scheduler-serverlanguage pythonmode diffbase 2ad04243861bhead cfa8f4a3292bmodel qwen38-27bstarted 2026-09-12T01:41:41+00:00harness 0.1.0run 76a3fa863e76
python-jose. This change updates python-jose from 3.3.0 to 3.4.0, which closes 3 known vulnerabilities. The harness proved 2 of the 3 with a test that fails on the vulnerable version and passes on the fixed one; the other 1 is unproven and marked so. Accept the 7 candidate tests if they look right, act on the findings below, and send the draft VEX statements to Product Security.

Who, what, why, where, when

WhoThe AI Test Harness 0.1.0 did the work without a person in the loop. A person decided the result: merged by croadfeldt at 2026-09-15T16:16:06Z.
WhatTests for python-jose 3.3.0 to 3.4.0, chosen from a work list of 64 packages, 5 of which this change touched. Repository frc-scheduler-server, change 2ad04243861b..cfa8f4a3292b, mode diff.
Whypython-jose (score 51, budget full): reachable from first-party code; 159 source lines changed; 5 advisories before the change, 0 after.
WhereEvery test ran in a sealed sandbox (podman, network none, image docker.io/library/python:3.12-slim); the model was qwen38-27b in agent mode; the tests were proposed to the overlay repository as https://github.com/croadfeldt/ai-test-harness-overlays/pull/1; the run's own files are in this folder.
WhenStarted 2026-09-12 01:41:41 UTC. The last stage to write was self-verification at 2026-10-13 18:34:45 UTC.

Decisions the harness made

Actions it took

Actions it did not take, by design

How it happened, stage by stage

What each stage read, did and wrote, with the file behind every number.

  1. Self-verificationstage 0

    Read

    the harness's own code and the failure register

    Did

    • 28 register checks, all passed; sandbox probe skipped; model probe skipped.

    Wrote

    selfcheck/selfcheck.json

  2. Intakestage 1

    Read

    the repository at base and head, its dependency manifest

    Did

    • Resolved 63 packages at head and 63 at base with ; looked every version up in OSV: 4 with advisories at head; 5 changed; work list of 64 rows.

    Wrote

    intake/worklist.json · intake/graph.new.json · intake/sbom.new.cdx.json · intake/vulns.json

  3. Analysisstage 2

    Read

    the work list; package source, advisories, the application's call sites

    Did

    • 1 package(s) analyzed: python-jose score 51, budget full, 0 advisories, 0 breaking changes.

    Wrote

    analyze/python-jose/facts.json

  4. Generationstage 3

    Read

    the fact bundle per package; the model; the sealed sandbox for baseline runs

    Did

    • Model qwen38-27b, mode agent.
    • python-jose: 10 tests kept in 4 file(s), 36 model calls.

    Wrote

    generate/python-jose/manifest.json · tests/ · model-calls/

  5. Validation executionstage 4

    Read

    the candidate tests; the environments at head, base or the fixed candidate

    Did

    • python-jose: sandbox runs new, new-rerun, old on podman (image digest not recorded); 10 tests, 9 pass on head, 0 flaky, 3 fix-pinning confirmed; 573 lines of the package covered.
    • Mutation: 139 sites on executed lines, 12 sampled, 3 killed, score 0.25.

    Wrote

    execute/python-jose/results.json · new/ · execute/python-jose/mutation/mutation.json · execute/python-jose/relevance.json

  6. Triagestage 5

    Read

    every verdict and finding from stage 4

    Did

    • python-jose: 1 findings, 7 tests to accept, 3 to discard or regenerate, 1 escalated to a person (threshold 0.7).

    Wrote

    triage/python-jose/triage.json

  7. Packetstage 6

    Read

    triage, the tests, the advisories

    Did

    • python-jose: 7 accepted tests in the patch; VEX drafts {"CVE-2024-29370": "under_investigation", "CVE-2024-33663": "fixed", "CVE-2024-33664": "fixed"}.

    Wrote

    packet/python-jose/packet.md · packet/python-jose/tests.patch · packet/python-jose/vex.openvex.json

  8. Attestationstage attest

    Read

    the manifest, the tests, the results; UDLM's chain code

    Did

    • python-jose: 7 provenance records; statement PASSED, key sha256:4f2defc2e375, verified at build; UDLM records sealed, schema problems 0.

    Wrote

    attest/python-jose/statement.dsse.json · attest/python-jose/MANIFEST.json · udlm/

  9. Assessmentstage assess

    Read

    every record the run wrote, against the blueprint's goals

    Did

    • 10 goals met, 1 not met, 1 not applicable, 1 measured without a target.

    Wrote

    assess/assess.md

  10. Test pull requeststage propose

    Read

    the packet and the records

    Did

    Wrote

    propose/python-jose/proposal.json

  11. Feedbackstage 7

    Read

    the pull request as a person decided it

    Did

    • python-jose: merged by croadfeldt at 2026-09-15T16:16:06Z; 7 accepted, 0 edited, 0 rejected; 14 requested and realized records; acceptance statement signed.

    Wrote

    feedback/python-jose/acceptance.json · feedback/python-jose/decisions.jsonl

python-jose 3.3.0 → 3.4.0

In plain terms. This change updates python-jose from 3.3.0 to 3.4.0, which closes 3 known vulnerabilities. The harness proved 2 of the 3 with a test that fails on the vulnerable version and passes on the fixed one; the other 1 is unproven and marked so. Accept the 7 candidate tests if they look right, act on the findings below, and send the draft VEX statements to Product Security.

Tests, one row each

TestCategoryOldNewVerdictAction
test_pysec_2025_185_fix_pinningcvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_pysec_2025_185_exposurecvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_ghsa_6c5p_j8vq_pqhj_fix_pinningcvefailpassfix-pinning confirmed: fails on vulnerable, passes on fixedaccept as CVE evidence; VEX status fixed
test_ghsa_6c5p_j8vq_pqhj_exposurecvefailfailfails on the fixed version: test bug or the advisory is misread; back to generationregenerate or discard
test_ghsa_cjwg_qfpm_7377_fix_pinningcvefailpassfix-pinning confirmed: fails on vulnerable, passes on fixedaccept as CVE evidence; VEX status fixed
test_ghsa_cjwg_qfpm_7377_exposurecvefailpassfix-pinning confirmed: fails on vulnerable, passes on fixedaccept as CVE evidence; VEX status fixed
test_jwt_encode_hs256_structureunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_decode_roundtrip_hs256unitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_decode_wrong_key_raises_jwt_errorunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_decode_expired_raises_expired_signature_errorunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate

Findings

ClassConfidenceFindingRoute
defect0.4python-jose: a code path raised the same internal error on both versions during CVE test generation (CVE-2024-33663); NOT corroborated by stage 4 verdictsescalate: below confidence threshold

Draft VEX statements, for Product Security

VulnerabilityStatusBasis
CVE-2024-33663, CVE-2026-85394fixedfix-pinning test(s) ['test_ghsa_6c5p_j8vq_pqhj_fix_pinning'] fail on pkg:pypi/python-jose@3.3.0 and pass on pkg:pypi/python-jose@3.4.0
CVE-2024-33664fixedfix-pinning test(s) ['test_ghsa_cjwg_qfpm_7377_fix_pinning', 'test_ghsa_cjwg_qfpm_7377_exposure'] fail on pkg:pypi/python-jose@3.3.0 and pass on pkg:pypi/python-jose@3.4.0
CVE-2024-29370under_investigationbump to a fixed version per advisory metadata; no confirmed fix-pinning test yet

Mutation: which test killed what

139 sites, 12 sampled, 3 killed, score 0.25.

TestMutants killedKilled only by this test
test_jwt_decode_expired_raises_expired_signature_error21
test_jwt_decode_roundtrip_hs25621
test_jwt_decode_wrong_key_raises_jwt_error10
test_jwt_encode_hs256_structure10

The records, in plain terms

30 UDLM records, each a fact from this run in the estate's own language. Intent is what the harness proposed, Requested is what a person was asked to decide, Realized is what landed and who made it so. All are sealed with a content hash.

RecordStateWhat it saysWritten
JobRequestedThe harness was asked to run on frc-scheduler-server from 2ad04243861b to cfa8f4a3292b in diff mode.2026-09-12 01:41:41 UTC
JobRealizedThe run finished; its outputs are the records this statement seals.2026-09-13 13:29:49 UTC
SoftwarePackageDiscoveredpython-jose 3.4.0 (pypi) is the version this change installs.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_ghsa_6c5p_j8vq_pqhj_fix_pinning is a fix-pinning test for CVE-2024-33663 in tests/test_python_jose_cve_cve_2024_33663.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_ghsa_cjwg_qfpm_7377_exposure is an exposure test for CVE-2024-33664 in tests/test_python_jose_cve_cve_2024_33664.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_ghsa_cjwg_qfpm_7377_fix_pinning is a fix-pinning test for CVE-2024-33664 in tests/test_python_jose_cve_cve_2024_33664.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_jwt_decode_expired_raises_expired_signature_error is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_jwt_decode_roundtrip_hs256 is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_jwt_decode_wrong_key_raises_jwt_error is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
TestEvidenceIntenttest_jwt_encode_hs256_structure is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-13 13:29:49 UTC
VexStatementIntentDraft: CVE-2024-33663 is fixed in this package. fix-pinning test(s) ['test_ghsa_6c5p_j8vq_pqhj_fix_pinning'] fail on pkg:pypi/python-jose@3.3.0 and pass on pkg:pypi/python-jose@3.4.02026-09-13 13:29:49 UTC
VexStatementIntentDraft: CVE-2024-33664 is fixed in this package. fix-pinning test(s) ['test_ghsa_cjwg_qfpm_7377_fix_pinning', 'test_ghsa_cjwg_qfpm_7377_exposure'] fail on pkg:pypi/python-jose@3.3.0 and pass on pkg:pypi/python-jose@3.4.02026-09-13 13:29:49 UTC
VexStatementIntentDraft: CVE-2024-29370 is under_investigation in this package. bump to a fixed version per advisory metadata; no confirmed fix-pinning test yet2026-09-13 13:29:49 UTC
VulnerabilityDiscoveredCVE-2024-33663 is a known vulnerability affecting <3.4.0. Also known as GHSA-6c5p-j8vq-pqhj, PYSEC-2024-232.2026-09-13 13:29:49 UTC
VulnerabilityDiscoveredCVE-2024-33664 is a known vulnerability affecting <3.4.0. Also known as GHSA-cjwg-qfpm-7377, PYSEC-2024-233.2026-09-13 13:29:49 UTC
VulnerabilityDiscoveredCVE-2024-29370 is a known vulnerability affecting versions the advisory does not pin. Also known as PYSEC-2025-185.2026-09-13 13:29:49 UTC
TestEvidenceRequestedA person was asked to decide test_ghsa_6c5p_j8vq_pqhj_fix_pinning on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_ghsa_6c5p_j8vq_pqhj_fix_pinning was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_cve_cve_2024_33663.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_ghsa_cjwg_qfpm_7377_fix_pinning on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_ghsa_cjwg_qfpm_7377_fix_pinning was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_cve_cve_2024_33664.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_ghsa_cjwg_qfpm_7377_exposure on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_ghsa_cjwg_qfpm_7377_exposure was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_cve_cve_2024_33664.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_jwt_encode_hs256_structure on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_jwt_encode_hs256_structure was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_unit.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_jwt_decode_roundtrip_hs256 on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_jwt_decode_roundtrip_hs256 was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_unit.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_jwt_decode_wrong_key_raises_jwt_error on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_jwt_decode_wrong_key_raises_jwt_error was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_unit.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC
TestEvidenceRequestedA person was asked to decide test_jwt_decode_expired_raises_expired_signature_error on the pull request.2026-09-15 16:16:06 UTC
TestEvidenceRealizedtest_jwt_decode_expired_raises_expired_signature_error was accepted by croadfeldt and landed at overlays/python/python-jose/3.4.x/test_python_jose_unit.py in merge 87109359f6c3.2026-09-15 16:16:06 UTC

The run against the blueprint's goals

IdGoalMeasuredVerdictFrom
G1Nothing merges without a personharness produced patches and drafts only; no git push, no VEX publishedmetpacket.md
G2Every kept test was compiled and run in the sandbox before a reviewer sees it10 tests ran on head, again for flakes, and on base; 0 flakymetresults.json
G3A test that passes on both versions is not presented as CVE evidence3 fix-pinning confirmed; pass-on-both CVE tests are classified test-bug and excluded from VEX 'fixed'mettriage.json
G4Stage 0 passed before evidence was produced28 of 28 register checks; sandbox probe skipped; model probe skippedmetselfcheck.json
G5Generated tests ran with no network, no secrets, read-only root, disposable container{'capabilities': 'all dropped', 'disposable': True, 'limits': {'cpus': '2', 'memory': '2g', 'pids': 512, 'timeout_s': 90metsandbox.json
G6Every accepted test carries a provenance record and a signed attestation7 records; envelopes verified locally: [True]; signer is a development keymet (development signer)statement.dsse.json
G7Every advisory on the work item has a CVE-targeted test attempt and a draft VEX statementpython-jose: {'CVE-2024-29370': 'under_investigation', 'CVE-2024-33663': 'fixed', 'CVE-2024-33664': 'fixed'}metvex.openvex.json
G8The run surfaces a finding the PR diff does not showpython-jose: ['python-jose: a code path raised the same internal error on both versions during ']mettriage.json
G9Packet within 2 hours of trigger for depth 0 and 1intake 2026-09-12T01:42:29+00:00 -> packet 2026-09-13T13:33:56+00:00see measurementrun.json summary.json
G10Every number in the packet points at a file in the work directorypacket sections cite artifact paths; attestation subjects are the patch and the manifest digestsmetstatement.json
G11Test strength measured by mutation and coverage deltapython-jose: score 0.25 on 12 sampled mutants; coverage delta: no baseline overlay suite yetnot met (target 0.6 on sampled mutants)mutation.json
G12The pipeline's own gaps are stated, never inherited silentlyanalysis predates the pipeline facts; rerun analysisnot applicable
G13Every behaviour the change reaches ends in exactly one state, and a gap carries its reason3 behaviour(s) reached: 1 validated, 0 for your environment, 2 stated gap(s) (2 strength-not-sampled); 2 reference(s) set aside with a reasonmetaccounting-summary.json accounting.json

What is in this folder, and who it is for

What is in this folder, and who it is for

Every file exists for a reader or a tool named here. The plain-English files come first; the machine files stay because they are the proof and the replay.

Everyone

  • README.md: The story of this run: who, what, why, where, when, the outcome, the decisions and the actions.
  • triage/accounting-summary.json: The accounting rolled up: how many behaviours were reached, validated, handed to the team, or left as stated gaps, and why.
  • assess/assess.md: The run against the blueprint's thirteen goals, each with the measurement and the file it came from.

Reviewers (the developer or team that owns the change)

Product Security

  • intake/vulns.json: Known vulnerabilities for every package version in either graph, from OSV, including malicious-package reports.
  • analyze/python-jose/vulns.json: The advisories on this package's versions, with the symbols they name and the fixed versions.
  • packet/python-jose/vex.openvex.json: One draft VEX statement per advisory, with its basis; a proposal for Product Security, never published by the harness.

Supply Chain Security and auditors

The person who ran the harness

Tools (kept for replay and proof; not meant to be read)