source: pr-fix-known-vulns-cluster-run3

What happened, why, and what came out of it. Written by the harness from its own records; a person decides what to do with it. The same story opens the run's folder as README.md; the example's write-up is on its example page.

repository sourcelanguage pythonmode diffbase 2ad04243861bhead cfa8f4a3292bmodel qwen/qwen3.8-27bstarted 2026-09-14T00:13:46+00:00harness 0.1.0run 333f5fa69d50
python-jose. This change updates python-jose from 3.3.0 to 3.4.0, which closes 3 known vulnerabilities. The harness could not prove any of the 3 with a test; that is stated, not hidden. Accept the 8 candidate tests if they look right, act on the findings below, and send the draft VEX statements to Product Security.

Who, what, why, where, when

WhoThe AI Test Harness 0.1.0 did the work without a person in the loop. A person decides the result; nothing in this run has been merged.
WhatTests for python-jose 3.3.0 to 3.4.0, chosen from a work list of 64 packages, 5 of which this change touched; 7 other package(s) were analyzed and scored but not selected for generation. Repository source, change 2ad04243861b..cfa8f4a3292b, mode diff.
Whypython-jose (score 51, budget full): reachable from first-party code; 159 source lines changed; 5 advisories before the change, 0 after.
WhereEvery test ran in a sealed sandbox (podman, network deny-all networkpolicy on the task pod (claim; verified by the stage 0 probe in this pod), image image-registry.openshift-image-registry.svc:5000/ai-test-harness/harness@sha256:e8276d99ada9a46acc64a703ffaa2449b09e73830f61b401d1483d311fdd5ed0); the model was qwen/qwen3.8-27b in agent mode; the run's own files are in this folder.
WhenStarted 2026-09-14 00:13:46 UTC. The last stage to write was self-verification at 2026-10-13 18:34:44 UTC.

Decisions the harness made

Actions it took

Actions it did not take, by design

How it happened, stage by stage

What each stage read, did and wrote, with the file behind every number.

  1. Self-verificationstage 0

    Read

    the harness's own code and the failure register

    Did

    • 28 register checks, all passed; sandbox probe skipped; model probe skipped.

    Wrote

    selfcheck/selfcheck.json

  2. Intakestage 1

    Read

    the repository at base and head, its dependency manifest

    Did

    • Resolved 63 packages at head and 63 at base with ; looked every version up in OSV: 4 with advisories at head; 5 changed; work list of 64 rows.

    Wrote

    intake/worklist.json · intake/graph.new.json · intake/sbom.new.cdx.json · intake/vulns.json

  3. Analysisstage 2

    Read

    the work list; package source, advisories, the application's call sites

    Did

    • 8 package(s) analyzed: ecdsa score 65, budget full, 2 advisories, 0 breaking changes; pdfminer-six score 65, budget full, 4 advisories, 0 breaking changes; pillow score 55, budget full, 0 advisories, 12 breaking changes; pyasn1 score 90, budget full, 8 advisories, 146 breaking changes; python-jose score 51, budget full, 0 advisories, 0 breaking changes; python-multipart score 55, budget full, 0 advisories, 82 breaking changes; starlette score 80, budget full, 14 advisories, 0 breaking changes; weasyprint score 70, budget full, 0 advisories, 152 breaking changes.

    Wrote

    analyze/ecdsa/facts.json · analyze/pdfminer-six/facts.json · analyze/pillow/facts.json · analyze/pyasn1/facts.json · analyze/python-jose/facts.json · analyze/python-multipart/facts.json · analyze/starlette/facts.json · analyze/weasyprint/facts.json

  4. Generationstage 3

    Read

    the fact bundle per package; the model; the sealed sandbox for baseline runs

    Did

    • Model qwen/qwen3.8-27b, mode agent.
    • python-jose: 14 tests kept in 4 file(s), 42 model calls.

    Wrote

    generate/python-jose/manifest.json · tests/ · model-calls/

  5. Validation executionstage 4

    Read

    the candidate tests; the environments at head, base or the fixed candidate

    Did

    • python-jose: sandbox runs new, new-rerun, old on podman (image sha256:e8276d99ada9a46acc6); 14 tests, 14 pass on head, 0 flaky, 0 fix-pinning confirmed; 573 lines of the package covered.
    • Mutation: 135 sites on executed lines, 25 sampled, 11 killed, score 0.44.
    • Relevance: 14 generated and 47 application tests examined; 0 obsolete, 7 redundant proposed, horizon this change.

    Wrote

    execute/python-jose/results.json · new/ · execute/python-jose/mutation/mutation.json · execute/python-jose/relevance.json

  6. Triagestage 5

    Read

    every verdict and finding from stage 4

    Did

    • python-jose: 9 findings, 8 tests to accept, 6 to discard or regenerate, 9 escalated to a person (threshold 0.7).

    Wrote

    triage/python-jose/triage.json

  7. Packetstage 6

    Read

    triage, the tests, the advisories

    Did

    • python-jose: 8 accepted tests in the patch; VEX drafts {"CVE-2024-29370": "under_investigation", "CVE-2024-33663": "under_investigation", "CVE-2024-33664": "under_investigation"}.

    Wrote

    packet/python-jose/packet.md · packet/python-jose/tests.patch · packet/python-jose/vex.openvex.json

  8. Attestationstage attest

    Read

    the manifest, the tests, the results; UDLM's chain code

    Did

    • python-jose: 8 provenance records; statement PASSED, key sha256:2b291bb6fc3c, verified at build; UDLM records sealed, schema problems 0.

    Wrote

    attest/python-jose/statement.dsse.json · attest/python-jose/MANIFEST.json · udlm/

  9. Assessmentstage assess

    Read

    every record the run wrote, against the blueprint's goals

    Did

    • 10 goals met, 1 not met, 1 not applicable, 1 measured without a target.

    Wrote

    assess/assess.md

  10. Test pull requeststage propose

    Read

    the packet and the records

    Did

    • did not run in this run.

    Wrote

    nothing

  11. Feedbackstage 7

    Read

    the pull request as a person decided it

    Did

    • did not run in this run.

    Wrote

    nothing

python-jose 3.3.0 → 3.4.0

In plain terms. This change updates python-jose from 3.3.0 to 3.4.0, which closes 3 known vulnerabilities. The harness could not prove any of the 3 with a test; that is stated, not hidden. Accept the 8 candidate tests if they look right, act on the findings below, and send the draft VEX statements to Product Security.

Tests, one row each

TestCategoryOldNewVerdictAction
test_pysec_2025_185_fix_pinningcvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_pysec_2025_185_exposurecvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_ghsa_6c5p_j8vq_pqhj_fix_pinningcvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_ghsa_6c5p_j8vq_pqhj_exposurecvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_ghsa_cjwg_qfpm_7377_fix_pinningcvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_ghsa_cjwg_qfpm_7377_exposurecvepasspassnot a fix-pinning test: passes on both versions; keep only as characterization if it covers the symboldiscard as CVE evidence; keep as characterization only if reviewer wants it
test_jwt_encode_decode_roundtripunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_decode_wrong_key_raises_jwt_errorunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_decode_expired_raises_expired_signature_errorunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_get_unverified_header_returns_algunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jwt_get_unverified_claims_returns_claimsunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_jws_sign_verify_roundtripunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_utils_base64url_roundtripunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate
test_utils_ensure_binary_coerces_str_to_bytesunitpasspasscandidate: passes on head, same on oldaccept as characterization candidate

Findings

ClassConfidenceFindingRoute
defect0.4python-jose: a code path raised the same internal error on both versions during CVE test generation (CVE-2024-33663); NOT corroborated by stage 4 verdictsescalate: below confidence threshold
defect0.4python-jose: a code path raised the same internal error on both versions during CVE test generation (CVE-2024-29370); NOT corroborated by stage 4 verdictsescalate: below confidence threshold
redundant0.6python-jose: test_jwt_decode_wrong_key_raises_jwt_error's 5 mutant kill(s) are all among test_jws_sign_verify_roundtrip's 6escalate: below confidence threshold
redundant0.6python-jose: test_jwt_decode_expired_raises_expired_signature_error's 8 mutant kill(s) are all among test_jwt_encode_decode_roundtrip's 9escalate: below confidence threshold
redundant0.6python-jose: test_jwt_get_unverified_header_returns_alg's 7 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8escalate: below confidence threshold
redundant0.6python-jose: test_jwt_get_unverified_claims_returns_claims's 6 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8escalate: below confidence threshold
redundant0.6python-jose: test_jws_sign_verify_roundtrip's 6 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8escalate: below confidence threshold
redundant0.6python-jose: test_utils_base64url_roundtrip's 1 mutant kill(s) are all among test_ghsa_cjwg_qfpm_7377_exposure's 5escalate: below confidence threshold
redundant0.6python-jose: test_utils_ensure_binary_coerces_str_to_bytes's 1 mutant kill(s) are all among test_ghsa_cjwg_qfpm_7377_exposure's 5escalate: below confidence threshold

Draft VEX statements, for Product Security

VulnerabilityStatusBasis
CVE-2024-33663, CVE-2026-85394under_investigationbump to a fixed version per advisory metadata; no confirmed fix-pinning test yet
CVE-2024-33664under_investigationbump to a fixed version per advisory metadata; no confirmed fix-pinning test yet
CVE-2024-29370under_investigationbump to a fixed version per advisory metadata; no confirmed fix-pinning test yet

Mutation: which test killed what

135 sites, 25 sampled, 11 killed, score 0.44.

TestMutants killedKilled only by this test
test_ghsa_6c5p_j8vq_pqhj_fix_pinning10
test_ghsa_cjwg_qfpm_7377_exposure50
test_ghsa_cjwg_qfpm_7377_fix_pinning50
test_jws_sign_verify_roundtrip60
test_jwt_decode_expired_raises_expired_signature_error80
test_jwt_decode_wrong_key_raises_jwt_error50
test_jwt_encode_decode_roundtrip91
test_jwt_get_unverified_claims_returns_claims60
test_jwt_get_unverified_header_returns_alg70
test_pysec_2025_185_exposure50
test_pysec_2025_185_fix_pinning50
test_utils_base64url_roundtrip10
test_utils_ensure_binary_coerces_str_to_bytes10

Retirement proposals

TestReasonEvidence
test_jwt_decode_wrong_key_raises_jwt_errorredundant-subsumedtest_jwt_decode_wrong_key_raises_jwt_error's 5 mutant kill(s) are all among test_jws_sign_verify_roundtrip's 6
test_jwt_decode_expired_raises_expired_signature_errorredundant-subsumedtest_jwt_decode_expired_raises_expired_signature_error's 8 mutant kill(s) are all among test_jwt_encode_decode_roundtrip's 9
test_jwt_get_unverified_header_returns_algredundant-subsumedtest_jwt_get_unverified_header_returns_alg's 7 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8
test_jwt_get_unverified_claims_returns_claimsredundant-subsumedtest_jwt_get_unverified_claims_returns_claims's 6 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8
test_jws_sign_verify_roundtripredundant-subsumedtest_jws_sign_verify_roundtrip's 6 mutant kill(s) are all among test_jwt_decode_expired_raises_expired_signature_error's 8
test_utils_base64url_roundtripredundant-subsumedtest_utils_base64url_roundtrip's 1 mutant kill(s) are all among test_ghsa_cjwg_qfpm_7377_exposure's 5
test_utils_ensure_binary_coerces_str_to_bytesredundant-subsumedtest_utils_ensure_binary_coerces_str_to_bytes's 1 mutant kill(s) are all among test_ghsa_cjwg_qfpm_7377_exposure's 5

The records, in plain terms

17 UDLM records, each a fact from this run in the estate's own language. Intent is what the harness proposed, Requested is what a person was asked to decide, Realized is what landed and who made it so. All are sealed with a content hash.

RecordStateWhat it saysWritten
JobRequestedThe harness was asked to run on source from 2ad04243861b to cfa8f4a3292b in diff mode.2026-09-14 00:13:46 UTC
JobRealizedThe run finished; its outputs are the records this statement seals.2026-09-14 01:12:27 UTC
SoftwarePackageDiscoveredpython-jose 3.4.0 (pypi) is the version this change installs.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jws_sign_verify_roundtrip is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jwt_decode_expired_raises_expired_signature_error is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jwt_decode_wrong_key_raises_jwt_error is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jwt_encode_decode_roundtrip is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jwt_get_unverified_claims_returns_claims is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_jwt_get_unverified_header_returns_alg is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_utils_base64url_roundtrip is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
TestEvidenceIntenttest_utils_ensure_binary_coerces_str_to_bytes is a unit test in tests/test_python_jose_unit.py; it passed on head in the sealed sandbox.2026-09-14 01:12:27 UTC
VexStatementIntentDraft: CVE-2024-33663 is under_investigation in this package. bump to a fixed version per advisory metadata; no confirmed fix-pinning test yet2026-09-14 01:12:27 UTC
VexStatementIntentDraft: CVE-2024-33664 is under_investigation in this package. bump to a fixed version per advisory metadata; no confirmed fix-pinning test yet2026-09-14 01:12:27 UTC
VexStatementIntentDraft: CVE-2024-29370 is under_investigation in this package. bump to a fixed version per advisory metadata; no confirmed fix-pinning test yet2026-09-14 01:12:27 UTC
VulnerabilityDiscoveredCVE-2024-33663 is a known vulnerability affecting <3.4.0. Also known as GHSA-6c5p-j8vq-pqhj, PYSEC-2024-232.2026-09-14 01:12:27 UTC
VulnerabilityDiscoveredCVE-2024-33664 is a known vulnerability affecting <3.4.0. Also known as GHSA-cjwg-qfpm-7377, PYSEC-2024-233.2026-09-14 01:12:27 UTC
VulnerabilityDiscoveredCVE-2024-29370 is a known vulnerability affecting versions the advisory does not pin. Also known as PYSEC-2025-185.2026-09-14 01:12:27 UTC

The run against the blueprint's goals

IdGoalMeasuredVerdictFrom
G1Nothing merges without a personharness produced patches and drafts only; no git push, no VEX publishedmetpacket.md
G2Every kept test was compiled and run in the sandbox before a reviewer sees it14 tests ran on head, again for flakes, and on base; 0 flakymetresults.json
G3A test that passes on both versions is not presented as CVE evidence0 fix-pinning confirmed; pass-on-both CVE tests are classified test-bug and excluded from VEX 'fixed'mettriage.json
G4Stage 0 passed before evidence was produced28 of 28 register checks; sandbox probe skipped; model probe skippedmetselfcheck.json
G5Generated tests ran with no network, no secrets, read-only root, disposable container{'disposable': 'the task pod', 'limits': {'cpus': '2', 'memory': '2g', 'pids': 512, 'timeout_s': 900}, 'network': 'metsandbox.json
G6Every accepted test carries a provenance record and a signed attestation8 records; envelopes verified locally: [True]; signer is a development keymet (development signer)statement.dsse.json
G7Every advisory on the work item has a CVE-targeted test attempt and a draft VEX statementpython-jose: {'CVE-2024-29370': 'under_investigation', 'CVE-2024-33663': 'under_investigation', 'CVE-2024-33664': 'under_investigation'}metvex.openvex.json
G8The run surfaces a finding the PR diff does not showpython-jose: ['python-jose: a code path raised the same internal error on both versions during ', 'python-jose: a code path raised the same internal error on both versions during ', "python-jose:mettriage.json
G9Packet within 2 hours of trigger for depth 0 and 1intake 2026-09-14T00:14:10+00:00 -> packet 2026-09-14T01:20:33+00:00see measurementrun.json summary.json
G10Every number in the packet points at a file in the work directorypacket sections cite artifact paths; attestation subjects are the patch and the manifest digestsmetstatement.json
G11Test strength measured by mutation and coverage deltapython-jose: score 0.44 on 25 sampled mutants; coverage delta: no baseline overlay suite yetnot met (target 0.6 on sampled mutants)mutation.json
G12The pipeline's own gaps are stated, never inherited silentlyanalysis predates the pipeline facts; rerun analysisnot applicable
G13Every behaviour the change reaches ends in exactly one state, and a gap carries its reason3 behaviour(s) reached: 0 validated, 0 for your environment, 3 stated gap(s) (3 strength-not-sampled); 2 reference(s) set aside with a reasonmetaccounting-summary.json accounting.json

What is in this folder, and who it is for

What is in this folder, and who it is for

Every file exists for a reader or a tool named here. The plain-English files come first; the machine files stay because they are the proof and the replay.

Everyone

  • README.md: The story of this run: who, what, why, where, when, the outcome, the decisions and the actions.
  • triage/accounting-summary.json: The accounting rolled up: how many behaviours were reached, validated, handed to the team, or left as stated gaps, and why.
  • assess/assess.md: The run against the blueprint's thirteen goals, each with the measurement and the file it came from.

Reviewers (the developer or team that owns the change)

Product Security

Supply Chain Security and auditors

The person who ran the harness

Tools (kept for replay and proof; not meant to be read)