Red-evidence: N-AALP verify-at-use guard¶
Mutation-survival evidence for the load-bearing properties of naalp_verify_at_use. Each row
was produced by actually editing naalp_verify_at_use/verify_at_use.py (never any
impl/python/naalp/* file, which is Part-1 and out of scope for this task), confirming the
named test(s) flip RED for the stated reason, then restoring the file from a .bak copy
(never git checkout), clearing __pycache__ (PYTHONDONTWRITEBYTECODE=1 set for every run
so no stale bytecode can mask a mutation or fake a revert), and re-running the full suite
GREEN with the file's hash re-confirmed identical to the pre-mutation baseline.
Baseline / reverted file hash for naalp_verify_at_use/verify_at_use.py (SHA-256), confirmed
identical before mutation 1 and after every one of the four reverts below:
Run command (from ecosystem/naalp-verify-at-use/, invoking the real installed Python
interpreter for this platform -- on Windows the Microsoft-Store python/python3
execution-alias stubs resolve ahead of a real install on PATH and hang, so on that platform
invoke the actual interpreter binary directly rather than the bare python command):
All commands below were run from ecosystem/naalp-verify-at-use/ with
PYTHONDONTWRITEBYTECODE=1 set and __pycache__ removed immediately before every test
invocation (mutate, revert, and re-confirm), so no cached .pyc could report a result that
did not come from the file on disk at that moment. Full suite size: 18 tests.
Non-circular vector provenance (established before any mutation test)¶
OracleGroundingTests confirms, before any fail-closed test relies on them, that the two
Part-1 wire primitives this guard's checks depend on match an authority OUTSIDE the code
under test (naalp.approval / naalp.identity / naalp.cbor):
- Approval body bytes:
tools/approval_oracle.py(an existing top-level oracle, read only) buildsvectors/approval/cases.json's approval "A" body withtools/cbor_oracle.py's from-scratch deterministic-CBOR encoder -- notnaalp.cbor. Re-deriving the identical body throughnaalp.approval.ApprovalRecord(...).bytes()and comparing to the oracle'srecord_hexis a genuine non-circular check on the wire encodingnaalp.approval.verify_approval(called insideconsume_approval, which this guard delegates to) depends on. - Signer id:
tools/signerid_oracle.py(an existing top-level oracle, read only) is an independent multicodec/multihash/multibase constructor for the self-certifying signer id, sharing no code withnaalp.identity. Checked against (1) the pinned, GRADEDvectors/identity/cases.jsonEd25519 fixture (built from RFC 8032's own published test key -- an id no part of this package derived), and (2) the same oracle invoked directly on a freshly generated ML-DSA-65 key this package's own tests sign with. - RevocationRecord body bytes: independently re-derived via
tools/cbor_oracle.py's{1: key, 2: not_after}map encoding and compared tonaalp.identity.RevocationRecord(...).bytes().
All three passed before any fail-closed mutation test below was written against them.
M1 -- fail-open bypass (test_valid_approval_executes_exactly_once +5 others)¶
Mutated: VerifyAtUseGuard.execute(), removed the call to the re-verify choke point
entirely:
- pos_time = self._clock() if pos_time is None else pos_time
- args_id = cbor.content_id(action.args)
- self.verify_and_consume(record, sig, args_id, action.effect, pos_time) # RESUME only past this line
- return action.run()
+ pos_time = self._clock() if pos_time is None else pos_time
+ args_id = cbor.content_id(action.args)
+ # MUTATION M1 (red-evidence): skip the re-verify entirely -- fail-open.
+ return action.run()
RED, confirmed: 6 of 18 tests failed --
FAIL: test_exactly_once_under_race
AssertionError: 8 != 1 : the action must run exactly once under a race
FAIL: test_expired_at_use_through_execute_never_runs
AssertionError: ApprovalError not raised
FAIL: test_reuse_through_execute_never_runs_a_second_time
AssertionError: ApprovalError not raised
FAIL: test_revoked_key_through_execute_never_runs
AssertionError: ApprovalError not raised
FAIL: test_valid_approval_executes_exactly_once
AssertionError: False is not true (ledger.is_consumed(rec.id()))
FAIL: test_wrong_audience_through_execute_never_runs
AssertionError: ApprovalError not raised
Every test driven THROUGH execute() (the concurrency test included -- it drives execute()
in 8 threads) flipped: with the re-verify skipped, execute() runs the action unconditionally
regardless of expiry, reuse, revocation, or audience, and a raced approval runs 8 times instead
of once. The 12 tests that call verify_and_consume() directly (the choke point itself, not
execute()) were unaffected -- exactly the boundary this mutation targets.
Reverted: the file was restored from its .bak copy (never git checkout), __pycache__
cleared, hash re-confirmed 75c51cb134a746108f402c93b53010bddf9fac25896ef83ef7e8484fc298a19c.
GREEN, confirmed: 18/18 tests pass.
M2 -- ignore the injected use-time clock (test_valid_at_issuance_expired_at_use_time_fails_closed +3 others)¶
Mutated: VerifyAtUseGuard.verify_and_consume(), the clock-injection line (the guard's
own now/pos_time plumbing -- the property the requirement text explicitly names: "valid at
issuance but expired ... at use-time"):
- pos_time = self._clock() if pos_time is None else pos_time
+ pos_time = 0 # MUTATION M2 (red-evidence): ignore the injected use-time clock entirely
RED, confirmed: 4 of 18 tests failed --
FAIL: test_valid_at_issuance_expired_at_use_time_fails_closed
AssertionError: ApprovalError not raised
FAIL: test_valid_at_issuance_key_revoked_at_use_time_fails_closed
AssertionError: ApprovalError not raised
FAIL: test_expired_at_use_through_execute_never_runs
AssertionError: ApprovalError not raised
FAIL: test_revoked_key_through_execute_never_runs
AssertionError: ApprovalError not raised
With the clock forced to epoch 0, pos_time > a.not_after is never true for any real
not_after (defeating expiry) and pos_time > revocation.not_after is never true for any
real revocation (defeating key liveness) -- both a genuine fail-open regression, not a crash.
Exactly the four clock-dependent tests flipped (14 of 18 still passed); every clock-independent
check (signature, audience, effect, single-use, args binding) was unaffected.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
75c51cb134a746108f402c93b53010bddf9fac25896ef83ef7e8484fc298a19c.
GREEN, confirmed: 18/18 tests pass.
M3 -- single-use-at-settlement (test_reuse_fails_closed +2 others)¶
Mutated: VerifyAtUseGuard.verify_and_consume(), wrapped the ledger consume call to
swallow exactly the single-use error and pretend it succeeded:
- return approval.consume_approval(
- record, self._approver_alg, self._approver_pubkey, sig, args_content_id,
- pos_time, required_effect, self._ledger, self._identity,
- )
+ try:
+ return approval.consume_approval(
+ record, self._approver_alg, self._approver_pubkey, sig, args_content_id,
+ pos_time, required_effect, self._ledger, self._identity,
+ )
+ except approval.ApprovalError as e:
+ if e.kind == "AlreadyConsumed":
+ # MUTATION M3 (red-evidence): silently swallow a replay as if it succeeded.
+ return None
+ raise
RED, confirmed: 3 of 18 tests failed --
FAIL: test_exactly_once_under_race
AssertionError: 8 != 1 : the action must run exactly once under a race
FAIL: test_reuse_fails_closed
AssertionError: ApprovalError not raised
FAIL: test_reuse_through_execute_never_runs_a_second_time
AssertionError: ApprovalError not raised
With AlreadyConsumed swallowed, a second (or eighth, under the race) use of the SAME
approval falls through as if it were fresh -- exactly the single-use-at-settlement violation
this guard exists to prevent. Exactly the three reuse-dependent tests flipped (15 of 18 still
passed).
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
75c51cb134a746108f402c93b53010bddf9fac25896ef83ef7e8484fc298a19c.
GREEN, confirmed: 18/18 tests pass.
M4 -- key-liveness re-check (test_valid_at_issuance_key_revoked_at_use_time_fails_closed +1 other)¶
Mutated: VerifyAtUseGuard.verify_and_consume(), disabled the key-revocation branch so it
is never taken:
approver_id = identity.signer_id(self._approver_alg, self._approver_pubkey)
revocation = self._revocations.get(approver_id)
- if revocation is not None and identity.revoked_at(revocation, pos_time):
+ if False: # MUTATION M4 (red-evidence): never honor a registered key revocation
raise approval.ApprovalError("KeyRevoked", "approver key is revoked as of use-time")
RED, confirmed: 2 of 18 tests failed --
FAIL: test_valid_at_issuance_key_revoked_at_use_time_fails_closed
AssertionError: ApprovalError not raised
FAIL: test_revoked_key_through_execute_never_runs
AssertionError: ApprovalError not raised
With the revocation branch dead, an approval signed under a key this guard has been told is
revoked is treated as if the key were still live -- the exact "revoked" property this guard names
alongside expired/consumed/out-of-audience. Exactly the two revocation-dependent tests flipped
(16 of 18 still passed); the sibling test asserting a FUTURE-dated revocation does NOT yet
block use (test_key_revoked_only_in_the_future_does_not_block_use_now) stayed green in both
the baseline and the mutant, since it never took the raise branch either way -- confirming
the mutation targets exactly the revocation check and nothing else.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
75c51cb134a746108f402c93b53010bddf9fac25896ef83ef7e8484fc298a19c.
GREEN, confirmed: 18/18 tests pass.
Why these four, and why every mutation targets verify_at_use.py rather than naalp.*¶
naalp_verify_at_use is glue: every cryptographic, identity, and ledger check (signature,
args-content-id binding, expiry, key-liveness, effect-ceiling, atomic single-use consume) is
delegated to the real Part-1 naalp.approval / naalp.identity primitives, which carry their
own graded red-evidence in impl/python/tests against vectors/approval/cases.json and
vectors/identity/cases.json (out of scope for this task -- Part-1 files are not touched
here). What IS this task's own code, and therefore what these four mutations target, is the
guard's OWN choices: that execute() actually re-verifies before it ever resumes (M1), that
every check is judged against a REAL injected use-time clock rather than a fixed value (M2),
that it does not paper over the one error naalp.approval raises specifically to prevent
replay (M3), and that it actually re-checks key liveness -- the "revoked" property that has no
Part-1 analogue and is this guard's own addition composing naalp.identity.revoked_at (M4).
All four are exactly the properties this guard must get right -- not the
underlying SDK.