Red-evidence: N-AALP HITL interceptor¶
Mutation-survival evidence for the three load-bearing tests described below. Each row
was produced by actually editing naalp_hitl/interceptor.py (never impl/python/naalp/*,
which is Part-1 and out of scope for this task), confirming the named test flips RED for the
stated reason, then restoring the file from a .bak copy (never git checkout), clearing
__pycache__ (PYTHONDONTWRITEBYTECODE=1 set for every run so no stale bytecode can mask a
mutation or fake a revert), and re-running the full suite GREEN.
Baseline / reverted file hash for naalp_hitl/interceptor.py (SHA-256), confirmed identical
before mutation 1 and after every one of the three reverts below:
Run command (from ecosystem/naalp-hitl/, invoking the real installed Python interpreter for
this platform -- on Windows the Microsoft-Store python/python3 execution-alias stubs
resolve ahead of a real install on PATH and hang, so on that platform invoke the actual
interpreter binary directly rather than the bare python command):
All commands below were run from ecosystem/naalp-hitl/ with PYTHONDONTWRITEBYTECODE=1 set
and __pycache__ removed immediately before every test invocation (mutate, revert, and
re-confirm), so no cached .pyc could report a result that did not come from the file on disk
at that moment.
M1 -- valid-resume (test_approved_action_pauses_then_resumes)¶
Mutated: HITLInterceptor.intercept(), the line reached only after a successful
verify_and_consume() (the RESUME step):
- return action.execute()
+ return None # MUTATION M1 (red-evidence): never actually resume the action
RED, confirmed:
result was None instead of "TRANSFERRED" -- the exact assertion the mutation defeats.
The other 13 tests were unaffected (this path is reached only by a real granted approval).
Reverted: the file was restored from its .bak copy (never git checkout), __pycache__
cleared, hash re-confirmed 9f5a90ac9703c60ce4f721aa79053d4fe4fe91f82bc2bbf4073c0704a17ae25a.
GREEN, confirmed: 14/14 tests pass.
M2 -- expired-fail-closed (test_expired_fails_closed_through_intercept)¶
Mutated: HITLInterceptor.verify_and_consume(), the clock-injection line (the interceptor's
own now/pos_time plumbing -- the property the task requires be enforced via injection,
never a hardcoded value):
- pos_time = self._clock() if pos_time is None else pos_time
+ pos_time = 0 # MUTATION M2 (red-evidence): ignore the injected clock entirely (never expires)
RED, confirmed:
FAIL: test_expired_fails_closed_through_intercept
AssertionError: ApprovalError not raised
FAIL: test_expired_approval_fails_closed
AssertionError: ApprovalError not raised
With the clock forced to epoch 0, pos_time > a.not_after is never true for any real
not_after, so an approval that is already expired is silently treated as still valid -- a
genuine fail-open regression, not a crash. Exactly the two expiry-dependent tests flipped
(12 of 14 still passed); every other check (signature, audience, effect, single-use) is
independent of the clock and was unaffected.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
9f5a90ac9703c60ce4f721aa79053d4fe4fe91f82bc2bbf4073c0704a17ae25a.
GREEN, confirmed: 14/14 tests pass.
M3 -- single-use-fail-closed (test_reuse_of_same_approval_fails_closed_through_intercept)¶
Mutated: HITLInterceptor.verify_and_consume(), wrapped the ledger consume call to swallow
exactly the single-use error and pretend it succeeded:
- return approval.consume_approval(
- record, self._approver_alg, self._approver_pubkey, sig, args_content_id,
- pos_time, required_effect, self._ledger, self._identity,
- )
+ try:
+ return approval.consume_approval(
+ record, self._approver_alg, self._approver_pubkey, sig, args_content_id,
+ pos_time, required_effect, self._ledger, self._identity,
+ )
+ except approval.ApprovalError as e:
+ if e.kind == "AlreadyConsumed":
+ # MUTATION M3 (red-evidence): silently swallow a replay as if it succeeded.
+ return None
+ raise
RED, confirmed:
FAIL: test_reuse_of_same_approval_fails_closed_through_intercept
AssertionError: ApprovalError not raised
FAIL: test_reuse_of_same_approval_fails_closed
AssertionError: ApprovalError not raised
With AlreadyConsumed swallowed, verify_and_consume() returns normally on the second use of
the SAME approval object, so intercept() falls through to return action.execute() a second
time -- the replayed approval would have executed the action again (the exact single-use
violation the interceptor exists to prevent). Exactly the two reuse-dependent tests flipped
(12 of 14 still passed).
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
9f5a90ac9703c60ce4f721aa79053d4fe4fe91f82bc2bbf4073c0704a17ae25a.
GREEN, confirmed: 14/14 tests pass.
The AU-2/AU-3 structured refusal log + AU-10 non-repudiation signed record¶
Mutation-survival evidence for the seven mutations recorded below. Each was produced by
actually editing naalp_hitl/interceptor.py or naalp_hitl/nonrepudiation.py (never
impl/python/naalp/*), confirming the named test(s) flip RED for the stated reason, then
restoring the file from a .bak copy (never git checkout, since this tree is uncommitted),
clearing __pycache__ (PYTHONDONTWRITEBYTECODE=1 set for every run), and re-running the full
suite (21/21) GREEN.
Baseline / reverted file hashes (SHA-256), confirmed identical before mutation M4 and after every one of the seven reverts below:
naalp_hitl/interceptor.py: 647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1
naalp_hitl/nonrepudiation.py: 5101bcc360a7a13d13137d983462c0ff9fadb03a5f1045e778002498b6e13f09
naalp_hitl/refusal_log.py: 1cc50d3dfe3d7ac3b63d0593721cc71978a120c4ae045000e4741f83cd1e12d8 (never mutated; hashed for completeness)
naalp_hitl/__init__.py: d24a8193a7d010c79206ace97dccee4d52043cb4fdb740a7256f76dfdfbd52cd (never mutated; hashed for completeness)
Run command (from ecosystem/naalp-hitl/, invoking the real installed Python interpreter for
this platform, not the Microsoft-Store python/python3 stubs):
M4 -- Layer-1 persistence (test_layer1_persists_every_au3_field_on_bad_signature)¶
Mutated: HITLInterceptor._on_refusal() in interceptor.py, the line that actually
persists the AU-3 entry:
- self._refusal_log.record(entry) # persist-before-propagate; a fault here propagates, not err
+ pass # MUTATION M4 (red-evidence): never actually persist the AU-3 log entry
RED, confirmed: test_layer1_persists_every_au3_field_on_bad_signature failed
(AssertionError: 0 != 1), together with test_layer2_absent_below_threshold_even_with_signing_key_configured,
test_denied_approval_fails_closed_and_never_executes, and 2 errors from
test_logging_failure_propagates_and_still_blocks_execution /
test_layer1_persists_on_direct_verify_and_consume_with_default_kind_and_principal (5 total).
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1. GREEN, confirmed: 21/21.
M5 -- non-repudiation predicate fail-open (test_layer2_absent_below_threshold_even_with_signing_key_configured)¶
Mutated: _on_refusal()'s Layer-2 gate: dropped the threshold check, so ANY refusal signs
whenever a key is configured, regardless of effect:
- if self._signing_alg is not None and self._signing_seed is not None \
- and self.requires_non_repudiation(required_effect):
+ if self._signing_alg is not None and self._signing_seed is not None:
+ # MUTATION M5: ignore the non-repudiation threshold entirely
RED, confirmed:
FAIL: test_layer2_absent_below_threshold_even_with_signing_key_configured
AssertionError: RefusalDecisionRecord(...) is not None : below threshold: NEVER mint a signed record, key or no key
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1. GREEN, confirmed: 21/21.
M6 -- non-repudiation predicate fail-closed-forever (test_layer2_mints_a_real_verifiable_signed_record_at_threshold)¶
Mutated: HITLInterceptor.requires_non_repudiation() forced to always return False:
- return effect >= self._non_repudiation_threshold
+ return False # MUTATION M6: never eligible, even at/above threshold
RED, confirmed:
FAIL: test_layer2_mints_a_real_verifiable_signed_record_at_threshold
AssertionError: unexpectedly None
ERROR: test_outcome_mapping_for_every_refusal_kind (AttributeError: 'NoneType' object has no attribute 'outcome')
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1. GREEN, confirmed: 21/21.
M7 -- fabricated signature (test_layer2_mints_a_real_verifiable_signed_record_at_threshold)¶
Mutated: sign_refusal_record() in nonrepudiation.py returns a fixed, bogus value instead
of a real signature:
- return cose.mldsa_sign(alg, seed, rec.bytes())
+ return b"\x00" * 64 # MUTATION M7: fabricate a bogus, unverifiable signature
RED, confirmed:
FAIL: test_layer2_mints_a_real_verifiable_signed_record_at_threshold
AssertionError: False is not true
verify_refusal_record(record, alg, interceptor_pubkey, sig) on the fabricated bytes correctly
failed to verify -- the exact real-cryptography assertion this task requires. 20 of 21 passed.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
5101bcc360a7a13d13137d983462c0ff9fadb03a5f1045e778002498b6e13f09. GREEN, confirmed: 21/21.
M8 -- wrong outcome mapping (test_outcome_mapping_for_every_refusal_kind)¶
Mutated: refusal_outcome_for_reason() in nonrepudiation.py collapses the whole closed
outcome set to REFUSAL_DENIED regardless of the reason kind:
- if kind in _HELD_KINDS:
- return approval.REFUSAL_HELD
- if kind in _UNVERIFIABLE_KINDS:
- return approval.REFUSAL_UNVERIFIABLE
- return approval.REFUSAL_DENIED
+ return approval.REFUSAL_DENIED # MUTATION M8: collapse the whole closed set to DENIED
RED, confirmed:
FAIL: test_outcome_mapping_for_every_refusal_kind
AssertionError: 0 != 1 (ApprovalRequired mapped to DENIED instead of HELD)
FAIL: test_layer2_mints_a_real_verifiable_signed_record_at_threshold
AssertionError: 0 != 2 (BadSignature mapped to DENIED instead of UNVERIFIABLE)
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
5101bcc360a7a13d13137d983462c0ff9fadb03a5f1045e778002498b6e13f09. GREEN, confirmed: 21/21.
M9 -- wrong record-id reference (test_layer2_mints_a_real_verifiable_signed_record_at_threshold)¶
Mutated: _on_refusal() in interceptor.py derives the coarse Refusal from the WRONG
bytes (empty) instead of the actual signed record:
- refusal = approval.refusal_from_record(outcome_code, record.bytes())
+ refusal = approval.refusal_from_record(outcome_code, b"")
+ # MUTATION M9: reference the wrong bytes instead of the real signed record
RED, confirmed:
FAIL: test_layer2_mints_a_real_verifiable_signed_record_at_threshold
AssertionError: b' 08\xb0...' != b' 0\xac\xdd...' (refusal.record != record.content_id())
Refusal no longer named the record that was actually signed -- exactly the
closure property (refusal.record == record.content_id()) this task requires. 20 of
21 still passed.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1. GREEN, confirmed: 21/21.
M10 -- the audit log bypassed on the intercept() human-decline path (test_denied_approval_fails_closed_and_never_executes)¶
Mutated: HITLInterceptor.intercept()'s decision is None branch stopped routing through
_on_refusal() entirely (raises the bare HITLError directly, as before this feature existed):
- err = HITLError(
- "ApprovalDenied", "the human operator declined to approve %r" % (action.kind,)
- )
- raise self._on_refusal(
- err, kind=action.kind, args_content_id=args_id, required_effect=action.effect,
- approver_identity="", source_principal=action.principal,
- )
+ # MUTATION M10: bypass the audit log entirely on the human-decline path.
+ raise HITLError(
+ "ApprovalDenied", "the human operator declined to approve %r" % (action.kind,)
+ )
RED, confirmed:
FAIL: test_denied_approval_fails_closed_and_never_executes
AssertionError: 0 != 1 (no Layer-1 entry was persisted for a human decline)
ERROR: test_outcome_mapping_for_every_refusal_kind (AttributeError: 'HITLError' object has no attribute 'd6')
intercept()'s OWN wiring into _on_refusal for the one
refusal reason (ApprovalDenied) that never passes through verify_and_consume at all --
distinct from M4, which targets _on_refusal's internals reached from either caller.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
647f3ca848e109186916262ef4fc7ed24aa524ad71090318fe57ca9516284fd1. GREEN, confirmed: 21/21.
Why these seven, and why the mutation targets stay inside naalp_hitl/¶
Exactly as for M1-M3 above: every piece of cryptography (naalp.cose.mldsa_sign /
cose_verify1_raw), the closed outcome vocabulary and its Refusal /
refusal_from_record / parse_refusal machinery, and canonical CBOR encoding are the real,
already-graded Part-1 naalp.approval / naalp.cose / naalp.cbor primitives -- untouched by
this task. What this feature adds, and therefore what M4-M10 target, is entirely naalp_hitl's own new
code: that it actually calls the log sink on every refusal path including the one
(ApprovalDenied) that never reaches verify_and_consume (M4, M10); that the non-repudiation
threshold gate is neither always-on nor always-off (M5, M6); that the signature _on_refusal
attaches is the interceptor's own REAL signature over the REAL record bytes, not a fabricated
placeholder (M7); that the reason->outcome mapping actually discriminates the three outcomes
rather than collapsing them (M8); and that the coarse Refusal references the record that was
actually signed (M9).
Why these three, and why the mutation targets interceptor.py rather than naalp.approval¶
naalp_hitl is glue: every cryptographic and ledger check (signature, args-content-id binding,
expiry, effect-ceiling, atomic single-use consume) is delegated to the real Part-1
naalp.approval primitive, which carries its own graded red-evidence in impl/python/tests
against vectors/approval/cases.json (out of scope for this task -- Part-1 files are not
touched here). What IS this task's own code, and therefore what these three mutations target,
is the interceptor's OWN choices: that it actually calls action.execute() only after a real
consume (M1), that it feeds the checks a REAL injected clock rather than a fixed value (M2),
and that it does not paper over the one error naalp.approval raises specifically to prevent
replay (M3). All three are exactly the properties this component depends on the interceptor --
not the underlying SDK -- to get right.