Red-evidence: N-AALP multi-agent interaction shapes¶
Mutation-survival evidence for three load-bearing properties described below. Each row
was produced by actually editing naalp_multiagent/pipeline.py or naalp_multiagent/
fanout.py (never ecosystem/naalp-react/* or ecosystem/naalp-plan/*, both out of scope
for this task, and never impl/python/naalp/*, which is Part-1), confirming the named test flips RED for the stated reason, then restoring the file
from a .bak copy (never git checkout), clearing __pycache__
(PYTHONDONTWRITEBYTECODE=1 set for every run so no stale bytecode can mask a mutation or
fake a revert), sha256-confirming the restored file matches the pre-mutation baseline, and
re-running the full suite GREEN.
Baseline / reverted file hashes (SHA-256), confirmed identical before mutation and after every revert below:
naalp_multiagent/pipeline.py d5c3b50d3ee8ca8c8fced4abb0f9b521224c8f577e5878f8019402035e6bddda
naalp_multiagent/fanout.py a58b4df9b7a113c5760824a07d77525b1064329ce4a24145d3a23411b1aff95a
Run command (from ecosystem/naalp-multiagent/, invoking the real installed Python
interpreter for this platform -- on Windows the Microsoft-Store python/python3
execution-alias stubs resolve ahead of a real install on PATH and hang, so on that platform
invoke the actual interpreter binary directly by its own install path rather than the bare
python command):
PYTHONDONTWRITEBYTECODE=1 python -m unittest -v tests.test_pipeline tests.test_fanout tests.test_reconcile
All commands below were run from ecosystem/naalp-multiagent/ with PYTHONDONTWRITEBYTECODE=1
set and __pycache__ removed immediately before every test invocation (mutate, revert, and
re-confirm), so no cached .pyc could report a result that did not come from the file on
disk at that moment. Baseline (pre-mutation) and every post-revert run: 14/14 tests GREEN
(test_pipeline 4, test_fanout 4, test_reconcile 6).
M1 -- causal-edge wiring dropped in the sequential pipeline¶
(test_three_stage_pipeline_runs_in_order_with_causal_chain)
Mutated: pipeline.py, SequentialPipeline.run() -- the line that sets a stage's causal
edge to its predecessor's signed request content id:
action = stage.build_action(prev_observation)
- causes = (prev_request_id,) if prev_request_id is not None else ()
+ causes = () # MUTATION M1 (red-evidence): drop causal-edge wiring to the prior stage
linked_action = replace(action, causes=causes)
RED, confirmed (1 of 4 test_pipeline tests flipped):
FAIL: test_three_stage_pipeline_runs_in_order_with_causal_chain
AssertionError: Lists differ: [] != [b' 0p\xb0uY\xaf\xa6\xa8F\xb7\x14\xd4\xa3]...']
With the wiring dropped, every stage's signed request carries an empty causes[] regardless
of which agent preceded it -- the exact failure this property exists to rule out ("agent A's signed
output feeds agent B feeds C", causally). The other three tests were unaffected:
test_single_stage_pipeline_has_no_prior_observation_and_empty_causes expects empty causes
anyway (no predecessor exists), test_stage_refusal_blocks_downstream_and_it_is_never_
signed_or_run does not assert on causal-edge VALUES, and test_downstream_build_action_
receives_prior_observation_body asserts on the Observation body, not on causes.
Reverted: the file was restored from its .bak copy (never git checkout),
__pycache__ cleared, hash re-confirmed
d5c3b50d3ee8ca8c8fced4abb0f9b521224c8f577e5878f8019402035e6bddda.
GREEN, confirmed: 4/4 test_pipeline tests pass (14/14 full suite).
M2 -- an excluded branch is included in the reconciled graph anyway¶
(test_branch_failing_verification_is_excluded_with_a_named_error)
Mutated: fanout.py, ParallelFanout.run() -- the else (non-"executed") branch of the
per-branch classification loop, which is supposed to record the branch as "excluded" and
never add its content id to graph_nodes:
branch_results[b.agent_id] = BranchResult(
agent_id=b.agent_id, status="excluded",
request=attempt.request, observation=attempt.observation,
error=error_name, detail=attempt.detail,
)
+ graph_nodes.append(CausalNode(id=attempt.request.id, causes=[shared_input_id])) # MUTATION M2 (red-evidence): include an excluded branch's node in the reconciled graph anyway
order = reconcile_nodes(graph_nodes)
RED, confirmed (1 of 4 test_fanout tests flipped):
FAIL: test_branch_failing_verification_is_excluded_with_a_named_error
AssertionError: b' 0\xaa.\x82\xa0\xd7\x02...' unexpectedly found in (... 5-element tuple ...)
With the exclusion skipped, the WrongAudience branch's own request content id -- despite
branch_results["agent-bad"].status == "excluded" still being correctly reported -- is
wrongly folded into the reconciled causal graph and appears in reconciled_order, exactly
the failure this property exists to rule out ("a branch failing verification is excluded ...
never included with a fabricated causal edge"). The other three test_fanout tests were
unaffected: test_fanout_branches_all_causally_link_to_the_shared_input and
test_reconcile_order_is_deterministic_across_many_runs have no excluded branch in their
scenario (the else arm is never reached), and test_duplicate_agent_id_is_a_named_error_
before_any_branch_runs fails before any branch ever runs.
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
a58b4df9b7a113c5760824a07d77525b1064329ce4a24145d3a23411b1aff95a.
GREEN, confirmed: 4/4 test_fanout tests pass (14/14 full suite).
M3 -- the reconcile step is bypassed; order follows thread-completion time¶
(test_reconcile_order_is_deterministic_across_many_runs)
Mutated: fanout.py, ParallelFanout.run() -- the line that hands the causal graph to
the real Part-1 reconcile primitive, replaced with an order built directly from attempts
(a dict populated inside the as_completed() loop, so its insertion order IS thread
completion order -- nondeterministic run to run under the branches' randomized jitter):
- order = reconcile_nodes(graph_nodes)
+ order = (shared_input_id,) + tuple( # MUTATION M3 (red-evidence): bypass the Part-1 reconcile primitive; order by thread-completion time instead of the deterministic linearization
+ a.request.id for a in attempts.values() if a.status == "executed"
+ )
return FanoutRun(shared_input_id=shared_input_id, branch_results=branch_results, reconciled_order=order)
RED, confirmed (1 of 4 test_fanout tests flipped):
FAIL: test_reconcile_order_is_deterministic_across_many_runs
AssertionError: Lists differ: [b' 0...', b' 0\x85\x15\xd3O9\xa3Ke...'] != [b' 0...', b' 0C\x07\xd6\x94C\x9c@Cj...']
The failure lands on the test's second, independent assertion -- that the order really IS
the bytewise-ascending content-id tie-break (computed by sorting the executed branches'
request ids independently of the fan-out) -- rather than always on the "N repeated runs
agree with each other" loop: since ThreadPoolExecutor scheduling for a small, fast branch
set can occasionally land in the same completion order across a handful of consecutive
invocations by chance, the "N runs agree with each other" check alone is not a fully
reliable RED signal for this specific mutation; the independently-computed sorted-order
comparison is what reliably catches it, and did. This is exactly the failure this property
exists to rule out: with the real Part-1 primitive bypassed, the reconciled order is a
function of scheduling timing, not of the causal graph's content ids -- so it would silently
vary between two runs of the identical parallel set whenever thread timing genuinely differs.
The other three test_fanout tests were unaffected (none of them compares two runs'
reconciled_order, and test_branch_failing_verification_is_excluded_with_a_named_error's
own assertNotIn still holds because the excluded branch is correctly absent from
attempts' "executed" subset regardless of ordering).
Reverted: restored from .bak, __pycache__ cleared, hash re-confirmed
a58b4df9b7a113c5760824a07d77525b1064329ce4a24145d3a23411b1aff95a.
GREEN, confirmed: 4/4 test_fanout tests pass (14/14 full suite).
A note on a real, witnessed concurrency defect this task's own design surfaced¶
Before any of the three mutations above, ParallelFanout's first real run under genuine
ThreadPoolExecutor concurrency produced corrupted ML-DSA signatures (BadSignature) on 3
of 5 branches. This was diagnosed, not assumed: an isolated repro with NO
naalp_multiagent code involved -- 8 threads, each looping naalp.cose.mldsa_sign +
naalp.cose.mldsa_verify with distinct seeds/messages, no lock -- reproduced 65 failed
verifications out of 160 calls. The cause is that the Part-1 Python reference SDK's ML-DSA
backend (dilithium_py) exposes ML_DSA_65/ML_DSA_87 as process-wide singleton objects
that are not thread-safe; naalp_plan.PlanOrchestrator never surfaces this because it signs
sequentially in one thread, and ParallelFanout is this ecosystem's first genuinely
concurrent caller. This is a Part-1 SDK defect out of this task's scope to fix (the task
scopes edits strictly to a new ecosystem/ package; impl/ is import-only). The scoped
mitigation is naalp_multiagent.CRYPTO_LOCK, a module-level (process-wide) lock that
serializes only the two calls into the shared backend (bridge.action_to_request /
bridge.response_to_observation), leaving branch dispatch and the injected executor I/O
genuinely concurrent; it is exported so a caller whose own executor/build_action also
calls naalp.envelope/naalp.cose locally (as this task's own test harness does, to
simulate a remote counterpart in-process) can serialize against the same lock. See
fanout.py's module docstring for the full writeup. This finding is real, load-bearing,
and belongs to Part-1, not to this package.