Skip to content

feat: bytes_sent on upstream_reset audit line, with H2 upstream tests - #16

Merged
chaodu-agent merged 2 commits into
mainfrom
feat/upstream-reset-bytes-sent
Nov 17, 2026
Merged

chaodu-agent merged 2 commits into
mainfrom
feat/upstream-reset-bytes-sent

Conversation

@chaodu-agent

Copy link
Copy Markdown
Contributor

Port of finding B from the feat/phase1-k8s investigation: design intent (docs/design/phase1.md, "Upstream error mid-stream") is that the audit line for a mid-stream upstream reset records how many response bytes were already forwarded.

Changes

  • ap-core/audit.rs: new bytes_sent: Option<u64>, skip_serializing_if = None. Present only on the upstream_reset line, omitted on every other line. No new secret-bearing field.
  • ap-proxy/pipeline.rs: StreamAudit counts response body bytes forwarded; on reset the Drop writes bytes_sent.
  • test_support.rs: tls_server_config(alpn) extracted from start_upstream (unchanged behaviour, still http/1.1) plus Upstream::from_parts, so a test can run its own server.
  • New pipeline/h2_tests.rs, using a real h2-only TLS server (ALPN h2) under the throwaway CA:
    • h2 relay works (upstream saw HTTP/2.0), one allow line, no bytes_sent.
    • mid-stream RST_STREAM: client body errors (no synthetic success), exactly one upstream_reset line, bytes_sent == 8 (partial-), no key, surrogate, credential or /ap/ in the audit text.
    • reset before any response byte: 502, one upstream_unavailable line, no bytes_sent, no upstream_reset.
  • Dev-dependency: h2 only (bytes, rustls, tokio-rustls were already present). No policy config change; no connect_to or test-resolve from the old branch.

Evidence (macmini, cargo 1.100)

  • cargo fmt --all --check: ok
  • cargo clippy --workspace --all-targets -- -D warnings: ok
  • cargo test --workspace: all pass (3 new h2 tests, 1 new audit test)
  • Release build has no test hooks; ap-egress does not depend on ap-broker
  • python3 tests/matrix/matrix.py: 41 PASS of 41 rows (count unchanged; audit field-shape rows pass)
  • Not verified here: cargo test -p ap-proxy --features matrix-upstream --test matrix fails on this host only because cargo 1.100 places the test exe outside deps/, so bin_dir() in tests/matrix.rs misses the binaries. Not caused by this PR; CI on ubuntu is the check.

…ream tests

Tests (macmini, cargo 1.100): cargo fmt --all --check ok; clippy --workspace --all-targets -D warnings ok;
cargo test --workspace all pass (incl. 3 new pipeline::h2_tests and audit bytes_sent test);
release build has no test hooks; ap-egress not dependent on ap-broker; tests/matrix/matrix.py 41/41 PASS.
cargo matrix target fails on this host's cargo layout (test exe not under deps/), unrelated to this change.
…, saturate the counter

Checked on macmini: fmt, clippy -D warnings, cargo test --workspace, matrix.py 41/41,
h2_tests x100 loop 0 failures, release binaries carry no AP_TEST_ hooks.
The cargo matrix test cannot find binaries on that host (pre-existing path lookup).
@chaodu-agent

Copy link
Copy Markdown
Contributor Author

Review follow-up, fixed in d425c6c (additive commit):

  • MEDIUM, timing sleep in the mid-stream reset test: the 150 ms sleep is gone. The h2 server sends headers and the body frames, then waits on a oneshot gate. The test reads and asserts all 8 body bytes, releases the gate, and only then polls for the reset. No elapsed-time assumption remains.
  • LOW, no test for a reset after headers with no body: new test a_reset_after_the_head_with_no_body_audits_zero_bytes asserts upstream_reset, status 200, one audit line, and bytes_sent == 0 (present, not omitted).
  • LOW, accumulation across frames not proven: /reset-mid now sends par and tial- as two DATA frames; the test asserts the exact sum of 8.
  • LOW, audit field list in docs: docs/design/phase1.md now lists optional bytes_sent and says it is omitted from every record except upstream_reset.
  • LOW, unchecked addition: the counter uses u64::try_from(len).unwrap_or(u64::MAX) and saturating_add.

Checked on macmini: fmt, clippy -D warnings, cargo test --workspace, matrix driver 41/41, the h2 tests 100 times with 0 failures, release binaries carry no AP_TEST_ hook strings, ap-egress has no ap-broker dependency. The cargo --test matrix target cannot locate binaries on that host (existing path lookup issue, unchanged by this PR); the Python driver covers the same 41 rows.

@chaodu-agent
chaodu-agent merged commit cf97f38 into main Nov 17, 2026
1 of 2 checks passed
chaodu-agent added a commit that referenced this pull request Nov 17, 2026
- passthrough audit record gets bytes_sent: None (field added by #16)
- drop dead __ap_probe reserved-name check; mark caller key header values sensitive
- evidence digest comment says what the digest covers
- PT-12: obs-fold row, GET CL+TE and bare-LF measured as known gap
- design 3.1/3.5/acc 12 and runbook: hyper behaviour, loopback + front reliance, stage 2 raw-head scanner

Checked on macmini: fmt, clippy (default and stage1-probe,matrix-upstream) -D warnings, cargo test --workspace, release hook greps clean, sidecar matrix 41/41, passthrough matrix 13/13, matrix_passthrough cargo target ok. cargo matrix target fails on this host only (new cargo test-exe layout; bin_dir lookup), driver run directly instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant