AVP

Added

• Host-validation hardening + wildcard opt-in. Binding hosts must name a real destination: empty, whitespace, bare/embedded *, malformed, and public-suffix wildcards (*.com, *.co.uk, *.github.io) are rejected at config-load. Wildcard hosts are now opt-in via the top-level allow_wildcard_hosts: true — off by default, since a wildcard widens a secret's blast radius to every subdomain.

• Full-feature docker-e2e harness. The tests/docker-e2e harness now covers all 12 features end-to-end on an isolated bridge (header/body/multi/composite injectors, scope-violation, fail-closed, deny, host-validation, forward_unmodified, SNI-mismatch, binding_source, and oauth2 with a hermetic mock token endpoint). Runs as the CI e2e-docker job.

• docs/ROADMAP.md. Forward-looking work, linked to the design ADRs (GCP backend, off-box audit shipping, TLS-passthrough scoping, metrics/healthz, artifact signing/SBOM, reserved injector types).

• OAuth2 RFC 6749 §5.1 + RFC 6750 §1.2 token_type validation (post-slice-9 hardening). The success-response parser now reads token_type and rejects anything other than case-insensitive "Bearer" with a new unsupported_token_type outcome on the token_exchange audit event. Without this, a token endpoint that returned a legacy MAC token, vendor DPoP, or "PoP" scheme would have been silently injected as Authorization: Bearer <mac-or-dpop-token> — either confusing the upstream with a 401 or, worse, leaking credentials in a malformed scheme. Strict on present-but-wrong; lenient (accept with warning log) on absent — being strict on absent would break real providers that omit the RFC-REQUIRED field, with no security gain since AVP only knows how to inject Bearer anyway. Three new tests pin the case-insensitive accept (Bearer/bearer/BEARER/BeArEr), the rejection set (MAC/DPoP/Basic/PoP), and the absent-token_type lenient path. Closes the deferred hardening item from the slice-9 blind-spot pass.

• Stateful no-leak property test for oauth2_refresh (tests/test_addon_oauth2_noleak_stateful.py). Mirrors the existing test_addon_noleak_stateful.py Hypothesis pattern but exercises the oauth2_refresh path — which has its own derived-token cache + write-back chain on top of the shared addon pipeline. A RuleBasedStateMachine interleaves request, rotate (stage an upstream rotation), reload (rebuild via configure_from_path and flush the derived-token cache), and scope changes. The independent oracle (hand-written, no policy import) catches the case where the matcher and the addon drift TOGETHER. Invariants: every secret value that EVER existed in the vault for this binding — initial refresh token, every rotated refresh token, every access token issued, the client secret — must be absent from the audit log across every interleaving; the vault always holds either the initial refresh token or one of the staged-rotation values (never malformed). 25 examples × 20 steps per example green.

• OAuth2 worked example + end-to-end test suite (ADR-0017 slice 9 + post-review hardening). bindings.example.yaml gains a worked GOOGLE_OAUTH entry using the google provider preset, with explicit operator-facing prose calling out the lockout footgun (read-only backend + auto-rotation = self-destruct on first request — avp doctor --probe-oauth flags it). tests/test_oauth2_refresh_e2e.py adds 7 end-to-end tests driving a real local http.server.HTTPServer through the full addon pipeline via the production configure_from_path codepath (NOT synthetic addon construction): happy-path with real backend + real urllib + real audit file; cache-hit (2 requests, 1 upstream call); refresh-token rotation persists to vault and audit; invalid_grant → 503 denial; forwarded-unmodified verbatim path; avp doctor --probe-oauth --exchange against the live server (rotation = WARN, no write-back); plus a companion regression test that the YAML fixture's backend: block actually constructs through build_backend without patches. The DNS-stub fixture passes through IP literals + loopback to the real resolver so the in-process mock server is reachable (the SSRF guard's socket.getaddrinfo monkey-patch globally affects http.client too; naive "everything is public" stub misroutes loopback). The TLS layer is deliberately out of scope (mitmproxy + urllib own it); the test docstring is explicit about plaintext-transport coverage. Post-Oracle / Silas adversarial-review hardenings applied inline: (1) post-call SSRF re-check on resp.geturl() — urllib follows 3xx by default, so a token endpoint that redirects to a private address (169.254.169.254, 10.x.x.x, ULA, …) would otherwise bypass the request-time guard; we now reject the response as ssrf_blocked when the final URL diverges from the configured one and resolves internal. The on-the-wire visit still happens (a fully-redirect-disabling fix requires test-suite refactoring around the urlopen patch boundary; tracked in the hardening patch series); (2) explicit User-Agent: agent-vault-proxy/oauth2-refresh — some providers block stock Python-urllib/3.X with confusing 4xx errors; identifying AVP also lets providers whitelist explicitly; (3) audit-field allowlist guard — the existing "secret bytes substring-absent" check is necessary but not sufficient (encoded variants — base64, hashed, unicode-escaped — would defeat substring search); a positive allowlist on event field names forces any new audit field through code review. Two new exchange tests cover the redirect rejection (vs. public-redirect acceptance) and the User-Agent header. Known pre-existing bug surfaced during this work: _ssrf_guard.check_url_not_internal's IP-literal short-circuit's except ValueError catches SsrfBlockedError (because the latter IS-A ValueError) — tracked separately since the production loop check at getaddrinfo time still blocks correctly under real DNS. Hardening patch series remaining: pending-audit-before-write (Oracle F2), BWS revision_date precondition (Oracle F3), per-binding write-back rate limit (Oracle F4), SsrfBlockedError exception inheritance refactor, fully-redirect-disabling implementation.

• avp doctor --probe-oauth CLI subcommand (ADR-0017 slice 8). Operator self-service: "is this OAuth2 binding wired correctly?" without driving a proxied request through an agent. Read-only by default. Per-binding checks: ssrf (re-runs the SSRF guard on token_url at probe time — config-load already passed, but DNS could rebind), input:<role> × 3 (the three vault inputs — client_id_secret, client_secret_secret, refresh_token_secret — fetched through the configured backend; empty value or fetch failure rolls up FAIL), writable (backend exposes update so slice-7 write-back is reachable — read-only backend with refresh_token_write_back: true is a FAIL since a rotation would lock the binding out; with refresh_token_write_back: false it is a WARN since the operator opted in to the lockout). Opt-in --exchange flag adds a fifth check that actually calls the upstream token endpoint — most providers will rotate the refresh token on a successful exchange, so a probe with --exchange is state-mutating against the upstream and the probe deliberately reports a rotation as WARN (the probe does NOT write-back to the vault; if rotation happens, the operator must manually update the vault secret with the new value). Status taxonomy: OK / WARN / FAIL / SKIP; FAIL alone rolls up to exit code 1. New CLI flags: --probe-oauth, --binding NAME (restrict to one binding), --exchange (live call), --config (the existing daemon config path). Default avp doctor (no --probe-oauth) MUST NOT load the config or build the backend — verified by test so a missing/mis-permissioned config can never break the CA-only flow. Implementation lives in cli/doctor_oauth.py (separation mirrors cli/env.py). Operator-visible messages report secret-name and byte-length only — never the secret value. 21 tests cover discovery, the unknown/wrong-type filter, every per-check happy and failure path, the rollup, SKIP-on-prior-input-failure, rotation-as-WARN on live exchange, and the two CLI integration paths (probe+OK = exit 0, probe+FAIL = exit 1, default = doesn't build backend).

• OAuth2 refresh-token shape guard (ADR-0017 slice 7 hardening, post-review). A compromised or MITM'd token endpoint that returns a junk refresh_token field (single byte, control chars, oversized blob) could otherwise drive the proxy into PUTing that value into BWS and permanently bricking the binding — the prior value is overwritten with no live backup, and an operator's only recovery is a manual rotation against the upstream. The new _is_well_formed_refresh_token guard in addon.py enforces RFC 6749 §A.17 vschar (printable ASCII 0x20-0x7E) and a generous length envelope (8 - 4096 bytes — covers every public provider's token format today). A rotated token failing the shape check skips the write-back entirely and emits a fifth outcome on the same refresh_token_rotated audit event: write_back_rejected_malformed (error_type: "malformed_refresh_token"). The current request still serves the upstream-issued access token; the next exchange will surface the upstream/vault drift via the standard token_endpoint_error:invalid_grant path. 6 parametrised malformed-input tests cover too-short, too-long, embedded NUL, tab, newline, and non-ASCII payloads — all reject without mutating the vault. Surfaced jointly by Oracle (F1, HIGH) and a separate offensive-security pass (vault-poisoning, MED). Two follow-up items remain deferred to a slice 7+ patch series: (1) emitting a refresh_token_rotated:pending audit BEFORE the backend PUT (closes the mid-write crash / audit-emit failure ordering gap — Oracle F2, Silas #6); (2) per-binding write-back rate limit to bound BWS PUT pressure under a forced-rotation DoS (Oracle F4, Silas #5); (3) BWS revision_date precondition on the PUT to detect operator-side manual rotation that would otherwise be silently clobbered (Oracle F3, Silas #3).

• OAuth2 refresh-token write-back (ADR-0017 slice 7). Closes the loop on refresh-token rotation. When the upstream issues a new refresh token alongside the access token (ExchangeResult.new_refresh_token is not None), the addon persists it back to the vault via the slice-3 backend.update Protocol method, then invalidates the cached read on the wrapping CachingSecretsClient so the next access-token expiry inside the existing ttl_seconds window doesn't reuse a value the upstream just invalidated (without that flush, the binding would silently lock out for up to ttl_seconds). New CachingSecretsClient.update_secret(name, value, ctx) is the write-through helper — delegates to the module-level update_secret dispatch (raises BackendNotWritableError for read-only adapters), then calls self.flush(name) on success so the generation counter bump catches any in-flight singleflight fetch. Four outcomes, each on a single refresh_token_rotated audit event: success (backend update returned), write_back_disabled (operator opted out via refresh_token_write_back: false), write_back_unavailable (backend is read-only — structural configuration error), write_back_failed (backend update raised — transient; the access token is STILL served on this request because killing it after we already hold a valid token would compound the failure). Audit event carries binding_name, refresh_token_secret (bindings.yaml reference name), outcome, and an error_type class-name field on failure outcomes — never the refresh-token VALUE (old or new), never the exception message (a backend could surface credential material in __str__). Audit ordering preserved end-to-end: token_exchange → refresh_token_rotated → inject_decision. Cache hits skip the rotation path entirely — the write-back already happened during the miss. The same ExchangeResult.new_refresh_token semantic from slice 5 (set only when upstream issued a DIFFERENT value, never on an echoed-back identical refresh token) keeps the path quiet for providers that don't rotate. 9 end-to-end tests pin per-outcome audit shape, ordering, vault-cache flush after success, best-effort serving on write-back failure, and the cache-hit silence invariant. avp doctor --probe-oauth (slice 8) will surface write_back_unavailable and write_back_disabled configurations distinctly.

• OAuth2 refresh-token addon dispatch (ADR-0017 slice 6). The mitmproxy addon now executes an oauth2_refresh allowed verdict end-to-end. policy.decide returns a populated oauth2_injector field on the Decision; the addon's _execute_header_decision branches on that field to _execute_oauth2_refresh_decision, which (1) fetches the three vault inputs (client_id_ref / client_secret_ref / refresh_token_ref) — vault failure routes to the existing secret_unavailable / secret_fetch_error audit shape, no token exchange ever happens; (2) builds a KeyInputs over the bindings, runs DerivedTokenCache.dedup_or_fetch so N concurrent agent requests for the same binding share a single upstream exchange; (3) on miss invokes the slice-5 exchange() and emits a new token_exchange audit event (outcome taxonomy from slice 5, plus binding_name, token_url_host, cache_ttl_effective_seconds, used_default_expiry for the under-spec providers; deliberately carries zero secret material), then the existing inject_decision audit; (4) on miss + failure routes a 503 + token_exchange_failed:<outcome> denial through the same inject_decision path so downstream audit consumers don't need a new event-type branch to count token-related denials. The token_exchange audit fires BEFORE the proxied request bytes leave AVP — the G6 audit-before-action invariant carries through to the new event. Cache hits skip both exchange() and the token_exchange audit (the event signifies an upstream call happened; a cached hit is a no-op event-wise). configure_from_path rebuilds the cache on every reload — a stale refresh token must not survive a config rotation. 8 end-to-end tests pin cache miss success + audit ordering, audit-shape metadata (no secrets), cache hit silence, invalid_grant denial path, network-failure denial path, runtime SSRF rebind (the slice-5 re-check fires before urlopen), vault input failure (never reaches exchange), and reload-resets-cache. Write-back path, avp doctor --probe-oauth, and the worked example land in slices 7-9.

• OAuth2 token exchange injectors/oauth2_refresh.py (ADR-0017 slice 5). The synchronous core of the resolution step. Re-runs the SSRF guard on spec.token_url (DNS rebinding defense — a name that resolved public at config-load can resolve private later), builds an RFC 6749 §6 refresh-token grant POST (form-encoded body, client_auth_method: body_post puts credentials in body, basic puts them in the Authorization header), issues one POST via stdlib urllib.request.urlopen with HTTPS enforced by the schema validator, parses the response per §5.1 / §5.2. Outcome taxonomy: success, token_endpoint_error:<code> (any RFC 6749 §5.2 error code surfaced verbatim — invalid_grant, invalid_client, invalid_scope, vendor extensions), token_endpoint_status:<status> (non-JSON 4xx/5xx body), token_endpoint_unreachable (network errors + TimeoutError), response_parse_failed, ssrf_blocked. One retry on 5xx + network errors with a 1 s backoff; no retry on 4xx. expires_in is honoured when present, capped by cache_ttl_max_seconds after subtracting cache_ttl_safety_seconds; when absent, the cache_ttl_max is used and used_default_expiry=True is surfaced so the audit layer can flag under-spec providers. Refresh-token rotation detected via not-equal check (echoed-back identical refresh tokens are NOT a rotation — write-back must skip in that case). exchange_async dispatches the sync core through loop.run_in_executor on a dedicated 8-thread pool so a slow token endpoint never blocks the mitmproxy asyncio loop. 18 tests pin success/rotation/error/retry/SSRF/off-thread behaviour. The addon wire-up and audit emission land in slice 6.

• SSRF guard _ssrf_guard.py (ADR-0017 slice 2). Module exposing check_url_not_internal(url) that resolves the host (or accepts an IP literal directly) and refuses any address in loopback, RFC 1918 private, link-local (incl. AWS IMDS at 169.254.169.254), CGNAT 100.64/10, IPv6 ULA fc00::/7 (incl. IMDSv6 at fd00:ec2::254), unspecified, multicast, or reserved ranges. Fail-closed on DNS resolution failure and on mixed-result resolutions (a hostname returning any private record is blocked — DNS-rebinding defense). Wired into Oauth2RefreshInjector at config-load on every operator-supplied token_url (both the fully-explicit form and the tenant-specific preset form). 27 tests pin per-range behaviour and the config-load integration; the runtime DNS re-check before each token exchange lands in slice 5.

• SecretsBackend.update(name, value, ctx) Protocol extension + update_secret dispatch helper (ADR-0017 slice 3). Optional method (same opt-in shape as fetch_with_meta) — backends that implement it support write-back, backends that don't raise the new BackendNotWritableError (subclass of BackendUnavailableError so the existing fail-closed catch-all in the addon already handles it). BitwardenBackend.update issues a GET-then-PUT preserving note and project_ids so binding metadata isn't blanked on refresh-token rotation. StaticSecretsBackend is intentionally read-only — confirmed by test. The runtime write-back call site lands in slice 7. 7 tests.

• _DerivedTokenCache (ADR-0017 slice 4). Sibling of CachingSecretsClient — vault secrets and derived OAuth access tokens have incompatible invalidation, expiry, and audit-attribution semantics so they live in separate clients. Cache key is HMAC-SHA256 (per-instance salt) over (binding_name, token_url, scopes, client_id_value, refresh_token_value) so any rotation, scope edit, or config reload produces a different key — a stale token can't be served after the inputs that minted it have changed. Per-entry TTL from the caller-supplied expires_at. Inflight Future-per-key dedup makes N concurrent agent requests for the same binding trigger exactly one upstream exchange. Lazy expiry on read + proactive sweep on put keep the dict bounded across rotations. Best-effort zero-on-flush; the G10 invariants (mlock, RLIMIT_CORE=0, no swap, UNIX-user isolation) carry the at-rest defence. 11 tests including a concurrency pin on the inflight dedup.

• inject.type: oauth2_refresh (schema only, slice 1 of ADR-0017). The OAuth2 refresh-token grant injector (RFC 6749 §6) — schema, discriminated-union dispatch, bundled provider preset catalogue (google, microsoft, auth0, slack, atlassian, okta), HTTPS-only token_url validator, preset-or-explicit XOR, three-required-secret refs, defaults pinned (Authorization: Bearer {access_token}, cache_ttl_safety_seconds=60, cache_ttl_max_seconds=3600, refresh_token_write_back=True). The prior planned: P1 config-load error for this type: no longer fires. Runtime resolution (derived-token cache, off-thread token exchange, audit emission, write-back, doctor probe) lands in subsequent slices; existing parity meta-test (test_canonical_schemas_cover_implemented_injector_types) gains two new canonical schemas (preset + explicit) so future drift surfaces immediately. ADR full text at docs/adrs/ADR-0017-oauth2-refresh-injector.md.

• docs/adrs/ directory with a README.md documenting the in-repo ADR convention (MADR-lite frontmatter; one filename per git log docs/adrs/-able decision). ADR-0017 is the first entry. Pre-ADR-0017 ADRs (0011 BWS-notes, 0012 narrow-trust CA, 0013 declarative policy fixtures) get backfilled here as the affected code paths next get touched.

• Composite body bindings. inject.type: body now accepts inject.template (Jinja2-sandboxed) alongside compose:, mirroring the header composite path. Same SecretSpec.compiled_template machinery; the placeholder in the request body is replaced with the rendered template output rather than a single secret value. Streaming-mode invariants (constant memory, chunk-boundary correctness, fail-closed on backend failure) are preserved — the composite resolver hooks into the _BodyReplacer's lazy per-target fetch and reuses the addon's existing _fetch_and_render_composite (composite_unavailable / composite_fetch_error / render_failed audit shapes carry across, with the compose: list on failure). The P0.6 BodyInjector validator's deferral message ("composite/Jinja is not yet supported") is removed; bindings that previously failed config-load with that message now compile and render.

• Template helpers hmac_sha1(key, msg) and totp(secret_b32). hmac_sha1 mirrors the existing hmac_sha256 / hmac_sha512 shape (UTF-8 in, lowercase hex out). totp implements RFC 6238 with the SHA-1 default, 30-second window, 6-digit output, dynamic truncation per RFC 4226 §5.3; base32 input is normalised (case-insensitive, whitespace and = padding tolerated). TOTP is the single non-deterministic helper in the sandbox — it depends on time.time() — and is documented as such in the helper's docstring. RFC 6238 §5.2 reference vectors are pinned in tests/test_template.py; the RFC 2202 §3 case-1 vector pins hmac_sha1. The motivating case is APIs that require a live 2FA code in the request — e.g. Kraken's private endpoints take the current TOTP as the otp body field — computed against a secret AVP holds, not the agent.

Upgrade notes

• Reserved helper names. Adding hmac_sha1 and totp to the sandbox extends the set of names a composite template's compose: entries must not collide with. Bindings whose compose: list contains a literal entry named hmac_sha1 or totp (case-sensitive) will now fail at config-load with allowed_vars entry … collides with reserved function …; rename your compose entry. The same rule has always applied to b64encode / b64decode / sha256 / urlencode / hmac_sha256 / hmac_sha512; rename the offending compose entry (e.g. HMAC_SHA1 → HMAC_SHA1_VALUE) and the binding will load again. AVP daemon refuses to start on a broken config, so this is an immediately-visible failure on upgrade — not a silent regression.
• TOTP wall-clock dependence. Bindings using totp(...) produce a different output every 30 seconds and depend on the host clock being in sync with the upstream's TOTP authority. Verify chronyc tracking is healthy on the AVP host before relying on a totp-using binding in production; a stale or drifted host clock will produce codes the upstream silently refuses.
• BWS-notes bindings (ADR-0011): bindings resolve from each secret's notes field (host: required; optional header / format / methods / paths, defaulting to Authorization: Bearer). Bundled host-keyed exception table for known providers ships tight defaults (GitHub read-only — no POST /gists). Fail-closed, with distinct audit reasons no_binding_in_notes vs invalid_binding_metadata.
• avp operator CLI: avp env projects salted placeholders to ~/.config/avp/env (sourced, never eval'd); avp doctor checks the CA is not in any OS trust store and the CA key is 0600 (ADR-0012).
• Deterministic salted placeholders (avp-PLACEHOLDER-…, HMAC keyed by a per-install salt) and binding_source: file | bws_notes | both.
• avp setup cross-platform installer (systemd/launchd, isolated service user, append-only audit log); --no-service provisions without activating. tests/setup-e2e/ bats suite + e2e-setup CI job assert the installed state in a disposable container.

Changed

• Default binding_source: both — BWS-notes resolve alongside bindings.yaml, notes winning per secret. file mode is byte-identical to prior behaviour.
• Audit contract v2: adds binding_source; the audit log still records no header values, bodies, or query strings.