Skip to content

Keep the raw URI when adding a Range header - #2393

Open
pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/range-raw-uri-60cbe1
Open

pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/range-raw-uri-60cbe1

Conversation

@pavel-ptashyts

Copy link
Copy Markdown
Contributor

Summary

Request.toBuilder() always uses the fixing URI encoder. So a request built with URL encoding disabled is sent re-encoded as soon as a Range header is added to it, on either of the two paths that add one:

Request request = new RequestBuilder("GET", true).setUrl("http://localhost/a|b?q={x}").setRangeOffset(5).build();
// request line on main:   GET /a%7Cb?q=%7Bx%7D HTTP/1.1
// request line with this: GET /a|b?q={x} HTTP/1.1
  1. DefaultAsyncHttpClient.preProcessRequest, for a request with a range offset, ran toBuilder(), setHeader("Range", ...) and build().
  2. ResumableAsyncHandler.adjustRequestRange did the same, and rebuilt the request on every call, also when it set no header.

This is the same toBuilder() problem that #2384 fixed for stored cookies; these two paths were listed there under "Not changed here".

Changes:

  1. preProcessRequest sets the range through a new package-private withRangeHeader. A DefaultRequest is copied through a new package-private RequestBuilder(Request, boolean disableUrlEncoding, boolean validateHeaders) constructor with (request, true, false). That is what toBuilder() does (RequestBuilderBase(prototype, false, false)) with only the encoder changed: the prototype's URI is already encoded, and the raw encoder returns it unchanged. Everything else build() does (scheme check, charset, empty lists for a hand-built request) still happens.
  2. adjustRequestRange returns the request itself when it needs no Range header (it has one, or no bytes were transferred yet). Otherwise a DefaultRequest is copied the same way, by a private nested RequestBuilderBase subclass, since the new RequestBuilder constructor is not visible from handler.resumable.
  3. Subclasses of DefaultRequest and other Request implementations still go through toBuilder() on both paths, as they may override what it copies.

Unlike #2384 this does not share fields with the original request: the range-offset path is not a per-request path (it runs only for requests with a range offset), so keeping build() in it is simpler than reproducing what build() checks and fills in.

Behaviour notes

  • Same Range semantics as before: the resumable handler sets it only when the request has none, and the range offset then replaces any earlier value (all values, as setHeader did).
  • adjustRequestRange now returns the same instance when it adds nothing, so such a request goes on exactly as it does without a resumable handler. Its callers (preProcessRequest, ResumableIOExceptionFilter) do not depend on identity. One exception keeps the old normalisation: a DefaultRequest built through its public constructor with a null charset or a null cookie, form param or body part list is still rebuilt, so build() fills those in as before. Other build()-only effects, such as the scheme check, do not run on this no-op path, just as they do not without the handler.
  • No public API change: the new RequestBuilder constructor and withRangeHeader are package-private, the nested builder is private. Revapi passes.

Also found, not changed here

executeRequest calls preProcessRequest only when the client has at least one request filter (the noRequestFilters fast path, since d61fb56 for #706). Without a request filter, setRangeOffset is ignored and ResumableAsyncHandler adds no Range header to the first request: on main a request with setRangeOffset(5) on a client without filters reaches the server with no Range header. That is a separate bug for a separate change. The wire tests here add a pass-through request filter to reach the path.

Tests

DefaultAsyncHttpClientTest, over a loopback ServerSocket that records the request head:

  • a request built with URL encoding disabled and a range offset goes out with its URI as built, Range: bytes=5- and its other headers, and the caller's request has no Range header afterwards (fails on main with GET /a%7Cb?q=%7Bx%7D);
  • the same through a ResumableAsyncHandler with a stored offset, without and with a range offset (the offset replaces the handler's Range);
  • the range offset replaces two earlier Range values;
  • a subclass of DefaultRequest goes through its toBuilder().

The raw socket is used because these tests assert the request-target bytes as sent. The Jetty server in org.asynchttpclient.testserver exposes the decoded path (getPathInfo), which reads the same for /a|b and /a%7Cb. CrossHostReplayTest and other tests use the same pattern.

Calling withRangeHeader directly:

  • every field of the result equals what toBuilder() produces, checked reflectively;
  • the request's headers are not validated, as with toBuilder() (a DefaultRequest with non-validating ReadOnlyHttpHeaders holding an illegal value and an illegal name keeps both);
  • a hand-built DefaultRequest with null lists and charset gets them filled in, with its raw URI kept;
  • a hand-built DefaultRequest with an ftp URI is still rejected.

ResumableAsyncHandlerTest:

  • a request built with URL encoding disabled keeps its URL (fails on main with http://localhost/a%7Cb?q=%7Bx%7D);
  • without an offset, or with a Range header already set, the request is returned as is (both fail on main);
  • every field of the rebuilt request equals what toBuilder() produces;
  • the request's headers are not validated;
  • a hand-built DefaultRequest with any one of the four fields null is rebuilt and gets it filled in, and with all four null its raw URL is kept, with and without a Range header added;
  • a subclass of DefaultRequest goes through its toBuilder(), also when it is rebuilt only for a null charset;
  • an existing Range header is kept when a hand-built request is rebuilt;
  • a hand-built DefaultRequest with an ftp URI is still rejected when it is rebuilt.

./mvnw clean verify -Dgpg.skip=true passes on JDK 17 (2083 tests; no JDK 11 and no gpg on the host); CI covers JDK 11.

Second-agent review

Both reviewers ran from fresh CLI sessions, read-only, with the intent and constraints in the prompt.

Reviewer Agent and model Effort Commit reviewed Result
Guided (against AGENTS.md, on git diff upstream/main...HEAD) Codex gpt-6-astra medium 59b27b7 No findings
Independent (no checklist, reads the files) Claude Code claude-opus-5-5 medium 59b27b7 No findings

Nine rounds in all, the commit amended before the push after each round with findings. Fixed from earlier rounds:

  • the wire tests bound the server to InetAddress.getLoopbackAddress() but sent to 127.0.0.1, which fails where IPv6 is preferred; the server now binds to 127.0.0.1 (round 1, guided);
  • the accepted socket had no read timeout (round 2, guided);
  • an earlier version shared fields with the original request through a package-private DefaultRequest.withHeaders, as withCookies does. Rounds 1 to 5 found what that skipped from build(): the header copy validated headers that toBuilder() copied without validation (the base HttpHeaders.copy() builds a validating DefaultHttpHeaders), null lists and a null charset of a hand-built request were no longer filled in, and the scheme check no longer ran. The range-offset path now keeps build(), with only the encoder changed;
  • tests added: every field of the rebuilt request on both paths, no header validation on both paths, each null field on its own, the scheme check, and the resumable handler and range offset together over the wire (rounds 1 to 5, independent);
  • a raw 0x01 byte in a test string literal is now written as \u0001 (round 3, independent);
  • javadoc no longer says the URI "is already encoded", which a hand-built request need not be, and a test comment no longer claims an order of steps the test cannot see (round 7, independent);
  • tests added for an existing Range header kept when a hand-built request is rebuilt (round 7), and for a DefaultRequest subclass with a null charset rebuilt through its toBuilder() and the scheme check on the resumable rebuild path (round 8, independent).

Not changed:

  • The no-op path of adjustRequestRange skips build()-only effects such as the scheme check or deriving the charset from Content-Type for a hand-built request with non-null fields (rounds 4 and 5, independent, risk and nit). Not rebuilding there is the point of the change, and the request then goes on exactly as without a resumable handler.

Claude Code on behalf of Pavel Ptashyts

🤖 Generated with Claude Code

Request.toBuilder() always uses the fixing URI encoder, so a request
built with URL encoding disabled, such as http://localhost/a|b?q={x},
went out as /a%7Cb?q=%7Bx%7D once a Range header was added, either for
its range offset in DefaultAsyncHttpClient.preProcessRequest or by
ResumableAsyncHandler.adjustRequestRange. The latter also rebuilt the
request on every call, even when it set no header.

Both paths now copy a DefaultRequest as toBuilder() does, but with URL
encoding disabled: its URI is already encoded, so it is kept as built,
and build() still validates and fills in everything it did before.
adjustRequestRange returns the request as is when it needs no header,
unless it is a DefaultRequest from the public constructor with a null
charset or null lists, which is still rebuilt so build() fills them
in. Subclasses and other Request implementations still go through
toBuilder(). No public API change.

Claude Code on behalf of Pavel Ptashyts

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant