Repository navigation
Upgrade Netty to 4.2.21.Final - #2395
Merged
Merged
Conversation
Netty 4.2.21 reverts a build mistake in 4.2.20: its native libraries for x86_64 and aarch64 required glibc 2.33. It also fixes ZstdDecoder and ZstdDecompressor at a non-positive io.netty.compression.defaultMaxForwardBytes (netty/netty#18289), so AHC's workarounds for that bug go: - the ZstdDecoder that HttpContentDecompressor creates is no longer replaced by one with buffers of MAX_PART_SIZE at such a value (AsyncHttpClient#2375); - zstd is no longer left to HttpContentDecompressor at such a value (AsyncHttpClient#2362). It goes through PullDecompression like br and snappy, so it waits for demand, and a truncated stream fails, at any value of the property. Claude Code on behalf of Matthias Kurz Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5 tasks done
hyperxpro
requested changes
Nov 20, 2026
Without the workarounds, zstd responses fail with Netty 4.2.17 to 4.2.20 if the system property io.netty.compression.defaultMaxForwardBytes is negative. At 0 they never end and keep the event loop busy, as Netty's ZstdDecompressor then hands out empty buffers while it reports more output. Say in the README and in ResponseBodyControl.suspend() that a zero or negative value needs Netty 4.2.21 or later. Claude Code on behalf of Matthias Kurz Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
pushed a commit
that referenced
this pull request
Nov 21, 2026
## Summary - Since #2362, AHC decompresses HTTP/1.1 `br`, `zstd` and `snappy` bodies with Netty's pull-based `Decompressor` API, so that they wait for demand. Netty added that API in 4.2.17. With an older Netty, `Http1ContentDecompressor` caught the `LinkageError` and left these codings to `HttpContentDecompressor`, which decompresses them as soon as they are read. - AHC declares a Netty that has the API (4.2.20 on `main`, 4.2.21 with #2395), so this PR drops that fallback. The README now says that AHC needs Netty 4.2.17 or later. ## Change - **`Http1ContentDecompressor`:** the `pullAvailable` flag and the `LinkageError` catch go. `newPullDecompression` now only creates the `PullDecompression`. - **Unchanged:** - `PullDecompression.create` still checks whether Brotli and zstd are available, since brotli4j and zstd-jni are optional dependencies. A body whose library is missing is still left undecoded, as `HttpContentDecompressor` does. - The size limit and the split into parts of at most 64 KiB are unchanged. - **Test:** `gzipAndDeflateDoNotNeedNettysDecompressorApi` goes, together with the class loader that hid the API from it, as an older Netty would. - **Docs:** the README, the Javadoc of `ResponseBodyControl.suspend()` and the class Javadoc of `Http1ContentDecompressor` and `PullDecompression` no longer describe the fallback. The README now says that AHC needs Netty 4.2.17 or later, as it uses the API Netty added there, and that on-demand decompression applies over HTTP/1.1, and its paragraph on truncated bodies no longer names Brotli and Snappy bodies decompressed as soon as they are read. `PullDecompression` still notes that Netty marks the API `@UnstableApi`, so a Netty upgrade may require changes there. ## Notes - With AHC's declared Netty dependency, the fallback for older Netty versions is unnecessary. With Netty older than 4.2.17, br, zstd and snappy responses now fail instead of being decompressed as soon as they are read. - This PR and #2395 rewrite the same sentences in the README, the Javadoc of `ResponseBodyControl.suspend()` and the class Javadoc of `Http1ContentDecompressor`. #2395 removes the exception for `zstd` at a non-positive `io.netty.compression.defaultMaxForwardBytes`, which this PR keeps. Whichever is merged second, I'll merge `main` into it and resolve that. ## AI disclosure Claude Code on behalf of Matthias Kurz. The commit includes `Co-Authored-By` per `AGENTS.md`. ## Test plan - [x] `./mvnw -B -ntp -Dgpg.skip=true clean verify` on JDK 11: 2,097 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used. That is one test fewer than on `main`: the removed class loader test. - [x] The `Http1ContentDecompressor*Test`, `Http2ContentDecompressorTest`, `Http1Decompression*Test` and `ResponseBodyControlTest` classes: 141 tests, 0 failures. - [x] With the README change from review (`82fde4d68`), the same `clean verify`: 2,097 tests, 0 failures, 0 errors, 22 skipped; Revapi passed. Docker was not running, so the 11 Testcontainers proxy tests skipped themselves; they ran and passed with `32bb77670` (2,097 tests, 11 skipped). - [x] Merges with #2394 without conflicts. - [x] With `-Dnetty.version=4.2.17.Final`, all client tests (`./mvnw -pl client clean test`, JDK 11): 2,097 tests, 2 failures, both in tests that compare AHC's output and read requests with Netty's own decoders, which changed after 4.2.17 (`decodesLikeNettyForEveryBodyAndSplit`, `requestsReadsLikeNettyWhenAutoReadIsOff`). --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings in AsyncHttpClient#2396, which conflicts with this branch in the README, the Javadoc of ResponseBodyControl.suspend() and the class Javadoc of Http1ContentDecompressor, and AsyncHttpClient#2394, which changed the same files. The resolution keeps the Netty 4.2.17 requirement from AsyncHttpClient#2396 and the note on Netty 4.2.21 from this branch. As this branch no longer sends zstd to HttpContentDecompressor, it also drops the clause on Zstd bodies decompressed as soon as they are read from the README's paragraph on truncated bodies, which merged without a conflict. Claude Code on behalf of Matthias Kurz Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Conflicts resolved 👍 |
hyperxpro
approved these changes
Nov 21, 2026
Member
|
Thanks! I will cut a release soon |
Contributor
Author
Great, thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
useNativeTransportis set. On 4.2.20 that would happen on systems with an older glibc, such as RHEL 8 or Ubuntu 20.04. AHC 3.0.14 used Netty 4.2.18, so no AHC release has 4.2.20.ZstdDecoderandZstdDecompressorat a non-positiveio.netty.compression.defaultMaxForwardBytes(Fix ZstdDecoder and ZstdDecompressor with a non-positive defaultMaxForwardBytes netty/netty#18289). This PR drops AHC's workarounds for that bug, so zstd now waits for demand at any value of the property, like br and snappy. A zero or negative value of the property now needs Netty 4.2.21 or later.Change
ZstdDecoderreplacement from Upgrade Netty to 4.2.20.Final #2375. At a non-positive value of the property,Http1ContentDecompressorreplaced theZstdDecoderthatHttpContentDecompressorcreates with one whose buffers holdMAX_PART_SIZE, because Netty's decoder handed on an empty body at 0 and failed below it.HttpContentDecompressorinstead ofPullDecompression, because Netty'sZstdDecompressorhanded out nothing at 0 while reporting more output. It now takes the pull path there too:Truncated zstd streaminstead of ending quietly.ResponseBodyControl.suspend()and ofHttp1ContentDecompressorno longer make an exception for zstd. The README andsuspend()say instead that a zero or negative value of the property needs Netty 4.2.21 or later. With Netty 4.2.17 to 4.2.20, zstd responses then fail at a negative value, and at 0 they never end and keep the event loop busy, as Netty'sZstdDecompressorhands out empty buffers while it reports more output.MAX_FORWARD_BYTESis private, since onlyHttp1ContentDecompressoruses it now.Http1ContentDecompressorForwardThresholdTestruns with the property at 0 and -1 in a fresh JVM. It now also checks zstd while suspending after every part (one part per resume, none larger thanMAX_PART_SIZE) and a truncated zstd body (it fails, and nothing is left allocated).Notes
With the property at its default (64 KiB), nothing changes for zstd.
Other changes in 4.2.21 that touch AHC's paths:
SslHandlerchanges: SslHandler: Stop wrapping when the transport can not keep up netty/netty#18253, #18323, #18363 and #18406.AHC's tests pass without changes.
AI disclosure
Claude Code on behalf of Matthias Kurz. The commit includes
Co-Authored-ByperAGENTS.md.Test plan
./mvnw -B -ntp -Dgpg.skip=true clean verifyon JDK 11: 2,098 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used.5fd35bad8), the sameclean verify: 2,098 tests, 0 failures, 0 errors, 22 skipped; Revapi passed. Docker was not running, so the 11 Testcontainers proxy tests skipped themselves; they ran and passed in the runs above, on the same code.clean verify: 2,098 tests, 0 failures.IllegalArgumentException: initialCapacity : -1.Http1ContentDecompressor*Test,Http2ContentDecompressorTest,Http1Decompression*TestandResponseBodyControlTestclasses: 142 tests, 0 failures.ef18b0b68): no conflicts; the same classes, 203 tests, 0 failures;clean verifyon JDK 11, 2,159 tests, 0 failures, 0 errors, 11 skipped; Revapi passed.