Skip to content

Upgrade Netty to 4.2.21.Final - #2395

Merged
hyperxpro merged 3 commits into
AsyncHttpClient:mainfrom
mkurz:netty-4.2.21
Nov 21, 2026
Merged

hyperxpro merged 3 commits into
AsyncHttpClient:mainfrom
mkurz:netty-4.2.21

Conversation

@mkurz

@mkurz mkurz commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Upgrades Netty from 4.2.20.Final to 4.2.21.Final. It mainly reverts a build mistake in 4.2.20: Netty's native libraries for x86_64 and aarch64 required glibc 2.33. AHC declares the native epoll and io_uring jars at its Netty version, and when they do not load, it falls back to NIO, silently unless useNativeTransport is set. On 4.2.20 that would happen on systems with an older glibc, such as RHEL 8 or Ubuntu 20.04. AHC 3.0.14 used Netty 4.2.18, so no AHC release has 4.2.20.
  • 4.2.21 also fixes ZstdDecoder and ZstdDecompressor at a non-positive io.netty.compression.defaultMaxForwardBytes (Fix ZstdDecoder and ZstdDecompressor with a non-positive defaultMaxForwardBytes netty/netty#18289). This PR drops AHC's workarounds for that bug, so zstd now waits for demand at any value of the property, like br and snappy. A zero or negative value of the property now needs Netty 4.2.21 or later.

Change

  • Netty 4.2.21.Final.
  • Drop the ZstdDecoder replacement from Upgrade Netty to 4.2.20.Final #2375. At a non-positive value of the property, Http1ContentDecompressor replaced the ZstdDecoder that HttpContentDecompressor creates with one whose buffers hold MAX_PART_SIZE, because Netty's decoder handed on an empty body at 0 and failed below it.
  • Drop the zstd guard from fix(netty): stop inflating a suspended HTTP/1.1 response #2362. At such a value, zstd went to HttpContentDecompressor instead of PullDecompression, because Netty's ZstdDecompressor handed out nothing at 0 while reporting more output. It now takes the pull path there too:
    • it waits for demand;
    • parts are at most 64 KiB;
    • the size limit applies;
    • a truncated stream fails with Truncated zstd stream instead of ending quietly.
  • Docs: the README and the Javadoc of ResponseBodyControl.suspend() and of Http1ContentDecompressor no longer make an exception for zstd. The README and suspend() say instead that a zero or negative value of the property needs Netty 4.2.21 or later. With Netty 4.2.17 to 4.2.20, zstd responses then fail at a negative value, and at 0 they never end and keep the event loop busy, as Netty's ZstdDecompressor hands out empty buffers while it reports more output. MAX_FORWARD_BYTES is private, since only Http1ContentDecompressor uses it now.
  • Test: Http1ContentDecompressorForwardThresholdTest runs with the property at 0 and -1 in a fresh JVM. It now also checks zstd while suspending after every part (one part per resume, none larger than MAX_PART_SIZE) and a truncated zstd body (it fails, and nothing is left allocated).

Notes

AI disclosure

Claude Code on behalf of Matthias Kurz. The commit includes Co-Authored-By per AGENTS.md.

Test plan

  • ./mvnw -B -ntp -Dgpg.skip=true clean verify on JDK 11: 2,098 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used.
  • With the README and Javadoc change from review (5fd35bad8), the same clean verify: 2,098 tests, 0 failures, 0 errors, 22 skipped; Revapi passed. Docker was not running, so the 11 Testcontainers proxy tests skipped themselves; they ran and passed in the runs above, on the same code.
  • The version change alone, without the other changes, passes the same clean verify: 2,098 tests, 0 failures.
  • The extended test fails where it should:
    • with the guard put back, at 0 and -1: "the suspended zstd body did not wait for demand after its first part";
    • without the workarounds on Netty 4.2.20: at 0, the check hangs until its 60 s timeout; at -1, it fails with IllegalArgumentException: initialCapacity : -1.
  • The Http1ContentDecompressor*Test, Http2ContentDecompressorTest, Http1Decompression*Test and ResponseBodyControlTest classes: 142 tests, 0 failures.
  • Merged with Wait for demand after the server closes #2394 (ef18b0b68): no conflicts; the same classes, 203 tests, 0 failures; clean verify on JDK 11, 2,159 tests, 0 failures, 0 errors, 11 skipped; Revapi passed.

Netty 4.2.21 reverts a build mistake in 4.2.20: its native libraries
for x86_64 and aarch64 required glibc 2.33. It also fixes ZstdDecoder
and ZstdDecompressor at a non-positive
io.netty.compression.defaultMaxForwardBytes (netty/netty#18289), so
AHC's workarounds for that bug go:

- the ZstdDecoder that HttpContentDecompressor creates is no longer
  replaced by one with buffers of MAX_PART_SIZE at such a value
  (AsyncHttpClient#2375);
- zstd is no longer left to HttpContentDecompressor at such a value
  (AsyncHttpClient#2362). It goes through PullDecompression like br and snappy, so
  it waits for demand, and a truncated stream fails, at any value of
  the property.

Claude Code on behalf of Matthias Kurz

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mkurz mkurz mentioned this pull request Nov 18, 2026
5 tasks done
Without the workarounds, zstd responses fail with Netty 4.2.17 to 4.2.20
if the system property io.netty.compression.defaultMaxForwardBytes is
negative. At 0 they never end and keep the event loop busy, as Netty's
ZstdDecompressor then hands out empty buffers while it reports more
output.

Say in the README and in ResponseBodyControl.suspend() that a zero or
negative value needs Netty 4.2.21 or later.

Claude Code on behalf of Matthias Kurz

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mkurz
mkurz requested a review from hyperxpro November 20, 2026 23:34
hyperxpro pushed a commit that referenced this pull request Nov 21, 2026
## Summary

- Since #2362, AHC decompresses HTTP/1.1 `br`, `zstd` and `snappy`
bodies with Netty's pull-based `Decompressor` API, so that they wait for
demand. Netty added that API in 4.2.17. With an older Netty,
`Http1ContentDecompressor` caught the `LinkageError` and left these
codings to `HttpContentDecompressor`, which decompresses them as soon as
they are read.
- AHC declares a Netty that has the API (4.2.20 on `main`, 4.2.21 with
#2395), so this PR drops that fallback. The README now says that AHC
needs Netty 4.2.17 or later.

## Change

- **`Http1ContentDecompressor`:** the `pullAvailable` flag and the
`LinkageError` catch go. `newPullDecompression` now only creates the
`PullDecompression`.
- **Unchanged:**
- `PullDecompression.create` still checks whether Brotli and zstd are
available, since brotli4j and zstd-jni are optional dependencies. A body
whose library is missing is still left undecoded, as
`HttpContentDecompressor` does.
- The size limit and the split into parts of at most 64 KiB are
unchanged.
- **Test:** `gzipAndDeflateDoNotNeedNettysDecompressorApi` goes,
together with the class loader that hid the API from it, as an older
Netty would.
- **Docs:** the README, the Javadoc of `ResponseBodyControl.suspend()`
and the class Javadoc of `Http1ContentDecompressor` and
`PullDecompression` no longer describe the fallback. The README now says
that AHC needs Netty 4.2.17 or later, as it uses the API Netty added
there, and that on-demand decompression applies over HTTP/1.1, and its
paragraph on truncated bodies no longer names Brotli and Snappy bodies
decompressed as soon as they are read. `PullDecompression` still notes
that Netty marks the API `@UnstableApi`, so a Netty upgrade may require
changes there.

## Notes

- With AHC's declared Netty dependency, the fallback for older Netty
versions is unnecessary. With Netty older than 4.2.17, br, zstd and
snappy responses now fail instead of being decompressed as soon as they
are read.
- This PR and #2395 rewrite the same sentences in the README, the
Javadoc of `ResponseBodyControl.suspend()` and the class Javadoc of
`Http1ContentDecompressor`. #2395 removes the exception for `zstd` at a
non-positive `io.netty.compression.defaultMaxForwardBytes`, which this
PR keeps. Whichever is merged second, I'll merge `main` into it and
resolve that.

## AI disclosure

Claude Code on behalf of Matthias Kurz. The commit includes
`Co-Authored-By` per `AGENTS.md`.

## Test plan

- [x] `./mvnw -B -ntp -Dgpg.skip=true clean verify` on JDK 11: 2,097
tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping
flags were used. That is one test fewer than on `main`: the removed
class loader test.
- [x] The `Http1ContentDecompressor*Test`,
`Http2ContentDecompressorTest`, `Http1Decompression*Test` and
`ResponseBodyControlTest` classes: 141 tests, 0 failures.
- [x] With the README change from review (`82fde4d68`), the same `clean
verify`: 2,097 tests, 0 failures, 0 errors, 22 skipped; Revapi passed.
Docker was not running, so the 11 Testcontainers proxy tests skipped
themselves; they ran and passed with `32bb77670` (2,097 tests, 11
skipped).
- [x] Merges with #2394 without conflicts.
- [x] With `-Dnetty.version=4.2.17.Final`, all client tests (`./mvnw -pl
client clean test`, JDK 11): 2,097 tests, 2 failures, both in tests that
compare AHC's output and read requests with Netty's own decoders, which
changed after 4.2.17 (`decodesLikeNettyForEveryBodyAndSplit`,
`requestsReadsLikeNettyWhenAutoReadIsOff`).

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings in AsyncHttpClient#2396, which conflicts with this branch in the README, the
Javadoc of ResponseBodyControl.suspend() and the class Javadoc of
Http1ContentDecompressor, and AsyncHttpClient#2394, which changed the same files.

The resolution keeps the Netty 4.2.17 requirement from AsyncHttpClient#2396 and the
note on Netty 4.2.21 from this branch. As this branch no longer sends
zstd to HttpContentDecompressor, it also drops the clause on Zstd
bodies decompressed as soon as they are read from the README's
paragraph on truncated bodies, which merged without a conflict.

Claude Code on behalf of Matthias Kurz

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mkurz

mkurz commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

Conflicts resolved 👍

@hyperxpro
hyperxpro merged commit 8f6d126 into AsyncHttpClient:main Nov 21, 2026
13 checks passed
@hyperxpro

Copy link
Copy Markdown
Member

Thanks! I will cut a release soon

@mkurz

mkurz commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

I will cut a release soon

Great, thanks!

@mkurz
mkurz deleted the netty-4.2.21 branch November 21, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants