[FLINK-38764] Upgrade lz4 to 1.8.1 due to security vulnerability

[pnowojski]

What is the purpose of the change

Upgrade lz4 to 1.8.1 due to security vulnerability

Verifying this change

Change should be covered by the existing tests.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (yes / no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
• The serializers: (yes / no / don't know)
• The runtime per-record code paths (performance sensitive): (yes / no / don't know)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
• The S3 file system connector: (yes / no / don't know)

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[flinkbot]

CI report:

• 42cb207 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[pnowojski]

lz4 1.8.0 is still being pulled from our Kafka connector, via Kafka client 🤔

[INFO] +- org.apache.flink:flink-connector-kafka:jar:3.0.0-1.17:compile
[INFO] |  +- org.apache.flink:flink-connector-base:jar:1.17.0:compile
[INFO] |  \- org.apache.kafka:kafka-clients:jar:3.2.3:compile
[INFO] |     +- com.github.luben:zstd-jni:jar:1.5.2-1:runtime
[INFO] |     \- org.lz4:lz4-java:jar:1.8.0:runtime

Kafka connector is pulled in from examples and in some tests, so on the one hand I think we should be fine just ignoring it until kafka connector upgrades it's own dependency 🤔 But on the other hand I'm worried about dependency convergence if someone tries to use Flink with lz4 1.8.1 with Kafka Connector with lz4 1.8.0.

I'm not 100% sure how to procede here.

I guess we need to fix this problem simultaneously in the two repos, and only the new flink kafka connector versions will be officially compatible with Flink versions released with this change/fix?