Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[fix][misc] Log Conscrypt security provider initialization warnings at debug level #23364

Merged

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Sep 28, 2024

Motivation

  • pulsar-admin output parsing breaks in 4.0.0-preview.1 testing because of the warnings

Example from Pulsar Helm chart testing

Creating subscription for output topic
2024-09-27T19:25:20,336+0000 [main] WARN  org.apache.pulsar.common.util.SecurityUtility - Conscrypt isn't available for Linux amd64. Using JDK default security provider.
Waiting for function to be ready
parse error: Invalid numeric literal at line 1, column 14

Modifications

  • switch logging level from warn to debug

Additional context

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

…t debug level

- script output parsing breaks in 4.0.0-preview.1 testing because of the warnings
@lhotari lhotari added this to the 4.0.0 milestone Sep 28, 2024
@lhotari lhotari self-assigned this Sep 28, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 28, 2024
@lhotari
Copy link
Member Author

lhotari commented Sep 28, 2024

This is a regression in the Alpine image since 3.3.x.

I did some manual testing to see why the native library cannot be loaded.

237d36ef299c:/tmp/META-INF/native# ldd libconscrypt_openjdk_jni-linux-x86_64.so
	/lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7fffff06b000)
	libpthread.so.0 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libm.so.6 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7fffff047000)
	libc.so.6 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2 (0x7fffff011000)
Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37

this is similar to #22804 error message.

and

237d36ef299c:/tmp/META-INF/native# /usr/glibc-compat/bin/ldd libconscrypt_openjdk_jni-linux-x86_64.so
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ffffef67000)
	libpthread.so.0 => /usr/glibc-compat/lib/libpthread.so.0 (0x00007fffff7bf000)
	libm.so.6 => /usr/glibc-compat/lib/libm.so.6 (0x00007fffff6df000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fffff6bb000)
	libc.so.6 => /usr/glibc-compat/lib/libc.so.6 (0x00007ffffed25000)
	/usr/glibc-compat/lib64/ld-linux-x86-64.so.2 (0x00007ffffffca000)
	libc.musl-x86_64.so.1 => /lib/libc.musl-x86_64.so.1 (0x00007ffffec82000)

I wonder if using Alpine is the correct solution when mixing musl and glibc at runtime:
sgerrand/alpine-pkg-glibc#80 (comment)

Another comment against mixing musl and glibc at runtime:
sgerrand/alpine-pkg-glibc#194 (comment)

Yeah, fair enough - I have also seen it in cases where there is an accidental attempt to mix musl and glibc-linked binaries. I got a similar error using a musl JVM build with a different glibc native load library. Tend to agree with you that this is a common mixing problem and probably always a risk to some extent when using glibc on Alpine. You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)

Problems like this are the reasons that a number of the folks on the Alpine team are quite against packages and approaches like this to put glibc back onto a musl-based OS such as Alpine (despite its utility they would seemingly prefer folks use a minimal glibc-based distro instead of Alpine)

The comment "You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)" is something that is concerning. @merlimat Any thoughts on that?

@lhotari lhotari closed this Sep 28, 2024
@lhotari lhotari reopened this Sep 28, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 28, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 74.58%. Comparing base (bbc6224) to head (7f27e89).
Report is 609 commits behind head on master.

Files with missing lines Patch % Lines
...org/apache/pulsar/common/util/SecurityUtility.java 50.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23364      +/-   ##
============================================
+ Coverage     73.57%   74.58%   +1.01%     
- Complexity    32624    33958    +1334     
============================================
  Files          1877     1934      +57     
  Lines        139502   145131    +5629     
  Branches      15299    15870     +571     
============================================
+ Hits         102638   108251    +5613     
+ Misses        28908    28581     -327     
- Partials       7956     8299     +343     
Flag Coverage Δ
inttests 27.58% <25.00%> (+2.99%) ⬆️
systests 24.58% <0.00%> (+0.26%) ⬆️
unittests 73.93% <25.00%> (+1.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...org/apache/pulsar/common/util/SecurityUtility.java 72.22% <50.00%> (-7.70%) ⬇️

... and 604 files with indirect coverage changes

@nodece nodece merged commit 950309b into apache:master Sep 29, 2024
106 of 108 checks passed
lhotari added a commit that referenced this pull request Oct 4, 2024
lhotari added a commit that referenced this pull request Oct 4, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Oct 15, 2024
…t debug level (apache#23364)

(cherry picked from commit 950309b)
(cherry picked from commit 988b884)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Oct 16, 2024
…t debug level (apache#23364)

(cherry picked from commit 950309b)
(cherry picked from commit 988b884)
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants