TestCLI#test_control_for_ssl and others are newly failing with OpenSSL errors

[pvalena]

Describe the bug
Tests are failing when test suite is executed in Fedora. Previously those passed (with same OpenSSL version).

  1) Error:                                                                                                                                                                                     
TestCLI#test_control_for_ssl:                                                                                                                                                                   
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello                                                                                               
    /usr/share/ruby/net/protocol.rb:44:in `connect_nonblock'                                                                                                                                    
    /usr/share/ruby/net/protocol.rb:44:in `ssl_socket_connect'                                                                                                                                  
    /usr/share/ruby/net/http.rb:1009:in `connect'                                                                                                                                               
    /usr/share/ruby/net/http.rb:943:in `do_start'                                                                                                                                               
    /usr/share/ruby/net/http.rb:932:in `start'                                                                                                                                                  
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_cli.rb:90:in `test_control_for_ssl'                                                                   
                                                                                                      
  7) Error:                                                                                                                                                                                     
TestPumaServerSSL#test_request_wont_block_thread:                                                                                                                                               
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello                                                                                               
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:100:in `connect'                                                                   
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:100:in `test_request_wont_block_thread'                                            
                                                                                                                                                                                                
  8) Error:                                                                                                                                                                                     
TestPumaServerSSL#test_very_large_return:                                                                                                                                                       
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello                                                                                               
    /usr/share/ruby/net/protocol.rb:44:in `connect_nonblock'                                                                                                                                    
    /usr/share/ruby/net/protocol.rb:44:in `ssl_socket_connect'                                                                                                                                  
    /usr/share/ruby/net/http.rb:1009:in `connect'                                                                                                                                               
    /usr/share/ruby/net/http.rb:943:in `do_start'                                                                                                                                               
    /usr/share/ruby/net/http.rb:932:in `start'                                                                                                                                                  
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:124:in `test_very_large_return'                                                    
                                                                                                                                                                                                
  9) Skipped:                                                                                                                                                                                   
TestPumaServerSSL#test_ssl_v3_rejection [/builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:151]:                                         
SSLv3 protocol is unavailable                                                                                                                                                                   
                                                                                                                                                                                                
 10) Error:                                                                                                                                                                                     
TestPumaServerSSL#test_url_scheme_for_https:                                                                                                                                                    
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello                                                                                               
    /usr/share/ruby/net/protocol.rb:44:in `connect_nonblock'                                                                                                                                    
    /usr/share/ruby/net/protocol.rb:44:in `ssl_socket_connect'                                                                                                                                  
    /usr/share/ruby/net/http.rb:1009:in `connect'                                                                                                                                               
    /usr/share/ruby/net/http.rb:943:in `do_start'                                                                                                                                               
    /usr/share/ruby/net/http.rb:932:in `start'                                                                                                                                                  
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:82:in `test_url_scheme_for_https'                                                  
                                                                                                                                                                                                
 11) Error:                                                                                                                                                                                     
TestPumaServerSSL#test_form_submit:                                                                                                                                                             
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello                                                                                               
    /usr/share/ruby/net/protocol.rb:44:in `connect_nonblock'                                                                                                                                    
    /usr/share/ruby/net/protocol.rb:44:in `ssl_socket_connect'                                                                                                                                  
    /usr/share/ruby/net/http.rb:1009:in `connect'                                                                                                                                               
    /usr/share/ruby/net/http.rb:943:in `do_start'                                                                                                                                               
    /usr/share/ruby/net/http.rb:932:in `start'                                                                                                                                                  
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:137:in `test_form_submit'                                                          
 
 16) Failure:                                   
TestPumaServerSSLClient#test_verify_client_cert [/builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:291]:
Expected: false                                 
  Actual: true                                  

 17) Error:                                     
TestPumaServerSSLClient#test_verify_fail_if_client_expired_cert:                                
NoMethodError: undefined method `message' for nil:NilClass                                      
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:256:in `assert_ssl_client_error_match'
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:281:in `test_verify_fail_if_client_expired_cert'

 18) Error:                                     
TestPumaServerSSLClient#test_verify_fail_if_client_unknown_ca:                                  
NoMethodError: undefined method `message' for nil:NilClass                                      
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:256:in `assert_ssl_client_error_match'
    /builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:271:in `test_verify_fail_if_client_unknown_ca'

 19) Failure:                                   
TestPumaServerSSLClient#test_verify_fail_if_no_client_cert [/builddir/build/BUILD/puma-5.0.0.beta1/usr/share/gems/gems/puma-5.0.0.beta1/test/test_puma_server_ssl.rb:265]:
Expected /peer\ did\ not\ return\ a\ certificate/ to match # encoding: ASCII-8BIT               
#    valid: true                                
"OpenSSL error: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher - 337092801".

343 runs, 831 assertions, 2 failures, 7 errors, 10 skips                                        

------------------------------------------------------------ Debugging Info                     
TestIntegrationCluster#test_term_closes_listeners_unix                                          
    10 successes, 0 resets, 30 refused, failures 0                                              
TestIntegrationCluster#test_term_closes_listeners_tcp                                           
    11 successes, 1 resets, 28 refused, failures 0                                              
---------------------------------------------------------------------------

While executed like so:

$ CI=1 ruby -e 'Dir.glob "./test/**/test_*.rb", &method(:require)' -- -v
ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [i386-linux]
RUBYOPT: -Ilib:/builddir/build/BUILD/puma-5.0.0.beta1/usr/lib/gems/ruby/puma-5.0.0.beta1
                         Puma::MiniSSL                   OpenSSL
OPENSSL_LIBRARY_VERSION: OpenSSL 1.1.1g FIPS  21 Apr 2020OpenSSL 1.1.1g FIPS  21 Apr 2020
        OPENSSL_VERSION: OpenSSL 1.1.1g FIPS  21 Apr 2020OpenSSL 1.1.1g FIPS  21 Apr 2020
Run options: -v --seed 61893
# Running:
TestCLI#test_control_for_ssl = 1.05 s = E
 [ . . . ]

Desktop (please complete the following information):

• OS: Fedora Rawhide
• Puma Version 5.0.0.beta1

[MSP-Greg]

@pvalena

Thanks for the report. You mentioned 'newly failing'. What passed previously? Also, do any of the tests for test_puma_server_ssl.rb pass?

[pvalena]

@pvalena

Thanks for the report. You mentioned 'newly failing'. What passed previously? Also, do any of the tests for test_puma_server_ssl.rb pass?

Hello, I've broadened the description when I realized all of them are newly failing with probably the same/similar OpenSSL error. This is likely related to #2329 as well.

All of the test suite passed at my earlier build 2020-Jun-10 . Maybe it's related to some Fedora ciphers Change (not sure TBH).

[MSP-Greg]

Sorry, I got mixed up with the two issues.

All of the test suite passed at my earlier build 2020-Jun-10

Were you running FIPS with that build? I might know what's causing this, not sure. I'll need a bit of time to check it...

[pvalena]

Sorry, I got mixed up with the two issues.

All of the test suite passed at my earlier build 2020-Jun-10

Were you running FIPS with that build? I might know what's causing this, not sure. I'll need a bit of time to check it...

AFAICT it was the same config.
Here're both logs (old | new): https://download.copr.fedorainfracloud.org/results/pvalena/rubygems/fedora-rawhide-x86_64/01441187-rubygem-puma/builder-live.log.gz | https://download.copr.fedorainfracloud.org/results/pvalena/rubygems/fedora-rawhide-x86_64/01595268-rubygem-puma/builder-live.log.gz

NP, no rush. I will check if this happens with 4.3.5 as well.

[MSP-Greg]

@pvalena

I think Fedora raises the OpenSSL security level. If I raise the level locally, I get test errors.

I'm going to regenerate the certs/pems used in the CI, which should fix the issue. I think.

[MSP-Greg]

@pvalena

I just switched CI over to Ubuntu 20.04, which raises the security level, and I'm seeing the same errors you have. Working on updatng the certs...

[dentarg]

Connecting this to #2147 (about Debian) were the security level was also discussed

[pvalena]

FYI: 4.3.5 (with 4.3.4 tests) suffers from the same issue:
https://gist.github.com/93ac6372280cb00af74586550d9a83ca

@MSP-Greg thanks!