Unknown macOS Sonoma CDash Submission Failures

[svenevs]

macOS Sonoma jobs were having trouble submitting to CDash.

• Workaround: [sonoma] Bypass SSL problems with CURL_SSL_BACKEND=SecureTransport drake-ci#261
• Left to do:
  • Eventually a fix will come out, likely via brew changes. See: cmake issue with system libcurl on macOS 14.2 Homebrew/homebrew-core#158759
  • Reprovision sonoma images.
  • Revert [sonoma] Bypass SSL problems with CURL_SSL_BACKEND=SecureTransport drake-ci#261

CC @BetsyMcPhail @jwnimmer-tri

[svenevs]

https://drake-jenkins.csail.mit.edu/view/Nightly%20Production/job/mac-arm-sonoma-clang-bazel-nightly-everything-release/21/consoleFull

[5:15:11 AM]  Executed 7677 out of 7677 tests: 7677 tests pass.
[5:15:11 AM]  There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
[5:15:11 AM]  INFO: Build completed successfully, 44627 total actions
[5:15:11 AM]  INFO: Build Event Protocol files produced successfully.
[5:16:08 AM]     Error when uploading file: /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/_bazel_admin/Testing/20240102-0937/Update.xml
[5:16:08 AM]     Error message was: LibreSSL/3.3.6: error:0DFFF0A1:lib(13):func(4095):reason(161)
[5:16:08 AM]     Problems when submitting via HTTP
[5:16:08 AM]  CMake Warning at /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel/step-build.cmake:121 (message):
[5:16:08 AM]    *** CTest submit update part was not successful
[5:16:08 AM]  Call Stack (most recent call first):
[5:16:08 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/functions.cmake:430 (include)
[5:16:08 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel.cmake:307 (execute_step)
[5:16:08 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/ctest_driver_script.cmake:129 (include)
[5:16:08 AM]  
[5:16:08 AM]  
[5:17:17 AM]     Error when uploading file: /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/_bazel_admin/Testing/20240102-0937/Upload.xml
[5:17:17 AM]     Error message was: LibreSSL/3.3.6: error:0DFFF0A1:lib(13):func(4095):reason(161)
[5:17:17 AM]     Problems when submitting via HTTP
[5:17:17 AM]  CMake Warning at /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel/step-build.cmake:135 (message):
[5:17:17 AM]    *** CTest submit upload part was not successful
[5:17:17 AM]  Call Stack (most recent call first):
[5:17:17 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/functions.cmake:430 (include)
[5:17:17 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel.cmake:307 (execute_step)
[5:17:17 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/ctest_driver_script.cmake:129 (include)
[5:17:17 AM]  
[5:17:17 AM]  
[5:18:15 AM]  Error in HttpRequest
[5:18:15 AM]  CMake Warning at /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel/step-build.cmake:146 (message):
[5:18:15 AM]    *** CTest submit CDash upload Bazel JSON was not successful
[5:18:15 AM]  Call Stack (most recent call first):
[5:18:15 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/functions.cmake:430 (include)
[5:18:15 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel.cmake:307 (execute_step)
[5:18:15 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/ctest_driver_script.cmake:129 (include)
[5:18:15 AM]  
[5:18:15 AM]  
[5:19:13 AM]     Error when uploading file: /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/_bazel_admin/Testing/20240102-0937/Done.xml
[5:19:13 AM]     Error message was: LibreSSL/3.3.6: error:0DFFF0A1:lib(13):func(4095):reason(161)
[5:19:13 AM]     Problems when submitting via HTTP
[5:19:13 AM]  CMake Warning at /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel/step-build.cmake:183 (message):
[5:19:13 AM]    *** CTest submit done part was not successful
[5:19:13 AM]  Call Stack (most recent call first):
[5:19:13 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/functions.cmake:430 (include)
[5:19:13 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/driver/configurations/bazel.cmake:307 (execute_step)
[5:19:13 AM]    /Users/admin/workspace/mac-arm-sonoma-clang-bazel-nightly-everything-release/ci/ctest_driver_script.cmake:129 (include)

Still a problem, discussing internally.

[BetsyMcPhail]

Possibly related to Homebrew/homebrew-core#158759 (comment)

[svenevs]

Issue description updated now that we understand the solution better. RobotLocomotion/drake-ci#261 is a temporary bypass, there's no further action for us to take right now. In the future, the workaround should be removed when the problem is fixed upstream (in brew).