Skip to content

Regression: undefined symbol __atomic_load_4 on risvc32imac-unknown-none-elf #85736

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
xobs opened this issue May 27, 2021 · 11 comments
Closed
Labels
A-atomic Area: Atomics, barriers, and sync primitives A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. O-riscv Target: RISC-V architecture P-medium Medium priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Comments

@xobs
Copy link
Contributor

xobs commented May 27, 2021

Description

When building for riscv32imac-unknown-none-elf, rustc is occasionally generating calls to __atomic_load_4. This symbol doesn't appear to exist, even though atomics are supported on this platform. Furthermore, I'm not sure if atomics are used anywhere in this codepath -- log is the only crate in the dependency tree that I haven't verified.

This works fine if I target riscv32i-unknown-none-elf, which disables atomic generation.

Furthermore, this builds fine if I use rustc 1.51, however it fails with rustc 1.52.1.

Background

I'm trying to work with the crate gdbstub to integrate it into a kernel running on baremetal RISC-V. The target is riscv32imac-unknown-none-elf, which has atomics. For some reason, the compiler is generating calls to __atomic_load_4, which doesn't exist.

Meta

rustc --version --verbose:

rustc 1.52.1 (9bc8c42bb 2021-05-09)
binary: rustc
commit-hash: 9bc8c42bb2f19e745a63f3445f1ac248fb015e53
commit-date: 2021-05-09
host: x86_64-unknown-linux-gnu
release: 1.52.1
LLVM version: 12.0.0
rustc 1.51

user@Cuboid:/mnt/d/Code/Xous/Core/kernel$ cargo +1.51 build --target riscv32imac-unknown-none-elf
warning: unused variable: `gdb_interrupt`
   --> src/debug.rs:168:13
    |
168 |             gdb_interrupt: GdbInterrupt<'_>,
    |             ^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_gdb_interrupt`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `register_thread`
   --> src/debug.rs:257:13
    |
257 |             register_thread: &mut dyn FnMut(Tid),
    |             ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_register_thread`

warning: field is never read: `pid`
   --> src/debug.rs:136:9
    |
136 |         pid: Option<xous_kernel::PID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: field is never read: `tid`
   --> src/debug.rs:137:9
    |
137 |         tid: Option<xous_kernel::TID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: field is never read: `thread_mask`
   --> src/debug.rs:138:9
    |
138 |         thread_mask: usize,
    |         ^^^^^^^^^^^^^^^^^^

warning: variant is never constructed: `Debug`
  --> src/services.rs:61:5
   |
61 |     Debug(usize),
   |     ^^^^^^^^^^^^

warning: 6 warnings emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.76s
user@Cuboid:/mnt/d/Code/Xous/Core/kernel$

rustc 1.52.1

user@Cuboid:/mnt/d/Code/Xous/Core/kernel$ cargo +stable build --target riscv32imac-unknown-none-elf
   Compiling kernel v0.8.2 (/mnt/d/Code/Xous/Core/kernel)
warning: unused variable: `gdb_interrupt`
   --> src/debug.rs:168:13
    |
168 |             gdb_interrupt: GdbInterrupt<'_>,
    |             ^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_gdb_interrupt`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `register_thread`
   --> src/debug.rs:257:13
    |
257 |             register_thread: &mut dyn FnMut(Tid),
    |             ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_register_thread`

warning: field is never read: `pid`
   --> src/debug.rs:136:9
    |
136 |         pid: Option<xous_kernel::PID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: field is never read: `tid`
   --> src/debug.rs:137:9
    |
137 |         tid: Option<xous_kernel::TID>,
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: field is never read: `thread_mask`
   --> src/debug.rs:138:9
    |
138 |         thread_mask: usize,
    |         ^^^^^^^^^^^^^^^^^^

warning: variant is never constructed: `Debug`
  --> src/services.rs:61:5
   |
61 |     Debug(usize),
   |     ^^^^^^^^^^^^

error: linking with `rust-lld` failed: exit code: 1
  |
  = note: "rust-lld" "-flavor" "gnu" "-plugin-opt=O0" "-plugin-opt=mcpu=generic-rv32" "-L" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.0.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.1.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.10.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.11.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.12.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.13.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.14.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.15.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.2.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.3.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.4.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.5.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.6.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.7.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.8.rcgu.o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61.kernel.2tnp50fl-cgu.9.rcgu.o" "-o" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/kernel-a49a1beda364ba61" "--gc-sections" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps" "-L" "/mnt/d/Code/Xous/Core/kernel/target/debug/deps" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/kernel-b2cf221a795a62c2/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/kernel-b2cf221a795a62c2/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/riscv-4ccdf2c7a1922587/out" "-L" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/build/xous-1a7daf4c65ecd197/out" "-L" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib" "-Bstatic" "--whole-archive" "-lkernel" "--no-whole-archive" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libgdbstub_arch-56fd0be17381d6a4.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libxous-39428ed2b3a43c31.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libriscv-be84ff84a45e1b85.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbit_field-e83fefc77bfd4655.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbare_metal-4609200987315921.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libgdbstub-517875492397d3c0.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libcfg_if-5e47378e83c91cff.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libnum_traits-34dfc229d6370397.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libmanaged-9d0d80e606c7b727.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/liblog-4269a8ed40a0932a.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libcfg_if-2c07cfa8a1d4ab2f.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libutralib-18b768caf3ee14fc.rlib" "/mnt/d/Code/Xous/Core/kernel/target/riscv32imac-unknown-none-elf/debug/deps/libbitflags-46bd593bb6538427.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/librustc_std_workspace_core-525cd988d4e88ede.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/libcore-d1be523e2bc04c86.rlib" "/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/riscv32imac-unknown-none-elf/lib/libcompiler_builtins-f400fa914b91ce44.rlib" "-Tlink.x" "-Bdynamic"
  = note: rust-lld: error: undefined symbol: __atomic_load_4
          >>> referenced by atomic.rs:0 (/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:0)
          >>>               lto.tmp:(core::sync::atomic::AtomicUsize::load::h0594e1bc6e9a0e87)
          >>> referenced by atomic.rs:0 (/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:0)
          >>>               lto.tmp:(core::sync::atomic::atomic_load::h507cc031d576ee38)


error: aborting due to previous error; 6 warnings emitted

error: could not compile `kernel`

To learn more, run the command again with --verbose.
user@Cuboid:/mnt/d/Code/Xous/Core/kernel$

@xobs xobs added the C-bug Category: This is a bug. label May 27, 2021
@xobs xobs changed the title rust-lld: error: undefined symbol: __atomic_load_4 on risvc32imac-unknown-none-elf Regression: undefined symbol __atomic_load_4 on risvc32imac-unknown-none-elf May 27, 2021
@jonas-schievink jonas-schievink added O-riscv Target: RISC-V architecture regression-from-stable-to-stable Performance or correctness regression from one stable version to another. labels May 27, 2021
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label May 27, 2021
@apiraino
Copy link
Contributor

@xobs if possible, could you provide a small repro of the regression? That would help in finding where the regression came from

@rustbot ping icebreakers-cleanup-crew

@rustbot rustbot added the ICEBreaker-Cleanup-Crew Helping to "clean up" bugs with minimal examples and bisections label May 27, 2021
@rustbot
Copy link
Collaborator

rustbot commented May 27, 2021

@xobs
Copy link
Contributor Author

xobs commented May 27, 2021

So this appears to be triggered by the "-C", "linker-plugin-lto=yes", flag. Has that been made obsolete?

I created a repro at https://github.com/xobs/rust-1.52.1-link-error but I couldn't get the repro any smaller. When I tried, the problem went away. This is demonstrated by using the bare-gdbstub crate instead of gdbstub.

My guess is that the function is getting large enough that it is... doing something. Even though the two calls are identical.

@SNCPlay42
Copy link
Contributor

Repro: build as a binary crate with -C linker-plugin-lto=yes (godbolt)

#![no_std]
#![no_main]

use core::sync::atomic::{Ordering, AtomicUsize};

#[panic_handler]
fn handle_panic(_arg: &core::panic::PanicInfo) -> ! {
    loop {}
}

#[no_mangle]
pub extern "C" fn _start() -> ! {
    let x = AtomicUsize::new(0);
    x.load(Ordering::SeqCst);
    loop { }
}

searched nightlies: from nightly-2021-02-05 to nightly-2021-05-04
regressed nightly: nightly-2021-03-18
searched commits: from f5d8117 to 36f1f04
regressed commit: 0c34122 (#83084)

bisected with cargo-bisect-rustc v0.6.0

Host triple: x86_64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --target=riscv32imac-unknown-none-elf --preserve --start=2021-02-05 --end=2021-05-04 -- rustc -- -C linker-plugin-lto=yes --target=riscv32imac-unknown-none-elf 

@apiraino
Copy link
Contributor

apiraino commented Jun 2, 2021

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-medium +I-nominated

@rustbot rustbot added P-medium Medium priority I-nominated and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Jun 2, 2021
@nagisa nagisa self-assigned this Jun 3, 2021
@nagisa nagisa added the A-LTO Area: Link-time optimization (LTO) label Jun 3, 2021
@nagisa nagisa removed their assignment Jul 8, 2021
@workingjubilee workingjubilee added the A-atomic Area: Atomics, barriers, and sync primitives label Aug 17, 2022
bors pushed a commit to rust-lang-ci/rust that referenced this issue Aug 5, 2023
This roughly reverts PR rust-lang#66548

Atomic "CAS" are still disabled for targets without the
*“A” Standard Extension for Atomic Instructions*.
However this extension only adds instructions for operations more complex
than simple loads and stores, which are always atomic when aligned.

In the [Unprivileged Spec v. 20191213](https://riscv.org/technical/specifications/)
section 2.6 *Load and Store Instructions* of
chapter 2 *RV32I Base Integer Instruction Set* (emphasis mine):

> Even when misaligned loads and stores complete successfully,
> these accesses might run extremely slowly depending on the implementation
> (e.g., when implemented via an invisible trap). Further-more, whereas
> **naturally aligned loads and stores are guaranteed to execute atomically**,
> misaligned loads and stores might not, and hence require
> additional synchronization to ensure atomicity.

Unfortunately PR rust-lang#66548 did not provide
much details on the bug that motivated it, but
rust-lang#66240 and
rust-lang#85736 appear related
and happen with targets that do have the A extension.
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 5, 2023
Re-enable atomic loads and stores for all RISC-V targets

This roughly reverts PR rust-lang#66548

Atomic "CAS" are still disabled for targets without the *“A” Standard Extension for Atomic Instructions*. However this extension only adds instructions for operations more complex than simple loads and stores, which are always atomic when aligned.

In the [Unprivileged Spec v. 20191213](https://riscv.org/technical/specifications/) section 2.6 *Load and Store Instructions* of chapter 2 *RV32I Base Integer Instruction Set* (emphasis mine):

> Even when misaligned loads and stores complete successfully, these accesses might run extremely slowly depending on the implementation (e.g., when implemented via an invisible trap). Further-more, whereas **naturally aligned loads and stores are guaranteed to execute atomically**, misaligned loads and stores might not, and hence require additional synchronization to ensure atomicity.

Unfortunately PR rust-lang#66548 did not provide much details on the bug that motivated it, but rust-lang#66240 and rust-lang#85736 appear related and happen with targets that do have the A extension.
@istankovic
Copy link
Contributor

Seems to be working with rustc 1.75.0-nightly (249624b 2023-10-20).

@istankovic
Copy link
Contributor

@xobs is this still a problem for you, with current rustc?

@xobs
Copy link
Contributor Author

xobs commented Jan 31, 2024

I haven't seen this problem in a while. Closing. Thank you for the fix!

@xobs xobs closed this as completed Jan 31, 2024
@xobs xobs reopened this Feb 1, 2024
@xobs
Copy link
Contributor Author

xobs commented Feb 1, 2024

We just had a user report an issue with Rust 1.75.0 today that was solved by changing LTO from "fat" to "thin", so this appears to still exist.

The error in question is:

  = note: rust-lld: error: undefined symbol: core::panicking::panic_nounwind::heb8ddfbf25c163db
          >>> referenced by compiler_builtins.3bf56cc9babbb60e-cgu.000
          >>>               compiler_builtins-caad9c73c64d0d39.compiler_builtins.3bf56cc9babbb60e-cgu.000.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h35d828d67d4905f6) in archive ~/.rustup/toolchains/1.75.0-aarch64-apple-darwin/lib/rustlib/riscv32imac-unknown-xous-elf/lib/libcompiler_builtins-caad9c73c64d0d39.rlib
          >>> referenced by compiler_builtins.3bf56cc9babbb60e-cgu.000
          >>>               compiler_builtins-caad9c73c64d0d39.compiler_builtins.3bf56cc9babbb60e-cgu.000.rcgu.o:(compiler_builtins::math::libm::rem_pio2_large::rem_pio2_large::h35d828d67d4905f6) in archive ~/.rustup/toolchains/1.75.0-aarch64-apple-darwin/lib/rustlib/riscv32imac-unknown-xous-elf/lib/libcompiler_builtins-caad9c73c64d0d39.rlib

Changing lto="thin results in a successful build.

@istankovic
Copy link
Contributor

This looks a bit different, this time it's not about __atomic_load_4 and seems to come from compiler_builtins.
Perhaps related to #112245?

@xobs could you please provide a minimal example that reproduces this issue? I.e. code and cargo and target configuration you're using, as well as the compiler invocation?

@jieyouxu jieyouxu added T-libs Relevant to the library team, which will review and decide on the PR/issue. and removed ICEBreaker-Cleanup-Crew Helping to "clean up" bugs with minimal examples and bisections labels Apr 30, 2025
@xobs
Copy link
Contributor Author

xobs commented Jun 4, 2025

I haven't heard of any issues related to this in a while. So I'm going to go ahead and close it.

@xobs xobs closed this as completed Jun 4, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
A-atomic Area: Atomics, barriers, and sync primitives A-LTO Area: Link-time optimization (LTO) C-bug Category: This is a bug. O-riscv Target: RISC-V architecture P-medium Medium priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

9 participants