Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Can't build on Ubuntu 18.04 #30

Closed
rspeer opened this issue Dec 2, 2019 · 27 comments
Closed

Can't build on Ubuntu 18.04 #30

rspeer opened this issue Dec 2, 2019 · 27 comments

Comments

@rspeer
Copy link

rspeer commented Dec 2, 2019

I added openblas-src as a dependency to my Cargo.toml as recommended by this issue: rust-ndarray/ndarray-linalg#171

The output I got is:

$ cargo build
   Compiling openblas-src v0.7.0
error: failed to run custom build command for `openblas-src v0.7.0`

Caused by:
  process didn't exit successfully: `/home/rspeer/code/skipgram-engine/target/debug/build/openblas-src-1acb45e3d7ad0634/build-script-build` (exit code: 101)
--- stdout
Running: `"make" "libs" "netlib" "shared" "BINARY=64" "YES_CBLAS=1" "YES_LAPACKE=1" "-j12"`

--- stderr
Makefile.system:47: Makefile.rule: No such file or directory
make: Makefile.prebuild: No such file or directory
make: *** No rule to make target 'Makefile.prebuild'.  Stop.
Makefile.system:195: Makefile.conf: No such file or directory
Makefile.system:1063: Makefile.: No such file or directory
make: *** No rule to make target 'Makefile.'.  Stop.
thread 'main' panicked at 'Failed: `"make" "libs" "netlib" "shared" "BINARY=64" "YES_CBLAS=1" "YES_LAPACKE=1" "-j12"` (exit code: 2)', /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/openblas-src-0.7.0/build.rs:68:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
@IvanUkhov
Copy link
Member

Are you sure source is checked out?

@rspeer
Copy link
Author

rspeer commented Dec 2, 2019

Sorry, what?

Is there documentation about how to compile this? Is this even the right package to satisfy ndarray_linalg's dependency on OpenBLAS?

@rspeer
Copy link
Author

rspeer commented Dec 2, 2019

In case this clarifies the situation: I'm trying to build another crate, ndarray_linalg, which depends on this one, and following their directions, which leads to a compilation error with this crate.

I'm not trying to check out the Git repository of this package and compile it, it's a dependency specified in Cargo.

@IvanUkhov
Copy link
Member

Yeah, this is what I was trying to get at. OK, then it’s not relevant.

Another thought is that the tool generating Makefile.rule and the rest is not available in your environment. Try installing OpenBLAS separately and see if you run into the same problem. If not, something is wrong in this package.

@rspeer
Copy link
Author

rspeer commented Dec 3, 2019

libopenblas-base is installed. I also tried apt installing libopenblas-dev, which would include the dev tools necessary to compile OpenBLAS from source, but I still get the same error.

@IvanUkhov
Copy link
Member

What command did you invoke to compile OpenBLAS? I guess the difference could be seen here.

@rspeer
Copy link
Author

rspeer commented Dec 3, 2019

I did not invoke any command specifically to compile OpenBLAS.

I've been running cargo build in my project that depends on ndarray-linalg, and if that strictly requires compiling OpenBLAS from source instead of using the Ubuntu library, the mechanism for doing so isn't one that I have intentional control of.

@IvanUkhov
Copy link
Member

Sorry, I probably misread your comment. You mean you tried to install libopenblas-dev, which does the compilation internally, and it failed with the same error as the one you get when you try to install this Cargo package?

@rspeer
Copy link
Author

rspeer commented Dec 3, 2019

I installed libopenblas-dev using Ubuntu's apt and it worked fine, but it has no effect on this crate. I still get the same error in cargo when it's time to build this crate.

@IvanUkhov
Copy link
Member

I have update the version of OpenBLAS used internally. Could you please give it another try?

@rspeer
Copy link
Author

rspeer commented Feb 5, 2020

Okay, what I did was to check out a version of ndarray-linalg where I updated the relevant dependencies to:

  [dependencies.blas-src]
  version = "0.5"
  default-features = false
  
  [dependencies.lapack-src]
  version = "0.5"
  default-features = false
  
  [dependencies.openblas-src]
  version = "0.8"
  default-features = false
  features = ["static"]
  optional = true

and it compiles! But then when I try to run cargo test on it, I get a long error message where the relevant part seems to be:

  = note: /home/rspeer/code/skipgram-engine/ndarray-linalg/target/debug/deps/libndarray_linalg-6d78f840bdf6a1db.rlib(ndarray_linalg-6d78f840bdf6a1db.1l3vkv62l5dxkj8z.rcgu.o): In function `lapacke::dgetrf':                        
          /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/lapacke-0.2.0/src/lib.rs:6130: undefined reference to `LAPACKE_dgetrf'                                                                                        
          /home/rspeer/code/skipgram-engine/ndarray-linalg/target/debug/deps/libndarray_linalg-6d78f840bdf6a1db.rlib(ndarray_linalg-6d78f840bdf6a1db.1l3vkv62l5dxkj8z.rcgu.o): In function `lapacke::dgetrs':                        
          /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/lapacke-0.2.0/src/lib.rs:6302: undefined reference to `LAPACKE_dgetrs'                                                                                        
          collect2: error: ld returned 1 exit status                                                                                                                                                                                 

and I'm not sure where the responsibility lies between this package and ndarray-linalg.

@IvanUkhov
Copy link
Member

You are disabling all features. Try adding cblas and lapacke. Plus try with the default features.

@rspeer
Copy link
Author

rspeer commented Feb 6, 2020

Adding cblas and lapacke to what?

I would like to emphasize that I didn't write this Cargo.toml, so if the problem is which features ndarray-linalg enables or disables, maybe I should take this bug report there.

@IvanUkhov
Copy link
Member

IvanUkhov commented Feb 6, 2020

I mean that

default-features = false
features = ["static"]

implies that LAPACKE is not build. In any case, if you want to use ndarray-linalg, have you tried to follow these instructions? I don’t get why you need to change its Cargo.toml.

@rspeer
Copy link
Author

rspeer commented Feb 12, 2020

The reason I was changing its Cargo.toml was to attempt to use the new version of openblas-src you mentioned and tell you if it worked.

If I just follow the instructions for ndarray-linalg, that says I should add this to my own Cargo.toml:

ndarray = "0.13"
ndarray-linalg = { version = "0.12", features = ["openblas"] }

And the result I get is this:

error: failed to run custom build command for `openblas-src v0.7.0`

Caused by:
  process didn't exit successfully: `/home/rspeer/code/skipgram-engine/target/debug/build/openblas-src-1acb45e3d7ad0634/build-script-build` (exit code: 101)
--- stdout
Running: `"make" "libs" "netlib" "shared" "BINARY=64" "YES_CBLAS=1" "YES_LAPACKE=1" "-j12"`

--- stderr
Makefile.system:47: Makefile.rule: No such file or directory
make: Makefile.prebuild: No such file or directory
make: *** No rule to make target 'Makefile.prebuild'.  Stop.
Makefile.system:195: Makefile.conf: No such file or directory
Makefile.system:1063: Makefile.: No such file or directory
make: *** No rule to make target 'Makefile.'.  Stop.
thread 'main' panicked at 'Failed: `"make" "libs" "netlib" "shared" "BINARY=64" "YES_CBLAS=1" "YES_LAPACKE=1" "-j12"` (exit code: 2)', /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/openblas-src-0.7.0/build.rs:68:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

@IvanUkhov
Copy link
Member

IvanUkhov commented Feb 12, 2020

I have tried to reproduce this in ubuntu:18.04. After starting the container, I did the following:

apt-get update
apt-get install -y build-essential curl gfortran git vim
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
export PATH="$PATH:$HOME/.cargo/bin"
export USER=root
cargo new foo
cd foo/
vim Cargo.toml # See below
vim src/main.rs # See below
cargo run

Cargo.toml:

[package]
name = "foo"
version = "0.1.0"
authors = ["root"]
edition = "2018"

[dependencies]
ndarray = "0.13"
ndarray-linalg = "0.12"
openblas-src = "0.7"

main.rs (from ndarray-linalg’s examples):

extern crate ndarray;
extern crate ndarray_linalg;
extern crate openblas_src;

use ndarray::*;
use ndarray_linalg::*;

// Solve `Ax=b`
fn solve() -> Result<(), error::LinalgError> {
    let a: Array2<f64> = random((3, 3));
    let b: Array1<f64> = random(3);
    let _x = a.solve(&b)?;
    Ok(())
}

// Solve `Ax=b` for many b with fixed A
fn factorize() -> Result<(), error::LinalgError> {
    let a: Array2<f64> = random((3, 3));
    let f = a.factorize_into()?; // LU factorize A (A is consumed)
    for _ in 0..10 {
        let b: Array1<f64> = random(3);
        let _x = f.solve_into(b)?; // solve Ax=b using factorized L, U
    }
    Ok(())
}

fn main() {
    solve().unwrap();
    factorize().unwrap();
}

It seems to work. Do you have a similar setup?

@rspeer
Copy link
Author

rspeer commented Feb 13, 2020

I made a project where I ran exactly what you said, and I got the same error about a Makefile.

@IvanUkhov
Copy link
Member

Did you also run the commands shown on top, including apt-get install build-essential?

@rspeer
Copy link
Author

rspeer commented Feb 14, 2020

Yes, and build-essential was already installed anyway.

@rspeer
Copy link
Author

rspeer commented Feb 14, 2020

I didn't run export USER=root and I kinda don't know what you're up to there.

@IvanUkhov
Copy link
Member

Hmm, mystery. Yeah, that USER=root is not necessary. It’s just that the Docker image doesn’t have it set by default, and Cargo didn’t like it.

My another suggestion is to run the Docker image, ensure that it works, and then compare the packages installed there with the ones you have.

@IvanUkhov
Copy link
Member

IvanUkhov commented Feb 14, 2020

Another guess is that something is wrong with the unpacked package. All the errors are about missing make-related files. Take a look at what you have in the following folder:

/home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/openblas-src-0.7.0

You should see source and then source_<platform>. Run ls in both and post here.

@rspeer
Copy link
Author

rspeer commented Feb 14, 2020

$ ls /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/openblas-src-0.7.0/source
appveyor.yml         CONTRIBUTORS.md                  lapack
BACKERS.md           cpuid_alpha.c                    lapack-netlib
cblas.h              cpuid_arm64.c                    LICENSE
c_check              cpuid_arm.c                      Makefile
Changelog.txt        cpuid.h                          Makefile.alpha
cmake                cpuid_ia64.c                     Makefile.arm
CMakeLists.txt       cpuid_mips64.c                   Makefile.arm64
common_alpha.h       cpuid_mips.c                     Makefile.generic
common_arm64.h       cpuid_power.c                    Makefile.ia64
common_arm.h         cpuid.S                          Makefile.install
common_c.h           cpuid_sparc.c                    Makefile.mips
common_d.h           cpuid_x86.c                      Makefile.mips64
common.h             cpuid_zarch.c                    Makefile.power
common_ia64.h        ctest1.c                         Makefile.prebuild
common_interface.h   ctest2.c                         Makefile.rule
common_lapack.h      ctest.c                          Makefile.sparc
common_level1.h      driver                           Makefile.system
common_level2.h      exports                          Makefile.tail
common_level3.h      f_check                          Makefile.x86
common_linux.h       ftest2.f                         Makefile.x86_64
common_macro.h       ftest3.f                         Makefile.zarch
common_mips64.h      ftest.f                          make.inc
common_mips.h        gen_config_h.c                   openblas_config_template.h
common_param.h       getarch_2nd.c                    openblas.pc.in
common_power.h       getarch.c                        param.h
common_q.h           GotoBLAS_00License.txt           quickbuild.32bit
common_reference.h   GotoBLAS_01Readme.txt            quickbuild.64bit
common_s.h           GotoBLAS_02QuickInstall.txt      quickbuild.win32
common_sparc.h       GotoBLAS_03FAQ.txt               quickbuild.win64
common_stackalloc.h  GotoBLAS_04FAQ.txt               README.md
common_thread.h      GotoBLAS_05LargePage.txt         relapack
common_x86_64.h      GotoBLAS_06WeirdPerformance.txt  segfaults.patch
common_x86.h         interface                        symcopy.h
common_x.h           kernel                           TargetList.txt
common_zarch.h       l1param.h                        USAGE.md
common_z.h           l2param.h                        version.h

$ ls /home/rspeer/.cargo/registry/src/github.heygears.com-1ecc6299db9ec823/openblas-src-0.7.0/source_x86_64-unknown-linux-gnu/
BACKERS.md           common_thread.h  GotoBLAS_00License.txt           Makefile.install
c_check              common_x86.h     GotoBLAS_01Readme.txt            Makefile.mips
Changelog.txt        common_x.h       GotoBLAS_02QuickInstall.txt      Makefile.power
common_alpha.h       common_zarch.h   GotoBLAS_03FAQ.txt               Makefile.system
common_arm64.h       common_z.h       GotoBLAS_04FAQ.txt               Makefile.tail
common_c.h           cpuid_arm.c      GotoBLAS_06WeirdPerformance.txt  Makefile.x86
common.h             cpuid_ia64.c     kernel                           Makefile.x86_64
common_ia64.h        cpuid_mips64.c   l2param.h                        make.inc
common_level2.h      cpuid.S          lapack                           openblas.pc.in
common_level3.h      cpuid_sparc.c    lapack-netlib                    param.h
common_macro.h       cpuid_zarch.c    LICENSE                          quickbuild.64bit
common_param.h       driver           Makefile                         quickbuild.win64
common_q.h           ftest2.f         Makefile.arm                     README.md
common_reference.h   ftest3.f         Makefile.arm64                   relapack
common_sparc.h       gen_config_h.c   Makefile.generic                 segfaults.patch
common_stackalloc.h  getarch_2nd.c    Makefile.ia64                    USAGE.md

@IvanUkhov
Copy link
Member

Yeah, that’s why; the files are not there. Please try to remove the whole openblas-src-0.7.0. Let’s see what happens if Cargo tries to do it again.

@rhysnewell
Copy link

Hi all,

I was having a pretty similar issue. Removing openblas-src from the cargo cache and src and then rebuilding fixed the issue for me on my local machines and remote servers. However, openblas-src still won't build on Travis CI even after clearing the cache there.

I've compared my .travis.yml to the openblas-src .travsi.yml and pretty much everything is the same.

Cheers,
Rhys

@IvanUkhov
Copy link
Member

I think it might be related to #31. The way the source code of OpenBLAS is handled at the moment is not ideal.

@IvanUkhov
Copy link
Member

Please give a try to version 0.9.0. It might reduce confusion with build directories. See also the new feature cache in the readme.

# for free to join this conversation on GitHub. Already have an account? # to comment
Development

No branches or pull requests

3 participants