Static compiling

[adrianlzt]

I'm trying to make a static binary using go-ceph library.

I have tried passing the static flag to cgo:

go build -ldflags "-extldflags '-static'" cinder_example.go
# command-line-arguments
/home/adrian/.gvm/gos/go1.6.2/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: cannot find -lrados
/usr/bin/ld: cannot find -lrados
/usr/bin/ld: cannot find -lrados
collect2: error: ld returned 1 exit status

Or without cgo:

CGO_ENABLED=0 go build -ldflags '-s' cinder_example.go
# command-line-arguments
./cinder_example.go:8: undefined: rados.NewConn

Is this possible?
Thanks

[dotnwat]

Hmm, good question. I've never tried to build go-ceph statically, or any software that statically links against librados. As long as librados plays nicely as a statically linked library then I suspect it is possible. Do you have any links on how to build static Go binaries? I can play around with it a little bit. In the first error you posted cannot find -lrados seems like a problem with paths (you need to tell cgo where librados is located if it isn't installed in a standard location).

[adrianlzt]

I have found some refs, but little bit outdated I think:

http://blog.hashbangbash.com/2014/04/linking-golang-statically/
golang/go#9344 (comment)
https://groups.google.com/forum/#!searchin/golang-dev/CGO_ENABLED$3D0/golang-dev/JKM7UaKRT6g/zvqesRjURy4J
golang/go#12808

[adrianlzt]

Maybe this...
https://github.com/shadowmint/go-static-linking

[adrianlzt]

Any news about this?

[dotnwat]

@adrianlzt sorry no I haven't had time to look into this. thanks for those links. have you hacked on this? i'd be happy to review and merge changes, but my bandwidth is a bit limited right now.

[adrianlzt]

I'll try, but first I need to understant how all this stuff works (linking, cgo, etc).

[palmamartin]

The error /usr/bin/ld: cannot find -lrados means that the linker ld cannot find the statically compiled librados (librados.a). By installing the native RADOS library and development headers you only obtain the dynamically linked one (librados.so).

[yaozongyou]

It is possible to build a static binary using go-ceph, but we have some obstacles to overcome.

Firstly, the librados library only has shared library by default, we cannot build statically unless we have static librados library.

Secondly, on centos, some libraries librados depends on only have shared library on system, we need to build them ourself.

So the steps for building a static library as follows:

1. Build static librados library from ceph source code, we need make some changes to the source code, here is the diff:

# git diff  src/librados/CMakeLists.txt
diff --git a/src/librados/CMakeLists.txt b/src/librados/CMakeLists.txt
index f15aa7d..404fc5b 100644
--- a/src/librados/CMakeLists.txt
+++ b/src/librados/CMakeLists.txt
@@ -27,6 +27,9 @@ else(ENABLE_SHARED)
     $<TARGET_OBJECTS:librados_api_obj>
     $<TARGET_OBJECTS:librados_objs>)
 endif(ENABLE_SHARED)
+add_library(rados-static STATIC
+  $<TARGET_OBJECTS:librados_api_obj>
+  $<TARGET_OBJECTS:librados_objs>)
 target_link_libraries(librados PRIVATE
   osdc ceph-common cls_lock_client
   ${BLKID_LIBRARIES} ${CRYPTO_LIBS} ${EXTRALIBS})

# git diff src/CMakeLists.txt
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 3868956..6b809d2 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -711,6 +711,7 @@ if(WITH_STATIC_LIBSTDCXX)
 endif()
 add_library(ceph-common SHARED ${ceph_common_objs})
+add_library(ceph-common-static STATIC ${ceph_common_objs})
 target_link_libraries(ceph-common ${ceph_common_deps})
 # appease dpkg-shlibdeps
 set_target_properties(ceph-common PROPERTIES

And then, we can build static librados library using the following command:

make rados-static 
make ceph-common-static

2. Make changes to go-ceph for using static rados library, here is the diff:

--- a/rados/conn.go
+++ b/rados/conn.go
@@ -1,6 +1,11 @@
 package rados
 
-// #cgo LDFLAGS: -lrados
+// #cgo LDFLAGS: -lz -lnspr4 -lnss3 -lresolv -ldl -lm -lssl3 -lssl -lcrypto
+// #cgo LDFLAGS: -L. -lrados-static -lceph-common-static  -ljson_spirit -lcls_lock_client
+// #cgo LDFLAGS: -L. -lboost_iostreams -lboost_system -lboost_thread -lboost_regex
+// #cgo LDFLAGS: -L. -libverbs -lnl-3 -lnl-route-3
+// #cgo LDFLAGS: -static-libstdc++ -static-libgcc
+// #cgo CPPFLAGS: -Iinclude/
 // #include <stdlib.h>
 // #include <rados/librados.h>

As the diff shows, we need to copy the static libraries to go-ceph/rados directory for cgo to link.

After these two steps, the binary we build does not depend on librados.so to run.

[dotnwat]

@yaozongyou which version of ceph are you looking at? I just peaked at the latest master and there seems to be support for building static versions of librados.

is this something that people are still interested in? @adrianlzt

[yaozongyou]

@noahdesu I have tested on the ceph current master branch. As far as i know, the default ceph rpm repo does not contain librados static development rpm. And i have tried to build ceph using the flag -DENABLE_SHARED=OFF, but in my environment, some errors occur during cmake configure phrase, i did not go deeper for the reason. So I add rados-static and ceph-common-static cmake target by hand.

[smileusd]

I have meet the same problem. I want to build the ceph rbd statically but failed, and then I use the go-ceph to work around, but it is also failed. Looks like the root case is the same: lack some static link libraries in ceph.

[phlogistonjohn]

Hello, I'm automatically closing all issues filed prior to 2018 that have not been commented on recently. If this issue is still relevant to you please reopen the issue and restart the discussion. Thank you for your understanding.