Merge pull request #1928 from easybuilders/4.1.x

release EasyBuild v4.1.1
easybuilders · Jan 16, 2020 · 8b6852c · 8b6852c
2 parents 4f75cd5 + e9784f5
commit 8b6852c
Show file tree

Hide file tree

Showing 242 changed files with 779 additions and 406 deletions.
diff --git a/.github/workflows/unit_tests.yml b/.github/workflows/unit_tests.yml
@@ -42,7 +42,10 @@ jobs:
 
     - name: install OS & Python packages
       run: |
-        sudo apt-get update
+        # apt-get update is disabled for now, because it was failing in GitHub since Fri Dec 13th 2019,
+        # due to 'Conflicting distribution' issue
+        # sudo apt-get update
+
         # for modules tool
         sudo apt-get install lua5.2 liblua5.2-dev lua-filesystem lua-posix tcl tcl-dev
         # fix for lua-posix packaging issue, see https://bugs.launchpad.net/ubuntu/+source/lua-posix/+bug/1752082

diff --git a/RELEASE_NOTES b/RELEASE_NOTES
@@ -3,7 +3,59 @@ For more detailed information, please see the git log.
 
 These release notes can also be consulted at http://easybuild.readthedocs.org/en/latest/Release_notes.html.
 
-The latest version of easybuild-easyblocks provides 211 software-specific easyblocks and 36 generic easyblocks.
+The latest version of easybuild-easyblocks provides 212 software-specific easyblocks and 36 generic easyblocks.
+
+v4.1.1 (January 16th 2020)
+--------------------------
+
+update/bugfix release
+
+- new software-specific easyblock for cryptography (to fix missing -pthread for all versions) (#1874)
+- minor enhancements, including:
+  - update WPS easyblock for v3.6 & newer (#1315)
+  - update FSL easyblock to support FSL v6.0.2 & newer (#1860)
+  - add setup_cmake_env function in CMakeMake easyblock which can be leveraged in other easyblocks, and use it for OpenFOAM (#1869)
+  - remove obsolete configure options for Python + build with optimizations/LTO enabled for recent Python versions (#1876)
+  - update WPS easyblock for recent versions: set $WRF_DIR to point to location of WRF installation (#1886)
+  - make sure $LIBLAPACK_MT is set before using it in ESMF easyblock (#1887)
+  - remove useless PATH entries + add PKG_CONFIG_PATH in imkl easyblock (#1900)
+  - enhance tbb easyblock to support building on POWER (#1912)
+  - enhance TensorFlow easyblock to pick up on --cuda-compute-capabilities, and issue a warning if no CUDA compute capabilities are specified (#1913)
+  - add custom easyconfig parameter 'build_type' to generic CMakeMake easyblock (#1915, #1922)
+- various bug fixes, including:
+  - fix for conda packages that rely on particular versions of Python (#1836)
+  - fix path for intel and netCDF lib directories in NCL easyblock (#1862)
+  - fix CUDA 10.1 installation on POWER (#1871)
+  - change Bazel easyblock to prefer using Java dependency rather than included JDK (fix for POWER9) (#1875)
+  - remove optarch warning in GROMACS for Cray toolchains (#1879)
+  - also fix $WM_COMPILE_OPTION in OpenFOAM rc scripts to make debug builds work correctly (#1880)
+  - limit MPI ranks used for running WRF test cases to max. 4 + include contents of rsl.error.0000 output file in case test failed (#1884)
+  - update $PYTHONPATH + add "python -c 'import mrtrix3'" as sanity check command for recent MRtrix versions (#1889)
+  - update sanity check in SAMtools easyblock for version 1.10 (#1890)
+  - make sure $PYTHONNOUSERSITE it set when performing sanity check for (bundles of) Python package(s) (#1891)
+  - fix install dir subdir for WPS v4.0+ that is considered for $PATH and $LD_LIBRARY_PATH (#1895)
+  - impi: don't rebuild libfabric if the source code is not present (#1896)
+  - also copy component patches to self.cfg in Bundle generic easyblock (#1897)
+  - skip patch step in Bundle generic easyblock (per-component patches are still applied) (#1898)
+  - derive easyblock for iccifort only from icc easyblock (not ifort), to avoid adding include subdir to $CPATH (#1899)
+  - add 'export LANG=C' to preinstallopts instead of install_script path in CUDA easyblock (#1902)
+  - stop setting updating $CPATH and $LIBRARY_PATH for GCC and GCCcore, not required (#1903)
+  - remove $LIBRARY_PATH entries in iccifort easyblock, already known to icc et al (#1904)
+  - use major/minor version of Python command being used if req_py_majver/req_py_minver are not specified (#1907)
+  - define $EB_PYTHON in module for EasyBuild installation, to make sure correct Python version is used at runtime (#1908)
+  - fix Python easyblock to allow configuring build of Python v3.8 (Setup.dist script was renamed to Setup) (#1909)
+  - fix netCDF easyblock for version 4.4.0 (#1911)
+  - correct comment about when we set RUNPARALLEL in HDF5 easyblock (#1914)
+  - do not sanity check on MATLAB compiler, since it requires a separate license (#1916)
+  - build HDF5 without MPI C++ extension to avoid breaking linkage for C software that requires HDF5 (#1918, #1919)
+  - override set_pylibdirs method in VersionIndependentPythonPackage to hard set self.pylibdir to 'lib' (#1924)
+- other changes:
+  - increase timeout for interactive installation command in CUDA easyblock to 1000 sec. (#1878)
+  - disable running of 'sudo apt-get update' in GitHub CI config, since it's failing (and we don't really need it) (#1882)
+  - stop requiring Python dep for SWIG, just configure with --without-python if Python is not a dependency (#1894)
+  - update copyright statements for 2020 (#1905)
+  - make Hound CI code style checker ignore "Black would make changes" produced by flake8-black (#1923)
+
 
 v4.1.0 (December 4th 2019)
 --------------------------
@@ -28,7 +80,7 @@ update/bugfix release
   - fix CUDA header paths for TensorFlow versions < 1.14 (#1854)
   - handle incorrect regex better in generic CmdCp easyblock (#1861)
   - add missing docstrings in cmakeninja easyblock (#1867)
-other changes:
+- other changes:
   - add GitHub Actions workflow to run easybuild-easyblocks test suite (#1844)
 
 

diff --git a/easybuild/__init__.py b/easybuild/__init__.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/__init__.py b/easybuild/easyblocks/__init__.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
@@ -43,7 +43,7 @@
 # recent setuptools versions will *TRANSFORM* something like 'X.Y.Zdev' into 'X.Y.Z.dev0', with a warning like
 #   UserWarning: Normalizing '2.4.0dev' to '2.4.0.dev0'
 # This causes problems further up the dependency chain...
-VERSION = LooseVersion('4.1.0')
+VERSION = LooseVersion('4.1.1')
 UNKNOWN = 'UNKNOWN'
 
 

diff --git a/easybuild/easyblocks/a/abaqus.py b/easybuild/easyblocks/a/abaqus.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/acml.py b/easybuild/easyblocks/a/acml.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/adf.py b/easybuild/easyblocks/a/adf.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2016-2019 Ghent University
+# Copyright 2016-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/advisor.py b/easybuild/easyblocks/a/advisor.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/aladin.py b/easybuild/easyblocks/a/aladin.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/allinea.py b/easybuild/easyblocks/a/allinea.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2013-2019 Ghent University
+# Copyright 2013-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/amber.py b/easybuild/easyblocks/a/amber.py
@@ -1,6 +1,6 @@
 ##
-# Copyright 2009-2019 Ghent University
-# Copyright 2015-2019 Stanford University
+# Copyright 2009-2020 Ghent University
+# Copyright 2015-2020 Stanford University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/anaconda.py b/easybuild/easyblocks/a/anaconda.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/ansys.py b/easybuild/easyblocks/a/ansys.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/ant.py b/easybuild/easyblocks/a/ant.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/arb.py b/easybuild/easyblocks/a/arb.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/armadillo.py b/easybuild/easyblocks/a/armadillo.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/a/atlas.py b/easybuild/easyblocks/a/atlas.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bamtools.py b/easybuild/easyblocks/b/bamtools.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 The Cyprus Institute
+# Copyright 2009-2020 The Cyprus Institute
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bazel.py b/easybuild/easyblocks/b/bazel.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
@@ -96,7 +96,15 @@ def configure_step(self):
             self.log.info("Not patching Bazel build scripts, installation prefix for binutils/GCC not found")
 
         # enable building in parallel
-        env.setvar('EXTRA_BAZEL_ARGS', '--jobs=%d' % self.cfg['parallel'])
+        bazel_args = '--jobs=%d' % self.cfg['parallel']
+
+        # Bazel provides a JDK by itself for some architectures
+        # We want to enforce it using the JDK we provided via modules
+        # This is required for Power where Bazel does not have a JDK, but requires it for building itself
+        # See https://github.com/bazelbuild/bazel/issues/10377
+        bazel_args += ' --host_javabase=@local_jdk//:jdk'
+
+        env.setvar('EXTRA_BAZEL_ARGS', bazel_args)
 
     def build_step(self):
         """Custom build procedure for Bazel."""

diff --git a/easybuild/easyblocks/b/binutils.py b/easybuild/easyblocks/b/binutils.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bioconductor.py b/easybuild/easyblocks/b/bioconductor.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bisearch.py b/easybuild/easyblocks/b/bisearch.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/blacs.py b/easybuild/easyblocks/b/blacs.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/blat.py b/easybuild/easyblocks/b/blat.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 the Cyprus Institute
+# Copyright 2009-2020 the Cyprus Institute
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/blender.py b/easybuild/easyblocks/b/blender.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/boost.py b/easybuild/easyblocks/b/boost.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bowtie.py b/easybuild/easyblocks/b/bowtie.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2013-2019 Ghent University
+# Copyright 2013-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/b/bzip2.py b/easybuild/easyblocks/b/bzip2.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/cblas.py b/easybuild/easyblocks/c/cblas.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2013-2019 Ghent University
+# Copyright 2013-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/cgal.py b/easybuild/easyblocks/c/cgal.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/charmm.py b/easybuild/easyblocks/c/charmm.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2013-2019 Ghent University
+# Copyright 2013-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/clang.py b/easybuild/easyblocks/c/clang.py
@@ -1,6 +1,6 @@
 ##
 # Copyright 2013 Dmitri Gribenko
-# Copyright 2013-2019 Ghent University
+# Copyright 2013-2020 Ghent University
 #
 # This file is triple-licensed under GPLv2 (see below), MIT, and
 # BSD three-clause licenses.

diff --git a/easybuild/easyblocks/c/comsol.py b/easybuild/easyblocks/c/comsol.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/cp2k.py b/easybuild/easyblocks/c/cp2k.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/cplex.py b/easybuild/easyblocks/c/cplex.py
@@ -1,5 +1,5 @@
 ##
-# Copyright 2009-2019 Ghent University
+# Copyright 2009-2020 Ghent University
 #
 # This file is part of EasyBuild,
 # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),

diff --git a/easybuild/easyblocks/c/cryptography.py b/easybuild/easyblocks/c/cryptography.py
@@ -0,0 +1,57 @@
+##
+# Copyright 2017-2019 Ghent University
+#
+# This file is part of EasyBuild,
+# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
+# with support of Ghent University (http://ugent.be/hpc),
+# the Flemish Supercomputer Centre (VSC) (https://www.vscentrum.be),
+# Flemish Research Foundation (FWO) (http://www.fwo.be/en)
+# and the Department of Economy, Science and Innovation (EWI) (http://www.ewi-vlaanderen.be/en).
+#
+# https://github.com/easybuilders/easybuild
+#
+# EasyBuild is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation v2.
+#
+# EasyBuild is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with EasyBuild.  If not, see <http://www.gnu.org/licenses/>.
+##
+"""
+EasyBuild support for building and installing cryptography, implemented as an easyblock
+
+@author: Alexander Grund
+"""
+from distutils.version import LooseVersion
+
+from easybuild.easyblocks.generic.pythonpackage import PythonPackage
+from easybuild.tools.run import run_cmd
+
+
+class EB_cryptography(PythonPackage):
+    """Support for building/installing cryptography."""
+
+    def __init__(self, *args, **kwargs):
+        """Initialize cryptography easyblock."""
+        super(EB_cryptography, self).__init__(*args, **kwargs)
+
+        # cryptography compiles a library using pthreads but does not link against it
+        # which causes 'undefined symbol: pthread_atfork'
+        # see https://github.com/easybuilders/easybuild-easyconfigs/issues/9446
+        # and upstream: https://github.com/pyca/cryptography/issues/5084
+        self.cfg['preinstallopts'] += 'CFLAGS="$CFLAGS -pthread"'
+        self.log.info("Adding -pthread to preinstallopts of cryptography. Final value: %s", self.cfg['preinstallopts'])
+
+    def sanity_check_step(self, *args, **kwargs):
+        """Custom sanity check"""
+        success, fail_msg = super(EB_cryptography, self).sanity_check_step(*args, **kwargs)
+        if success:
+            # Check module added in v0.7 leading to issue #9446 (see above)
+            if LooseVersion(self.version) >= LooseVersion("0.7"):
+                run_cmd("python -c 'from cryptography.hazmat.bindings.openssl import binding'")
+        return success, fail_msg