allow escaping with --ignore-osdependencies

[fgeorgatos]

Hi,

the point of this is to ensure that there is no strict need to rewrite an .eb file when the sysadmin knows that the
osdependencies are somehow available. This may be the case with items like tcsh, zlib, devel-libs etc.

This option is practically a must for people trying easybuild in a non-redhat environment!

Fotis

[boegel]

Worthwhile and should not be hard to implement.

[fgeorgatos]

Hi there,

I've been following the thread on zlib and it just proves once more how messy this osdep business can be.

Request to ignore osdependencies (see #102), or to support them better, is a sensible workaround?

OK, I tried to play a bit with it, to add the --ignore-osdependencies parameter, but was lost a little bit in the source:

• easybuild/build.py: is the obvious place to scan for the parameter and set a python var about it
• easybuild/framework/easyconfig.py: where def validate() calls def validate_os_deps(); here it should take effect

I just haven't understood too well the logic of the code, to pass down the variable to easyconfig.py;
my suspicion is that there must be some other "better" way to prevent validate_os_deps() to be used!

Your offers?

tia.

[boegel]

There's a few things you can do.

You can add an extra named parameter to EasyConfig's __init__ that's enabled by default, and that you can disable when --ignore-osdependencies is used, e.g. check_osdeps=True.
You can then pass that to validate (define the same named variable there), and then use that to check whether validate_os_deps needs to be called.

Another alternative is to create a function in easyconfig.py that let's you change the list of os dependencies, e.g. set_osdeps(osdeps_list); that way you can just empty the list which yield the same result.

We don't really have rules on what's better, that's up to the guy who implements it. As long as it's not too crazy, we'll merge it. ;-)

[boegel]

This is an important issue for anyone using EasyBuild on non-RedHat systems, so let's bump the milestone to v1.1.

[fgeorgatos]

We never managed to get to really discuss this during the hackathon,
yet it is recognized as important one for cross-platform compatibility.

Since the interpretation of osdependencies is very system-specific,
I think it would be ideal if an external tool is being called for it
(eg. on the mac you could have macports/homebrew, while freebsd is with ports etc).

The simplest interface might be that you call a tool and it returns true
IFF all of its parameters are available as installed packages.

In that case, the answer to this thread would be (the fictional option is installtool):

installtool=/bin/true

and then, you have good orthogonality and need not increase the SLOC by
adding --ignore-osdependencies, which can also be served by --strict=ignore !

Fotis

[JensTimmerman]

I think our current fallback tool is which
This works fine for binaries, but not for libraries, maybe we could even fall deeper down to locate (however the locate database is not created by default on OS X iirc)

And yes, it would be amazing if someone wrote a completey cross platform is_installed tool, and if we allow for
configuring this in easybuild. But writing this ourselves is currently not on our roadmap. (maybe hashdist might ever provide something like this)

But yes, your suggested option would be a great workaround for now for people who know what they are doing.

[boegel]

We do fall back on locate already in v1.0.

It will first try rpm or dpkg (whatever is available), then which, and if needed also locate.

However, using locate for libraries or header files doesn't make much sense, because it doesn't tell you whether they're included in the library (-L) / header file (-I) search path...

[fgeorgatos]

Hi there,

On Mon, Dec 3, 2012 at 11:20 AM, Kenneth Hoste notifications@github.heygears.comwrote:

It will first try rpm or dpkg (whatever is available), then which, and if
needed also locate.

Avoid trying to bring the solution as part of easybuild because that's
going to be a mess:
look eg. the packaging of boost libraries across distros... AFAIR, it was a
horror story.

More generally, outsource the problem because then, we MAY also deliver a
hook
to provide eg. tcsh via a module instead (which is a really smart choice
actually!).

cheers,
F

[boegel]

The problem is that there's no tool out there that solves this cross-platform, imho.

We need a more general solution for this, but up until now, I don't know how to tackle this. Mac OS X doesn't have a package manager to talk to, so we might as well go for a more generic solution (e.g. actually try to use commands, header files, libraries, etc.).

Where possible, we should use explicit dependencies, but sometimes it's just not an option.

[JensTimmerman]

Fotis, an update on this?
I thought you had already mentioned you had a way of doing this?

[fgeorgatos]

Basically, I do "--strict=ignore" which does the job fine for me,
so, I've lowered the priority for this one.

More generally, for this particular issue, I had this fancy idea at some
point:

Use decorators on functions to make a codeblock "@escapable";
ie., let's not pollute the "eb --help" parameter space with all possible
small quirks and bits of tuning we need, and provide instead something like:
--escape=_os_dependency_check OR --escape=sanity_check_step

I think that would allow to experiment for more with the code without
putting excessive pressure to extend it with features and command line
switches...
...anyway, as said, --ignore-osdependencies does have its short-term
solution,
so this issue can be discussed in more detail in a face to face meeting.

On Tue, Feb 19, 2013 at 11:33 AM, Jens Timmerman
notifications@github.heygears.comwrote:

Fotis, an update on this?
I thought you had already mentioned you had a way of doing this?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/102#issuecomment-13766090.

[fgeorgatos]

Jens wrote:

it would be amazing if someone wrote a completey cross platform is_installed tool,

another belayed bit of info, on this thread: (if it fits elsewhere, notify)

This project seems to care for the mappings of packages across distributions:
http://enricozini.org/2011/debian/distromatch/
Perhaps there is a solution to osdependencies by either using that
or, other software projects that depend on it.

[boegel]

distromatch looks like a very promising solution for the problem at hand, because in some cases we won't be able to get rid of all the osdependencies...

[fgeorgatos]

addressed via #799