Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

is it possible to access a sge cluster with julia from outside ? #117

Open
samusz opened this issue May 10, 2019 · 4 comments
Open

is it possible to access a sge cluster with julia from outside ? #117

samusz opened this issue May 10, 2019 · 4 comments

Comments

@samusz
Copy link

samusz commented May 10, 2019

The cluster facility I have access to has Julia v0.5 at the moment.

Since I'd rather not have to worry with version compatibility is it possible to use this "cluster power" from another julia process (eg from my laptop) ?
I mostly FFTs calculations (sound analysis).

@vchuravy
Copy link
Member

You have to install the same Julia version on the Cluster. You can change which version of Julia is being started, but it has to be the same one accross the system

@samusz
Copy link
Author

samusz commented May 12, 2019

Is there a way to just use the cluster to send requests from my laptop's julia directly ? (in R there's a qsub package that does that : https://cran.r-project.org/web/packages/qsub/index.html, not sure if it should work with RCall, I had no success)

@vchuravy
Copy link
Member

I don't know hoe the R package does that, but as long as you have qsub installed we should be able to do something similar.

The only caveat is that we currently expect a shared file-system, but it might be feasible to relax that. Contributions are always welcomed!

@samusz
Copy link
Author

samusz commented May 16, 2019

I believe I have found it in the Distributed.jl package :
(from the doc https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed-Computing-1 )
addprocs(manager::ClusterManager; kwargs...) -> List of process identifiers
Launches worker processes via the specified cluster manager.
or may be
addprocs(machines; tunnel=false, sshflags=``, max_parallel=10, kwargs...) -> List of process identifiers
Add processes on remote machines via SSH. Requires julia to be installed in the same location on each node, or to be available via a shared file system.

I'll try to make it work. A bit confused now as to why there is 2 functions.
Thanks again for your help @vchuravy

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants