Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Conflict between Flux and ScatterNNLib inside GeometricFlux #169

Closed
ilancoulon opened this issue Apr 1, 2021 · 4 comments · Fixed by #170
Closed

Conflict between Flux and ScatterNNLib inside GeometricFlux #169

ilancoulon opened this issue Apr 1, 2021 · 4 comments · Fixed by #170

Comments

@ilancoulon
Copy link
Contributor

When trying to test GeometricFlux on my computer using Julia 1.6, I get a conflict for the use of gather between Flux and ScatterNNLib. The beginning of the output is given at the end but the following line would be the only interesting one:
WARNING: both ScatterNNlib and Flux export "gather"; uses of it in module GeometricFlux must be qualified.

Maybe this is related to yuehhua/ScatterNNlib.jl#32 but I am not sure. I have the same kind of errors in my own package that uses GeometricFlux. Any idea about that error?

(GeometricFlux) pkg> test
     Testing GeometricFlux
┌ Warning: Could not use exact versions of packages in manifest, re-resolving
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1524
      Status `/tmp/jl_x6mnS2/Project.toml`
  [052768ef] CUDA v2.6.2
  [864edb3b] DataStructures v0.18.9
  [1a297f60] FillArrays v0.11.7
  [587475ba] Flux v0.11.6
  [7e08b658] GeometricFlux v0.7.5 `/mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl`
  [21828b05] GraphMLDatasets v0.1.2
  [3ebe565e] GraphSignals v0.1.13
  [093fc24a] LightGraphs v1.3.5
  [626554b9] MetaGraphs v0.6.7
  [189a3867] Reexport v1.0.0
  [ae029012] Requires v1.1.3
  [b1168b60] ScatterNNlib v0.1.7
  [47aef6b3] SimpleWeightedGraphs v1.1.1
  [e88e6eb3] Zygote v0.6.8
  [700de1a5] ZygoteRules v0.2.1
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [9a3f8284] Random `@stdlib/Random`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_x6mnS2/Manifest.toml`
  [621f4979] AbstractFFTs v1.0.1
  [1520ce14] AbstractTrees v0.3.4
  [79e6a3ab] Adapt v3.2.0
  [ec485272] ArnoldiMethod v0.1.0
  [ab4f0b2a] BFloat16s v0.1.0
  [b99e7846] BinaryProvider v0.5.10
  [a74b3585] Blosc v0.7.0
  [e1450e63] BufferedStreams v1.0.0
  [fa961155] CEnum v0.4.1
  [336ed68f] CSV v0.8.4
  [052768ef] CUDA v2.6.2
  [082447d4] ChainRules v0.7.57
  [d360d2e6] ChainRulesCore v0.9.36
  [944b1d66] CodecZlib v0.7.0
  [3da002f7] ColorTypes v0.10.12
  [5ae59095] Colors v0.12.6
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v3.25.0
  [8f4d0f93] Conda v1.5.1
  [9a962f9c] DataAPI v1.6.0
  [124859b0] DataDeps v0.7.7
  [864edb3b] DataStructures v0.18.9
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.0.3
  [b552c78f] DiffRules v1.0.2
  [e2ba6199] ExprTools v0.1.3
  [1a297f60] FillArrays v0.11.7
  [53c48c17] FixedPointNumbers v0.8.4
  [587475ba] Flux v0.11.6
  [f6369f11] ForwardDiff v0.10.17
  [d9f16b24] Functors v0.1.0
  [0c68f7d7] GPUArrays v6.2.0
  [61eb1bfa] GPUCompiler v0.10.0
  [7e08b658] GeometricFlux v0.7.5 `/mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl`
  [a1251efa] GraphLaplacians v0.1.2
  [21828b05] GraphMLDatasets v0.1.2
  [3ebe565e] GraphSignals v0.1.13
  [f67ccb44] HDF5 v0.14.3
  [cd3eb016] HTTP v0.9.5
  [7869d1d1] IRTools v0.4.2
  [d25df0c9] Inflate v0.1.2
  [83e8ac13] IniFile v0.5.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [033835bb] JLD2 v0.3.3
  [692b3bcd] JLLWrappers v1.2.0
  [682c06a0] JSON v0.21.1
  [e5e0dc1b] Juno v0.8.4
  [929cbde3] LLVM v3.6.0
  [093fc24a] LightGraphs v1.3.5
  [23992714] MAT v0.9.2
  [1914dd2f] MacroTools v0.5.6
  [739be429] MbedTLS v1.0.3
  [e89f7d12] Media v0.5.0
  [c03570c3] Memoize v0.4.4
  [626554b9] MetaGraphs v0.6.7
  [e1d29d7a] Missings v0.4.5
  [872c559c] NNlib v0.7.17
  [77ba4419] NaNMath v0.3.5
  [bac558e1] OrderedCollections v1.4.0
  [69de0a69] Parsers v1.1.0
  [2dfb63ee] PooledArrays v1.2.1
  [438e738f] PyCall v1.92.2
  [189a3867] Reexport v1.0.0
  [ae029012] Requires v1.1.3
  [b1168b60] ScatterNNlib v0.1.7
  [6c6a2e73] Scratch v1.0.3
  [91c51154] SentinelArrays v1.2.16
  [699a6c99] SimpleTraits v0.9.3
  [47aef6b3] SimpleWeightedGraphs v1.1.1
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v1.3.0
  [90137ffa] StaticArrays v1.1.0
  [2913bbd2] StatsBase v0.33.4
  [3783bdb8] TableTraits v1.0.0
  [bd369af6] Tables v1.4.1
  [a759f4b9] TimerOutputs v0.5.8
  [3bb67fe8] TranscodingStreams v0.9.5
  [5c2747f8] URIs v1.2.0
  [81def892] VersionParsing v1.2.0
  [a5390f91] ZipFile v0.9.3
  [e88e6eb3] Zygote v0.6.8
  [700de1a5] ZygoteRules v0.2.1
  [0b7ba130] Blosc_jll v1.14.3+1
  [0234f1f7] HDF5_jll v1.12.0+1
  [5ced341a] Lz4_jll v1.9.2+2
  [458c3c95] OpenSSL_jll v1.1.1+6
  [efe28fd5] OpenSpecFun_jll v0.5.3+4
  [3161d3a3] Zstd_jll v1.4.8+0
  [0dad84c5] ArgTools `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8bb1440f] DelimitedFiles `@stdlib/DelimitedFiles`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads `@stdlib/Downloads`
  [9fa8497b] Future `@stdlib/Future`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [4af54fe1] LazyArtifacts `@stdlib/LazyArtifacts`
  [b27032c2] LibCURL `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [a63ad114] Mmap `@stdlib/Mmap`
  [ca575930] NetworkOptions `@stdlib/NetworkOptions`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [9abbd945] Profile `@stdlib/Profile`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [1a1011a3] SharedArrays `@stdlib/SharedArrays`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [fa267f1f] TOML `@stdlib/TOML`
  [a4e569a6] Tar `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll `@stdlib/p7zip_jll`
     Testing Running tests...
┌ Warning: CUDA unavailable, not loading CUDA support
└ @ ScatterNNlib ~/.julia/packages/ScatterNNlib/AFYe7/src/ScatterNNlib.jl:47
┌ Warning: CUDA unavailable, not testing GPU support
└ @ Main /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/runtests.jl:41
WARNING: both ScatterNNlib and Flux export "gather"; uses of it in module GeometricFlux must be qualified
WARNING: both LightGraphs and GraphSignals export "laplacian_matrix"; uses of it in module GeometricFlux must be qualified
layer with graph: Error During Test at /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:127
  Got exception outside of a @test
  UndefVarError: gather not defined
  Stacktrace:
    [1] (::GeometricFlux.var"#1#2"{Vector{Int64}})(Δ::Matrix{Float32})
      @ GeometricFlux /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/pool.jl:72
    [2] (::GeometricFlux.var"#4#back#3"{GeometricFlux.var"#1#2"{Vector{Int64}}})(Δ::Matrix{Float32})
      @ GeometricFlux ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
    [3] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/pool.jl:137 [inlined]
    [4] (::typeof(∂(pool)))(Δ::Matrix{Float32})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
    [5] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/layers/msgpass.jl:49 [inlined]
    [6] (::typeof(∂(aggregate_neighbors)))(Δ::Matrix{Float32})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
    [7] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/layers/gn.jl:64 [inlined]
    [8] (::typeof(∂(propagate)))(Δ::Tuple{Nothing, Fill{Float32, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Nothing})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
    [9] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/layers/msgpass.jl:58 [inlined]
   [10] (::typeof(∂(propagate)))(Δ::Tuple{Nothing, Fill{Float32, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
   [11] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/src/layers/conv.jl:237 [inlined]
   [12] (::typeof(∂(λ)))(Δ::Fill{Float32, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
   [13] Pullback
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:141 [inlined]
   [14] (::typeof(∂(λ)))(Δ::Float32)
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface2.jl:0
   [15] (::Zygote.var"#41#42"{typeof(∂(λ))})(Δ::Float32)
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface.jl:41
   [16] gradient(f::Function, args::Matrix{Float32})
      @ Zygote ~/.julia/packages/Zygote/lwmfx/src/compiler/interface.jl:59
   [17] macro expansion
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:141 [inlined]
   [18] macro expansion
      @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
   [19] macro expansion
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:128 [inlined]
   [20] macro expansion
      @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
   [21] macro expansion
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:125 [inlined]
   [22] macro expansion
      @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
   [23] top-level scope
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/layers/conv.jl:18
   [24] include(fname::String)
      @ Base.MainInclude ./client.jl:444
   [25] macro expansion
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/runtests.jl:46 [inlined]
   [26] macro expansion
      @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
   [27] top-level scope
      @ /mnt/c/Users/ilanc/Dropbox/Stage-MTL/julia/GeometricFlux.jl/test/runtests.jl:45
   [28] include(fname::String)
      @ Base.MainInclude ./client.jl:444
   [29] top-level scope
      @ none:6
   [30] eval
      @ ./boot.jl:360 [inlined]
   [31] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:261
   [32] _start()
      @ Base ./client.jl:485
@ilancoulon
Copy link
Contributor Author

It actually also happens with Julia 1.5: https://github.com/FluxML/GeometricFlux.jl/pull/168/checks?check_run_id=2247858662#step:6:156

I don't really know what should be done here... Should we change the gather name in ScatterNNLib? Or are both functions the same and we could then delete it in ScatterNNLib?

@yuehhua
Copy link
Member

yuehhua commented Apr 2, 2021

Yes, I am migrating scatter and gather to NNlib.jl, which is the dependency of Flux.jl.
While I still have to maintain the functionality of GeometricFlux.jl, I cannot purge ScatterNNlib.jl from GeometricFlux.jl.
Thus, an embarrassing situation happens............

The old functionality of ScatterNNlib.jl should be OK. Maybe I put a namespace of ScatterNNlib in GeometricFlux would work.

@ilancoulon
Copy link
Contributor Author

I'll do a PR with the namespaces of ScatterNNlib if that's ok!

@ilancoulon ilancoulon mentioned this issue Apr 2, 2021
Merged
@yuehhua yuehhua mentioned this issue Apr 2, 2021
Merged
@yuehhua
Copy link
Member

yuehhua commented Apr 2, 2021

Sorry, I could not push to your branch, so I create another PR for fixing advanced bugs.

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ScatterNNlib namespace for gather and scatter methods ilancoulon/GeometricFlux.jl
2 participants
@ilancoulon @yuehhua