Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

libloading support #63

Open
chertov opened this issue Feb 26, 2021 · 1 comment
Open

libloading support #63

chertov opened this issue Feb 26, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@chertov
Copy link

chertov commented Feb 26, 2021

Hi! I'm trying to use bindgen dynamic libloading for onnx shared library runtime.
I want to be able load and unload onnx shared library at runtime.
my current work is here chertov@b94870f

Can we make onnx shared library features check before loading?
For example we can check the existence of symbols like this

OrtSessionOptionsAppendExecutionProvider_CoreML
OrtSessionOptionsAppendExecutionProvider_CUDA
OrtSessionOptionsAppendExecutionProvider_Nnapi
etc..

in the ONNX runtime shared library and wrap this information with methods like "is_cuda_support()"

@chertov
Copy link
Author

chertov commented Feb 26, 2021

Simple CoreML demo in this branch
https://github.com/chertov/onnxruntime-rs/tree/libloading
i removed global variables g_ort(), G_ENV
running Session with CoreML https://github.com/chertov/onnxruntime-rs/blob/libloading/onnxruntime/src/session.rs#L117
but it falls with error:

OrtSessionOptionsAppendExecutionProvider_CoreML is ok = true
OrtSessionOptionsAppendExecutionProvider_CUDA is ok = false
OrtSessionOptionsAppendExecutionProvider_Nnapi is ok = true
Feb 26 19:14:29.957 DEBUG new{api=OrtApi name="test" log_level=Info}: onnxruntime::environment: Environment created. env_ptr="0x7feda0f84500"
Feb 26 19:14:29.958 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="226" function="operator()" logid=""}: onnxruntime::onnxruntime: "Flush-to-zero and denormal-as-zero are off"
Feb 26 19:14:29.958 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="233" function="ConstructorCommon" logid=""}: onnxruntime::onnxruntime: "Creating and using per session threadpools since use_per_session_threads_ is true"
Feb 26 19:14:29.966 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for CoreML with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for CoreML with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="1107" function="Initialize" logid=""}: onnxruntime::onnxruntime: "Initializing session."
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="1132" function="Initialize" logid=""}: onnxruntime::onnxruntime: "Adding default CPU execution provider."
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.970 DEBUG onnxruntime{category="onnxruntime" file="reshape_fusion.cc" line_number="37" function="ApplyImpl" logid=""}: onnxruntime::onnxruntime: "Total fused reshape node count: 0"
Feb 26 19:14:29.971 DEBUG onnxruntime{category="onnxruntime" file="reshape_fusion.cc" line_number="37" function="ApplyImpl" logid=""}: onnxruntime::onnxruntime: "Total fused reshape node count: 0"
Feb 26 19:14:29.972 DEBUG onnxruntime{category="onnxruntime" file="coreml_execution_provider.cc" line_number="166" function="GetCapability" logid=""}: onnxruntime::onnxruntime: "CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 10 number of nodes in the graph: 65 number of nodes supported by CoreML: 53"
Feb 26 19:14:29.987 DEBUG drop{self=SessionBuilder { env: Environment { env: Mutex { data: EnvironmentSingleton { name: "test", env_ptr: 0x7feda0f84500, api: OrtApi } } }, session_options_ptr: 0x7feda2904460, allocator: Arena, memory_type: Default }}: onnxruntime::session: Dropping the session options.
Error: Failed to create session: Error calling ONNX Runtime C function: Error compiling model compiler error:  Invalid network: Layer name missing.

@nbigaouette nbigaouette added the enhancement New feature or request label Mar 10, 2021
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants