Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

onnxruntime-extensions support #88

Open
rmarrugat opened this issue Jul 28, 2021 · 0 comments
Open

onnxruntime-extensions support #88

rmarrugat opened this issue Jul 28, 2021 · 0 comments

Comments

@rmarrugat
Copy link

Following the onnxruntime-extensions build script, I am able to use some community custom operations as sentencepiece_tokenizer.

In order to register those operations, I must call first the EnableOrtCustomOps functions, present at the main c and c++ APIs.
This call generates an OrtSessionOptions object which is used to invoke a new session containing the custom operations.

The current rust crate is missing those 2 calls: EnableOrtCustomOps and a SessionBuilder which can be fed with a OrtSessionOptions object.

For visibility purposes, the code below shows the implementation of the first call:

fn enable_custom_operations() -> *mut OrtSessionOptions {
    let g_ort = unsafe { OrtGetApiBase().as_ref().unwrap().GetApi.unwrap()(ORT_API_VERSION) };
    assert_ne!(g_ort, std::ptr::null_mut());
    let mut session_options_ptr: *mut OrtSessionOptions = std::ptr::null_mut();
    let status = unsafe { g_ort.as_ref().unwrap().CreateSessionOptions.unwrap()(&mut session_options_ptr) };
    check_status(g_ort, status).unwrap();
    let status = unsafe { g_ort.as_ref().unwrap().EnableOrtCustomOps.unwrap()(session_options_ptr) };
    check_status(g_ort, status).unwrap();
    session_options_ptr
}

And the next one the session building process:

    let session_options_ptr = enable_custom_operations();
    let model_bytes = std::fs::read("model.onnx").unwrap();
    let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

    let mut session = SessionBuilder {
        env: &environment,
        session_options_ptr,
        allocator: AllocatorType::Arena,
        memory_type: MemType::Default,
    }.with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_memory(model_bytes)?;
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant