Skip to content

[js][feature tracking] JavaScript library consume #14529

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Open
fs-eire opened this issue Feb 1, 2023 · 10 comments
Open

[js][feature tracking] JavaScript library consume #14529

fs-eire opened this issue Feb 1, 2023 · 10 comments
Labels
documentation improvements or additions to documentation; typically submitted using template feature request request for unsupported feature or enhancement platform:web issues related to ONNX Runtime web; typically submitted using template

Comments

@fs-eire
Copy link
Contributor

fs-eire commented Feb 1, 2023 •
edited
Loading

This issue is for tracking tasks and separated issues that related to JavaScript library consumption. This includes TODO items, known issues and feature works that affect any forms of consuming ONNX Runtime JavaScript API via the following library:

  • onnxruntime-web
  • onnxruntime-node
  • onnxruntime-react-native

Work items tracking table

Works to be done for ORT-Web:

  1. support ES6 export. [web] ES6 modules #10913
    • currently the "module" field is not working in js/web/package.json. Need to:
      a. switch to ES module for webassembly
      b. use standard module export
      c. make webpack behavior correct
  2. support loading from worker/audio worklet. Initializing inference session on worker thread for web applications using onnxruntime-web #12589 [Web] Running onnxruntime in an audio worklet #13072
  3. support service worker. [Web] chrome V3 extension TypeError: URL.createObjectURL is not a function #14445
  4. support electron [Web] ort-wasm-simd.wasm can't be loaded in Electron renderer (using webpack) #13933
  5. support export for Node.js
  6. update package.json (exports field) so that different projects can consume onnxruntime-web with minimal modification to its webpack configuration. [Web] NPM package include ts files in the output #13736

Works to be done for ORT-Node:

  1. support GPU EP (Cuda/DML/...) Running onnxruntime with a gpu in nodejs? #14127
  2. electron compatibility onnxruntime-node crash the electron app[Web]  #13086

Example tracking table

The following table is used to track tasks for updating examples to consume ONNX Runtime JavaScript API

Category Status Description PR/Issue # Comments
onnxruntime-node Need update version JavaScript / CommonJS / No_Bundler link
onnxruntime-node Not started JavaScript / ESModule / No_Bundler
onnxruntime-node Not started JavaScript / CommonJS / Webpack Webpack does not deal with node native modules correctly #14505
onnxruntime-node Not started JavaScript / ESModule / Webpack
onnxruntime-web Need update version JavaScript / Web / scriptTag link
onnxruntime-web Need update version JavaScript / Web / CommonJS / Webpack link
onnxruntime-web Not started JavaScript / Web / ESModule / Webpack not working #14504
onnxruntime-web Not started JavaScript / Node / CommonJS / No_Bundler multi-thread CPU EP not available for onnxruntime-web in Node.js binding
onnxruntime-web Not started JavaScript / Node / ESModule / No_Bundler ESModule not available for onnxruntime-web in Node.js binding
onnxruntime-web Not started JavaScript / Node / CommonJS / Webpack
@fs-eire fs-eire added feature request request for unsupported feature or enhancement documentation improvements or additions to documentation; typically submitted using template platform:web issues related to ONNX Runtime web; typically submitted using template labels Feb 1, 2023
@github-actions github-actions bot removed the platform:web issues related to ONNX Runtime web; typically submitted using template label Feb 1, 2023
@fs-eire fs-eire changed the title [js] update various examples to consume ONNX Runtime JavaScript API [js][feature tracking] JavaScript library consume Feb 1, 2023
@github-actions github-actions bot added ep:CUDA issues related to the CUDA execution provider ep:DML issues related to the DirectML execution provider labels Feb 1, 2023
@fs-eire fs-eire added platform:web issues related to ONNX Runtime web; typically submitted using template and removed ep:DML issues related to the DirectML execution provider ep:CUDA issues related to the CUDA execution provider labels Feb 1, 2023
fs-eire added a commit that referenced this issue Feb 2, 2023
@fs-eire
[js/web] remove 'module' field from package.json (#14532)
cfb6e52 
### Description
this is a workaround for
[#14529](#14504) when
consuming onnxruntime-web as ES module.
@aexposit
Copy link

Hi is there any news on point 2 - support loading from worker/audio worklet ?

@kungfooman kungfooman mentioned this issue Apr 25, 2023
Closed
@chsteinhauer
Copy link

Hi is there any news on point 2 - support loading from worker/audio worklet ?

I'm also interested in this 👀

@dakenf dakenf mentioned this issue May 23, 2023
Merged
snnn pushed a commit that referenced this issue Aug 25, 2023
@dakenf
Added DML and CUDA provider support in onnxruntime-node (#16050)
c262879 
### Description
I've added changes to support CUDA and DML (only on Windows, on other
platforms it will throw an error)



### Motivation and Context
It fixes this feature request
#14127 which is tracked
here #14529

I was working on StableDiffusion implementation for node.js and it is
very slow on CPU, so GPU support is essential.

Here is a working demo with a patched and precompiled version
https://github.com/dakenf/stable-diffusion-nodejs

---------
@MountainAndMorning
Copy link

Is the cuda or dml providers available in onnxruntime-node v1.16?

@fs-eire
Copy link
Contributor Author

fs-eire commented Sep 25, 2023

Is the cuda or dml providers available in onnxruntime-node v1.16?

They are not supported out-of-the-box and need a few manual steps to enable. We will consider to optimize this in next release.

@MountainAndMorning
Copy link

I am developing a tool that relies heavily on onnxruntime-node. Due to the lack of GPU support, the tool runs very slowly. I tried to compile onnxruntime-node, but encountered many problems. Is there a time plan for the release of the npm package, or can you provide the manual steps? Thank you.
In the other hand, is there any plan for the coreml backend for onnxruntime-node on the apple silicon platform.

@fs-eire
Copy link
Contributor Author

fs-eire commented Oct 11, 2023

we are working on the pipeline changes for Node.js binding.

@fs-eire
Copy link
Contributor Author

fs-eire commented Oct 11, 2023

UPDATE: Items (1) and (6) for onnxruntime-web is done by recent change merged into main branch.

@MountainAndMorning
Copy link

Thanks for your help.

@MountainAndMorning
Copy link

MountainAndMorning commented Oct 26, 2023 •
edited
Loading

Is there any progress about the coreml backend for onnxruntime-node?@fs-eire

kleiti pushed a commit to kleiti/onnxruntime that referenced this issue Mar 22, 2024
@dakenf @kleiti
Added DML and CUDA provider support in onnxruntime-node (microsoft#16050
2809a3f 
)

### Description
I've added changes to support CUDA and DML (only on Windows, on other
platforms it will throw an error)



### Motivation and Context
It fixes this feature request
microsoft#14127 which is tracked
here microsoft#14529

I was working on StableDiffusion implementation for node.js and it is
very slow on CPU, so GPU support is essential.

Here is a working demo with a patched and precompiled version
https://github.com/dakenf/stable-diffusion-nodejs

---------
siweic0 pushed a commit to siweic0/onnxruntime-web that referenced this issue May 9, 2024
@fs-eire
[js/web] remove 'module' field from package.json (microsoft#14532)
975ff12 
### Description
this is a workaround for
[microsoft#14529](microsoft#14504) when
consuming onnxruntime-web as ES module.
@EmergentOrder
Copy link

Checking in here, are there plans to address these issues?

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
documentation improvements or additions to documentation; typically submitted using template feature request request for unsupported feature or enhancement platform:web issues related to ONNX Runtime web; typically submitted using template
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@EmergentOrder @fs-eire @MountainAndMorning @aexposit @chsteinhauer