Include files coming from engines

[josepjaume]

Will JS files coming from engines be available to be required in pack?

My use case is a library that's composed by many pluggable engines, some of them having complex react components I'm currently building with webpack. I'd like to somehow be able to create a single compiled file in the end at the application level.

One solution would be to add a bundle.js file for each engine and version control it into the repo, but then I'm duplicating dependencies - react, for example, as well as any other one.

In my scenario, the ideal solution is to have a single entry point in my application that requires files from engines, the same way they would be required if they were npm packages.

So, in my application.js file, I'd like to have:

require('plugin1/application.js');
require('plugin2/application.js');

Thanks for making the effort to integrate webpack into rails!

[dhh]

Yeah, we need a way to add plugin paths to the node load path, just like asset pipeline does it. Not quite sure yet how. But you can see how things are setup to use vendor/node_modules and app/javascript in the config files.

[dhh]

@guilleiguaran Is it possible to use the erb loader on the webpack config? Because if so, we could scan all the plugins for a specific directory to add to webpack's load path and just insert it it directly.

[jcoyne]

It seems like we'd want to set resolve.modulesDirectories, but I don't see a CLI option for setting that.

[jcoyne]

Would a PR like this be helpful? webpack/webpack@master...jcoyne:resolve-modules-directories

[tf]

One of the main issues, from my point of view, is how to handle npm dependencies when extracting JavaScript into engines. Just adding engine paths to the load path won't ensure dependencies are installed. Including engine package.json files in the main app's yarn install can lead to problems with conflicts between bundler's and yarn's dependency resolution.

[renchap]

I am working on https://github.com/renchap/webpacker-react and this is my biggest issue at the moment.
Currently (push to the repo coming soon) I created a webpacker-react NPM package and you need to manually include it into your package.json in addition to adding the gem to your project, and it would be great to be able to avoid this step.

[josepjaume]

I'm also having this issue! It should be possible to depend on a package.json from the

[josepjaume]

🤓😂... gem the same way we do with gemspecs, right?

[tf]

Well, yes, but that would leave Yarn with dependency choices already made by Bundler, which in some cases might not be optimal. In particular, to really get to the most up to date versions, users might have to first bundle update and then yarn update. We would probably also need to apply some magic to the application's package.json to insert the (transitive) engine dependencies. I'm not sure all of this will make for a very intuitive workflow.

[mengqing]

Wondering if there is any updates to this issue. We're currently facing this particular issue and is trying to find out if there are any viable solutions

[dhh]

I don't think we have a good path for doing this given that such inclusion wouldn't be able to resolve any dependencies. I don't see a way out of having NPM packages for the plugins. That's how we've gone with Action Cable and Active Storage for now. I'm closing this lest someone else comes up with an alternative idea alongside a proof-of-concept PR that deals with the dependency issue.