Allow to use filesystem polling in watch mode (was: Debug in Docker)

[barfet]

Hi guys,

Thanks for your efforts in this project.
I have created a question in related sls project serverless-offline and will duplicate it here, because of adjacent functionality.
dherault/serverless-offline#300

It would be great to hear some thoughts regarding the topic.

Thanks!

[HyperBrain]

@barfet Thanks for the kick-off of the topic 🙌 . You should base your tests on version 3 of the plugin which will be released end of this week (a stable rc is already available at 3.0.0-rc.2).
The new version features full integration with sls invoke local which also can be executed with --inspect to run any function locally with a given event.json. Additionally it integrates much better with serverless-offline because it sets the --location option automatically and it turned out that source mapping now works correctly.

[barfet]

@HyperBrain thanks for the info, great stuff in V3! But, the use of serverless-offline is preferable for us, because we could emulate dev environment with it. Actually I'm in the final step for creating an interacting development setup. I moved from node-inspector to VS Code Debug and was able to create desired functionality but only with tsc compilation with serverless-plugin-typescript. (running sls offline in docker, attaching to Dokcer with VS Code and recompiling the sources and reattaching the debug automatically in VS Code).

But I have an issue with serverless-webpack. I run Docker container with next script nodemon --legacy-watch --watch ./.build --debug=0.0.0.0:5858 --nolazy node_modules/.bin/serverless offline start witch need to restart sls offline in case of file changes in .build folder. This is perfectly working with serverless-offline and serverless-plugin-typescript, but when I changing sls typescript to sls webpack it didn't fire any webpack recompilation in case of changes.

Do you support this kind of scenario? (recompilation source files when sls offline processes is already started)? As I said this is perfectly working but just with tsc compilation.

Thanks!

[HyperBrain]

In general, serverless-offline start should automatically rebuild and recompile the code automatically. We use that locally a lot. You do not need the serverless webpack command at all, as the hooks of serverless-offline should take care of that in the background (when serverless-webpack is enabled as plugin in your project).

You should not need to trigger serverless-offline itself with a watch command. Just let it run.

[HyperBrain]

FYI: This is the only command we run:
serverless offline --noTimeout --dontPrintOutput

This starts the APIG endpoint emulation and automatically rebuilds the sources with webpack on any source changes. It should not matter if you use offline or offline start here. So you have the APIG emulation running and the automatic rebuilds.

In theory this should also work, if you enable Node's debug mode.

[HyperBrain]

Here's a sample screen output of serverless-offline:

Serverless: Offline listening on http://localhost:3000
Hash: 41cebafab87a10be924d
Version: webpack 3.5.5
Time: 7822ms
                        Asset     Size  Chunks                    Chunk Names
    handlers/serve/handler.js   597 kB       0  [emitted]  [big]  handlers/serve/handler
     handlers/auth/handler.js  10.5 kB       1  [emitted]         handlers/auth/handler
handlers/serve/handler.js.map   390 kB       0  [emitted]         handlers/serve/handler
 handlers/auth/handler.js.map  6.93 kB       1  [emitted]         handlers/auth/handler
[...]

!!!!================================
!!!! HERE I CHANGED A SOURCE FILE AND SAVED IT
!!!!================================

Hash: 41cebafab87a10be924d
Version: webpack 3.5.5
Time: 307ms
                        Asset     Size  Chunks         Chunk Names
    handlers/serve/handler.js   597 kB       0  [big]  handlers/serve/handler
     handlers/auth/handler.js  10.5 kB       1         handlers/auth/handler
handlers/serve/handler.js.map   390 kB       0         handlers/serve/handler
 handlers/auth/handler.js.map  6.93 kB       1         handlers/auth/handler
[...]

So you see, as soon as the file changed, serverless offline + serverless webpack automatically triggered a recompile and the new handler is immediately testable on the local APIG endpoint

[barfet]

@HyperBrain Thanks for the detailed explanation! I tried your approach and it's perfectly working without Docker (on file changes Webpack is triggered and recompiled), but when I run the same inside of Docker container with volumes mapping than webpack doesn't recompile when files was modified in host machine. But is recompiling when I manually modify them inside the container. Maybe you faced something similar?

[HyperBrain]

That's an interesting behavior. I never used the watch mechanism on a mapped volume though.
I'll do some tests and try to find out why the filesystem events do not seem to work on mapped volumes, or if there's a special trick to enable that.

[barfet]

It may be the same issue as nodemon dealt with --legacy-watch flag. (Chokidar's polling)
https://github.com/remy/nodemon#application-isnt-restarting
https://github.com/paulmillr/chokidar#performance

[HyperBrain]

I see. Just checked the webpack API watch configuration options here.

It seems that it is possible to switch webpack's file watch to poll mode too.
I think we can add this as a feature to serverless-webpack, so that you can explicitly enable polling instead of event listening. I'm not quite sure what would be the best integration of such a switch.

Maybe a commandline option --webpack-use-polling would be the best way, that you can set on you invocation of serverless.

I will change the topic of this issue - because eventually it is about "Allow to use polling for watch mode" and flag it as feature. This should solve your problem completely.

[HyperBrain]

Can you comment our findings in dherault/serverless-offline#300, because it can be solved completely in serverless-webpack as far as I see it.

[barfet]

--webpack-use-polling command will be great to have. Thanks again for such quick feedback!
I'll add a comment in sls offline repo.

[HyperBrain]

Released with 3.1.0