Dynamic module or function loading across runspaces

[fatherofinvention]

Question

Hey @Badgerati - it's been awhile! Still loving using Pode. Wondering if you've run into this or have any ideas?

I'm trying to dynamically reload PowerShell modules/functions at runtime to avoid server restarts. Trying a middleware approach using Use-PodeScript fails with a null reference exception, for example:

Add-PodeMiddleware -Name 'LoadFunctions' -ScriptBlock {
    # Get all PS1 files in functions/abc123
    $abc123Functions = Get-ChildItem -Path "/home/user/web/project/functions/abc123" -Filter "*.ps1" -Recurse
    foreach ($abc123Function in $abc123Functions) {
        Use-PodeScript -Path $abc123Function.FullName
# Also tried dot-sourcing here too but no luck
    }
    return $true
}

Category: InvalidOperation: (:) [Import-PodeFunctionsIntoRunspaceState], RuntimeException
Message: You cannot call a method on a null-valued expression.
StackTrace: at Import-PodeFunctionsIntoRunspaceState, /home/user/.local/share/powershell/Modules/Pode/2.11.1/Private/AutoImport.ps1: line 50
at Use-PodeScript, /home/user/.local/share/powershell/Modules/Pode/2.11.1/Public/Utilities.ps1: line 215
at <ScriptBlock>, /home/user/web/project/middleware/dotsourceFunctions.ps1: line 7
at Invoke-PodeScriptBlock, /home/user/.local/share/powershell/Modules/Pode/2.11.1/Public/Utilities.ps1: line 573
at Invoke-PodeMiddleware, /home/user/.local/share/powershell/Modules/Pode/2.11.1/Private/Middleware.ps1: line 41
at <ScriptBlock>, <No file>: line 95

Up to this point, I've been using Use-PodeFolder -DefaultPath $config.Paths.Functions in main.ps1, but this only loads functions once across runspaces, requiring server restart to apply changes each time a function is updated. This takes on average about 10-12 seconds per restart, which creates a slow feedback loop. Related....I've managed to accomplish a similar "hot reloading" behavior with my routes by dot-sourcing route logic in (good for dev, probably not such a great practice in Prod), but the same thing isn't working here since the key difference is that I need current function definitions across all runspaces and it seems Use-PodeFolder and Use-PodeScript and Import-PodeModule aren't designed to support that.

Are there any ways around this to effectively reload function definitions across all runspaces without server restart (perhaps using middleware or some other approach)?

[mdaneri]

Is Restart-PodeServer too slow for your need?

[fatherofinvention]

@mdaneri yeah it’s way too slow unfortunately. Same goes for having Pode watch files and restart on change. It makes for quite a painful development process. That’s why dot sourcing route logic is so nice, you get to see changes reflected instantly without server restart. Again not something I’d do in Prod but for quick iterations in dev it’s perfect. Now I need to figure out a way to do this with dot-sourced functions and imported modules across runspaces.

[Badgerati]

You're right that those functions aren't really built for this use case. Technically dot-sourcing them in Add-PodeMiddleware dooooes work, however the new function definitions would be scoped only to the middleware's scriptblock - obviously not what you're after! For it to work the updated script/module would have to be reloaded in the top level scriptblock the runspace is using.

I have an idea for Use-PodeScript and Import-PodeModule that might work: some form of -Monitor switch which keeps track of a hash/timestamp for when the files/modules were updated 🤔 FYI, the Use-PodeFolder is a private internal function, so Id be careful in case it changes one day! If it's useful, we could probably make it public.

I'll test out the -Monitor idea next week; just working on the next release atm!

[fatherofinvention]

Hey @Badgerati that sounds great! Thanks for looking into it…and can’t wait for the next release.