Linux Kernel Module fails to build and load on WSL 2 #260
Comments
|
As of now, it's impossible to run Darling in WSL for various reasons, the two most important ones being:
|
|
Hmmm 2 would be the more likely to be resolved as work on wsl progresses more features are supported. As far as 1 goes idk. Probably be more likely for you guys to make it work in some other fashion. But oh well I guess for the time being. Can just vm linux or somethin. |
|
would be nice if you submit type 2 issues as suggestions to Microsoft here: |
|
So I put an issue in WSL site for help.. |
Seems so indeed! That's pretty cool, thanks for bringing it to my attention. overlayfs is still missing, but we could do without it (the way Wine does it). PID namespaces are missing, too, though these could be faked quite easily in LKM (or whatever replaces it). |
|
@bugaevc Mount, PID, IPC, and UTS namespaces are all implemented now (they appear to be pursuing container support on the hush hush). I think the main problem here is going to be lack of 'kernel' headers or real module-loading capabilities. The trouble is that WSL's modules, if they ever are implemented for arbitrary modules, will need to be a special kind of Windows signed driver (because the Windows Kernel enforces driver signing with either a Microsoft or very high-level microsoft partner signature. Drivers submitted for loading have to be signed after code review within Microsoft or a trusted partner). Alex Ionescu wrote up a little article on his github about it, and he was able to write a very basic out-of-tree kernelmode driver to expose some virtual hardware inside WSL. However, the WSL driver API is still neither finalized and versioned or even exposed with symbols, so even if you did want to write a driver for it, it would be a hack with no Kernel shim engine support, and it would only work if the system was set up to boot in 'Test Mode'. There are several people with signatures who might be willing to sign an opensource drivers for free (the people who sign dokany and who will sign the btrfs driver in the future), but it's unlikely they would do it if you're programming against an unstable and undocumented API. So for the foreseeable future, it's going to be a no-go. |
|
Seeing that WSL2 uses the actual Linux kernel, I wonder if there will be support for building and installing modules. |
|
Yeah, that's what I'm wondering as well. If modules are supported, there's a high chance Darling will work. |
|
WSL2 early preview released today(Insider Fast Ring). Can anybody confirm if it works work Darling? |
|
I wish I could confirm Darling works in WSL2. I have yet to build the kernel module to confirm if that works or not, but I will explain my experience thus far and see if the developers here can make anything of it. |
This issue was fixed with darlinghq/darling-foundation@0a00d64, but the darling repo has not been updated to use the newer commits from the submodule. You are going to have to either manually update the files, or use an older commit that didn't introduce the build breaking changes. Edit: @bugaevc would it be possible for you to update the submodule? |
Yeah, sure, but I need to get home & deal with other stuff first :) |
|
Can we bump to open this back up? |
|
Got back to attempting to re-compile Darling. One thing that I learned the first time reattempting to rebuild is that environment variables $PATH and $LD_LIBRARY_PATH do NOT remain set after exiting WSL2. This is important because as I mentioned above, I had to manually install the Clang binary as it isn't included in the package manager in Ubuntu 18.04 on WSL2. If I don't manually re-add Clang to these environment variables, it will attempt to compile with GCC and G++, which will result in build errors. I still am researching the issue to see if there's a way I can set the environment variables once and keep them set. Above all, I would just like to fix these build errors so I can get to testing the kernel module element. I tried researching if there was a similar issue to the darling-foundation issue I had above, but I can't find the SecurityInterface equivalent module repository. Any help is appreciated! |
/home/ricorobbins/darling/src/external/security/include/Security/Security.h:95:2: error: invalid preprocessing directive #i<AE>clude <<F3>ecurity<U+000F>Authori That sounds like you have either a corrupted file or a buggy filesystem. Try re-fetching Security.
The repo is here, the file is here, and it's a symlink to here. |
|
As of now, I can confirm that Darling on WSL2 is a no-go. Re-fetching JavaScriptCore fixed this problem, and I was able to build most of Darling. And before you ask, yes, I confirmed that I am, in fact, on WSL2: As of right now, apart from opening a new issue within the MicrosoftDocs/WSL repository, I'm at a loss for what to do next. |
That simply means that you don't have sources/headers for the kernel version you're running. Try this. |
|
Okay, I've downloaded and extracted the kernel source from Microsoft, but I still cannot compile the kernel module: |
|
You should have |
|
I built the entire kernel as well and was now able to build the kernel module and install it, but I still can't seem to run darling: Perhaps I didn't build the kernel and/or kernel module properly? The only thing I did that was different between my first attempt at building the kernel module and this new attempt was restarting the make lkm command after fixing more kernel-related dependency issues. |
ahyattdev
changed the title
|
I have WSL2 running, and my results are the exact opposite. The guy above clearly has a corrupted or broken system, given the sheer number of issues he ran into. The only thing I had to do to build it was One thing to note is that the build system creates many symlinks, meaning this can only be built on linux filesystems, i.e. the native WSL2 area. |
|
Oh, and there are some very bizarre warnings throughout the build, which is why I'm worried about it running. |
|
Quick question: Which variant of Ubuntu did you install in WSL2: Ubuntu 16.04, Ubuntu 18.04, or the one simply named Ubuntu? I currently have the Ubuntu 18.04 installed, which could explain some issues if you're using a different version of Ubuntu. And I did get a lot of warnings during the build process, actually. The only thing I couldn't get to compile properly was the kernel module. I'm not a developer by any stretch of the imagination, but I am tech-savvy and have messed around compiling other things already written before. Honestly, my past issues might be caused by other hardware-related issues. If I left WSL2 running for a prolonged period of time, my system tended to crash. Sometimes after a few hours, sometimes after 15 minutes. I didn't think anything of it because WSL2 is new and not in a stable build of Windows, but it could be that my system is broken. Honestly, I'd keep testing it myself, but right now I'm just a macOS enthusiast. In fact, the Windows system I tested it on was a multiboot Hackintosh, so I have no need to use Darling in my daily workflow. Depending on the extent of GUI support, once it's figured out and added, I might be able to create a "poor man's Hackintosh", if you will, and use that instead. Far more legal and less EULA issues to worry about. It's a pipe dream, for now, but maybe GUI support will be figured out by the time WSL2 becomes mainstream. For now, I just wanted to see if it could work, and since no one had started attempting it yet, I decided to give it a go myself. I'm glad you got it working! Maybe I'd be better off uninstalling WSL2 and going back to a stable build of Windows. |
|
I'm on 18.04 as well. In particular, I'm referring to things like having to reinstall simple packages, and the fact that clang is actually available as a normal package. Looking at my comment, I may have given the impression that I got the whole thing running. I did not 😄 . I did get it to build very smoothly though. |
|
@JohnHolmesII this is exciting. In order to test it, just run |
|
Hi tested today with latest on Debian 10 on WSL2 on latest 18956 insider (ships with kernel 4.19.57) and then lots of errors even when I close with ctr+c
did a dmesg log: sudo dmesg EDITED: attached dmesg as file |
|
ok guys.. finally success!
*Once module was built from this "running" kernel this time still produced similar errors.. Note I don't remember if in my last test I ran darling with sudo.. perhaps it should have worked with original kernel as long as you built a kernel " CONFIG_EMBEDDED not set" but not used as boot kernel.. |
|
@oscarbg Glad to see that you are able to get it working on WSL2; however, it does bring up a few questions.
|
|
|
@oscarbg Would you be willing to open up an issue over there? Edit: Or we could just fix the darling's code to use an alternative type when Edit2: It is strange that darling can't find Especially since |
|
@CuriousTommy yep I believe task_watch_t issue was on osfmk/kern/task.h file.. |
EDITPlease refer to the official instructions now. I'm leaving the original message here for posterity's sake. Hi, first of all, big thanks to all the Darling contributors for this amazing piece of software! Great job! Second, I'd just like to confirm that @oscarbg's method does in fact work, with the catch that I must run Other than that, everything runs as expected! I'm pretty sure Curl not being able to download over HTTPS because of SSL errors is normal in Darling, right? (According to #463, at least it is). I'm just asking since the first thing I tried to do was install Homebrew (which failed due to no support for HTTPS). DIY InstructionsWant to try out Darwin in the WSL? Here are some instructions to try this out for yourself: 1. Build most of the componentsOpen up a WSL shell and paste the following code. This will download, compile, and install most of the Darling components, except the kernel module, and then it'll download, compile, and install a modified WSL kernel (and copy it to your C: drive so it can be used). Note that parts where it says # make sure APT can find 32-bit libraries
sudo dpkg --add-architecture i386
sudo apt-get update
# install the necessary dependencies
sudo apt-get install cmake clang bison flex xz-utils libfuse-dev libudev-dev pkg-config libc6-dev:i386 linux-headers-generic gcc-multilib libcap2-bin libcairo2-dev libgl1-mesa-dev libtiff5-dev libfreetype6-dev libfreetype6-dev:i386 git libelf-dev libxml2-dev libegl1-mesa-dev libfontconfig1-dev libbsd-dev
# clone and configure Darling
git clone --recursive https://github.com/darlinghq/darling.git
cd darling
mkdir build && cd build
cmake ..
# symlink some files that are missing from `/usr/include`
# i had to do this on my system, maybe on your system you don't
# in any case, it won't hurt to symlink them even if you do have them in `/usr/include`
sudo ln -s /usr/include/x86_64-linux-gnu/jconfig.h /usr/include/jconfig.h
sudo ln -s /usr/include/x86_64-linux-gnu/tiffio.h /usr/include/tiffio.h
sudo ln -s /usr/include/x86_64-linux-gnu/tiff.h /usr/include/tiff.h
sudo ln -s /usr/include/x86_64-linux-gnu/tiffconf.h /usr/include/tiffconf.h
sudo ln -s /usr/include/x86_64-linux-gnu/tiffvers.h /usr/include/tiffvers.h
sudo ln -s /usr/include/x86_64-linux-gnu/tiffio.hxx /usr/include/tiffio.hxx
# build most of Darling
make
sudo make install
# before we can build the kernel module, we need to build the modified kernel
cd ../..
# clone and modify the kernel
git clone --depth 1 --single-branch --branch linux-msft-wsl-5.4.y https://github.com/microsoft/WSL2-Linux-Kernel.git kernel
cd kernel
sed -i 's/CONFIG_EMBEDDED=y/CONFIG_EMBEDDED=n/g' Microsoft/config-wsl
# build and install the kernel
make KCONFIG_CONFIG=Microsoft/config-wsl
sudo make modules_install
sudo make install
sudo update-initramfs -c -k 5.4.81-microsoft-standard-WSL2+
# copy the kernel to your C: drive
mkdir /mnt/c/linux-kernels
cp /boot/vmlinuz-5.4.81-microsoft-standard-WSL2+ /mnt/c/linux-kernels/vmlinuz-5.4.81-microsoft-standard-WSL2+2. Configure WSL to use your new kernelNow, you need to edit (or create) your [wsl2]
# ... other settings (if you have any) ...
kernel=C:\\linux-kernels\\vmlinuz-5.4.81-microsoft-standard-WSL2+Then, in PowerShell or CMD, shutdown the WSL by doing the following: wsl --shutdown3. Build the Darling kernel moduleOpen up another WSL prompt, and paste the following code. This will compile and install the kernel module # get back to the directory you built Darling in
cd darling/build
make lkm
sudo make lkm_install4. Enjoy!darling shellYou now have a functioning Darwin-in-Ubuntu-in-Windows abomination 😁 ! Happy hacking! Update |
|
Jeebus, MSFT builds their kernel as embedded? Do they strip symbols too? :) |
|
@facekapow I followed your instruction but failed at step 3. The compiler cannot find First errors: Also it seems that the |
|
@Berrysoft Unfortunately, as of now, older kernels like the one Microsoft is using are not supported. They might still build, but they're not officially supported. Maybe you could try again with the latest code from master; @LubosD recently pushed a fix for the
Actually, no. See this Microsoft doc, particularly this:
|
|
I know there have been some workarounds found for this, but is there any hope of Darling getting official support for WSL2?Especially with all the new developments as of Build 2020. |
|
I know that this may be slightly off-topic, but is any WSL downloaded from the Microsoft Store WSL2? Is WSL2 available/can it be obtained on Win 10 LTSC (build 17763)? |
@tajetaje Our position on this at the moment is pretty much "if it works, it works": we'll accept patches to make it work, but we won't support it officially.
@The-SamminAter The apps you download from the Microsoft Store are just the Linux distros that run atop the WSL; the actual WSL is distributed as an optional Windows feature. To be able to enable WSL 2, you need to be on Windows build number 19041 or higher (see https://docs.microsoft.com/en-us/windows/wsl/wsl2-index). |
Ok, thanks for the information. |
|
Any update on this issue ? |
See the response from facekapow, basically whenever WSL gets close enough to bare-metal Linux that it works out-of-the box, that'll be when you might expect support. It's the same way that you CAN run x86 apps on ARM, but don't expect help from Adobe for photoshop crashing, until that is, ARM's capabilities include full emulation of x86. |
|
Closing this as we no longer use a kernel module. |
|
Has anyone gotten it to work since? |
|
It should work normally in the WSL now, just like it does on Linux. |
What would it take to get darling to function on WSL? I can make and install it just fine, however I cannot build the kernel submodule since its a similar issue to a closed issue I came across regarding docker.
make -C /lib/modules/4.4.0-43-Microsoft/build M=/home/DuIslingr/darling/src/lkm modules make[1]: *** /lib/modules/4.4.0-43-Microsoft/build: No such file or directory. Stop. Makefile:36: recipe for target 'default' failed make: *** [default] Error 2I know it was said that it wouldn't work due to docker being simply a chroot environment essentially basically so it has no real kernel to work with. Which the same can be said for wsl. just would be nice if i could run Apples toolchain without having to wait for ports to linux or windows etc.
The text was updated successfully, but these errors were encountered: