Matt Greer

Windows Subsystem For Linux's Chicken and Egg Problem

17 March 2018

I’ve been excited for the Windows Subsystem for Linux ever since Microsoft announced it as Bash on Ubuntu on Windows (yikes that name was crazy). Although it has come a long ways, it still has some road blocks that prevent me from really using it.

A Tale of Two File Systems

When you are inside WSL, Linux sees two different file systems depending on whether you’re inside the Linux portion or have wandered out into the greater Windows world.

Windows’s NTFS and Linux’s Ext4 are wildy different in just about every possible way. So the WSL team has the crazy task of emulating Ext4 on top of NTFS, and they pull this off using the DrvFs file system. Details on how this works can be seen here. But the short of it is Windows files attempt to masquerade as Linux files as best they can.

You can see things are amiss just by listing some Windows files from within Linux.

Here is one of my projects, listed from within WSL and living on the WSL file system

wsl on wsl

This looks great, exactly as you’d expect.

And here is another copy of that same project, except this time out in the Windows file system, as seen from WSL

windows on wsl

You can see that when WSL is dealing with Windows files, DrvFs pretends root owns them, and that their persmissions are always 777. This is an understandable choice for the WSL team to make, given the great differences between the file systems. Pretending anyone can use the files is probably the best chance for success here.

I have simplified things a lot here. Permissions may not be 777 depending on how the files are setup on the NTFS side.

Except Weird Things Happen

The project I just listed is a really bog standard node based web app. It uses CoffeeScript, React and Jest. Nothing crazy going on at all. With the copy that lives on the Windows file system, I can fully build and launch the app in dev mode. This project uses Docker when running the app locally, and even that works just fine in WSL (I am using the Windows version of Docker from within WSL).

But running Jest does not.

I get an error deep inside Babel complaining about an unknown option coming from babel-preset-jest.

 FAIL  tests/store/budget-spec.coffee
  ● Test suite failed to run

    ReferenceError: [BABEL] unknown: Unknown option: /mnt/c/Users/matte/dev/cudgie/node_modules/jest/build/jest.js.SearchSource. Check out http://babeljs.io/docs/usage/options/ for more information about options.

    A common cause of this error is the presence of a configuration options object without the corresponding preset name. Example:

    Invalid:
      `{ presets: [{option: value}] }`
    Valid:
      `{ presets: [['presetName', {option: value}]] }`

    For more detailed information on preset configuration, please see http://babeljs.io/docs/plugins/#pluginpresets-options. (While processing preset: "/mnt/c/Users/matte/dev/cudgie/node_modules/jest/build/jest.js")

If I do the exact same thing from the copy of my project that lives inside WSL, Jest works just fine. The only difference is the location of the files. And of course Jest works just fine when I’m on Linux proper.

So? Just Keep the Files Inside WSL

That sounds like the simple solution, but there are caveats with keeping files inside WSL. In short, Windows apps should not touch them. I believe it’s safe for a Windows app to read files from WSL, but it is not safe at all to write them.

This means to use WSL with my project, I need to 100% confine myself to Linux apps. Technically Microsoft does not support X based apps on WSL (although they generally work). So technically, my editor needs to be terminal based. I don’t mind terminal editors, but I do mind being limited to them and only them.

This also means losing out on a lot of the promise of WSL. I can’t use Photoshop to edit my images for example. If I’m going to be 100% confinded inside the Linux subsystem, then why not just use Linux itself?

Scott Hanselman has a video showing how you should store your project files in Windows out in /mnt/c/..., and then use Windows apps to edit them. That sounds ideal, but even just a simple little node app can’t run Jest in this configuration. So it’s really not ideal at all.

I debugged into the problem for a while. Nothing obvious jumped out at me. My point isn’t this specific problem is blocking me. My point is the file system difference can cause weird unexpected behavior. Even if I fixed this particular issue, who knows what problem would be next.

OK, just use Linux X based editors

This actually is a decent solution. Install an X Server on Windows, then use say the Linux version of VS Code. Currently the Linux version of VS Code does not work on WSL, but progress is being made. People on Insiders Windows builds do report VS Code can work, but it’s a bit slow. People report that Linux Sublime and Linux gvim work fine in WSL. I have no idea how well things like IntelliJ would work. But again, if you’re going this far, I’d argue just use Linux proper.

More advanced Linux tools

The tools I use at work are provided by internal teams and they officially support Ubuntu and MacOS. Many of these tools are gnarly, for example using Docker in very advanced ways. So far I’ve not had much luck at all using them in WSL. When I try to use them on projects that live out on the Windows file system, they blow up spectacularly. There seems to be a giant world of hurt waiting for me if I dig into that. When I try them from within WSL on the Linux file system, they do work better and I think I could push through and get them completely working. But if I hit any problems, I’m on my own. I don’t blame those teams at all for saying “uhhh WSL? yeah good luck with that.”

Still Hopeful

I don’t mean this post to be just me complaining. Just want to point out WSL still has some kinks before it can be a truly reliable solution. I think WSL is very exciting and I am so happy MS is pursuing it. Thanks WSL team, keep it up!