test_unistd_io fails on Windows.

[juj]

Running test_unistd_io on Windows with latest Node.js 0.10.20, there are several errors, which looks like

======================================================================
ERROR: test_unistd_io (test_core.default)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\slaves\emscripten\win-emcc-incoming-tests\build\tests\test_core.py", line 7863, in test_unistd_io
    self.do_run(src, expected, js_engines=[NODE_JS])
  File "C:\Users\Administrator\Desktop\slaves\emscripten\win-emcc-incoming-tests\build\tests\runner.py", line 442, in do_run
    self.assertContained(expected_output, js_output.replace('\r\n', '\n'))
  File "C:\Users\Administrator\Desktop\slaves\emscripten\win-emcc-incoming-tests\build\tests\runner.py", line 311, in assertContained
    additional_info
Exception: Expected to find 'read from folder: -1
errno: 9
write to folder: -1
errno: 9

read from broken device: -1
errno: 5
write to broken device: -1
errno: 5

read from device: 4
data: ABCD
errno: 0
TO DEVICE: 119
TO DEVICE: 114
TO DEVICE: 105
TO DEVICE: 116
TO DEVICE: 101
TO DEVICE: 109
TO DEVICE: 101
TO DEVICE: 0
write to device: 8
errno: 0

open read-only device from createDevice for read, errno: 0
open read-only device from createDevice for write, errno: 13
open write-only device from createDevice for read, errno: 13
open write-only device from createDevice for write, errno: 0

read from file: 10
data: 1234567890
errno: 0

pread past end of file: 0
data: 
errno: 0

seek: 3
errno: 0

partial read from file: 3
data: 456
errno: 0

seek: 8
errno: 0
partial read from end of file: 2
data: 90
errno: 0

seek: -1
errno: 22
partial read from before start of file: 0
data: 
errno: 0

seek: 0
write to start of file: 3
errno: 0

seek: 10
write to end of file: 3
errno: 0

seek: 23
write after end of file: 8
errno: 0

seek: 0
read after write: 31
errno: 0
final: wri4567890wri\0\0\0\0\0\0\0\0\0\0writeme\0
' in 'exception thrown: Error: Permission denied,Error
    at stackTrace (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:898:15)
    at new ErrnoError (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:3030:24)
    at Object.FS.mknod (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2485:17)
    at Object.FS.mkdir (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2500:19)
    at eval (eval at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1212:91), <anonymous>:1:1104)
    at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1213:14)
    at Object._main (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:4907:2)
    at Object.callMain (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5242:30)
    at doRun (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5282:25)
    at run (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5295:5)

C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5259
      throw e;
            ^
Error
    at stackTrace (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:898:15)
    at new ErrnoError (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:3030:24)
    at Object.FS.mknod (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2485:17)
    at Object.FS.mkdir (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2500:19)
    at eval (eval at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1212:91), <anonymous>:1:1104)
    at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1213:14)
    at Object._main (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:4907:2)
    at Object.callMain (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5242:30)
    at doRun (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5282:25)
    at run (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5295:5)
', diff:

--- expected
+++ actual
@@ -1,73 +1,27 @@
-read from folder: -1
-errno: 9
-write to folder: -1
-errno: 9
+exception thrown: Error: Permission denied,Error
+    at stackTrace (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:898:15)
+    at new ErrnoError (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:3030:24)
+    at Object.FS.mknod (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2485:17)
+    at Object.FS.mkdir (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2500:19)
+    at eval (eval at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1212:91), <anonymous>:1:1104)
+    at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1213:14)
+    at Object._main (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:4907:2)
+    at Object.callMain (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5242:30)
+    at doRun (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5282:25)
+    at run (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5295:5)

-read from broken device: -1
-errno: 5
-write to broken device: -1
-errno: 5
+C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5259
+      throw e;
+            ^
+Error
+    at stackTrace (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:898:15)
+    at new ErrnoError (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:3030:24)
+    at Object.FS.mknod (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2485:17)
+    at Object.FS.mkdir (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:2500:19)
+    at eval (eval at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1212:91), <anonymous>:1:1104)
+    at _emscripten_asm_const (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:1213:14)
+    at Object._main (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:4907:2)
+    at Object.callMain (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5242:30)
+    at doRun (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5282:25)
+    at run (C:\Program Files (x86)\Emscripten\temp\emscripten_test_default_tj5fw9\src.cpp.o.js:5295:5)

-read from device: 4
-data: ABCD
-errno: 0
-TO DEVICE: 119
-TO DEVICE: 114
-TO DEVICE: 105
-TO DEVICE: 116
-TO DEVICE: 101
-TO DEVICE: 109
-TO DEVICE: 101
-TO DEVICE: 0
-write to device: 8
-errno: 0
-
-open read-only device from createDevice for read, errno: 0
-open read-only device from createDevice for write, errno: 13
-open write-only device from createDevice for read, errno: 13
-open write-only device from createDevice for write, errno: 0
-
-read from file: 10
-data: 1234567890
-errno: 0
-
-pread past end of file: 0
-data:
-errno: 0
-
-seek: 3
-errno: 0
-
-partial read from file: 3
-data: 456
-errno: 0
-
-seek: 8
-errno: 0
-partial read from end of file: 2
-data: 90
-errno: 0
-
-seek: -1
-errno: 22
-partial read from before start of file: 0
-data:
-errno: 0
-
-seek: 0
-write to start of file: 3
-errno: 0
-
-seek: 10
-write to end of file: 3
-errno: 0
-
-seek: 23
-write after end of file: 8
-errno: 0
-
-seek: 0
-read after write: 31
-errno: 0
-final: wri4567890wri\0\0\0\0\0\0\0\0\0\0writeme\0
-

To fix these, I noticed two issues in the library_nodefs.js code.

1. Looks like on Windows, lstatSync() for directories will never return execute bits set on the directory, even if the user has all security permissions enabled on that directory. I hacked in a change that propagates the write bits over to execute bits on Windows to fix this. This is probably a Windows-Unix discrepancy in node.js itself.

2. I see that openSync() gets called with integer values for the flags field. 2 == O_RDWR, and 65 == O_TRUNC|O_CREAT|O_WRONLY. library_fs.js has the function modeStringToFlags to explicitly convert the mode strings to the integer permission values, but looks like the node.js openSync() does not support the integer values, but only the string values, see http://nodejs.org/api/fs.html#fs_fs_open_path_flags_mode_callback . On Windows, the calls to openSync() will then fail, apparently on linux either ones are supported?

@inolen could you take a look at juj@b2913bc and see what you think?

(note that there are other unistd failures in the test runner on both Windows and Linux bots, this is just the first one I got now to debugging)

[inolen]

Regarding #2, check out:

https://github.com/joyent/node/blob/master/lib/fs.js#L397

and stringToFlags looks something like this:

// Used by binding.open and friends
function stringToFlags(flag) {
  // Only mess with strings
  if (!util.isString(flag)) {
    return flag;
  }

it looks like the numerical value is valid there, so I'm not quite sure what the problem is (perhaps the constant values are different for Windows and we need to map them in library_nodefs for Windows).

[juj]

Hmm interesting. Not sure what's the case, but since the numerical flags are not documented, I think I'll convert them back to strings before opening, so that we only utilize documented paths.

Also, I submitted a bug report about the stat discrepancy, see here: nodejs/node-v0.x-archive#6381 but no matter what happens to that, I think it's safe to have the first workaround present.

I am still seeing more nodefs-related failures on the Windows bot, working on sorting those out atm.

[juj]

The above pull request fixed this, closing.