Incorrect subsystem on windows with VS2022 #126

octopoulos · 2023-10-11T09:38:16Z

Just following the basic guide:

> mkdir ~/scratch
> cd ~/scratch
> git clone https://github.com/floooh/sokol-samples
> cd sokol-samples
> ./fips set config sapp-d3d11-win64-vstudio-debug
> ./fips build

=> error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ)

The reason is that the main hijack is not working, because it's being overridden by the project subsystem, which is set to Console (instead of NotSet).

    <Link>
      ...
      <SubSystem>Console</SubSystem>
    </Link>

If I change the SubSystem to Windows or NotSet then it works, but it would be nice if the tool could do it automatically, of if the hijack could override that setting.

The text was updated successfully, but these errors were encountered:

octopoulos · 2023-10-11T10:13:39Z

Note that a solution might just be to always define main( whenever WinMain( is defined.
(in sokol_app.h)

floooh · 2023-10-11T15:26:15Z

This is weird, the subsystem should be selected via fips_begin_app(... windowed) (requires WinMain) vs fips_begin_app(... cmdline) (requires main). Let me check if I have a CMakeLists error somewhere...

...also the CI should actually fail in that case...

floooh · 2023-10-11T15:37:28Z

Hmm, I just tried the process from scratch and it works here.

Can you give a bit more context?

the output lines from cmake following === generating: sapp-d3d11-win64-vstudio-debug
the output lines around the actual error, e.g. which of the samples is failing
cmake and Visual Studio version
output of fips diag tools

floooh · 2023-10-11T15:40:19Z

For instance on my system:

=== generating: sapp-d3d11-win64-vstudio-debug
config file: E:/scratch/sokol-samples/fips-files/configs/sapp-d3d11-win64-vstudio-debug.yml
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  FIPS_AUTO_IMPORT="OFF"
  FIPS_CONFIG="sapp-d3d11-win64-vstudio-debug"
  FIPS_LOCAL_BUILD="OFF"
  SOKOL_USE_D3D11:BOOL="ON"
  USE_SOKOL_APP:BOOL="ON"

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.34.31935.0
-- The CXX compiler identification is MSVC 19.34.31935.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE: Debug
FIPS_ROOT_DIR: E:/scratch/fips
FIPS_PROJECT_DIR: E:/scratch/sokol-samples
FIPS_BUILD_DIR: E:/scratch/fips-build
FIPS_DEPLOY_DIR: E:/scratch/fips-deploy
FIPS_PROJECT_NAME: sokol-samples
FIPS_AUTO_IMPORT: OFF
FIPS_CONFIG: sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_BUILD_DIR: E:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_DEPLOY_DIR: E:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64
-- Found Python: C:/Users/floh/scoop/apps/python/current/python.exe (found version "3.12.0") found components: Interpreter
-- Configuring done (4.5s)
-- Generating done (0.5s)
-- Build files have been written to: E:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
cmake --build . --config Debug --
MSBuild version 17.4.0+18d5aef85 for .NET Framework

cmake version 3.27.1

Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.4.2

λ fips diag tools
=== tools:
git:    found
cmake:  found
cmake-gui:      found
make:   found
ninja:  found
javac:  found
java:   found
vscode: found
vscode-cpptools:        found
vscode-cmaketools:      found
vscode-codelldb:        found
wasm-dwarf-debugging:   OPTIONAL, NOT FOUND (https://marketplace.visualstudio.com/items?itemName=ms-vscode.wasm-dwarf-debugging)
clion:  OPTIONAL, NOT FOUND (used as IDE with clion configs)
http-server:    found
wasmtime:       OPTIONAL, NOT FOUND (required for running WASI executables)

floooh · 2023-10-11T15:43:37Z

Also, when I open Visual Studio (via fips open), and look at the properties of the clear-sapp sample, the subsystem is set to Windows:

(none of the samples under sapp/ should actually use the Console subsystem)

...looking at the generated .vcxproj files, all targets are set to SubSystem Windows (which is correct but seems to be different on your side):

Something must be wrong about the cmake generation step I think...

octopoulos · 2023-10-11T20:36:09Z

Ok redoing it from scratch:
C:> mkdir scratch
C:> cd scratch
C:\scratch> git clone https://github.com/floooh/sokol-samples
C:\scratch> cd sokol-samples
C:\scratch\sokol-samples> ./fips set config sapp-d3d11-win64-vstudio-debug
...
C:\scratch\sokol-samples> ./fips build
...

=== building: sapp-d3d11-win64-vstudio-debug
=== generating: sapp-d3d11-win64-vstudio-debug
config file: C:/scratch/sokol-samples/fips-files/configs/sapp-d3d11-win64-vstudio-debug.yml
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  FIPS_AUTO_IMPORT="OFF"
  FIPS_CONFIG="sapp-d3d11-win64-vstudio-debug"
  FIPS_LOCAL_BUILD="OFF"
  SOKOL_USE_D3D11:BOOL="ON"
  USE_SOKOL_APP:BOOL="ON"

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.37.32824.0
-- The CXX compiler identification is MSVC 19.37.32824.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE: Debug
FIPS_ROOT_DIR: C:/scratch/fips
FIPS_PROJECT_DIR: C:/scratch/sokol-samples
FIPS_BUILD_DIR: C:/scratch/fips-build
FIPS_DEPLOY_DIR: C:/scratch/fips-deploy
FIPS_PROJECT_NAME: sokol-samples
FIPS_AUTO_IMPORT: OFF
FIPS_CONFIG: sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_BUILD_DIR: C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_DEPLOY_DIR: C:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
FIPS_PLATFORM:
FIPS_PLATFORM_NAME:
-- Found Python: C:/Code/Python38/python.exe (found version "3.8.1") found components: Interpreter
-- Configuring done
-- Generating done
-- Build files have been written to: C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
cmake --build . --config Debug --
MSBuild version 17.7.2+d6990bcfa for .NET Framework

All the samples fail, ex: first 2 errors:

  ## sokol-shdc: C:/scratch/sokol-samples/sapp/noentry-sapp.glsl C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug/sapp/noentry-sapp.gls
  l.h {'slang': 'hlsl4', 'compiler': 'MSVC'}
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  Building Custom Rule C:/scratch/sokol-samples/libs/sokol/CMakeLists.txt
  sokol.c
  sokol.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\libs\sokol\Debug\sokol.lib
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  arraytex-sapp.c
MSVCRTD.lib(exe_main.obj) : error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ) [
C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp.vcxproj]
C:\scratch\fips-deploy\sokol-samples\sapp-d3d11-win64-vstudio-debug\arraytex-sapp.exe : fatal error LNK1120: 1 unresolved externals [C:\scratch\fips-buil
d\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp.vcxproj]
  Building Custom Rule C:/scratch/fips-imgui/CMakeLists.txt
  imgui.cpp
  imgui_demo.cpp
  imgui_draw.cpp
  imgui_tables.cpp
  imgui_widgets.cpp
  Generating Code...
  imgui.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\fips-imgui_imgui\Debug\imgui.lib
  Building Custom Rule C:/scratch/sokol-samples/libs/dbgui/CMakeLists.txt
  dbgui.cc
  dbgui.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\libs\dbgui\Debug\dbgui.lib
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  arraytex-sapp.c
MSVCRTD.lib(exe_main.obj) : error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ) [
C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp-ui.vcxproj]
C:\scratch\fips-deploy\sokol-samples\sapp-d3d11-win64-vstudio-debug\arraytex-sapp-ui.exe : fatal error LNK1120: 1 unresolved externals [C:\scratch\fips-b
uild\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp-ui.vcxproj]
  Building Custom Rule C:/scratch/sokol-samples/libs/basisu/CMakeLists.txt
  sokol_basisu.cpp

octopoulos · 2023-10-11T20:43:27Z

PS C:\scratch\sokol-samples> cmake --version
cmake version 3.25.0

=> Oh I see, it's not the latest one, I guess I can upgrade to 3.27, who knows maybe that will fix, I'll keep you updated.
Edit: I tried with cmake 3.27.7 ... same thing.

Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.7.4

=> yours is showing stack reserve size, while mine shows nothing there. I'll check with the latest CMake. (edit: done)

By the way, I forgot to thank you for the prompt reply!

octopoulos · 2023-10-11T21:14:02Z

After comparing your file and mine, I see that the main difference is this, you have:

FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64

and I have:

FIPS_PLATFORM:
FIPS_PLATFORM_NAME:

That's probably the issue, right?

floooh · 2023-10-11T21:44:57Z

That's probably the issue, right?

Wow yeah, let me check how this is detected...

floooh · 2023-10-11T21:59:41Z

Hmm, still weird. When FIPS_PLATFORM isn't set it means it's not including this windows.cmake file here:

https://github.com/floooh/fips/blob/master/cmake-toolchains/windows.cmake

That should happen here when FIPS_HOST_WINDOWS is set:

https://github.com/floooh/fips/blob/36ae6b60a5d6d8c6a5661f8cf56b474c28d9bdaa/cmake/fips.cmake#L164-L172

...but that is definitely set because the message Host system: Windows shows up in your log:

https://github.com/floooh/fips/blob/36ae6b60a5d6d8c6a5661f8cf56b474c28d9bdaa/cmake/fips.cmake#L115-L118

Your Visual Studio seems to be a bit more recent... currently updating.

One more thing, are you building on PowerShell? If yes can you try on a "vanilla" cmd.exe (e.g. also not a "Visual Studio Command Prompt"). It should still work, but these are the only other differences I can think of.

I'll continue investigating tomorrow, it's pretty late now :)

floooh · 2023-10-11T22:04:18Z

Btw, can you also check this project:

https://github.com/floooh/pacman.c

This isn't using my fips cmake wrapper but a regular CMakeLists.txt instead. Build instructions are in the readme. If this also fails (which I don't expect), then it would be some incompatibility between cmake and the latest VS (unlikely though).

floooh · 2023-10-11T22:08:11Z

Hmm, updated to VS 17.7.5 and it's still working here, both on cmd.exe and in Powershell.

Running out of ideas...

floooh · 2023-10-11T22:18:07Z

Ok, one last thing before I hit the sack:

Can you add a couple of debug-messages to the fips directory that's been checked out next to sokol-samples:

...in fips/cmake/fips.cmake around line 164:

    if (NOT CMAKE_TOOLCHAIN_FILE)
        message("######### in NOT CMAKE_TOOLCHAIN_FILE")
        if (FIPS_HOST_WINDOWS)
            message("######### including windows.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/windows.cmake)
        elseif (FIPS_HOST_OSX)
            message("######### including osx.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/osx.cmake)
        else()
            message("######### including linux.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/linux.cmake)
        endif()
    endif()

... then run fips gen in the sokol-samples directory. Here it looks like this:

FIPS_PROJECT_DEPLOY_DIR: E:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
######## in NOT CMAKE_TOOLCHAIN_FILE
######### including windows.cmake
FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64
-- Configuring done (0.5s)

I basically want to check if windows.cmake is actually included.

floooh · 2023-10-11T22:22:06Z

PS: do you have an environment variable CMAKE_TOOLCHAIN_FILE set by any chance? Because this would definitely confuse this piece of code.

floooh · 2023-10-11T22:22:19Z

Anyway logging out now :)

octopoulos · 2023-10-12T15:22:19Z

Hi, just letting you know that I will check everything tomorrow, I just had a full day trip.
Note: I think the best way to fix would be to have CMakeLists.txt specify the subsystem:Windows.
So, expect some news tomorrow.

octopoulos · 2023-10-13T20:28:38Z

Pacman works fine.

message("######### in NOT CMAKE_TOOLCHAIN_FILE")

Bingo!

I edited the fips.cmake to remove the toolchain check, and then it compiles fine on windows. I also tried emscripten a few days ago and that worked fine too.

floooh · 2023-10-14T15:05:36Z

message("######### in NOT CMAKE_TOOLCHAIN_FILE")

But this is entirely correct.

What's concerning is that there is no ######### including windows.cmake message (or is there?).

When the windows.cmake file is included, it should work because this should set the correct subsystems, otherwise such problems are expected.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incorrect subsystem on windows with VS2022 #126

Incorrect subsystem on windows with VS2022 #126

octopoulos commented Oct 11, 2023

octopoulos commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023 •

edited

Loading

octopoulos commented Oct 11, 2023

octopoulos commented Oct 11, 2023 •

edited

Loading

octopoulos commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

octopoulos commented Oct 12, 2023

octopoulos commented Oct 13, 2023 •

edited

Loading

floooh commented Oct 14, 2023 •

edited

Loading

Incorrect subsystem on windows with VS2022 #126

Incorrect subsystem on windows with VS2022 #126

Comments

octopoulos commented Oct 11, 2023

octopoulos commented Oct 11, 2023 • edited Loading

floooh commented Oct 11, 2023 • edited Loading

floooh commented Oct 11, 2023 • edited Loading

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023 • edited Loading

octopoulos commented Oct 11, 2023

octopoulos commented Oct 11, 2023 • edited Loading

octopoulos commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023 • edited Loading

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

floooh commented Oct 11, 2023

octopoulos commented Oct 12, 2023

octopoulos commented Oct 13, 2023 • edited Loading

floooh commented Oct 14, 2023 • edited Loading

octopoulos commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

octopoulos commented Oct 11, 2023 •

edited

Loading

floooh commented Oct 11, 2023 •

edited

Loading

octopoulos commented Oct 13, 2023 •

edited

Loading

floooh commented Oct 14, 2023 •

edited

Loading