Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Unable to use MinGW instead of MSVC for building packages on Windows #2337

Open
VDovidaytis-HORIS opened this issue Oct 18, 2024 · 5 comments
Open

Unable to use MinGW instead of MSVC for building packages on Windows #2337

VDovidaytis-HORIS opened this issue Oct 18, 2024 · 5 comments
Labels
question

Comments

@VDovidaytis-HORIS
Copy link

Your question:

Hello!

I am trying to contribute the Lets-Plot package to conda-forge.
That project is based on Kotlin Native with Python API.

The issue I am facing is that conda-build defaults to using MSVC on Windows, but the library needs to be built with MinGW, because Kotlin Native supports only MinGW as a build target for the Windows platform.

Even after configuring meta.yaml, bld.bat, and setup.py to explicitly call MinGW, conda-build continues to invoke MSVC.

Steps taken:

  • Modified bld.bat to configure MinGW.
  • Adjusted setup.py to link to MinGW binaries.
  • Set appropriate environment variables.
  • Despite these modifications, conda-build continues to use MSVC.

Is there an official way to force conda-build to use MinGW for Windows builds? Or are there any known workarounds or future plans to support this?
@mfansler
Copy link
Member

Do you have a PR on staged-recipes? One need not have a fully worked out recipe to start a draft and let others see how it fails.

All R packages build through MinGW, so maybe have a look at their recipes. Recently we've been transitioning to UCRT, so some of the recipes might be in flux. For example, here's a recent submission without UCRT.

I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').

@VDovidaytis-HORIS
Copy link
Author

Hello! Thank you for the reply!

Do you have a PR on staged-recipes?

I have just made one:

conda-forge/staged-recipes#27974

For example, here's conda-forge/staged-recipes#27938.

I tried many things from that repository, but they didn't help. I also noticed that there is a problem with the CI build of that PR on Windows. However, I am not sure if it’s caused by MinGW problems.

I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').

When I try something like {{ compiler('m2w64_cxx') }} instead of {{ compiler('cxx') }}, conda starts searching for the m2w64_cxx_win-64 package, which doesn't exist. This issue occurs with any attempt to use m2w64_... instead of standard compiler options. I even tried to create a m2w64_cxx_win-64 package that links to the original m2w64_cxx package locally, but it didn’t help override the MSVC call.

I tried to build conda packages locally on my Windows machine by running the command (according to the documentation):

conda build recipes\lets-plot\ -m .\.ci_support\win64.yaml

I also tried running just conda build:

conda build recipes\lets-plot\

But there was no difference.

Build machine info:

Edition Windows 11 Pro
Version 23H2
Installed on    ‎18.‎01.‎2023
OS build    22631.4317
Experience  Windows Feature Experience Pack 1000.22700.1041.0

Conda build environment info:

active environment : test-conda-build
    active env location : D:\Util\miniconda3\envs\test-conda-build
            shell level : 2
       user config file : C:\Users\Valentin Dovidaytis\.condarc
 populated config files :
          conda version : 24.7.1
    conda-build version : not installed
         python version : 3.12.4.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake
                          __conda=24.7.1=0
                          __win=0=0
       base environment : D:\Util\miniconda3  (writable)
      conda av data dir : D:\Util\miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\Util\miniconda3\pkgs
                          C:\Users\Valentin Dovidaytis\.conda\pkgs
                          C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\pkgs
       envs directories : D:\Util\miniconda3\envs
                          C:\Users\Valentin Dovidaytis\.conda\envs
                          C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/24.7.1 requests/2.32.3 CPython/3.12.4 Windows/11 Windows/10.0.22631 solver/libmamba conda-libmamba-solver/24.7.0 libmambapy/1.5.8 aau/0.4.4 c/. s/. e/.
          administrator : False
             netrc file : None
           offline mode : False

Conda environment packages:

# packages in environment at D:\Util\miniconda3\envs\test-conda-build:
#
# Name                    Version                   Build  Channel
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     24.2.0             pyh71513ae_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py312h275cf98_2    conda-forge
bzip2                     1.0.8                h2466b09_7    conda-forge
ca-certificates           2024.8.30            h56e8100_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h4389bb4_0    conda-forge
chardet                   5.2.0           py312h2e8e312_2    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.9.2          py312h2e8e312_0    conda-forge
conda-build               24.9.0          py312h2e8e312_0    conda-forge
conda-index               0.5.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.9.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.4.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.11.0             pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
filelock                  3.16.1             pyhd8ed1ab_0    conda-forge
fmt                       11.0.2               h7f575de_0    conda-forge
frozendict                2.4.6           py312h4389bb4_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h2e8e312_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_0    conda-forge
krb5                      1.21.3               hdf4eb48_0    conda-forge
libarchive                3.7.4                haf234dc_0    conda-forge
libcurl                   8.10.1               h1ee3ff0_0    conda-forge
libexpat                  2.6.3                he0c23c2_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
liblief                   0.14.1               he0c23c2_2    conda-forge
libmamba                  1.5.10               h81425b0_1    conda-forge
libmambapy                1.5.10          py312h643a1bd_1    conda-forge
libsolv                   0.7.30               hbb528cf_0    conda-forge
libsqlite                 3.47.0               h2466b09_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libxml2                   2.12.7               h0f24e4e_4    conda-forge
libzlib                   1.3.1                h2466b09_2    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              hcfcfb64_1001    conda-forge
m2-conda-epoch            20230914               0_x86_64    conda-forge
m2-msys2-runtime          3.4.9.1              hd8ed1ab_4    conda-forge
m2-patch                  2.7.6.2              hd8ed1ab_4    conda-forge
markupsafe                3.0.2           py312h31fea79_0    conda-forge
menuinst                  2.1.2           py312h275cf98_1    conda-forge
more-itertools            10.5.0             pyhd8ed1ab_0    conda-forge
openssl                   3.3.2                h2466b09_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pkginfo                   1.11.2             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.6              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
psutil                    6.0.0           py312h4389bb4_2    conda-forge
py-lief                   0.14.1          py312h275cf98_2    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py312he70551f_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.12.7          hce54a09_0_cpython    conda-forge
python-libarchive-c       5.1             py312h2e8e312_1    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pytz                      2024.2             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py312h4389bb4_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hcfcfb64_1    conda-forge
reproc-cpp                14.2.4.post0         h63175ca_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               ha073cba_1    conda-forge
rpds-py                   0.20.0          py312h2615798_1    conda-forge
ruamel.yaml               0.18.6          py312h4389bb4_1    conda-forge
ruamel.yaml.clib          0.2.8           py312h4389bb4_1    conda-forge
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
truststore                0.9.2              pyhd8ed1ab_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_1    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
vc                        14.3                h8a93ad2_22    conda-forge
vc14_runtime              14.40.33810         hcc2c482_22    conda-forge
vs2015_runtime            14.40.33810         h3bf8584_22    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyh7428d3b_7    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yaml-cpp                  0.8.0                h63175ca_0    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zstandard                 0.23.0          py312h7606c53_1    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

@VDovidaytis-HORIS
Copy link
Author

I have just made one:

conda-forge/staged-recipes#27974

As I expected, the CI build on the Windows server crashed at the extension compilation step because MSVC was called instead of the MinGW compiler:

 creating build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge
  "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -ID:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic -ID:\bld\lets-plot_1729682723651\_h_env\include -ID:\bld\lets-plot_1729682723651\_h_env\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -ID:\bld\lets-plot_1729682723651\_h_env\Library\include -ID:\bld\lets-plot_1729682723651\_h_env\Library\include /d1trimfile:D:\bld\lets-plot_1729682723651\work /TcD:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c /Fobuild\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj
  lets_plot_kotlin_bridge.c
  D:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c(47): warning C4090: 'function': different 'const' qualifiers
  "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\libs /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib lets_plot_python_extension.lib stdc++.lib /EXPORT:PyInit_lets_plot_kotlin_bridge build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj /OUT:build\lib.win-amd64-cpython-311\lets_plot_kotlin_bridge.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.cp311-win_amd64.lib -static-libgcc -static -lbcrypt -lpthread -lz
  LINK : warning LNK4044: unrecognized option '/static-libgcc'; ignored
  LINK : warning LNK4044: unrecognized option '/static'; ignored
  LINK : warning LNK4044: unrecognized option '/lbcrypt'; ignored
  LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
  LINK : warning LNK4044: unrecognized option '/lz'; ignored
  LINK : fatal error LNK1181: cannot open input file 'lets_plot_python_extension.lib'
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX64\\x64\\link.exe' failed with exit code 1181
  error: subprocess-exited-with-error
  Building wheel for lets-plot (setup.py): finished with status 'error'

The compiler('m2w64_cxx') setting was used in the recipe, but it was ignored by conda.

What steps should I try next to fix the Windows build?

@alshan alshan mentioned this issue Dec 30, 2024
Open
@owinebar
Copy link

I believe the issue is that those definitions are derived from conda_build_config.yamlin the conda-forge-pinning-feedstock repo. Those still use the old definitions because this migration https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/r-base44_and_m2w64-ucrt.yaml is still not completed. If I understand correctly, you need to manually enter the new definitions from that migration in your project's conda_build_config.yaml.

@VDovidaytis-HORIS
Copy link
Author

I believe the issue is that those definitions are derived from conda_build_config.yamlin the conda-forge-pinning-feedstock repo. Those still use the old definitions because this migration https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/r-base44_and_m2w64-ucrt.yaml is still not completed. If I understand correctly, you need to manually enter the new definitions from that migration in your project's conda_build_config.yaml.

Thank you for the answer! Unfortunately, it didn’t work. It seems that we need to wait for official MinGW support from the conda-forge team.

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
question
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mfansler @VDovidaytis-HORIS @owinebar