-
Notifications
You must be signed in to change notification settings - Fork 13.2k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Add a new cygwin target #79854
Comments
@mintty something strange is going on in your environment: |
No, the file is there. |
What's the status now? Seems the cygwin support was asked many years ago in other threads. I'm installing a python package in cygwin, that thing depends on cryptography, which depends on a rust compier. |
First attempt (python x.py): but the file does exist Second attempt (./x.py): Installed package mingw64-i686-gcc-core Third attempt (./x.py): |
It builds fine in MSYS2 so there should be no problem with Cygwin.
This is unexpected, seems Cygwin path was passed instead of Windows path.
This output doesn't contain the root cause of the failure. You should look higher up in the terminal (or even paste whole output somewhere) and if there is nothing interesting re-run with |
Maybe, but for building a cygwin package a cygwin toolchain is essential. |
Ah, I think I know what it is. I think all Cygwin packages do use UNIX style paths but native Windows applications use only Windows style paths. So there are two problems here:
So to do it that way you'd need to convert paths using Side question: do you want Cygwin host+target or Cygwin host but targeting only MinGW? |
I'm only interested in cygwin target. I have no capacity to work on your build, anyway. I was just reporting as I think (quoting from above)
|
This statement is not true when it comes to the compilers though. |
Compilers translate source code to machine code. It doesn't involve low level OS specific things like the device drivers. I don't see why this particular compiler is special. No official support of cygwin rust compiler package is understandable, but not even be able to build from source code is hard to understand. |
No matter if its C++ or Rust, std library is all about using OS specific code to give users cross-platform and easy to use abstractions though.
All the errors in this issue are about the paths, you can take any Win32 native application and it won't be able to take or output UNIX paths used by Cygwin because Win32 API itself doesn't support these paths. |
I guess something in the Rust build scripts is confused by the cygwin environment, misinterpreting it as a Windows environment and thus using wrong path syntax. It should be easy to find that place for those familiar with the build scripts. |
I think it has nothing to do with build scripts, instead the issue is within Python that I mentioned earlier. IIUC this line passes Cygwin path to Cargo: rust/src/bootstrap/bootstrap.py Line 759 in 9fb32dc
Still there is 2nd part of path issue, Rust calls linker (GCC in this case) with Win32 paths but since it's Cygwin based executable it can work only with Cygwin paths. |
This composes os.path.join(self.rust_root, "src/bootstrap/Cargo.toml") which works fine in cygwin, with cygwin python. |
No, it's created using relative paths to Python file and then normalised: rust/src/bootstrap/bootstrap.py Line 846 in 9fb32dc
|
Cygwin often (but not always) can understand and use Windows-style paths just fine; the problems I've run into have mostly been of the "Windows executable doesn't understand Cygwin-style path" variety (although that's started to change recently; I'm seeing more The first question that occurs to my is why the build wants to use rust/src/bootstrap/bootstrap.py Lines 268 to 289 in 41ad4d9
and Lines 47 to 49 in fee3a45
which seems to indicate that the rust source pretends that, say i686-pc-cygwin on a 64-bit Windows is actually x86_64-w64-mingw32 , that is, the code base as-is explicitly disallows attempting to create a true Cygwin-native Rust; it will create either a Windows-native version of Rust or maybe an MSYS version of Rust, both already available from https://forge.rust-lang.org/infra/other-installation-methods.html#other-ways-to-install-rustup.
To build an actual Cygwin version of Rust, one would first need to change the code in both the above places so the code sees Cygwin as a thing distinct from MSVC, MinGW, and MSYS, then figure out which of the tests for "is this Windows?" are actually for:
Some of these questions have the same answers in MSVC and Cygwin, so that treating them the same way makes sense; others have different answers in MSVC and Cygwin, and must be treated differently. I've run into both kinds of problems in different packages. It's probably possible to disentangle those questions; as a starting point, the API (C-level and above) is closest to BSD (not usually too far from Linux aside from not using In short, earlier comments suggest OP wants to build a Rust compiler that is itself a Cygwin executable that generates Cygwin executables; the alternative is that they want to use Cygwin to provide a build platform to (cross-)compile their own version of the Rust compilers available from |
As I already said there is no Cygwin target right now so you have to add it first to build Rust as Cygwin application. As for the build system detection or MinGW you can override it with |
I'd like to have a go building Rust for Cygwin. I see the bootstrapping process begins with x.py, but even it seems to start by downloading some pre-built binaries. Can anyone show me full bootstrap instructions from nothing but source code? |
Sorry but any chance of a little more hand-holding? Whatever I do x.py seems to start by downloading |
@jschultz this target doesn't exist yet, so you need to cross compile to it from an existing target. That could be Linux or windows-gnu, but you can't build natively on cygwin until it's been upstreamed (and we'll likely not make it a tier 2 target right away so you'll still need to cross compile). See https://doc.rust-lang.org/nightly/rustc/platform-support.html for more info about targets. |
DWEST asked these questions above., thought I would take a stab at them:
file /usr/bin/cat/usr/bin/cat: PE32+ executable (console) x86-64, for MS Windows, 12 sections does the linker need to be able to resolve all symbols, or can that wait until run time?AFAIK, links are resolved at link time against known libs.
I think it is PATH.
I don't think those calls are part of the POSIX API.
LF
I don't know that Windows-native executables behavior is defined in POSIX.
Not unless the Windows API funcs are defined in POSIX or some compatibility lib.
Is that something that POSIX specifies? Usually config scripts test things like
Cygwin's raison d'etre is to provide a POSIX compat layer so POSIX apps can easily be run
That's not specified by POSIX, but I would lean toward 8-bits/char. No
Cygwin automatically translates such usage into neutered pathnames suitable for the
PATH is used as it would be under linux. Generally I can take 'rpm's from linux and build them in Cygwin |
Your point-by-point answers will likely be appreciated by any Windows- or Linux-centric developer trying to get Rust compiling on Cygwin As you may have noticed, the Windows option is the first given in each question except the last: the reason I posed the question that way was that rust, at the time, assumed that when I executed I have seen configuration tie the answers to all of those questions to a simple binary "Does the platform name contain the string 'win'?" or "Is the platform Windows?". Cygwin complicates the issue, with some questions answered the same as on Windows (the first three questions), and others closer to their answers on BSD (most of the rest, except the declspec and Windows API ones). Most of the answers above were correct, just a few clarifications:
Apparently Cygwin defines these in a compatibility header, which you can technically use if you are very, very careful about types because of the type mismatch mentioned below. I wouldn't recommend trying it if there is any other option.
Windows is LLP64, while Linux and Cygwin are LP64 so
Windows defaults to UTF-16, Cygwin seems to default to UTF-8 these days, though both probably used to vary by region.
Usually, yes, but not always. Many packages assume that linking some library will pull in it's dependent library's symbols at runtime, so they don't need to bother with linking those. On Windows and Cygwin, this is an error; the dependent library must be specified at link time. A couple will try some ELF-specific thing and crash, or try to use functions from GNU Libc and also crash. Rust assumes that Cygwin == Windows, perhaps because the former includes "win" as a substring. Many packages, when first porting from Linux to Windows, will assume for simplicity that answering one of these questions will provide the answer for all the others. Porting to Cygwin involves the process of decoupling those answers into two to three groups. (Requiring all symbols be resolvable at runtime and refusing to recognize |
On 2023/03/27 17:15, DWesl wrote:
are the type sizes LLP64 or LP64? (is long 32 bits or 64 on a
64-bit platform?)
Is that something that POSIX specifies? Usually config scripts
test things like that.
Windows is LLP64, while Linux is LP64 so |sizeof(long)| won't match
assumptions.
I suspect this is outside the scope of POSIX.
---
But those sizes have never been constants -- every source needs to
test for
sizes if they rely on them.
Windows defaults to UTF-16, Cygwin seems to default to UTF-8 these
days, though this used to vary by region.
---
Internally Windows defaults to UCS-2LE, not UTF-16. UCS-2LE
allocates 2 bytes/character
but no way to extend beyond 16-bits -- that came later in user compat
libs outside the
kernel. Windows is mostly UTF-8 to be compatible with the web. It's
rare to find
a website hosted on windows in UTF-16. What can be worse is that many
people have
the mistaken assumption that USC-2 will always include a BOM so LE can
be differentiated
from BE. Files coming directly out of the kernel (log files,
especially) often
don't have BOM's. It's smarter to assume LE because of Windows being
native to
LE processors, though some BE processors have been supported over the years.
But none of that has anything to do with POSIX compatibility.
Generally I can take 'rpm's from linux and build them in Cygwin
with no changes (presuming they put binaries in places like
'/usr/bin'.
Usually, yes, but not always.
---
Nothing is ever "always", otherwise it is identical.
Many packages assume that linking some library will pull in it's
dependent library's symbols at runtime, so they don't need to bother
with linking those. On Windows and Cygwin, this is an error; the
dependent library must be specified at link time. A couple will try
some ELF-specific thing and crash, or try to use functions from GNU
Libc and also crash.
---
Of course, windows is COFF, not ELF. But you can use DLL's (Delayed
Load Libraries), to
have symbols linked at runtime (even if they had to be resolved at link
time).
Many packages, when first porting from Linux to Windows, will assume
for simplicity that answering one of these questions will provide the
answer for all the others.
---
That would be a naive assumption.
Porting to Cygwin involves the process of decoupling those answers
into two to three groups. (Requiring all symbols be resolvable at
runtime and refusing to recognize |rpath| as a concept doesn't imply a
thing about whether tests should expect CRLF line endings or 32-bit
long on 64-bit platforms).
---
That's why Gnu's 'configure' was invented -- it tested each item so
programs using it
could be run on a large number of target platforms. Now people try to
make due with
less precise auto-config tools that make invalid assumptions by trying
to group all target
platforms into 2-3 groups -- which just won't work in reality.
|
The compiler needs to know the size of pointers on the target for which it's generating object code. If you want to support both LLP64 and and LP64 on cygwin, you will need two different target specs. |
hi, if someone want to start a Rust port for Cygwin, my previous attempt may be somewhat useful: https://gist.github.com/ookiineko/057eb3a91825313caeaf6d793a33b0b2 i have added a initial cygwin target in my fork just for fun But I was unable to fix the stack unwinding problem (i am just lack of that exp), currently it will crash on panicking and will not show the back trace...
so currently it uses a Fedora host for cross compiling programs (unfortunately Fedora Cygwin is also a bit outdated, but the cross GCC is still working, things can still compile if i steal the sysroot from a Cygwin installation |
I sure hope a solution can be found for this as Cygwin will be in a slow degrade as a result of the underlying issue. Anybody make a write-up on this; seems like a good hacker news post ;^) |
currently the main issue is Cygwin lacks of maintainers for LLVM packages and the those packages are outdated (doesn't affect cross-compiling from other platform with latest LLVM like Fedora Linux), this should get fixed with some efforts. a rough example is here (9.0 -> 15.0, cygwin-apps seems to require Ocaml binding to work when upgrading the packages, which involves reviving not just the LLVM packages, but also the Ocaml ones. that's quite a lot of efforts. after that some initial work for adding the Cygwin target for Rust/STD support/libc crate is done in here (Rust 1.67.0 nightly, because that's the latest version when i started to work on it, currently also see the related thread i started at the cygwin-apps mailing list (July, Aug, Sep): https://cygwin.com/pipermail/cygwin-apps/2023-July/043048.html i do want to share my journey porting Rust and cross-compiling rust programs for running on Cygwin, but it's buggy and full of hack right now, and i don't have a blog so i don't know when i will do that... (EDIT: maybe i will write one after i get the host tools (like rustc and cargo) working directly on cygwin xd |
finally now i have managed to get an experimental llvm15 toolchain patched (including clang and libc++) for cygwin and get it to do sth btw i have now cross-compiled host rustc for cygwin, it starts, but currently not good enough to compile programs (as all kinds of small bugs are starting to show up
also there seems to be a guy in the mailing list who is planning to become new maintainer of the llvm pkgs |
did you manage create a rust toolchain for cygwin? |
for now i have a working cross toolchain on linux, that is able to cross-compile some cygwin programs. for a rust toolchain on cygwin, i haven't yet got it working. although i have i have a weird issue with LLVM initialization in rustc, it jumps to I'm still trying to figure out why. I don't have a lot of debugging experience with gdb, but i will try to learn more and diagnose |
can you document your steps, maybe someone can help |
sure, im currently setting up a new set of cygwin cross compilers at arch-cygwin (since the fedora cygwin i was using is inactive), i will write down my journey and a build guide in details (probably make it a blog) once im ready |
that's great to hear |
I should note we'd probably love to have more maintainer support for the |
They were accepted as tier 2: #121712 but they are blocked on |
i just have achieved cross-compiling on a mingw host in the last few days: https://github.com/ookiineko-cygport/rust/releases/test-build so now it should be easier to test and debug this thing (cross compiling from linux host is no longer required i have also tested a temp quick hack to make it link against |
@ghost : the gist gives me a 404 , can you reshare it again? |
ghost means the user deactivated the account |
It looks like the repositories moved to @ookiineko-cygport at least. |
yep, we enjoy this https://github.com/ookiineko-cygport/rust |
now the only thing left is try to bootrstrap it from mrustc |
It'd be a lot easier to cross compile from another existing target than to bootstrap from mrustc once the necessary changes are merged. |
We write pull requests for mrustc or patch it |
Since I've seen people be confused by being linked to this issue: a cygwin target has been merged with #134999 |
yes. was closed as wontfix (or I suppose "PRs welcome"?). closing as completed. |
Opened a new issue for "the target works enough to be built as a host", since there's been so many requests for it: #137819 |
According to the sparse build instructions available, rust should be built by running x.py.
After a bunch of secondary downloads, removing obstacles with curl, it fails at:
error: manifest path
/cygdrive/c/tmp/rust/src/bootstrap/Cargo.toml
does not exist
failed to run: /cygdrive/c/tmp/rust/build/i686-pc-windows-gnu/stage0/bin/cargo build --manifest-path /cygdrive/c/tmp/rust/src/bootstrap/Cargo.toml
I found #5526 (comment) but actually any package that builds on Linux should generally also be buildable in the cygwin Posix environment.
The text was updated successfully, but these errors were encountered: