gh-115119: Default to --with-system-libmpdec=yes

[erlend-aasland]

Mark _decimal as missing if libmpdecimal is not found.

• Issue: Unvendor libmpdec sources #115119

---

📚 Documentation preview 📚: https://cpython-previews--118539.org.readthedocs.build/

[erlend-aasland]

$ ./configure --with-system-libmpdec=no | grep -E "(decimal|mpdec)"
checking for --with-system-libmpdec... no
checking for --with-decimal-contextvar... yes
checking for decimal libmpdec machine... universal
checking for stdlib extension module _decimal... yes

$ ./configure | grep -E "(decimal|mpdec)"
checking for --with-system-libmpdec... yes
checking for libmpdec... yes
checking for --with-decimal-contextvar... yes
checking for stdlib extension module _decimal... yes
$ grep -E "_DECIMAL_(CFLAGS|LDFLAGS)=" Makefile
MODULE__DECIMAL_CFLAGS=-I/opt/homebrew/Cellar/mpdecimal/4.0.0/include
MODULE__DECIMAL_LDFLAGS=-L/opt/homebrew/Cellar/mpdecimal/4.0.0/lib -lmpdec -lm

[erlend-aasland]

@zware: I'm not sure if we should fall back to the internal copy if libmpdec is missing. For now, it is marked as missing if it is missing :) We could change it so configure fails if --with-system-libmpdec=yes and no mpdecimal library was found.

[erlend-aasland]

Things to consider:

• Ubuntu 20.04 provides libmpdecimal 2.4 (no pkgconfig support): apt install libmpdec2
• Ubuntu 22.04 provides libmpdecimal 2.5 (no pkgconfig support): apt install libmpdec3 (!)
• Ubuntu 24.04 does not appear to provide libmpdecimal at all

For now, we can consider to use the bundled version for the Ubuntu CI.

[zware]

@zware: I'm not sure if we should fall back to the internal copy if libmpdec is missing. For now, it is marked as missing if it is missing :) We could change it so configure fails if --with-system-libmpdec=yes and no mpdecimal library was found.

Right, an explicit --with-system-libmpdec argument should be followed, failing if the system library is not there. --with-system-libmpdec=no/--without-system-libmpdec should warn, and no argument with no system library available should also warn. Whatever I said in the issue was really shorthand for "do whatever we did when we ripped out libffi because that seemed to work¹" which I think is basically what I've now laid out here :)

Footnotes

1. though that one was complicated by differing timelines on Linux and macOS which we don't need to deal with here ↩

[erlend-aasland]

Right, an explicit --with-system-libmpdec argument should be followed, failing if the system library is not there.

Yep

[...] --with-system-libmpdec=no/--without-system-libmpdec should warn, [..]

Yes, we need to warn for that case.

[...] and no argument with no system library available should also warn.

Hm, I think this case should fail, similar to how we handle --with-system-libmpdec=yes and no library found.

[zware]

[...] and no argument with no system library available should also warn.

Hm, I think this case should fail, similar to how we handle --with-system-libmpdec=yes and no library found.

I don't want a bare ./configure to start failing if mpdecimal isn't found, since that's not going to be the behavior even after the bundled library is removed. I could maybe get behind allowing the _decimal module build to fail rather than build the bundled library in that case, but even then we should also emit a warning from configure to make the issue more obvious.

[erlend-aasland]

Ok, how about this:

• bare (no --with-system-libmpdec): warn if system library is not found, mark _decimal as missing
• --with-system-libmpdec=yes: mark _decimal as missing
• --with-system-libmpdec=no: use bundled version, warn that it will be removed in Python 3.15

[zware]

I have a couple of questions and qualms, but overall I'm okay with this as is. We can tweak it through the beta phase if needed.

[erlend-aasland]

@hugovk: FYI, this will fail for Ubuntu 24.04, as libmpdec-dev is unavailable there.

[zware]

I've been trying to figure out why that is, and have come up short (I get lost quickly in the Debian/Ubuntu development process). All I've found is a note from @doko42 in the Ubuntu python3.11 3.11.2-4 changelog, saying "Build with internal mpdecimal library, so that mpdecimal can be removed for bookworm." (and indeed, Debian bookworm does not have a libmpdec package either).

[erlend-aasland]

Found it:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013395

[erlend-aasland]

... and:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056785

[zware]

I've been trying to figure out why that is, and have come up short (I get lost quickly in the Debian/Ubuntu development process). All I've found is a note from @doko42 in the Ubuntu python3.11 3.11.2-4 changelog, saying "Build with internal mpdecimal library, so that mpdecimal can be removed for bookworm." (and indeed, Debian bookworm does not have a libmpdec package either).

[erlend-aasland]

Thanks for helping out, Zach!

[asottile]

fwiw this is going to be painful for ubuntu and debian which has removed the libmpdec system library entirely (as it was only used for building python). also this was done pretty close to the beta freeze so it definitely caught me off guard (and broke the deadsnakes builds!)

[thesamesam]

That seems to have been mentioned at #118539 (comment) already, but I'm not sure what the problem is. We didn't have libmpdec packaged in Gentoo until now but we've done it and now all is fine...

Why can't Debian/Ubuntu just set --with-system-libmpdec=no or --without-system-libmpdec (whichever it is) if they can't package libmpdec quickly?

[erlend-aasland]

@thesamesam, @asottile: we're adding a fallback to the bundled version if an external libmpdec cannot be found.

FTR, I find the recent removal of libmpdec from Ubuntu/Debian very strange.