Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

PyROOT update broke some existing code #15117

Closed
1 task done
tmadlener opened this issue Apr 3, 2024 · 7 comments · Fixed by #15125
Closed
1 task done

PyROOT update broke some existing code #15117

tmadlener opened this issue Apr 3, 2024 · 7 comments · Fixed by #15125

Comments

@tmadlener
Copy link
Contributor

tmadlener commented Apr 3, 2024

Check duplicate issues.

  • Checked for duplicates

Description

The recent update to PyROOT, broke some existing code on our end. It's not entirely trivial but I have managed to put together a "minimal" reproducer that triggers the issue. I am fairly certain that the new behavior is wrong, but I am not completely certain.

This might also be related to #15085, but I am not entirely sure, as that could also be a c++ standard issue(?). I am also pretty sure that this is an issue in PyROOT and not in cppyy (see at the very bottom).

Reproducer

import ROOT

ROOT.gInterpreter.LoadText(
    """
#include <memory>
#include <iostream>
#include <type_traits>

namespace repro {

struct Base {
  virtual ~Base() = default;
  virtual int func() const = 0;
};

struct Derived : public Base {
  Derived(int i) : m_i(i) {}
  ~Derived() = default;
  Derived(const Derived&) = delete;
  Derived& operator=(const Derived&) = delete;
  Derived(Derived&&) = default;
  Derived& operator=(Derived&&) = default;

  int func() const final { return m_i; }
private:
  int m_i{42};
};

void foo(std::unique_ptr<Base> basePtr) {
   std::cout << "via unique_ptr: " << basePtr->func() << std::endl;
}

template<typename T, typename = std::enable_if_t<std::is_base_of_v<Base, T> &&
                                                !std::is_lvalue_reference_v<T>>>
void foo(T&& t) {
  std::cout << "via template func: " << t.func() << std::endl;
}

} // namespace repro
"""
)

from ROOT import repro

c = repro.Derived(123)
repro.foo(ROOT.std.move(c))

ROOT version

This works with 6.30.04 (and previous) but starts to fail with the current ROOT master (via dev3 on ubuntu2204).

Installation method

6.30.04 via spack, ROOT master via LCG dev3 nightly build

Operating system

Ubuntu 22.04

Additional context

With 6.30.04 I get

$ python repro.py
via template func: 123

However, with the updated cppyy I get:

$ python repro.py 
input_line_42:6:92: error: call to deleted constructor of 'std::unique_ptr<repro::Base, std::default_delete<repro::Base> >'
   ((void (&)(std::unique_ptr<repro::Base, std::default_delete<repro::Base> >))repro::foo)(*(std::unique_ptr<repro::Base, std::default_delete<repro::Base> >*)args[0]);
                                                                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/11/bits/unique_ptr.h:468:7: note: 'unique_ptr' has been explicitly marked deleted here
      unique_ptr(const unique_ptr&) = delete;
      ^

Which is in principle correct, but I don't understand why it chooses that overload in the first place, as it should use the templated function instead.

Curiously, one thing that makes things "work" with the current master branch is to change

- void foo(std::unique_ptr<Base> basePtr) {
+ void foo(std::unique_ptr<Base>&& basePtr) {

However, that changes behavior slightly because now this calls the unique_ptr overload instead:

$ python repro.py 
via unique_ptr: 123

Additionally, this change breaks with root 6.30.04 leading to a segmentation because it also tries to call the wrong overload.

stack trace
$ python repro.py
via unique_ptr:  *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 2 (Thread 0x7af5721ff640 (LWP 292770) "python"):
#0  __futex_abstimed_wait_common64 (private=-392944720, cancel=true, abstime=0x7af5721fe520, op=137, expected=0, futex_word=0x7af5875bf86c <_PyRuntime+428>) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=-392944720, abstime=0x7af5721fe520, clockid=-2029319085, expected=0, futex_word=0x7af5875bf86c <_PyRuntime+428>) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word
entry=0x7af5875bf86c <_PyRuntime+428>, expected=expected
entry=0, clockid=clockid
entry=1, abstime=abstime
entry=0x7af5721fe520, private=private
entry=0) at ./nptl/futex-internal.c:139
#3  0x00007af586e93e9b in __pthread_cond_wait_common (abstime=0x7af5721fe520, clockid=1, mutex=0x7af5875bf870 <_PyRuntime+432>, cond=0x7af5875bf840 <_PyRuntime+384>) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_timedwait64 (cond=cond
entry=0x7af5875bf840 <_PyRuntime+384>, mutex=mutex
entry=0x7af5875bf870 <_PyRuntime+432>, abstime=abstime
entry=0x7af5721fe520) at ./nptl/pthread_cond_wait.c:652
#5  0x00007af5873c5420 in PyCOND_TIMEDWAIT (us=<optimized out>, mut=0x7af5875bf870 <_PyRuntime+432>, cond=0x7af5875bf840 <_PyRuntime+384>) at Python/condvar.h:73
#6  take_gil (tstate=tstate
entry=0x5fe0e89423b0) at Python/ceval_gil.h:256
#7  0x00007af5873c59a2 in PyEval_RestoreThread (tstate=tstate
entry=0x5fe0e89423b0) at Python/ceval.c:547
#8  0x00007af58747886a in pysleep (secs=<optimized out>) at ./Modules/timemodule.c:2077
#9  time_sleep (self=<optimized out>, obj=<optimized out>) at ./Modules/timemodule.c:370
#10 0x00007af587315dcd in cfunction_vectorcall_O (func=0x7af58650bc40, args=0x7af5723a2d98, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:516
#11 0x00007af5872750af in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af5723a2d98, callable=0x7af58650bc40, tstate=0x5fe0e89423b0) at ./Include/cpython/abstract.h:114
#12 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=0x7af58650bc40) at ./Include/cpython/abstract.h:123
#13 call_function (kwnames=0x0, oparg=1, pp_stack=<synthetic pointer>, trace_info=0x7af5721fe6f0, tstate=0x5fe0e89423b0) at Python/ceval.c:5893
#14 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4181
#15 0x00007af5873c6e54 in _PyEval_EvalFrame (throwflag=0, f=0x7af5723a2c20, tstate=0x5fe0e89423b0) at ./Include/internal/pycore_ceval.h:46
#16 _PyEval_Vector (tstate=0x5fe0e89423b0, con=0x7af586cd8050, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5067
#17 0x00007af58726d6f0 in do_call_core (kwdict=0x7af5722c2680, callargs=0x7af586cc3be0, func=0x7af586cd8040, trace_info=0x7af5721fe8d0, tstate=0x5fe0e89423b0) at Python/ceval.c:5945
#18 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4277
#19 0x00007af5873c6e54 in _PyEval_EvalFrame (throwflag=0, f=0x7af5722543c0, tstate=0x5fe0e89423b0) at ./Include/internal/pycore_ceval.h:46
#20 _PyEval_Vector (tstate=0x5fe0e89423b0, con=0x7af5864a55b0, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5067
#21 0x00007af5872729fc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af586cb1130, callable=0x7af5864a55a0, tstate=0x5fe0e89423b0) at ./Include/cpython/abstract.h:114
#22 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=0x7af5864a55a0) at ./Include/cpython/abstract.h:123
#23 call_function (kwnames=0x0, oparg=1, pp_stack=<synthetic pointer>, trace_info=0x7af5721feab0, tstate=0x5fe0e89423b0) at Python/ceval.c:5893
#24 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4198
#25 0x00007af5873c6e54 in _PyEval_EvalFrame (throwflag=0, f=0x7af586cb0fc0, tstate=0x5fe0e89423b0) at ./Include/internal/pycore_ceval.h:46
#26 _PyEval_Vector (tstate=0x5fe0e89423b0, con=0x7af5864a5880, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5067
#27 0x00007af5872729fc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af5723f1950, callable=0x7af5864a5870, tstate=0x5fe0e89423b0) at ./Include/cpython/abstract.h:114
#28 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=0x7af5864a5870) at ./Include/cpython/abstract.h:123
#29 call_function (kwnames=0x0, oparg=1, pp_stack=<synthetic pointer>, trace_info=0x7af5721fec90, tstate=0x5fe0e89423b0) at Python/ceval.c:5893
#30 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4198
#31 0x00007af5873c6e54 in _PyEval_EvalFrame (throwflag=0, f=0x7af5723f17e0, tstate=0x5fe0e89423b0) at ./Include/internal/pycore_ceval.h:46
#32 _PyEval_Vector (tstate=0x5fe0e89423b0, con=0x7af5864a5640, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:5067
#33 0x00007af5872cd9bc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7af5721feda8, callable=0x7af5864a5630, tstate=0x5fe0e89423b0) at ./Include/cpython/abstract.h:114
#34 method_vectorcall (method=<optimized out>, args=0x7af586bf8088, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:61
#35 0x00007af58747b509 in thread_run (boot_raw=0x7af5722ab3f0) at ./Modules/_threadmodule.c:1100
#36 0x00007af58741c5ab in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:248
#37 0x00007af586e94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#38 0x00007af586f26850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 1 (Thread 0x7af58764d740 (LWP 292754) "python"):
#0  0x00007af586eea42f in __GI___wait4 (pid=292773, stat_loc=stat_loc
entry=0x7ffea3a8ed78, options=options
entry=0, usage=usage
entry=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
#1  0x00007af586eea3ab in __GI___waitpid (pid=<optimized out>, stat_loc=stat_loc
entry=0x7ffea3a8ed78, options=options
entry=0) at ./posix/waitpid.c:38
#2  0x00007af586e50bdb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:171
#3  0x00007af586112468 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x5fe0e479def0) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/core/unix/src/TUnixSystem.cxx:2120
#4  TUnixSystem::StackTrace (this=0x5fe0e479def0) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/core/unix/src/TUnixSystem.cxx:2411
#5  0x00007af58760c5a4 in (anonymous namespace)::do_trace (sig=1) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:183
#6  (anonymous namespace)::TExceptionHandlerImp::HandleException (this=<optimized out>, sig=1) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:196
#7  0x00007af586111799 in TUnixSystem::DispatchSignals (this=0x5fe0e479def0, sig=kSigSegmentationViolation) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/core/unix/src/TUnixSystem.cxx:3626
#8  <signal handler called>
#9  0x00007af5858a404a in ?? ()
#10 0x00007af585c29540 in ?? () from /home/tmadlener/work/.spack/spackages/gcc-runtime/12.3.0/skylake-ubuntu22.04-gcc12.3.0/hio32wy/lib/libstdc++.so.6
#11 0x00005fe0e89d9090 in ?? ()
#12 0x00007ffea3a91a60 in ?? ()
#13 0x00007af5858a1017 in ?? ()
#14 0x0000000000000001 in ?? ()
#15 0x00007af587611dfe in WrapperCall (method=<optimized out>, nargs=140731644189264, args_=0x7ffea3a91d50, self=0x0, result=0x0) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:785
#16 0x00007af5870bd70b in GILCallV (ctxt=0x7ffea3a91d30, self=0x0, method=105419582700640) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:68
#17 CPyCppyy::(anonymous namespace)::VoidExecutor::Execute (this=<optimized out>, method=105419582700640, self=0x0, ctxt=0x7ffea3a91d30) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:410
#18 0x00007af5870a95eb in CPyCppyy::CPPMethod::ExecuteFast (self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>, this=<optimized out>) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:74
#19 0x00007af5870ac098 in CPyCppyy::CPPMethod::ExecuteProtected (this=this
entry=0x5fe0e8c18c70, self=0x0, offset=0, ctxt=0x7ffea3a91d30) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:149
#20 0x00007af5870aa84d in CPyCppyy::CPPMethod::Execute (this=0x5fe0e8c18c70, self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:786
#21 0x00007af5870b0eff in CPyCppyy::(anonymous namespace)::mp_call (pymeth=0x7af5722cc820, args=0x7af5722a9750, kwds=0x7af5722d6280) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPOverload.cxx:567
#22 0x00007af5870dddb3 in CPyCppyy::tpp_call (pytmpl=0x7af5722e0040, args=0x7af5722a9750, kwds=0x7af5722d6280) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/TemplateProxy.cxx:621
#23 0x00007af5872cacc4 in _PyObject_MakeTpCall (tstate=tstate
entry=0x5fe0e474d0c0, callable=callable
entry=0x7af5722e0040, args=args
entry=0x7af586c1dbb0, nargs=1, keywords=keywords
entry=0x0) at Objects/call.c:215
#24 0x00007af5872750f2 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af586c1dbb0, callable=0x7af5722e0040, tstate=0x5fe0e474d0c0) at ./Include/cpython/abstract.h:112
#25 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af586c1dbb0, callable=0x7af5722e0040, tstate=0x5fe0e474d0c0) at ./Include/cpython/abstract.h:99
#26 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=0x7af5722e0040) at ./Include/cpython/abstract.h:123
#27 call_function (kwnames=0x0, oparg=1, pp_stack=<synthetic pointer>, trace_info=0x7ffea3a920b0, tstate=0x5fe0e474d0c0) at Python/ceval.c:5893
#28 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4181
#29 0x00007af5873c6ce1 in _PyEval_EvalFrame (throwflag=0, f=0x7af586c1da40, tstate=0x5fe0e474d0c0) at ./Include/internal/pycore_ceval.h:46
#30 _PyEval_Vector (kwnames=0x0, argcount=0, args=0x0, locals=0x5fe0e474d0c0, con=0x7ffea3a921a0, tstate=0x5fe0e474d0c0) at Python/ceval.c:5067
#31 PyEval_EvalCode (co=co
entry=0x7af586c71b00, globals=globals
entry=0x7af586550200, locals=locals
entry=0x7af586550200) at Python/ceval.c:1134
#32 0x00007af58740d379 in run_eval_code_obj (locals=0x7af586550200, globals=0x7af586550200, co=0x7af586c71b00, tstate=0x5fe0e474d0c0) at Python/pythonrun.c:1291
#33 run_mod (mod=<optimized out>, filename=filename
entry=0x7af5865a5e30, globals=globals
entry=0x7af586550200, locals=locals
entry=0x7af586550200, flags=flags
entry=0x7ffea3a92318, arena=arena
entry=0x7af5865026f0) at Python/pythonrun.c:1312
#34 0x00007af58740e5a8 in pyrun_file (flags=0x7ffea3a92318, closeit=-2040898000, locals=0x7af586550200, globals=0x7af586550200, start=257, filename=0x7af5865a5e30, fp=0x5fe0e477c7f0) at Python/pythonrun.c:1208
#35 _PyRun_SimpleFileObject (fp=fp
entry=0x5fe0e477c7f0, filename=filename
entry=0x7af5865a5e30, closeit=closeit
entry=1, flags=flags
entry=0x7ffea3a92318) at Python/pythonrun.c:456
#36 0x00007af58740ec10 in _PyRun_AnyFileObject (fp=fp
entry=0x5fe0e477c7f0, filename=filename
entry=0x7af5865a5e30, closeit=closeit
entry=1, flags=flags
entry=0x7ffea3a92318) at Python/pythonrun.c:90
#37 0x00007af58742f688 in pymain_run_file_obj (skip_source_first_line=0, filename=0x7af5865a5e30, program_name=0x7af586553cb0) at Modules/main.c:353
#38 pymain_run_file (config=0x5fe0e4731230) at Modules/main.c:372
#39 pymain_run_python (exitcode=0x7ffea3a92310) at Modules/main.c:587
#40 Py_RunMain () at Modules/main.c:666
#41 0x00007af58742fc4e in pymain_main (args=0x7ffea3a92400) at Modules/main.c:696
#42 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:720
#43 0x00007af586e29d90 in __libc_start_call_main (main=main
entry=0x5fe0e3303060 <main>, argc=argc
entry=2, argv=argv
entry=0x7ffea3a92588) at ../sysdeps/nptl/libc_start_call_main.h:58
#44 0x00007af586e29e40 in __libc_start_main_impl (main=0x5fe0e3303060 <main>, argc=2, argv=0x7ffea3a92588, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffea3a92578) at ../csu/libc-start.c:392
#45 0x00005fe0e3303095 in _start ()
===========================================================


The lines below might hint at the cause of the crash. If you see question
marks as part of the stack trace, try to recompile with debugging information
enabled and export CLING_DEBUG=1 environment variable before running.
You may get help by asking at the ROOT forum https://root.cern/forum
preferably using the command (.forum bug) in the ROOT prompt.
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern/bugs or (preferably) using the command (.gh bug) in
the ROOT prompt. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#9  0x00007af5858a404a in ?? ()
#10 0x00007af585c29540 in ?? () from /home/tmadlener/work/.spack/spackages/gcc-runtime/12.3.0/skylake-ubuntu22.04-gcc12.3.0/hio32wy/lib/libstdc++.so.6
#11 0x00005fe0e89d9090 in ?? ()
#12 0x00007ffea3a91a60 in ?? ()
#13 0x00007af5858a1017 in ?? ()
#14 0x0000000000000001 in ?? ()
#15 0x00007af587611dfe in WrapperCall (method=<optimized out>, nargs=140731644189264, args_=0x7ffea3a91d50, self=0x0, result=0x0) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:785
#16 0x00007af5870bd70b in GILCallV (ctxt=0x7ffea3a91d30, self=0x0, method=105419582700640) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:68
#17 CPyCppyy::(anonymous namespace)::VoidExecutor::Execute (this=<optimized out>, method=105419582700640, self=0x0, ctxt=0x7ffea3a91d30) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:410
#18 0x00007af5870a95eb in CPyCppyy::CPPMethod::ExecuteFast (self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>, this=<optimized out>) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:74
#19 0x00007af5870ac098 in CPyCppyy::CPPMethod::ExecuteProtected (this=this
entry=0x5fe0e8c18c70, self=0x0, offset=0, ctxt=0x7ffea3a91d30) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:149
#20 0x00007af5870aa84d in CPyCppyy::CPPMethod::Execute (this=0x5fe0e8c18c70, self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:786
#21 0x00007af5870b0eff in CPyCppyy::(anonymous namespace)::mp_call (pymeth=0x7af5722cc820, args=0x7af5722a9750, kwds=0x7af5722d6280) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/CPPOverload.cxx:567
#22 0x00007af5870dddb3 in CPyCppyy::tpp_call (pytmpl=0x7af5722e0040, args=0x7af5722a9750, kwds=0x7af5722d6280) at /home/tmadlener/work/.spack/spack-build-stage/spack-stage-root-6.30.04-26bzoxf2purznzcogecypb6xctskrjc3/spack-src/bindings/pyroot/cppyy/CPyCppyy/src/TemplateProxy.cxx:621
#23 0x00007af5872cacc4 in _PyObject_MakeTpCall (tstate=tstate
entry=0x5fe0e474d0c0, callable=callable
entry=0x7af5722e0040, args=args
entry=0x7af586c1dbb0, nargs=1, keywords=keywords
entry=0x0) at Objects/call.c:215
#24 0x00007af5872750f2 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af586c1dbb0, callable=0x7af5722e0040, tstate=0x5fe0e474d0c0) at ./Include/cpython/abstract.h:112
#25 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7af586c1dbb0, callable=0x7af5722e0040, tstate=0x5fe0e474d0c0) at ./Include/cpython/abstract.h:99
#26 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=<optimized out>, callable=0x7af5722e0040) at ./Include/cpython/abstract.h:123
#27 call_function (kwnames=0x0, oparg=1, pp_stack=<synthetic pointer>, trace_info=0x7ffea3a920b0, tstate=0x5fe0e474d0c0) at Python/ceval.c:5893
#28 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:4181
#29 0x00007af5873c6ce1 in _PyEval_EvalFrame (throwflag=0, f=0x7af586c1da40, tstate=0x5fe0e474d0c0) at ./Include/internal/pycore_ceval.h:46
#30 _PyEval_Vector (kwnames=0x0, argcount=0, args=0x0, locals=0x5fe0e474d0c0, con=0x7ffea3a921a0, tstate=0x5fe0e474d0c0) at Python/ceval.c:5067
#31 PyEval_EvalCode (co=co
entry=0x7af586c71b00, globals=globals
entry=0x7af586550200, locals=locals
entry=0x7af586550200) at Python/ceval.c:1134
#32 0x00007af58740d379 in run_eval_code_obj (locals=0x7af586550200, globals=0x7af586550200, co=0x7af586c71b00, tstate=0x5fe0e474d0c0) at Python/pythonrun.c:1291
#33 run_mod (mod=<optimized out>, filename=filename
entry=0x7af5865a5e30, globals=globals
entry=0x7af586550200, locals=locals
entry=0x7af586550200, flags=flags
entry=0x7ffea3a92318, arena=arena
entry=0x7af5865026f0) at Python/pythonrun.c:1312
#34 0x00007af58740e5a8 in pyrun_file (flags=0x7ffea3a92318, closeit=-2040898000, locals=0x7af586550200, globals=0x7af586550200, start=257, filename=0x7af5865a5e30, fp=0x5fe0e477c7f0) at Python/pythonrun.c:1208
#35 _PyRun_SimpleFileObject (fp=fp
entry=0x5fe0e477c7f0, filename=filename
entry=0x7af5865a5e30, closeit=closeit
entry=1, flags=flags
entry=0x7ffea3a92318) at Python/pythonrun.c:456
#36 0x00007af58740ec10 in _PyRun_AnyFileObject (fp=fp
entry=0x5fe0e477c7f0, filename=filename
entry=0x7af5865a5e30, closeit=closeit
entry=1, flags=flags
entry=0x7ffea3a92318) at Python/pythonrun.c:90
#37 0x00007af58742f688 in pymain_run_file_obj (skip_source_first_line=0, filename=0x7af5865a5e30, program_name=0x7af586553cb0) at Modules/main.c:353
#38 pymain_run_file (config=0x5fe0e4731230) at Modules/main.c:372
#39 pymain_run_python (exitcode=0x7ffea3a92310) at Modules/main.c:587
#40 Py_RunMain () at Modules/main.c:666
#41 0x00007af58742fc4e in pymain_main (args=0x7ffea3a92400) at Modules/main.c:696
#42 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:720
#43 0x00007af586e29d90 in __libc_start_call_main (main=main
entry=0x5fe0e3303060 <main>, argc=argc
entry=2, argv=argv
entry=0x7ffea3a92588) at ../sysdeps/nptl/libc_start_call_main.h:58
#44 0x00007af586e29e40 in __libc_start_main_impl (main=0x5fe0e3303060 <main>, argc=2, argv=0x7ffea3a92588, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffea3a92578) at ../csu/libc-start.c:392
#45 0x00005fe0e3303095 in _start ()
===========================================================


Traceback (most recent call last):
  File "/home/tmadlener/work/playground/python/pyroot_update_issue/repro.py", line 52, in <module>
    repro.foo(cppyy.gbl.std.move(c))
TypeError: Template method resolution failed:
  void repro::foo(unique_ptr<repro::Base,default_delete<repro::Base> >&& basePtr) =>
    SegmentationViolation: segfault in C++; program state was reset
  Failed to instantiate "foo(repro::Derived&)"
  Failed to instantiate "foo(repro::Derived*)"
  void repro::foo(repro::Derived&& t) =>
    ValueError: could not convert argument 1 (object is not an rvalue)

Finally, I have checked this with "pure" cppyy (3.1.2). There I observe the following:

  • If I have void foo(std::unique_ptr<Base>) (the original), then cppyy calls the templated version
  • If I have void foo(std::unique_ptr<Base>&&), then cppyy also calls the unique_ptr version
@guitargeek
Copy link
Contributor

Thank you so much for the detailed report! I see, so cppyy upstream made a change to core/metacling to make this case work. I have opened a PR suggesting the same change to ROOT. Let's see if it works.

At least the problem is understood!

guitargeek added a commit to guitargeek/root that referenced this issue Apr 4, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
@guitargeek
Copy link
Contributor

Okay, the updates to CallFunc would be too controversial. See also: #14426

Also, I don't like it that the unique_ptr overload is wrongly chosen, even if it doesn't harm in you example.

I suggest to disable the implicit promotion to smart pointers now:
#15125

Then the behavior of PyROOT is the same as in ROOT 6.30 for your reproducer, which I think is safest.

What do you think?

@guitargeek
Copy link
Contributor

And about the debatable overload choice, I suggest to open an issue in wlav/cppyy.

@guitargeek guitargeek added this to the 6.32/00 milestone Apr 4, 2024
@tmadlener
Copy link
Contributor Author

Thanks for the quick action on this one. Actually looking into #14425, I think this might be considered a duplicate of that. Sorry, I didn't see that before I opened this one.

Regarding the proposed changes in #15125, I am OK with that obviously, since it doesn't require work on our end ;) I am not sure how cppyy handles the lifetimes of objects that are referred to via a regular pointer, but an implicit conversion to a smart pointer seems like a good source of all kinds of issues there. So also from that point of view, I think it is safe to not have that conversion.

I will report the overload resolution issue upstream.

guitargeek added a commit that referenced this issue Apr 4, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes #15117.
@guitargeek
Copy link
Contributor

Marked a "Fixed in: not applicable" because this was a regression and not a bug in any released version of ROOT.

kristupaspranc pushed a commit to kristupaspranc/root that referenced this issue Apr 10, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
lobis pushed a commit to lobis/root that referenced this issue Apr 10, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
@andresailer
Copy link
Contributor

This needs a backport to 6.32, or maybe an additional fix, see #15125 (comment)

@andresailer andresailer reopened this Apr 16, 2024
guitargeek added a commit to guitargeek/root that referenced this issue Apr 23, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
@guitargeek
Copy link
Contributor

Thank you for the reminder @andresailer! Sorry that this one slipped through the cracks

guitargeek added a commit that referenced this issue Apr 23, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes #15117.
kristupaspranc pushed a commit to kristupaspranc/root that referenced this issue May 21, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
silverweed pushed a commit to silverweed/root that referenced this issue Aug 19, 2024
The automatic conversion of ordinary obejcts to smart pointers is
disabled for PyROOT because it can cause trouble with overload
resolution. If a function has overloads for both ordinary objects and
smart pointers, then the implicit conversion to smart pointers can
result in the smart pointer overload being hit, even though there would
be an overload for the regular object. Since PyROOT didn't have this
feature before 6.32 anyway, disabling it was the safest option.

Closes root-project#15117.
# for free to join this conversation on GitHub. Already have an account? # to comment