Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) #417

valassi · 2022-04-04T08:19:47Z

I am progressing in the integration of madevent+cudacpp (#400 and beyond, PR #401 and beyond).

The main issue I am facing now is that the MEs per event are different in fortran and in cuda/cpp. I am using the same type of debugging Stefan had used in his earlier tests, just caculate OUT with Fortran and OUT2 with cuda/cpp and print out the ratio.

In gg_tt, the results were wildly different. The ratios were huge: event number (in iteration with 16), OUT, OUT2, ratio

           1   2.7179477360093959E-006   3.8048997495080039        1399916.4513349019     
           2   2.4038671516498269E-003  0.49642094931550251        206.50931103858960     
           3   3.8625573348408305E-005   1.0461548930420270        27084.514282948174     
           4   1.5598005763660177E-003  0.44890337193428720        287.79536226363660     
           5   2.1373159576718779E-005  0.62881610141698518        29420.830325055824     
           6   3.1008689435420177E-005  0.54918963982598723        17710.830409964587     
           7   2.9484864958814827E-003  0.57067969683783715        193.55004597612245     
           8   6.2609580619723756E-004   1.0769120999539372        1720.0436247846069     
           9   6.0233031303646806E-005  0.47369098861178799        7864.3059855947913     
          10   1.6038037166377266E-006   1.1317896927132225        705690.90280321101     
          11   1.0426826876806022E-007   2.1018860475392533        20158443.909860998     
          12   1.1518358808597667E-005   2.5083158683912758        217766.77650630136     
          13   1.1234114251542622E-005  0.98776012624391241        87925.056139452849     
          14   2.4224907882032997E-003  0.51894860539999155        214.22108514389134     
          15   2.9500647392138135E-008   3.0360682133944707        102915308.02824269     
          16   6.2546173728503316E-004  0.32241178922449132        515.47803807151672     
           1   1.7029951282941172E-003  0.53612877811686066        314.81521538696268     
           2   2.9291733453278772E-006   1.2607876351277905        430424.38479743304     
           3   4.9347016092575566E-004  0.32938191546612056        667.48091687691169     
           4   1.6280666234303769E-004  0.49474672397459363        3038.8604302454773     
           5   1.4736053161971475E-006   1.1342960426203157        769742.09454369463     
           6   6.9846696062340651E-006  0.80332578412761235        115012.71060990711     
           7   5.4275677335086569E-006   1.2173073582027885        224282.29696469562     
           8   4.0578872911806462E-007   1.5364484218704257        3786326.0155345383     
           9   3.0325171903159342E-010   21.050238541199739        69415067483.942871     
          10   2.8918905478919862E-007   1.6530004691860378        5715985.5873209843     
          11   4.1024090171637163E-004  0.37673263740361568        918.32051808446306     
          12   4.3009140456843768E-005  0.57648363678235448        13403.746986313519     
          13   5.4088560373472470E-007   1.6124294061024296        2981091.3712046947     
          14   2.1578285273715184E-008   3.2148077890067244        148983468.71531665     
          15   1.7296118741249129E-003  0.53221685547275532        307.70883539523999     
          16   5.8969429096509206E-004  0.32227914777498462        546.51902301367625

In eemumu, the ratio is just a factor 2, constant, which is more reassuring

           1   9.8004965672104129E-003   1.9600993297408009E-002   2.0000000166305023     
           2   1.1724719298602063E-002   2.3449438792192107E-002   2.0000000166305032     
           3   1.4554142097642922E-002   2.9108284437328553E-002   2.0000000166305032     
           4   1.4477004470808850E-002   2.8954009182377564E-002   2.0000000166305028     
           5   1.6689698547106752E-002   3.3379397371771591E-002   2.0000000166305032     
           6   1.5519499291890769E-002   3.1038998841878617E-002   2.0000000166305028     
           7   1.6557597554898929E-002   3.3115195385159030E-002   2.0000000166305028     
           8   1.6681976010953982E-002   3.3363952299337613E-002   2.0000000166305028     
           9   5.1679312758123701E-003   1.0335862637570036E-002   2.0000000166305028     
          10   1.6214777951799174E-002   3.2429556173258253E-002   2.0000000166305023     
          11   1.0143021732339707E-002   2.0286043633362963E-002   2.0000000166305028     
          12   1.4779976051690746E-002   2.9559952349179922E-002   2.0000000166305028     
          13   4.9002280616957297E-003   9.8004562048847159E-003   2.0000000166305028     
          14   1.5500804764423547E-002   3.1001609786633282E-002   2.0000000166305036     
          15   1.4684660194587036E-002   2.9369320633387361E-002   2.0000000166305032     
          16   1.4714929896109829E-002   2.9429860036936338E-002   2.0000000166305028     
           1   5.1282892677137152E-003   1.0256578620713460E-002   2.0000000166305032     
           2   1.4281474270700567E-002   2.8562948778909240E-002   2.0000000166305036     
           3   1.6684724008223736E-002   3.3369448293922820E-002   2.0000000166305028     
           4   1.6102256423163881E-002   3.2204513114116380E-002   2.0000000166305028     
           5   1.1111533543840167E-002   2.2223067272470726E-002   2.0000000166305028     
           6   1.4573721940112907E-002   2.9147444122594134E-002   2.0000000166305028     
           7   1.5999470038856736E-002   3.1998940343792706E-002   2.0000000166305032     
           8   1.6356374513047667E-002   3.2712749298110064E-002   2.0000000166305028     
           9   1.2383004835806181E-002   2.4766009877547960E-002   2.0000000166305028     
          10   1.6245472393907993E-002   3.2490945057986345E-002   2.0000000166305019     
          11   9.6885666245603384E-003   1.9377133410246411E-002   2.0000000166305028     
          12   1.5589385332815228E-002   3.1178770924889763E-002   2.0000000166305023     
          13   6.5371165317142333E-003   1.3074233172144003E-002   2.0000000166305032     
          14   9.0139452412322137E-003   1.8027890632370869E-002   2.0000000166305028     
          15   1.4676589174709815E-002   2.9353178593498688E-002   2.0000000166305028     
          16   1.6668413222278331E-002   3.3336826721760755E-002   2.0000000166305028

(By the way for eemumu I get much more than the 32 events I required, also to be debugged)

@roiser does this ring any bell?

Thanks Andrea

The text was updated successfully, but these errors were encountered:

valassi · 2022-04-04T08:51:40Z

ok ok, probably this is the famous single diagram enhahncement factor?... that indeed we do not hav ein cudacpp yet

roiser · 2022-04-04T09:17:14Z

@valassi the constant factor was what I observed initially, I think it was also on eemumu at the time

valassi · 2022-04-04T09:28:54Z

Ok this is definitely the multichannel issue

I retried with

32 1 1      ! Number of events and max and min iterations
0.000001    ! Accuracy (ignored because max iterations = min iterations)
0           ! Grid Adjustment 0=none, 2=adjust (NB if = 0, ftn26 will still be used if present)
0           ! Suppress Amplitude 1=yes (i.e. use MadEvent single-diagram enhancement)
0           ! Helicity Sum/event 0=exact
1           ! Channel number for singla-diagram enhancement multi-channel (IGNORED as suppress amplitude is 0?)

Now I get for eemumu

LD_LIBRARY_PATH=../../lib:$LD_LIBRARY_PATH ./cmadevent_cudacpp < ../../../tmad/input_app_32_NOMULTICHANNEL.txt 
...
           1   1.9600993297408012E-002   1.9600993297408009E-002  0.99999999999999978     
           2   2.3449438792192103E-002   2.3449438792192107E-002   1.0000000000000002     
           3   2.9108284437328547E-002   2.9108284437328553E-002   1.0000000000000002     
           4   2.8954009182377567E-002   2.8954009182377564E-002  0.99999999999999989     
           5   3.3379397371771584E-002   3.3379397371771591E-002   1.0000000000000002     
           6   3.1038998841878617E-002   3.1038998841878617E-002   1.0000000000000000     
           7   3.3115195385159030E-002   3.3115195385159030E-002   1.0000000000000000     
           8   3.3363952299337613E-002   3.3363952299337613E-002   1.0000000000000000     
           9   1.0335862637570036E-002   1.0335862637570036E-002   1.0000000000000000     
          10   3.2429556173258260E-002   3.2429556173258253E-002  0.99999999999999978     
          11   2.0286043633362966E-002   2.0286043633362963E-002  0.99999999999999978     
          12   2.9559952349179928E-002   2.9559952349179922E-002  0.99999999999999978     
          13   9.8004562048847159E-003   9.8004562048847159E-003   1.0000000000000000     
          14   3.1001609786633272E-002   3.1001609786633282E-002   1.0000000000000004     
          15   2.9369320633387354E-002   2.9369320633387361E-002   1.0000000000000002     
          16   2.9429860036936338E-002   2.9429860036936338E-002   1.0000000000000000     
           1   1.0256578620713460E-002   1.0256578620713460E-002   1.0000000000000000     
           2   2.8562948778909233E-002   2.8562948778909240E-002   1.0000000000000002     
           3   3.3369448293922820E-002   3.3369448293922820E-002   1.0000000000000000     
           4   3.2204513114116387E-002   3.2204513114116380E-002  0.99999999999999978     
           5   2.2223067272470726E-002   2.2223067272470726E-002   1.0000000000000000     
           6   2.9147444122594138E-002   2.9147444122594134E-002  0.99999999999999989     
           7   3.1998940343792706E-002   3.1998940343792706E-002   1.0000000000000000     
 ...

And for ggtt

           1   3.5665107131744436        3.8048997495080039        1.0668409702101742     
           2  0.45750753393963106       0.49642094931550251        1.0850552449722197     
           3   1.0034869585255990        1.0461548930420270        1.0425196701899535     
           4  0.42736056050249699       0.44890337193428720        1.0504089834739543     
           5  0.62208668391557365       0.62881610141698518        1.0108174916380701     
           6  0.54427805795509532       0.54918963982598723        1.0090240306385769     
           7  0.42777117870194153       0.57067969683783715        1.3340770141867602     
           8  0.90719516691836111        1.0769120999539372        1.1870787447116646     
           9  0.46940856376493523       0.47369098861178799        1.0091230224103820     
          10   1.1211030838709168        1.1317896927132225        1.0095322267827569     
          11   2.0814056569772625        2.1018860475392533        1.0098396919857198     
          12   2.3377317419203960        2.5083158683912758        1.0729699320978328     
          13  0.96639301107661646       0.98776012624391241        1.0221101714544600     
          14  0.47219747839080861       0.51894860539999155        1.0990075744760541     
          15   2.9999390801816128        3.0360682133944707        1.0120432889626114     
          16  0.31943068952450482       0.32241178922449132        1.0093325400399820     
           1  0.49352008663987490       0.53612877811686066        1.0863362862636179     
           2   1.2387754214196172        1.2607876351277905        1.0177693335915139     
           3  0.32641446184474765       0.32938191546612056        1.0090910605020444     
           4  0.48419754170306206       0.49474672397459363        1.0217869389308072     
           5   1.1241654439875644        1.1342960426203157        1.0090116616614870     
           6  0.79551174010509906       0.80332578412761235        1.0098226633606702     
           7   1.1905375807398626        1.2173073582027885        1.0224854535430035     
           8   1.5222331121596433        1.5364484218704257        1.0093384578204416     
           9   20.527314687202445        21.050238541199739        1.0254745378031986     
          10   1.6377557144388479        1.6530004691860378        1.0093083202902535     
          11  0.37093260175527626       0.37673263740361568        1.0156363598694029     
          12  0.56844879445413321       0.57648363678235448        1.0141346809186866     
          13   1.5926397221564033        1.6124294061024296        1.0124257129033749     
          14   3.1771304607040958        3.2148077890067244        1.0118589175888857     
          15  0.49032034992343876       0.53221685547275532        1.0854472092701404     
          16  0.31941863974661144       0.32227914777498462        1.0089553572410250

The difference can be up to 30% in ggtt, I guess this is the running of alphas?

valassi · 2022-04-04T09:43:59Z

@oliviermattelaer have you already implemented multichannel in cudacpp, at least as a draft patch? Otherwise I can have a look, it should be easy.

The only problem is that I see we may need "get_channel_cut", see #419. Is this something we can ignore?

Thanmks
Andrea

oliviermattelaer · 2022-04-04T09:55:10Z

Hi,

I did implement the multi-channel in the standalone_gpu output.
The change/support should be straightforward to pass to the cudacpp output.

However, you can ignore (for the moment at least) the function get_channel_cut.
If you do observe change due to that function, then one should ensure that sde_strategy=1 in the run_card (as mention in the other thread

valassi · 2022-04-19T15:13:49Z

This is a relatively old result and I think I reported it somewhere already, anyway indeed if I use a fixed renormalization scale in the runcard then the numbers ar emuch better

17c0317

eemumu
           1   7.0122530298854253E-003   7.0122530298854253E-003   1.0000000000000000
           2   2.3218937589107692E-002   2.3218937589107695E-002   1.0000000000000002
           3   3.3242912896103267E-002   3.3242912896103261E-002  0.99999999999999978
           4   3.1473694471549470E-002   3.1473694471549470E-002   1.0000000000000000
           5   3.1606619479595172E-002   3.1606619479595179E-002   1.0000000000000002
           6   6.2142537348977038E-003   6.2142537348977029E-003  0.99999999999999989
           7   3.3242595080926932E-002   3.3242595080926939E-002   1.0000000000000002
           8   2.2506195169388281E-002   2.2506195169388281E-002   1.0000000000000000
           9   3.1672885635931201E-002   3.1672885635931194E-002  0.99999999999999978
          10   3.2924582565326467E-002   3.2924582565326460E-002  0.99999999999999978
          11   3.0926391975085288E-002   3.0926391975085288E-002   1.0000000000000000
          12   2.8135738050448882E-002   2.8135738050448878E-002  0.99999999999999989
          13   6.2584396832608387E-003   6.2584396832608379E-003  0.99999999999999989
          14   1.0485674646729875E-002   1.0485674646729873E-002  0.99999999999999989
          15   1.5945066839427015E-002   1.5945066839427015E-002   1.0000000000000000
          16   3.3219374126500487E-002   3.3219374126500487E-002   1.0000000000000000

ggtt
           1  0.78423397473520462       0.64611860501637686       0.82388499584519759
           2  0.62950033971308184       0.51864866917331454       0.82390530465751288
           3  0.57526360418136868       0.47395662197405852       0.82389467807288885
           4   1.5120621250124833        1.2457761317964153       0.82389216103546703
           5   2.2906505759378888        1.8872473632427160       0.82389142327829612
           6  0.59898514542325931       0.49350026859418389       0.82389400198808449
           7   1.1835801201095073       0.97512153045541194       0.82387454291238993
           8   2.5869362144872889        2.1312907080199439       0.82386674092865075
           9  0.43667584919510116       0.35978023082632749       0.82390686704906890
          10   2.3893710083336148        1.9685124263697180       0.82386218779083198
          11  0.42849681058716904       0.35304118150456576       0.82390620602471587
          12  0.44363781067750596       0.36551508924098952       0.82390427606427297
          13  0.47491717634446251       0.39128502405826593       0.82390160547586244
          14  0.73158763306450869       0.60274790874796458       0.82389023748685553
          15   1.1234792216675360       0.92560731746131641       0.82387577768236242
          16  0.60247725395640073       0.49637685519408703       0.82389310456857212

valassi · 2022-04-19T15:14:09Z

I will now slowly start integrating Stefan's alphas...

…running alphas madgraph5#373 and madgraph5#417 All ok! Now the ratio is 1 even with running alphas ./cmadevent_cudacpp < ../../../tmad/input_app_32_NOMULTICHANNEL.txt ... RESET CUMULATIVE VARIABLE 1 3.5665107131744436 3.5664076084950689 0.99997109088191050 2 0.45750753393963106 0.45750669938725042 0.99999817587183004 3 1.0034869585255990 1.0034632350091610 0.99997635891903081 4 0.42736056050249699 0.42735994776248010 0.99999856622235761 5 0.62208668391557365 0.62207238390567887 0.99997701283395946 6 0.54427805795509532 0.54426778363100825 0.99998112302354114 7 0.42777117870194153 0.42777104754863132 0.99999969340311656 8 0.90719516691836111 0.90719169458154836 0.99999617244784877 9 0.46940856376493523 0.46940250853893922 0.99998710030778426 10 1.1211030838709168 1.1210569608644638 0.99995885926359795 11 2.0814056569772625 2.0812943357966138 0.99994651634568421 12 2.3377317419203960 2.3376751859689477 0.99997580733903979 13 0.96639301107661646 0.96636328419795126 0.99996923934845916 14 0.47219747839080861 0.47219648282845950 0.99999789163984421 15 2.9999390801816128 2.9997680469889678 0.99994298777806023 16 0.31943068952450482 0.31943233120584363 1.0000051393976617 1 0.49352008663987490 0.49351851086041021 0.99999680706113625 2 1.2387754214196172 1.2387275810481315 0.99996138091646114 3 0.32641446184474765 0.32641575510565102 1.0000039620208494 4 0.48419754170306206 0.48419268874481292 0.99998997731745587 5 1.1241654439875644 1.1241187554052836 0.99995846822855972 6 0.79551174010509906 0.79548665978611088 0.99996847272299860 7 1.1905375807398626 1.1904955772543695 0.99996471889155569 8 1.5222331121596433 1.5221598419580713 0.99995186663528290 9 20.527314687202445 20.526129514290172 0.99994226361653571 10 1.6377557144388479 1.6376744770942142 0.99995039715391154 11 0.37093260175527626 0.37093237398102596 0.99999938594167992 12 0.56844879445413321 0.56843834295622875 0.99998161400286811 13 1.5926397221564033 1.5925649447066856 0.99995304810706565 14 3.1771304607040958 3.1769466958870427 0.99994216012866766 15 0.49032034992343876 0.49031882610983851 0.99999689220812371 16 0.31941863974661144 0.31942025174076361 1.0000050466502313 Iteration 1 Mean: 0.6464E+03 Abs mean: 0.6464E+03 Fluctuation: 0.271E+03 0.645E+04 100.0%

…oper test of running alphas madgraph5#373 and madgraph5#417 All ok! Now the ratio is 1 even with running alphas ./cmadevent_cudacpp < ../../../tmad/input_app_32_NOMULTICHANNEL.txt ... RESET CUMULATIVE VARIABLE 1 3.5665107131744436 3.5664076084950689 0.99997109088191050 2 0.45750753393963106 0.45750669938725042 0.99999817587183004 3 1.0034869585255990 1.0034632350091610 0.99997635891903081 4 0.42736056050249699 0.42735994776248010 0.99999856622235761 5 0.62208668391557365 0.62207238390567887 0.99997701283395946 6 0.54427805795509532 0.54426778363100825 0.99998112302354114 7 0.42777117870194153 0.42777104754863132 0.99999969340311656 8 0.90719516691836111 0.90719169458154836 0.99999617244784877 9 0.46940856376493523 0.46940250853893922 0.99998710030778426 10 1.1211030838709168 1.1210569608644638 0.99995885926359795 11 2.0814056569772625 2.0812943357966138 0.99994651634568421 12 2.3377317419203960 2.3376751859689477 0.99997580733903979 13 0.96639301107661646 0.96636328419795126 0.99996923934845916 14 0.47219747839080861 0.47219648282845950 0.99999789163984421 15 2.9999390801816128 2.9997680469889678 0.99994298777806023 16 0.31943068952450482 0.31943233120584363 1.0000051393976617 1 0.49352008663987490 0.49351851086041021 0.99999680706113625 2 1.2387754214196172 1.2387275810481315 0.99996138091646114 3 0.32641446184474765 0.32641575510565102 1.0000039620208494 4 0.48419754170306206 0.48419268874481292 0.99998997731745587 5 1.1241654439875644 1.1241187554052836 0.99995846822855972 6 0.79551174010509906 0.79548665978611088 0.99996847272299860 7 1.1905375807398626 1.1904955772543695 0.99996471889155569 8 1.5222331121596433 1.5221598419580713 0.99995186663528290 9 20.527314687202445 20.526129514290172 0.99994226361653571 10 1.6377557144388479 1.6376744770942142 0.99995039715391154 11 0.37093260175527626 0.37093237398102596 0.99999938594167992 12 0.56844879445413321 0.56843834295622875 0.99998161400286811 13 1.5926397221564033 1.5925649447066856 0.99995304810706565 14 3.1771304607040958 3.1769466958870427 0.99994216012866766 15 0.49032034992343876 0.49031882610983851 0.99999689220812371 16 0.31941863974661144 0.31942025174076361 1.0000050466502313 Iteration 1 Mean: 0.6464E+03 Abs mean: 0.6464E+03 Fluctuation: 0.271E+03 0.645E+04 100.0%

…#417 and madgraph5#373

valassi · 2022-05-11T15:42:15Z

I am about to merge PR #453 where I move back to variable renormalization scales (ie truly running alphas event by event) and all results are the same in Fortran and Cudacpp. I will close #373 about running alphas.

I will keep this #417 open for the moment as I want to implement some more automatic tests that frtran and cudacpp are the same, in a less verbse way - and possibly add performance comparisons.

valassi · 2022-05-11T17:29:47Z

@oliviermattelaer I am doing a more fine-grained comparison of Fortran and cudacpp. This is with running alphas (variable renormalization scale) but no multichannel yet.

CUDA and C++ are always in excellent agreement with each other. The agreement of Fortran with them is good, but not spectacular. For ggtt there are often deviations by more than 5E-5, and what is worrying is that they are ALWAYS IN THE SAME DIRECTION... can this be a systematic bug?

I am printing the min and max at the end, and I am printing indivdidual events only if deviation >5E-5 in absolute value
See fd6605b

./gmadevent_cudacpp < ../../../tmad/input_app_64_NOMULTICHANNEL.txt
...
 RESET CUMULATIVE VARIABLE
 WARNING! Deviation more than 5E-5          10   5.0208307517816033        5.0205082151740275       0.99993576031068931
 WARNING! Deviation more than 5E-5          14   6.1412376054978886        6.1408422471646871       0.99993562236822631
 WARNING! Deviation more than 5E-5          19   2.8196698917337062        2.8195133794161817       0.99994449268051433
 WARNING! Deviation more than 5E-5          26   1.7000555878730745        1.6999701821900661       0.99994976300562310
 WARNING! Deviation more than 5E-5          31   8.9853421261395425        8.9847441073480176       0.99993344507274962
 RESET CUMULATIVE VARIABLE
 WARNING! Deviation more than 5E-5           1   2.9952828789093009        2.9951147060108139       0.99994385408481079
 WARNING! Deviation more than 5E-5           5   3.2674597739718134        3.2672949541081930       0.99994955718661538
 WARNING! Deviation more than 5E-5          11   23.906164531244425        23.904572425204815       0.99993340186220470
 WARNING! Deviation more than 5E-5          12   7.2115125554333357        7.2111024304164699       0.99994312912669658
 WARNING! Deviation more than 5E-5          18   5.0277582816984809        5.0274636035675559       0.99994138975773805
 WARNING! Deviation more than 5E-5          21   5.1118977339682887        5.1115741556696488       0.99993670094444775
 WARNING! Deviation more than 5E-5          28   1.8208877147178686        1.8207945747421526       0.99994884913827298
 WARNING! Deviation more than 5E-5           9   7.5632375656327859        7.5628103733018515       0.99994351726661668
 WARNING! Deviation more than 5E-5          15   4.1469688581967885        4.1467152375571983       0.99993884192327875
 WARNING! Deviation more than 5E-5          26   4.8212544866832658        4.8209820679043265       0.99994349628718171
 WARNING! Deviation more than 5E-5          27   20.810883398215640        20.809421655551752       0.99992976066244199
...
 ME ratio CudaCpp/Fortran: MIN =   0.99992976066244199
 ME ratio CudaCpp/Fortran: MAX =    1.0000040201393361
 ME ratio CudaCpp/Fortran: 1-MIN =    7.0239337558009041E-005
 ME ratio CudaCpp/Fortran: MAX-1 =    4.0201393360916882E-006

./cmadevent_cudacpp < ../../../tmad/input_app_64_NOMULTICHANNEL.txt
...
 ME ratio CudaCpp/Fortran: MIN =   0.99992976066244166
 ME ratio CudaCpp/Fortran: MAX =    1.0000040201393365
 ME ratio CudaCpp/Fortran: 1-MIN =    7.0239337558342108E-005
 ME ratio CudaCpp/Fortran: MAX-1 =    4.0201393365357774E-006

valassi · 2022-05-11T17:34:46Z

Above was double, Below is for FPTYPE=f ie float. Bigger discrepancies, but always an imbalance in direction
See 0543430

./gmadevent_cudacpp < ../../../tmad/input_app_64_NOMULTICHANNEL.txt
...
 RESET CUMULATIVE VARIABLE
 WARNING! Deviation more than 5E-5          10   5.0208307517816033        5.0204830169677734       0.99993074157823258
 WARNING! Deviation more than 5E-5          14   6.1412376054978886        6.1409120559692383       0.99994698958914097
 WARNING! Deviation more than 5E-5          19   2.8196698917337062        2.8195171356201172       0.99994582482366579
 WARNING! Deviation more than 5E-5          23   1.3917471502668604        1.3916516304016113       0.99993136694030182
 WARNING! Deviation more than 5E-5          30  0.59946736522418109       0.59986174106597900        1.0006578770833512
 WARNING! Deviation more than 5E-5          31   8.9853421261395425        8.9848155975341797       0.99994140138483645
 RESET CUMULATIVE VARIABLE
 WARNING! Deviation more than 5E-5           1   2.9952828789093009        2.9951152801513672       0.99994404576639029
 WARNING! Deviation more than 5E-5          12   7.2115125554333357        7.2111072540283203       0.99994379800327604
 WARNING! Deviation more than 5E-5          18   5.0277582816984809        5.0274577140808105       0.99994021836356684
 WARNING! Deviation more than 5E-5          21   5.1118977339682887        5.1116113662719727       0.99994398015938124
 WARNING! Deviation more than 5E-5          28   1.8208877147178686        1.8207850456237793       0.99994361591148129
 WARNING! Deviation more than 5E-5           9   7.5632375656327859        7.5628528594970703       0.99994913472803448
 WARNING! Deviation more than 5E-5          12  0.34758494237401355       0.34689116477966309       0.99800400561194635
 WARNING! Deviation more than 5E-5          13  0.35451575056455586       0.35453897714614868        1.0000655163601500
 WARNING! Deviation more than 5E-5          15   4.1469688581967885        4.1467485427856445       0.99994687314550035
 WARNING! Deviation more than 5E-5          26   4.8212544866832658        4.8209867477416992       0.99994446695516570
 WARNING! Deviation more than 5E-5          27   20.810883398215640        20.813144683837891        1.0001086588002528
 WARNING! Deviation more than 5E-5          28   1.5052106057124179        1.5051305294036865       0.99994680059492835
 Iteration  1   Mean: 0.4017E+03 Abs mean: 0.4017E+03   Fluctuation:  0.119E+03   0.525E+04   100.0%
...
 ME ratio CudaCpp/Fortran: MIN =   0.99800400561194635
 ME ratio CudaCpp/Fortran: MAX =    1.0006578770833512
 ME ratio CudaCpp/Fortran: 1-MIN =    1.9959943880536457E-003
 ME ratio CudaCpp/Fortran: MAX-1 =    6.5787708335118822E-004

./cmadevent_cudacpp < ../../../tmad/input_app_64_NOMULTICHANNEL.txt
...
 ME ratio CudaCpp/Fortran: MIN =   0.99800323394186297
 ME ratio CudaCpp/Fortran: MAX =    1.0006573799366487
 ME ratio CudaCpp/Fortran: 1-MIN =    1.9967660581370339E-003
 ME ratio CudaCpp/Fortran: MAX-1 =    6.5737993664871652E-004

valassi · 2022-06-05T10:11:39Z

I have renamed this issue to mention differences by factors or orders of magnitude. The comparison at the permille level is now moved as a separet issue to #476

valassi · 2022-06-08T10:11:41Z

Ok this is a long issue about large differences between fortran and cudacpp physics results, which included

huge differences coming from alphas
huge differences coming from multichannel, more details in Debug fortran vs cpp differences in multichannel (using SD1 to bypass "get_channel_cut" issues and LIMHEL=0 to bypass helicity filtering issues) #419

The differences at the level of orderes of magnitude or factors have now gone. There are still differences at the permille or percent level (from alphas?) that I will investigate in a separate #476

This can now be closed (by PR #465)

valassi mentioned this issue Apr 4, 2022

Debug fortran vs cpp differences in multichannel (using SD1 to bypass "get_channel_cut" issues and LIMHEL=0 to bypass helicity filtering issues) #419

Closed

valassi mentioned this issue Apr 19, 2022

Incomplete patches ("v311") with progress towards madevent+cudacpp integration #401

Merged

valassi mentioned this issue May 11, 2022

Running of alphas (QCD coupling) #373

Closed

valassi mentioned this issue May 11, 2022

Enable properly running alphas again (variable renormalization scale) + Makefile fixes #453

Merged

valassi added a commit to valassi/madgraph4gpu that referenced this issue May 11, 2022

[alphas2] in codegen, keep the default run_card_default for madgraph5…

7cb298d

…#417 and madgraph5#373

valassi mentioned this issue May 11, 2022

First patch for automatic Fortran vs cuda/cpp comparisons #454

Merged

valassi mentioned this issue May 11, 2022

Use single precision or even half precision #5

Open

valassi mentioned this issue May 23, 2022

Multichannel: Fortran and Cudacpp MEs differ? (invalid, I was using the wrong SD strategy) #467

Closed

valassi changed the title ~~Physics results are different in fortran and cudacpp MEs~~ Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) Jun 5, 2022

valassi mentioned this issue Jun 5, 2022

Physics results are different in fortran and cudacpp MEs at the ~permille level: different handling of top quark width #476

Closed

valassi mentioned this issue Jun 8, 2022

First patch to integrate multichannel (single diagram enhancement) into cudacpp #465

Merged

valassi linked a pull request Jun 8, 2022 that will close this issue

First patch to integrate multichannel (single diagram enhancement) into cudacpp #465

Merged

valassi self-assigned this Jun 8, 2022

valassi closed this as completed Jun 8, 2022

valassi mentioned this issue Aug 5, 2024

Move LIMHEL=0 to madgraph4gpu only and add a runcard allowing LIMHEL=0 #950

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) #417

Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) #417

valassi commented Apr 4, 2022

valassi commented Apr 4, 2022

roiser commented Apr 4, 2022

valassi commented Apr 4, 2022

valassi commented Apr 4, 2022

oliviermattelaer commented Apr 4, 2022

valassi commented Apr 19, 2022

valassi commented Apr 19, 2022

valassi commented May 11, 2022

valassi commented May 11, 2022

valassi commented May 11, 2022

valassi commented Jun 5, 2022

valassi commented Jun 8, 2022

Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) #417

Physics results are different in fortran and cudacpp MEs (by factors or orders of magnitude) #417

Comments

valassi commented Apr 4, 2022

valassi commented Apr 4, 2022

roiser commented Apr 4, 2022

valassi commented Apr 4, 2022

valassi commented Apr 4, 2022

oliviermattelaer commented Apr 4, 2022

valassi commented Apr 19, 2022

valassi commented Apr 19, 2022

valassi commented May 11, 2022

valassi commented May 11, 2022

valassi commented May 11, 2022

valassi commented Jun 5, 2022

valassi commented Jun 8, 2022