Merge stdext and reverse deps into XAPI #5378

edwintorok · 2024-01-22T11:31:26Z

Merges as git subtree:

xapi-inventory
xapi-rrd
xapi-stdext*

Opening PR to see whether the CI works.

Make safe-string compliant in a backward-compatible way

…, std, threads, unix} 1.1.0 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Move quicktest_encodings from xapi into a unit test

See discussion in https://ocaml.github.io/ocamlunix/ocamlunix.html#sec118 and xapi-project#3570 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

really_write: remove deprecation and make robust against EINTR

See e.g. https://github.com/ocaml/ocaml/blob/trunk/otherlibs/unix/unix.ml#L328-L332 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

unixext: update interface to mimick the ocaml Unix one (fd-send-recv >= 2.0.0)

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

`Backtrace.is_important` was not called. We have a `finally` function that does the proper thing wrt to backtraces, use it. xapi-stdext-pervasives is a dependency of xapi-stdext-threads already, so this does not introduce a new dependency. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

We've got a `with_file` in this same file, use it instead of reimplementing it, so we get the improved backtraces. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

We cannot use finally here, because the socket is only closed on failure. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

CP-28365: improve backtraces by using finally

…nal one with it Use the Logs library to report the cleanup exception we caught. This way, we don't have to depend on xcp-idl. In xcp-idl, a Logs reporter will be registered, which collects the log messages from Logs and reports them in the usual way. Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

CA-292641: Use Logs to log cleanup exn instead of shadowing the origi…

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Prepare to release 4.4.0

Optionally use rpclib if it's available

Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Simplify PPX processing

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>

…a7c07e997c7' Also move *.opam files to root, update dune install/uninstall rules and quality gate. git-subtree-dir: ocaml/libs/xapi-rrd git-subtree-mainline: da5059d git-subtree-split: 7e7e838

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

…merge-xapi-rrd' and 'private/edvint/merge-inventory'

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

…f1a8522c8e6399' git-subtree-dir: ocaml/libs/xapi-stdext git-subtree-mainline: c0ed37e git-subtree-split: 4cf5cbe

edwintorok · 2024-01-22T15:54:14Z

#5383 and #5382 as separate draft PRs if you want to see how xapi-inventory and xapi-rrd got merged.
I tried to keep the merge commits buildable by using 'git commit -a --amend' when I did build fixes like moving opam files around or updating Makefiles/quality gates.
Also includes #5381

edwintorok · 2024-01-22T15:56:12Z

Is it worth it, wouldn't it be easier to drop the dune definiton of the packages of the stdext libs?

I found another way of doing it (by using .opam.template files). In general I'd like to move to generating files from dune-project, because then we can also use tools like 'opam dune lint' that checks the consistency of dependencies between opam files and dune files, and we can also share some metadata (e.g. consistent homepage, build rules and bugreport urls). And it'll also encourage us to have fewer opam files, we have way too many now....

Otherwise it fails because it doesn't about the 'xapi-stdext' packge (it is not part of xs-opam). But ocamlformat shouldn't need depexts anyway, so disable them. Signed-off-by: Edwin Török <edwin.torok@cloud.com>

psafont · 2024-01-22T16:21:15Z

And it'll also encourage us to have fewer opam files, we have way too many now....

I agree, also at some point dune should support stashing them away in the opam subdir

edwintorok · 2024-01-22T16:29:59Z

Tests are passing now, and I've also done a successful koji build.
(Haven't tried a chainbuild after removing the packages from xs-opam yet).

psafont · 2024-01-23T11:07:51Z

Can the unneeded files in ocaml/libs/xapi-stdext/, ocaml/libs/xapi-inventory/ and ocaml/libs/xapi-rrd/? (like the github workflows or gitignore) be removed

Drop redundant: * .github * .ocamlformat * .travis.yml * INSTALL * MAINTAINERS * .gitattributes * .gitarchive-info * Makefile For consistency I've removed these from other subprojects too, not just recently merged ones. The only one I didn't touch was vhd-tool, which we may intend to upstream again at some point. I've kept the README/LICENSE and Changelog files. Signed-off-by: Edwin Török <edwin.torok@cloud.com>

edwintorok · 2024-01-23T17:20:15Z

I've cleaned up the merged subprojects, I've kept .gitignore/ChangeLog/LICENSE/README:

ocaml/libs/xapi-inventory/:
.  ..  .gitignore  ChangeLog  LICENSE  README.md  lib

ocaml/libs/xapi-rrd:
.  ..  .gitignore  ChangeLog  LICENSE  README.md  lib  lib_test  unix

ocaml/libs/xapi-stdext:
.  ..  .gitignore  CHANGES.md  LICENSE  README.md  lib

psafont · 2024-01-30T12:38:30Z

I'd like to push this forward, but after the fixed release is done.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

codecov · 2024-01-31T10:09:06Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (4ea474b) 45.38% compared to head (478ec07) 45.38%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5378   +/-   ##
=======================================
  Coverage   45.38%   45.38%           
=======================================
  Files          18       18           
  Lines        2937     2937           
=======================================
  Hits         1333     1333           
  Misses       1604     1604

Flag	Coverage Δ
python2.7	`52.09% <ø> (ø)`
python3.11	`50.15% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

psafont · 2024-01-31T10:09:22Z

Overrriding the DCO check, all commits are from Xenserver / Citrix employees at the time of committing

psafont · 2024-01-31T10:17:47Z

I've checked that the tests in the new packages have been run

github-actions · 2024-01-31T10:26:26Z

pytype_reporter extracted 49 problem reports from pytype output.

You can check the results of the job here

mseri and others added 30 commits March 15, 2018 15:12

Merge pull request xapi-project#27 from mseri/master

001b4b7

Make safe-string compliant in a backward-compatible way

Prepare to release stdext 4.0.0 and xapi-stdext-{bigbuffer, encodings…

22e68f8

…, std, threads, unix} 1.1.0 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Move quicktest_encodings from xapi into a unit test

713d5aa

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Merge pull request xapi-project#28 from gaborigloi/test_encodings

5a05644

Move quicktest_encodings from xapi into a unit test

really_write: remove deprecation and make robust against EINTR

0e0f010

See discussion in https://ocaml.github.io/ocamlunix/ocamlunix.html#sec118 and xapi-project#3570 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

unixext_open_stubs: fix use of uninitialised variable

b98891a

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

really_write: use single_write_substring and avoid an unsafe coercion

85f5556

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

.travis.yml: don't run revdeps on the coverage build

6fe6bef

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Merge pull request xapi-project#29 from mseri/fix-really-write

3a73fbb

really_write: remove deprecation and make robust against EINTR

unixext: update interface to mimick the ocaml Unix one

1b708d1

See e.g. https://github.com/ocaml/ocaml/blob/trunk/otherlibs/unix/unix.ml#L328-L332 Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

xapi-stdext-unix: update opam file

6fd0d53

Signed-off-by: Marcello Seri <marcello.seri@citrix.com>

Merge pull request xapi-project#31 from mseri/master

d474a58

unixext: update interface to mimick the ocaml Unix one (fd-send-recv >= 2.0.0)

CP-28365: do not loose the backtrace in Mutex.execute

f00572e

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

CP-28365: improve backtraces in with_file

90178da

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

CP-28365: improve backtraces in daemonize

5401c04

We've got a `with_file` in this same file, use it instead of reimplementing it, so we get the improved backtraces. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

CP-28365: improve backtraces in Unixext.open_connection*

adba34e

We cannot use finally here, because the socket is only closed on failure. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

CP-28365: improve stacktraces in Semaphore.execute_with_weight

b78eb0e

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Merge pull request xapi-project#32 from edwintorok/backtraces

163c59a

CP-28365: improve backtraces by using finally

Update .travis.yml after move to ocaml 4.06

3b3f070

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Merge pull request xapi-project#33 from gaborigloi/logs

3423c18

CA-292641: Use Logs to log cleanup exn instead of shadowing the origi…

Prepare to release 4.4.0

66baac0

Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>

Merge pull request xapi-project#34 from gaborigloi/new_release

552e4c8

Prepare to release 4.4.0

Use rpclib if it's available

03ab7c0

Merge pull request xapi-project#26 from samoht/rpclib

d518b29

Optionally use rpclib if it's available

Simplify PPX processing

ec275d9

Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Merge pull request xapi-project#27 from lindig/master

d31660e

Simplify PPX processing

Ported build to dune and deprecated the old package xcp-inventory.

200c85d

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>

Removed conflict between xcp-inventory and xapi-inventory.

98f2dc9

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>

edwintorok added 5 commits January 22, 2024 15:28

Add 'ocaml/libs/xapi-rrd/' from commit '7e7e83822bf633de0169b72bad48e…

c7668e2

…a7c07e997c7' Also move *.opam files to root, update dune install/uninstall rules and quality gate. git-subtree-dir: ocaml/libs/xapi-rrd git-subtree-mainline: da5059d git-subtree-split: 7e7e838

[maintenance]: make format

9b31ae5

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

Merge branches 'private/edvint/generate_opam_files', 'private/edvint/…

1671c17

…merge-xapi-rrd' and 'private/edvint/merge-inventory'

[maintenance]: avoid deprecation warning on Uuid.create

c0ed37e

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

Add 'ocaml/libs/xapi-stdext/' from commit '4cf5cbe00189b9f5ea96a3b0eb…

dd1bc5e

…f1a8522c8e6399' git-subtree-dir: ocaml/libs/xapi-stdext git-subtree-mainline: c0ed37e git-subtree-split: 4cf5cbe

This was referenced Jan 22, 2024

merge xapi-inventory #5382

Merged

merge xapi-rrd #5383

Merged

edwintorok force-pushed the private/edvint/merge-stdext2 branch from 68abce8 to dd1bc5e Compare January 22, 2024 15:57

fix(ci): disable opam depext for ocamlformat

6d41706

Otherwise it fails because it doesn't about the 'xapi-stdext' packge (it is not part of xs-opam). But ocamlformat shouldn't need depexts anyway, so disable them. Signed-off-by: Edwin Török <edwin.torok@cloud.com>

edwintorok marked this pull request as ready for review January 22, 2024 16:29

edwintorok mentioned this pull request Jan 22, 2024

Move xapi-rrd, xapi-inventory and xapi-stdext* to xs-extra xapi-project/xs-opam#665

Merged

psafont approved these changes Jan 24, 2024

View reviewed changes

edwintorok mentioned this pull request Jan 29, 2024

CP-47001: stdext: quickcheck-style tests and select->epoll conversion #5402

Merged

lindig approved these changes Jan 31, 2024

View reviewed changes

Resolve quality-gate conflict between PR and master branch

478ec07

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

psafont force-pushed the private/edvint/merge-stdext2 branch from 4bd7423 to 478ec07 Compare January 31, 2024 10:07

psafont merged commit eff32e4 into xapi-project:master Jan 31, 2024
8 checks passed

psafont mentioned this pull request Jan 31, 2024

Revert "Merge stdext and reverse deps into XAPI" #5407

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge stdext and reverse deps into XAPI #5378

Merge stdext and reverse deps into XAPI #5378

edwintorok commented Jan 22, 2024

edwintorok commented Jan 22, 2024 •

edited

Loading

edwintorok commented Jan 22, 2024

psafont commented Jan 22, 2024

edwintorok commented Jan 22, 2024

psafont commented Jan 23, 2024

edwintorok commented Jan 23, 2024

psafont commented Jan 30, 2024

codecov bot commented Jan 31, 2024

psafont commented Jan 31, 2024

psafont commented Jan 31, 2024

github-actions bot commented Jan 31, 2024 •

edited

Loading

Merge stdext and reverse deps into XAPI #5378

Merge stdext and reverse deps into XAPI #5378

Conversation

edwintorok commented Jan 22, 2024

edwintorok commented Jan 22, 2024 • edited Loading

edwintorok commented Jan 22, 2024

psafont commented Jan 22, 2024

edwintorok commented Jan 22, 2024

psafont commented Jan 23, 2024

edwintorok commented Jan 23, 2024

psafont commented Jan 30, 2024

codecov bot commented Jan 31, 2024

Codecov Report

psafont commented Jan 31, 2024

psafont commented Jan 31, 2024

github-actions bot commented Jan 31, 2024 • edited Loading

pytype_reporter extracted 49 problem reports from pytype output.

edwintorok commented Jan 22, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading