End-to-end test for XRootD #328
Conversation
There was a problem hiding this comment.
Obviously waiting on the SciTokens config, but definitely good to see how the end-to-end setup would work. Very much like how I imagined after you described it for me and definitely a template that can be used for later integration tests (say for the cache, registry, and director).
|
Looking at this further, I was under the impression you didn't want this to be a unit test and instead a test that ran during integration testing. At least that what issue #170 seems to imply. Yet the TestOrigin code will run as part of the |
There was a problem hiding this comment.
@bbockelm I ran this in a local dev container with the correct versions of sciTokens-cpp and the correct xrootd running with some minor changes (see my other review comments.)
The major problem I see is that if the xrootd server is not up, it will constantly get into a loop of the following error: INFO[2023-11-08T20:08:51Z] Failed to send request to XRootD; likely, server is not up (will retry in 50ms): Get "https://ccd8174190bf:8443": dial tcp 172.17.0.3:8443: connect: connection refused
That is only visible if running go test in the xrootd folder. Otherwise the tests will just hang indefinitely if running go test ./... from the base directory with any good error output. Another issue is that you can't cancel out of the test easily if this is happening ctrl-c doesn't work. I had to open another terminal and kill multiple processes by pid.
This could be a major issue in a dev environment where we just want to make sure the tests still pass after pushing. I think my earlier comment about not making this a unit test still stands.
The passed context allows the external caller (such as a unit test) to shutdown the daemons running in the go routine.
Makes the cache specific to pelican and not shared with other applications. Particularly, it avoids polluting the cache with remnants of prior unit test runs.
Starts the server, waits for it to be available then performs the upload/download/delete test cycle again xrootd.
|
@bbockelm It passes! I can't request a review from you as it's your own PR originally, but if you could take a look at my changes, that would be appreciated. |
|
Merging PR as both @turetske and I are happy with it. |
@turetske - not sure if this is helpful but I was tinkering with testing xrootd while on the airplane and cobbled together this PR.
Now, the tests shouldn't pass for the exact same thing you're struggling with -- waiting on the new release of
scitokens-cpp. But perhaps the approach is interesting for you?One item we'll need to think through: How do we want to approach Mac OS X? The homebrew-packaged version of XRootD comes without SciTokens, meaning that maintaining a test there would be fairly difficult.