Pelican cache with auth #423
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was
linked to
issues
Nov 29, 2023
turetske
force-pushed
the
pelican-cache-with-auth
branch
from
ad11dbc to
4052af8
Compare
-- Created a Cache.XrootdPort which will overwrite XrootdPort if
initServer is called by a cache
-- Created a ServerType const for config that is used by initServer
to do any server type specific actions
-- Refactored the calls to initServer as needed
-- Adjusted parameters.yaml
-- Adjusted the default parameters to use a cache port
-- Issuer is now the IssuerURL as opposed to the NamespaceURL
-- The basepath is now the prefix rather than the hardcoded "/"
-- Added scitokens tracing to the xrootd-origin.cfg
-- Added auth checking to the xrootd-cache.cfg
-- Added a WriteCacheSciTokensConfig file
-- Adjusted to Authfile generation to handle empty nsAds
-- Added testing of the new scitokens generation and authfile generation
-- This is part of a refactor to allow much greater code reuse
-- The code will use this struct and composition to reuse code for
namespace registration, periodic advertisement, xrootd config
generation, etc.
-- Refactored common code into server_ui/advertise.go
-- Individual cache/origin ad creation is now in cache_ui/advertise.go and origin_ui/advertise.go respectively
-- moved common code out of origin_ui into server_ui
-- Adjusted cache_serve and origin_serve
-- Origin/Cache xrootd environment configuration are now in their own functions
-- Common code moved from origin_ui to server_ui/xrootd_servers.go
-- Some cleanup of cmd/cache_serve.go
turetske
force-pushed
the
pelican-cache-with-auth
branch
from
4052af8 to
760d8d4
Compare
turetske
requested review from
jhiemstrawisc
and removed request for
bbockelm
bbockelm
requested changes
Dec 2, 2023
xrootd/authorization.go
Outdated
| foundPublicLine = true | ||
| } else { | ||
| output.Write([]byte(lineContents + "\n")) | ||
| if nsAds == nil { |
There was a problem hiding this comment.
This could probably use some explaining. It looks like the input auth file is ignored nsAds isn't null. Why?
To support some legacy use cases, we'll need to merge in an authfile for the cache.
This tidies up the definitions to use a golang interface instead of a struct with a pointer. Also switches the type struct to a bitmask to allow multi-type servers in the future.
VOMS-based authentication may be needed by some specific caches. Add it back to the cache config, defaulting to false. Add its existence to the documentation.
The director mode did not duplicate the logic for CLI-based cache overrides. Instead of copy/pasting, this refactors the cache selection logic to a single location, fixing the underlying bug.
There was a missing "return" after a failure to invoke `HEAD`; without this, a null pointer panic ensued.
If a port is not specified (e.g., https://foo.com), then XRootD 5.6.3 will segfault when parsing the pss.origin. This adds the default port if not present (e.g., https://foo.com:443).
The multi-space setup appears to not work if the different directories are on the same device (it appears that XRootD is doing some device discovery here?). Disable it for now, leaving a note for later.
bbockelm
approved these changes
Dec 4, 2023
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds authentication to the pelican cache
-- Adds scitokens authenticated to the pelican cache
-- Fixes some errors that were in the namespace registration
-- Does a refactor to move common code out of origin specific folders/functions to be used by both the cache and origin
For ease of understanding, the first 6 commits (up through b0606ce) are the actual cache with auth functionality and all the commits afterwards are the refactor.