diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000000..29e28465fa --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,299 @@ +# Architecture + +* _Updated: December 2021_ + +This document describes the repository organization and the kustomize +build process. It's meant to lower the barrier to learning and +contributing to the code base. + +If not kept up to date, it will just be a historical snapshot. + + +## Repository layout + +[human-edited docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/site +[generated docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/docs +[rendered docs]: https://kubectl.docs.kubernetes.io +[openapi]: https://kubernetes.io/blog/2016/12/kubernetes-supports-openapi + +[`api` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/api/go.mod +[`api`]: #the-api-module +[`cmd/config` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/go.mod +[`cmd/config`]: #the-cmdconfig-module +[`kustomize` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kustomize/go.mod +[`kustomize`]: #the-kustomize-module +[`kyaml` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kyaml/go.mod +[`kyaml`]: #the-kyaml-module +[`kyaml/kio.Filter`]: https://github.com/Kubernetes-sigs/kustomize/blob/master/kyaml/kio/kio.go +[`go-yaml`]: https://github.com/go-yaml/yaml/tree/v3 + + +[3922]: https://github.com/kubernetes-sigs/kustomize/issues/3922 + + + + | directory | purpose | + | ---------: | :---------- | + | `api` | The [`api`] module, holding high level kustomize code, suitable for import by other programs. | + | `cmd` | Various Go programs aiding repo management. See also `hack`. As an outlier, includes the special [`cmd/config`] module. | + | `docs` | Old home of documentation; contains pointers to new homes: [human-edited docs], [generated docs] and [rendered docs]. | + | `examples` | Full kustomization examples that run as pre-merge tests. | + | `functions` | Examples of plugins in KRM function form. TODO([3922]): Move under `plugin`. | + | `hack` | Various shell scripts to help with code management. | + | `kustomize` | The [`kustomize`] module holds the `main.go` for kustomize. | + | `kyaml` | The [`kyaml`] module, holding Kubernetes-specific YAML editing packages used by the [`api`] module. Wraps [`go-yaml`] v3.| + | `plugin` | Examples of Kustomize plugins. | + | `releasing` | Instructions for releasing the various modules. | + | `site` | Old generated documentation, kept to provide redirection links to the new docs. | + + +## Modules + +[semantically versioned]: https://semver.org +[Go modules]: https://github.com/golang/go/wiki/Modules + +The [Go modules] in the kustomize repository are [semantically versioned]. + + +### `kustomize` + +> _Depends on [`api`], [`cmd/config`], [`kyaml`]_ + +The [`kustomize` module] contains the `main.go` for `kustomize`, buildable with + +``` +(cd kustomize; go install .) +``` + +[appears in kubectl]: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kubectl/pkg/cmd/kustomize/kustomize.go + +Below this are packages containing +[cobra](http://github.com/spf13/cobra) commands implementing `build`, +`edit`, `fix`, etc., packages linked together by `main.go`. + +These command packages are intentionally public, semantically +versioned, and can be used in other programs. Specifically, the +`kustomize build` command [appears in kubectl] as `kubectl kustomize`. + +The code in the `build` package is dominated by flag validation, +with minimal business logic. The critical lines are something +like + +``` +# Make a kustomizer. +k := krusty.MakeKustomizer( + HonorKustomizeFlags(krusty.MakeDefaultOptions()), +) + +# Run the kustomizer, sending location of kustomization.yaml +m := k.Run(fSys, "/path/to/dir") + +# Write the result as YAML. +writer.Write(m.AsYaml()) +``` + +The `krusty` package is in the [`api`] module. + +### `api` + +> _Depends on [`kyaml`] and code generated from builtin plugin modules_ + +The [`api` module] is used by CLI programs like `kustomize` and `kubectl` +to read and honor `kustomization.yaml` files and all that implies. + +The main public packages in the [`api` module] are + + | package | | + | --------: | :---------- | + | `filters` | Implementations of [`kyaml/kio.Filter`] used by kustomize to transform Kubernetes objects. | + | `konfig` | Configuration methods and constants in the kustomize API. | + | `krusty` | Primary API entry point. Holds the kustomizer and hundreds of tests for it. | + | `loader` | Loads kustomization files and the files they refer to, enforcing security rules. | + | `resmap` | The primary internal data structure over which the kustomizer and filters work. | + | `types` | The `Kustomization` object and ancillary structs. | + +### `cmd/config` + +> _Depends on [`kyaml`]_ + +This module contains cobra commands and kyaml-based functionality to +provide unix-like file manipulation commands to kustomize like `grep` +and `tree`. These commands may be included in any program that +manipulates k8s YAML (e.g. kustomize). + +### `kyaml` + +> _Has no in-repo dependence_ + +The [`kyaml` module] is a kubernetes-focussed enhancement of [go-yaml]. + +The YAML manipulation performed by a kustomize is based on these libraries. + +These libraries evolve independently of kustomize, and other programs depend on them. + +The key public packages in the [`kyaml` module] include + + | package | | + | --------: | :---------- | + | `errors` | Wrapper for the go-errors/errors lib | + | `filesys` | A kustomize-specific file system abstraction, to ease writing tests | + | `fn/framework` | An SDK for writing KRM Functions in Go | + | `fn/runtime` | Implements the runtime for KRM Function extensions | + | `kio` | Libraries for reading and writing collections of Kubernetes resources as RNodes | + | `openapi` | Loads and accesses openapi schemas for schema-aware resource manipultaion | + | `resid` | Representations to aid in unique identification of Kubernetes resources | + | `yaml` | A Kubernetes-focused wrapper of [go-yaml], notably including the RNode object | + + +------- + +## How _kustomize build_ works + +The command `kustomize build` accepts a single string argument, +which must resolve to a directory, possibly in a git repository, +called the _kustomization root_. + +This directory must contain a file called `kustomization.yaml`, with +YAML that marshals into a single instance of a `Kustomization` object. + +For the remainder of this document, the word _kustomization_ refers to +either of these things. + +This kustomization is the access point to a directed, acyclic graph of +Kubernetes objects, including other kustomizations, to include in a +build. + +Execution of `build` starts and ends in the [`api`] module, +frequently dipping into the [`kyaml`] module for lower level +YAML manipulation. + +### The `build` flow + + - Validate command lines arguments and flags. + + - Make a `Kustomizer` as a function of those arguments. + + - Call `Run` on the kustomizer, passing it the path to the + kustomization. + + `Run` returns an instance of `ResMap`, the `api` package's + representation of a set of kubernetes `Resource` objects. + + This structure offers resource lookup methods (map behavior), + but also retains the resources in the order they were + specified in kustomization files (list behavior). + + Post-run, the objects are fully hydrated, per the + instructions in the kustomization. + + - Marshal the objects as YAML to a file or `stdout`. + + +### The `Run` function + + - Create various objects + + - A `ResMap` factory. + + Makes `ResMaps` from byte streams, other `ResMaps`, etc. + + - A file `loader.Loader`. + + It's fed an appropriate set of restrictions, and the path to the kustomization. + + - A plugin loader. + + It finds plugins (transformers, generators or validators) + and prepares them for running. + + - A `KustTarget` encapsulating all of the above. + + A KustTarget contains one `Kustomization` and represents + everything that kustomization can reach. This will include + other `KustTarget` instances, each having a smaller purview than + the one referencing it. + + - Call `KustTarget.Load` to load its kustomization. + + This step deals with deprecations and field changes. + + - Load [openapi] data specified by the kustomization. + + This is needed to recognize k8s kinds and their special + properties, e.g. which kinds are cluster-scoped, which kinds + refer to others, etc. + + - Call `KustTarget.makeCustomizedResmap` to create the `ResMap` result. + + This visits everything referenced by the kustomization, + performing all generation, transformation and validation. + + - Finish the `Run` with + + - Optional reordering of objects in `ResMap`, overriding the + FIFO rule. + + - Optional addition of _kustomize build annotations_ to the + resources. E.g. from which repo and file the resource was + read, the fact that kustomize touched the resource, etc. + These kustomize-specific annotations are intended for + server-side data analytics, file structure traceability and + reconstruction, etc. + +### The `makeCustomizedResmap` function + + This function starts the process of object transformation, + as well as accumulation of recursively referenced data. + + - Call `ra := KustTarget.AccumulateTarget`. + + The result, `ra`, is a resource accumulator that contains + everything referred to by the current kustomization, now fully + hydrated. + + - Uniquify names of generated objects by appending content hashes. + + This cannot be done until the objects are complete. + + - Fix all name references (given that names may have changed). + + E.g. if a ConfigMaps was given a generated name, all objects that + refer to that ConfigMap must be given its name. + + - Resolve vars, replacing them with whatever they refer to (a legacy feature). + +### The `AccumulateTarget` function + + - Call `AccumulateResources` over the `resources` field (this can recurse). + - Call `AccumulateComponents` over the `components` field (this can recurse), + - Load legacy (pre-plugin) global kustomize configuration, + - Load legacy (pre-openapi) _Custom Resource Definition_ data. + - In the context of the data loaded above, run the kustomization's + - generators, + - transformers, + - and validators. + - Accumulate `vars` (make note of them for later replacement). + +### `AccumulateResources` and component accumulation + + - If the path is a file: + - Accumulate the objects in the file (treating them + as opaque kubernetes objects). + + - If the path is a directory: + - Create a new `KustTarget` referring to that directory's kustomization. + - Call `subRa := KustTarget.AccumulateTarget`. + - Call `ra.MergeAccumulator(subRa)` + This completes a recursion. + + - If the path is a git URL: + - Clone the repository to a temporary directory. + - Process the path optionally specified in the URL + as a path in the clone. + - If no path specified, work from the repository root. + + +That's as deep as this discussion will go. + +The deeper this document goes into the details, the faster +it will get out of date.