Proposal: Improve Makefile

[dio]

Note: Sorry that I put this proposal as a draft PR instead of an issue since it is easier for me to present the idea via code.

This proposes Makefile improvement. To let the transition smooth, this PR puts the proposal into a new file called: Next.mk. If this is something that we are looking for, we can "replace" the main Makefile.

To invoke a target inside it, we can run it with make -f Next.mk.

This proposal is broken down into three main commits:

• Add Next.mk 0eae5da, this adds the basic structure.
• Add test target 6295a15, this adds the test target. Basically runs the bazel test //tests/....
• Add sanity tests f83fd5a, this adds checks for bazel-related tasks.

In this proposal, the Next.mk has several targets, that can be inspected via the "help" target.

Next.mk depends on several *.mk files:

• Tools.mk provides dependency tools versions.
• tools/build/Help.mk provides the "help" target.
• tools/build/Env.mk sets some required env vars.
• tools/build/Installer.mk provides targets that install tools. This is done on-demand.

Some quick notes:

1. This proposal moves the top BUILD file to BUILD.bazel hence "build" directory can be used to store the current plugin binary.
2. This proposal also adds a new workflow: .github/workflows/commit.yaml.
3. To run "build" and "check". The build step runs clean before
   build hence the *.pb.go is regenerated and its consistency can be
   checked via make check.
4. Some packaging helpers are not yet included in this proposal since we do packaging via workflow and not calling the make targets.

Next

After this, the plan is to:

1. Do hygiene and code-quality tasks, by adding format and lint targets. Also, a way to measure the coverage. We can add the format and lint check here, but that will actually introduce more file changes due to that (hence the PR gets bigger and will be hard to review).
2. Fresh up the bazel/*. Seems like at least we can (the work on this is already started):
   a. Upgrade protobuf dependency to match the one with non-bazel, and use compiled protoc whenever possible to reduce the time cost (similar to what we have with the envoy project).
   b. Use @maven via https://github.com/bazelbuild/rules_jvm_external.
   c. Make sure the python version and modules we use are pinned.

Signed-off-by: Dhi Aurrahman dio@rockybars.com

[dio]

The "CI-runs" on my "fork" can be observed from: dio#2. 🙇🏾

[dio]

@elliotmjackson do you think this makes sense? Thank you!

[elliotmjackson]

yes, it absolutely makes sense - I just want some time to take a look at the github action changes. Right now, im a touch busy but i will look at this soon

[dio]

Thanks, @elliotmjackson. Sure, take your time.

[elliotmjackson]

Im quite happy with the changes to the makefile, they are very much needed and this is a good first step. My concerns relate to this new github action, its run time far exceeds our newly tweaked CI and somewhat undoes a lot of the gains we have achieved.

[dio]

@elliotmjackson seems like we need to keep the current one and iterate on the new one. I'll take a deeper look at the disparity list.

Also, if you can expand on the following points:

• its run time far exceeds our newly tweaked CI (do you mean it is too slow?)
• somewhat undoes a lot of the gains we have achieved.

That would be great.

[dio]

Ah yeah, it is too slow, will take a look (seems like we don't have to do all the stuff "sequentially" and via bazel).

The current one is cached bazel, I think when we have this cached (the bazel stuff), it will be better.

[elliotmjackson]

Ok cool, I assume caching only happens from the base branch. Happy to see where it goes... thanks for the contribution @dio and I apologise for not being overly focused on this repo to date, will endeavour to give you more attention next time.

[dio]

@elliotmjackson no worries at all :). Love to contribute more if you can help me on merging this PR. Thanks!

[dio]

(sorry was out too for personal reason)