Use K8 ingress for TLS termination

[gdbelvin]

Use K8 ingress for TLS termination

Kubernetes ingress objects support

• Managed TLS Certificates
• Central monitoring and metrics
• Static IP address, which allows DNS setup for sandbox.keytransparency.dev.

The backend protocol from the ingress reverse proxy to the binaries use HTTP/2 + TLS

This PR uses kustomize to configure k8 configs for both GCE and baremetal deployments. The baremetal deployment is used for local testing and the kubernetes test in travis. Baremetal deployments must locally supply an ingress implementation, this PR uses nginx. The GCE deployment uses the cloud provided GCE ingress implementation.

I chose 'kustomizeoverhelm` because kustomize has better support for multiple deployment environments. We can revisit this decision later if needed.

The directory structure:

deploy
├── kubernetes
│   ├── base
│   └── overlays
│       ├── gke // Google Cloud
│       └── local // Baremetal for local and travis tests
│           └── ingress-nginx
│               ├── baremetal // nginx configs required for non-cloud deployments
│               ├── cloud-generic // Basic nginx config required for most deployments
│               └── static  // Basic nginx config required for all nginx deployments

To verify that this PR worked:

$ ./scripts/deploy.sh
$ curl https://sandbox.keytransparency.dev -w '%{http_version}\n'
ok
2 

$ go install github.com/fullstorydev/grpcurl/cmd/grpcurl
$ grpcurl sandbox.keytransparency.dev:443 list
google.keytransparency.v1.KeyTransparency
grpc.reflection.v1alpha.ServerReflection

Fixes #1396

[codecov]

Codecov Report

Merging #1443 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1443      +/-   ##
==========================================
+ Coverage   66.39%   66.41%   +0.02%     
==========================================
  Files          54       54              
  Lines        4026     4026              
==========================================
+ Hits         2673     2674       +1     
  Misses        960      960              
+ Partials      393      392       -1

Impacted Files	Coverage Δ
core/sequencer/server.go	73.61% <0%> (-0.33%)	⬇️
core/sequencer/trillian_client.go	61.42% <0%> (+2.85%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 112863f...bd3af4e. Read the comment docs.

[gdbelvin]

After researching Helm, I've decided to stick with Kustomize for the following reasons

1. Helm supports environment specific customization through Go Templating. This makes small changes possible, but larger changes quickly become convoluted.
2. I couldn't see a way to optionally depend on subcharts in an environment specific manner.

• To support local testing, we need bare metal features that cloud service providers typically provide.

3. The more complex uses cases for setting up let's encrypt certs can be managed by relying on GCE. This reduces the effectiveness of local testing, but the local testing story probably deserves a re-think.

[NatalieDoduc]

For your main PR description, can you make the first line more descriptive? These guidelines are really useful: https://chris.beams.io/posts/git-commit/

Could you also add in the PR description more details about the additional files and structure under ingress-nginx?

Also, i'd recommend if you would explain the use of Kustomize (+investigation into Helm results in the PR description itself, rather than inline in the comments).

Finally if this adds a mechanism to route from outside to our cluster, can you add sample commands or reference to how one might validate that the setup works?

[gdbelvin]

Thanks for your suggestions @NatalieDoduc

Please see

• Updated README.md
  • Dir structure
  • Testing instructions
• A README.md for the ingress-nginx directory

[NatalieDoduc]

LGTM - can you just update the PR title to match your first line of the description.

Also, a few questions below, but only for clarification, no changes required. Thanks!