Hashicorp Labs

This repo contains tools for testing and experimenting with Hashicorp software.

But not only that..you can play with OpenFaas serverless stuff too! Keep reading.

It will deploy a cluster for Vault, Consul and Nomad where each component will be connected together to form a perfect environment for testing your applications with service mesh.

Table of Contents

• Hashicorp Labs
  • Install cluster with Vagrant
  • Provision and deploy
  • Play with microservices
  • OpenFaas in Nomad
    • Monitoring OpenFaas
    • Docker login (for OpenFAAS)
  • Clean up

Install cluster with Vagrant

WARNING: At the moment the clusters will be loaded in dev mode. If the services will fail, you will lose every data since the backends are loaded in memory.

Vagrant file is taken partially from here hashicorp's nomad-guides It's been revisited and modified (and it will be upgraded in the future).

You can choose respective software versions and VM specs with these environment variables:

ENV	description	default value	Ubuntu ARM alternative	Centos 7 alternative
BASE_BOX	Vagrant Base Box	generic/ubuntu2110	rkrause/ubuntu-21.10-arm64	bento/centos-7
BOX_VERSION	Vagrant Base Box Version	3.5.2	1.0.0	202103.18.0
VAGRANT_CPU_NUM	Number of cpu used by VM	2
VAGRANT_MEM	Memory used by VM	8192
VAGRANT_VMWARE	If set, uses VMware provider	false
VAULT_VERSION	Vault version	1.8.4
CONSUL_VERSION	Consul version	1.10.4
NOMAD_VERSION	NOMAD version	1.1.6
CNI_VERSION	CNI plugin version	0.9.1
CONTAINERD_VERSION	Containerd version	1.4.11-1		1.4.9-3.1
DOCKER_CE_VERSION	Docker CE version	20.10.10~3-0~ubuntu		20.10.8-3
DOCKER_SCAN_VERSION	Docker Scan version	Not installed		0.8.0-3
ENVOY_VERSION	Envoy version	1.18.2
TF_VAR_faasd_version	Faasd provider version	0.13.0
TF_VAR_faas_nats_version	NATS version version	0.22.0
TF_VAR_faas_auth_plugin_version	Faas Auth plugin version	0.21.0
TF_VAR_faas_gateway_version	Faas Gateway version	0.21.0
TF_VAR_faas_queue_worker_version	Faas Queue Worker version	0.12.2

Provision and deploy

You need Hashicorp Terraform (version >= 0.13.1, < 1.0.0 ) and Vagrant installed (version >= 2.2.1).

To provision and deploy the workload simply do:

./deploy.sh

or through Makefile:

make

with make help you'll find some other useful make targets to launch.

By default it will use VirtualBox. To use VMware add VAGRANT_VMWARE=true to you're environment variables.

Now you can install it also in ARM based hosts (like Macbook M1 pro on hypervisor VMWare Fusion). With Ubuntu 21.10.

Tested with Centos 7, Ubuntu 21.04, 21.10 and Ubuntu 21.10 ARM edition Boxes: bentos/centos-7, generic/ubuntu2104, generic/ubuntu2110, rkrause/ubuntu-21.10-arm64

Makefile useful tips

• You can run make tests to perform some tests to the Vault, Consul and Nomad endpoints.
• With make provision and with the env var TAGS_ONLY set as a comma-separated list, you can provision only certain Ansible roles (ex.: TAGS_ONLY="consul,nomad" make provision).
• To deploy only Hashicorp stack (Vault, Consul and Nomad) without Terraforming any services, run make vagrant.

At the end, if everything went fine, you can reach the services Vault, Consul and Nomad at localhost, respectively at 8200, 8500, 4646. While Consul Ingress Gateway is listening at port 8080, where you can find some preinstalled services.

In Consul you should see something like:

While in Nomad:

You can disable terraform applying by set the environment variable TERRAFORM_LABS to false. This will only provision the Hashicorp stack without deploying in Nomad.

Play with microservices

With the provided code you could deployed two microservices minimal-service and minimal-service-2 that can talk to each other.

If you go inside terraform folder and do:

terraform apply -var deploy_example_jobs=true -auto-approve

you'll find both in the cluster.

Both are deployed by Nomad with Envoy as sidecar, so they are inside the Consul service mesh.

Do some intra-services communication test:

nomad alloc exec -task minimal-service \
  $(curl -s http://127.0.0.1:4646/v1/job/minimal-service/allocations | \
  jq -r '.[0].ID'| \
  cut -c -8) curl -s 127.0.0.1:8080 | \
  jq
{
  "host": "127.0.0.1:9090",
  "statuscode": 200,
  "headers": {
    "Accept": "*/*",
    "Content-Length": "0",
    "Duration": "0.030891",
    "Request-time": "2021-02-13 19:08:03.673759094 +0000 UTC",
    "Response-time": "2021-02-13 19:08:03.673789985 +0000 UTC",
    "User-Agent": "curl/7.69.1",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Forwarded-Proto": "http",
    "X-Request-Id": "cd6dcc66-f73c-4d16-8783-e4c7690bb929"
  },
  "protocol": "HTTP/1.1",
  "requestURI": "/",
  "servedBy": "6194ce0dea8d",
  "method": "GET"
}

with this command we go inside minimal-service container and execute a GET request to localhost at port 8080. At that port Envoy proxy is listening to requests, in this case it will proxy the request to minimal-service-2. This is just an example.

OpenFaas in Nomad

In this cluster, by default, we have also deployed faasd! So now we can reach OpenFaas gateway and use OpenFaas for our serverless testing!

Just add 127.0.0.1 faasd-gateway to your /etc/hosts file and you're done.

Go to http://faasd-gateway:8080 to enjoy the beautiful OpenFaas homepage.

Monitoring OpenFaas

Add 127.0.0.1 grafana to your /etc/hosts file and go to http://grafana:8080. Both dashboard are taken from Grafana dashboard repos with few modifications:

• Nomad jobs
• Faasd

Docker login (for OpenFAAS)

OpenFaas needs docker credentials to pull and push images.

The utility script docker-login-faasd.sh does that:

$ ./docker-login.sh <username> <password>
Docker login for Faasd..
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/#/#credentials-store

Login Succeeded
Connection to 127.0.0.1 closed.
 
Done.

Clean up

For cleaning up just execute clean.sh or make clean