Skip to content

Edda

Jump to bottom
Matthew Johnson edited this page Sep 13, 2019 · 26 revisions

Introduction

Netflix/Edda is a caching service for AWS API calls that can help mitigate the effects of throttling. Instead of querying AWS APIs for information about your systems, use Edda and save your AWS API calls for creating and modifying resources.

The configuration of this container drops the use of MongoDB in favor of using DynamoDB and S3.

Caveats

The current Edda Docker image is not production ready. This guide will help you get up and running quickly, but take note of the following:

  • Ensure the docker host has the correct time. Communication with the AWS API's require the container to be within 15 minutes of actual.
  • The docker images currently do not export logs off the container. Logging is set to INFO level, which can be verbose. Logs are written to /jetty.log within the container. Override the log4j.category.com.netflix.edda property to set a less verbose logging level, if you need to save disk space.
  • The Tomcat JVM configuration has not been tuned for large collections. The Java default max heap size is used, which is the smaller of either 1/4 system RAM or 1GB.
  • The IAM role guidance has not been tweaked to the absolute minimum set of permissions.

Pre-Requisites

  • Create an S3 bucket to store the current resource crawl state.

Configure

  1. Copy the edda.properties template file to edda.properties.mine and make modifications there.
    1. Set edda.region= to the region that you are using.
    2. Set edda.s3current.bucket= to your new S3 bucket name.
  2. For local workstation execution, configure keys that have the PowerUser role:
    1. Set edda.aws.accessKey=
    2. Set edda.aws.secretKey=
  3. For AWS execution:
    1. Comment out edda.aws.accessKey=
    2. Comment out edda.aws.secretKey=
    3. Launch an instance with an IAM Role that grants the equivalent of the PowerUser role.

No configuration is necessary for DynamoDB; the service will create the tables needed at startup. If you decide to change your S3 bucket name after you have started the service for the first time, then you will need to delete the edda-s3current-collection-index-dev table in DynamoDB before starting the service with the new configuration.

If you want to minimize the rights associated with the role used for Edda, then follow these guidelines:

  • Read/write access is required for DynamoDB and S3.
  • Read access is required for the rest of the resources you want to crawl (i.e. describe... calls).

Run the Container

docker run -d \
  --name edda \
  -p 8080:8080 \
  -v `pwd`/edda.properties.mine:/tomcat/webapps/ROOT/WEB-INF/classes/edda.properties \
  netflixoss/edda:2.1

Test the Container

Test the Edda service by querying for instances and security groups.

curl -g 'localhost:8080/api/v2/view/instances;_pp'
curl -g 'localhost:8080/api/v2/aws/securityGroups;_pp'

See the Edda REST API documentation for additional use cases.

Inspect the Container

docker exec -i -t edda bash

tail -f jetty.log
Clone this wiki locally