Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: Create a new blob-store service for handling large temporal payloads #642

Merged
merged 3 commits into from
Oct 13, 2024

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented Oct 13, 2024

  • feat(blob-store): Add blob store for storing large blobs for agent workflows
  • feat(blob-store): Add authentication

Important

Introduces a new blob-store service using SeaweedFS for handling large temporal payloads, with Docker and authentication setup.

  • Blob Store Service:
    • Introduces a new blob-store service using SeaweedFS in blob-store/Dockerfile, docker-compose.yml, and docker-compose-ha.yml.
    • Adds entrypoint.sh for environment variable checks and configuration file generation.
    • Includes s3.json.template for S3 configuration.
  • Environment Configuration:
    • Updates .env.example with S3_ENDPOINT, S3_ACCESS_KEY, and S3_SECRET_KEY for blob store configuration.
    • Adds USE_BLOB_STORE_FOR_TEMPORAL and BLOB_STORE_CUTOFF_KB for temporal payload handling.
  • Authentication:
    • Implements authentication in s3.json.template with access and secret keys for the julep account.
  • Miscellaneous:
    • Adds .gitignore entry for /s3.json in blob-store/.gitignore.
    • Updates main docker-compose.yml to include blob-store/docker-compose.yml.

This description was created by Ellipsis for 736b9df. It will automatically update as commits are pushed.

…rkflows

Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 736b9df in 25 seconds

More details
  • Looked at 301 lines of code in 8 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. blob-store/Dockerfile:14
  • Draft comment:
    Consider removing build dependencies after use to reduce image size. You can do this by combining the apk del build_deps command with the previous commands in a single RUN statement.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
2. blob-store/entrypoint.sh:5
  • Draft comment:
    Consider checking if the DEBUG environment variable is set before using it. This will prevent potential issues if the variable is not set.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The entrypoint script checks for environment variables but does not check for the DEBUG variable, which is used later in the script.
3. blob-store/s3.json.template:4
  • Draft comment:
    Ensure that allowing anonymous read access is intentional and secure for your use case. If not, consider removing or restricting this access.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_ooMQtoDKoQIffK0E


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@creatorrr creatorrr merged commit e02ee06 into dev Oct 13, 2024
6 checks passed
@creatorrr creatorrr deleted the f/blob-store branch October 13, 2024 20:33
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant