Skip to content

Latest commit

 

History

History
52 lines (34 loc) · 1.53 KB

data.md

File metadata and controls

52 lines (34 loc) · 1.53 KB
Raw

Data

Big Data

See big-data.md

Data Validation

Start by validating data formats for correctness.

Scripts for this can be found in both the DevOps-Python-tools and DevOps-Bash-tools repos.

Then proceed to more advanced content validation.

Data Integration

  • DBT - open-source data pipeline workflow tool
  • Informatica - proprietary legacy now available via SaaS, with self-hosted agents on VMs or Kubernetes
  • Airbyte - open source self-hosted or SaaS proprietary with 300+ connectors
  • Apache Camel - open source with 100+ connectors
  • Spring Integration - XML config, only use for Spring heavy shops
  • Mulesoft - XML config, only use for proprietary connectors

Mulesoft

  • lightweight enterprise service bus + integration framework
  • proprietary connectors
  • Anypoint Studio (Eclipse-based IDE)
  • Anypoint Enterprise Security - security features, transactions

Spring Integration

TODO

Diagram - Top 9 Systems Integrations

Ported from private Knowledge Base pages 2016+