OLake

OLake is an open-source EL engine that replicates databases into Apache Iceberg and Parquet on S3, solving the problem of slow and fragile data ingestion pipelines. It targets data engineers and engineering teams in mid-market and enterprise companies who need reliable, high-performance data movement. It is positioned as faster than traditional tools, with benchmarks showing up to 500x faster ingest, and offers self-hosted deployment for full control over data.

Key features

• CDC (Change Data Capture)
• Parallel chunking
• Incremental sync
• Supports 8 sources: Postgres, MySQL, MongoDB, Oracle, Kafka, S3, DB2 LUW, MSSQL
• Writes to Apache Iceberg or Parquet
• Supports AWS Glue, Hive Metastore, JDBC/SQL, REST catalogs
• Self-hosted deployment
• Scheduled compaction and cleanup