dltHub

The product is an AI-native data engineering platform and open-source Python library that builds, runs, and governs data pipelines on top of existing warehouses such as Snowflake, Databricks, and BigQuery, solving the problem of creating and maintaining production ingestion, transformation, and orchestration at scale. It is for developers, data and platform teams, and engineering teams in B2B organizations that need to ship and operate pipelines. It is delivered as managed infrastructure with agentic primitives and a context catalog for lineage, governance, and observability, with agents handling building, deployment, and maintenance.

Key features

• Schema inference
• Data normalization
• Incremental loading
• Data contracts
• Data ingestion
• Data transformation
• Pipeline orchestration
• Lineage tracking
• Data quality monitoring
• Governance controls
• Run state observability
• Scheduling and alerting
• 5000+ source connectors
• Warehouse integration for Snowflake Databricks BigQuery