In-depth guides and interactive tools for building real-time data pipelines across Oracle, Debezium, GoldenGate, Kafka, Redpanda, and Apache Iceberg.
Explore guidesResiliency and recovery architectures for Debezium Server with Oracle LogMiner in Kubernetes: monitoring visualizations, root-cause analysis, recovery playbooks, and K8s topology patterns.
Interactive tuning studio that generates optimized configurations for throughput, latency, CPU, and memory profiles, with trade-off analysis for each.
Dashboard-style explorer for Debezium Server with Oracle XStream: connection pooling, JDBC state storage, SMT pipelines, CloudEvents formatting, HTTP sink batching, and retry strategies.
XStream vs LogMiner for multiple concurrent readers, same schema vs different schemas, horizontal scaling patterns, and high availability in Kubernetes.
Change Data Capture (CDC) is a design pattern that tracks row-level changes in a database, inserts, updates, and deletes, and delivers them as a stream of events to downstream systems in real time. Instead of periodically querying for differences, CDC reads the database's transaction log (redo logs in Oracle, WAL in PostgreSQL, binlog in MySQL), capturing every change the moment it's committed.
CDC is the foundation of modern event-driven architectures, enabling real-time data replication, cache invalidation, audit logging, microservice synchronisation, and streaming analytics. In a typical CDC data stack, change events flow from the source database into a message broker, Apache Kafka or Redpanda, and from there into downstream stores such as data lakes built on Apache Iceberg, analytical warehouses, or search indexes. Debezium (open-source) and Oracle GoldenGate (commercial) are the most widely deployed CDC tools for Oracle Database environments.
Open-source CDC platform that streams database changes as events. Runs as a standalone Quarkus application in Kubernetes, supporting Oracle, PostgreSQL, MySQL, SQL Server, and MongoDB.
Oracle's built-in utility for reading redo logs via SQL. No additional license required. Used by Debezium with the online_catalog or redo_log_catalog strategies.
Oracle's high-performance streaming API built on GoldenGate internals. Provides lower latency than LogMiner but requires an Oracle GoldenGate license. Used by Debezium as an alternative capture adapter.
Enterprise-grade real-time data integration platform for heterogeneous replication, zero-downtime migrations, and active-active database synchronization across Oracle and non-Oracle targets.
The most widely used event streaming platforms for CDC pipelines. Kafka Connect hosts Debezium as a source connector, publishing change events to topics. Redpanda is a Kafka-compatible alternative with no ZooKeeper dependency.
Open table format for large-scale data lakes in object storage (S3, Azure Data Lake, GCS). CDC events flow through Kafka Connect sink connectors into Iceberg tables, enabling real-time analytics on operational Oracle data.
log.mining.batch.size.default (increase to 100,000+ for larger mining windows), max.queue.size (buffer between source and sink), poll.interval.ms (reduce to minimize idle time), and HTTP sink batch size. Use ZGC for garbage collection on large heaps, and ensure the Kubernetes pod has guaranteed QoS with requests equal to limits.