System Architecture

This view visualizes the data pipeline defined in application.properties. Data flows from the Oracle PDB, through Debezium's transformation logic (SMTs), and is pushed via HTTP to the event gateway.

🗃 Oracle 19c

my-db_1

PDB Source

LogMiner
→

⚙ Debezium 3.0

my-app-debezium

Kubernetes Pod

CloudEvents
JSON
→

🌐 HTTP Sink

Event Gateway

:9090/v1/events

Connector Class

...oracle.OracleConnector

Output Format

CloudEvents (JSON)

Topic Prefix

my-app.cdc

Oracle Source Configuration

Configuration for the OracleConnector. This section defines how Debezium connects to the PDB, handles connection pooling, and manages snapshots.

Connection Settings

DB Name

my-db_1

PDB Name

my-db_1

Snapshot Mode

no_data

Decimal Handling

double

Note: LogMiner is the presumed adapter. User has requested research into XStream (see Research Notes).

Connection Pool Lifecycles

Comparing Max Idle Time vs Max Lifetime (ms)

Signal Channel

Debezium signals are enabled via a signaling table, allowing ad-hoc snapshots.

debezium.source.signal.enabled.channels=source
debezium.source.signal.data.collection=MY_DB.C##DEBEZIUM.DEBEZIUM_SIGNAL

State Management (JDBC)

Debezium persists its offsets and schema history in Oracle tables rather than local files or Kafka topics. This ensures the connector state resides alongside the data.

Offset Storage

JdbcOffsetBackingStore

Tracks the last read position in the transaction logs.

Table DDL

CREATE TABLE DEBEZIUM_OFFSET_STORAGE (
    id VARCHAR2(36) NOT NULL,
    offset_key VARCHAR2(1255),
    offset_val VARCHAR2(1255),
    record_insert_ts TIMESTAMP NOT NULL,
    record_insert_seq INTEGER NOT NULL
)

Flush Interval: 60,000 ms

Schema History

JdbcSchemaHistory

Tracks DDL changes to captured tables to interpret logs correctly.

Table DDL

CREATE TABLE DEBEZIUM_SCHEMA_HISTORY (
    id VARCHAR2(36) NOT NULL,
    history_data CLOB,
    history_data_seq INTEGER,
    record_insert_ts TIMESTAMP NOT NULL,
    record_insert_seq INTEGER NOT NULL
)

Store Only Captured: true

Pipeline & Transformations

This section analyzes the Single Message Transformations (SMTs) and data formatting applied to every event before it reaches the sink.

Transformation Simulator: RegexRouter

The configuration defines a route transformation. It uses regex .* to match ANY topic and replaces it with my-app.cdc.

Input Topic (Simulated)

Regex: .* → Replacement: my-app.cdc

➔

Final Output Topic

my-app.cdc

Data Format: CloudEvents

The payload adheres to the CloudEvents specification.

Key Format JSON (Schemas Enabled)
Value Format CloudEvents (JSON)
Value Converter io.debezium.converters.CloudEventsConverter

Topic Structure

Topic naming strategy controls.

Prefix: my-app.cdc

Note: The RegexRouter transformation overrides the default topic naming, forcing all events into a single topic or gateway endpoint logic.

HTTP Sink Configuration

Events are batched and sent via HTTP POST to a downstream gateway. This section analyzes the reliability and throughput settings.

Batch Capacity

Configured Batch Size vs. Max Queue Size

Resilience: Retry Backoff

Simulated delay for 10 retries (Base: 1000ms)

Endpoint Details

Target URL http://localhost:9090/api-gateway/v1/events

Content Type application/cloudevents-batch+json

Request Timeout 3,600,000 ms (1 Hour)

Error Handler io.debezium.server.http.HttpBatchedChangeConsumer$ErrorHandler

Research & Action Items

Based on the provided notes, the following items require investigation for the migration from LogMiner to XStream.

1. XStream vs. LogMiner Configuration

Current: Config is set for LogMiner (implied default).
Target: XStream.
Impact: Requires changing `debezium.source.connector.adapter` to `xstream`. Requires Oracle XStream license and setup (XStream Outbound server).

2. Message Structure Analysis

Question: Will structure differ between LogMiner/XStream or Debezium 3.0/3.4?
Insight: The abstract `CloudEvents` output structure generally remains stable across adapters, as Debezium normalizes the event. However, specific `source` metadata fields (like SCN vs LCR position) might differ.

3. Debezium Server 3.4.0 Upgrade

Requirement: Compare 3.0.0 config with 3.4.0.
Action: Check `quarkus.*` namespace changes and any deprecation in the `http-batched` sink properties.

Debezium XStream Configuration Explorer

System Architecture

Connector Class

Output Format

Topic Prefix

Oracle Source Configuration

Connection Settings

Connection Pool Lifecycles

Signal Channel

State Management (JDBC)

Offset Storage

Schema History

Pipeline & Transformations

Transformation Simulator: RegexRouter

Data Format: CloudEvents

Topic Structure

HTTP Sink Configuration

Batch Capacity

Resilience: Retry Backoff

Endpoint Details

Research & Action Items

1. XStream vs. LogMiner Configuration

2. Message Structure Analysis

3. Debezium Server 3.4.0 Upgrade