changedatacapture.net

Oracle Archive Log Manager

Interactive guide to preventing missing archived log errors in Debezium Oracle CDC pipelines.

Oracle Archive Log Management for Debezium

LogMiner reads change data from archived redo logs. RMAN deletes them. Left uncoordinated, this conflict produces fatal "missing archived log" errors that halt your CDC pipeline. This tool covers the prerequisite Oracle setup, explains the retention versus downtime conflict with an interactive simulator, and provides the SCN-aware RMAN cleanup strategy that prevents outages.

How Debezium reads Oracle changes

The Debezium Oracle connector uses the LogMiner utility to read changes from the redo stream. LogMiner needs access to both online redo logs (for recent changes) and archived redo logs (for historical data and restarts). This dependency on raw archive log files on disk is the key operational constraint.

1. Oracle Database

DML changes are written to the active online redo log by the LGWR process.

2. Archive Log Destinations

On each log switch, ARCn copies the full redo log to one or more archive destinations on disk.

3. Debezium via LogMiner

Debezium calls DBMS_LOGMNR.ADD_LOGFILE and queries V$LOGMNR_CONTENTS to stream changes.

The conflict: If RMAN deletes an archived log (step 2) before Debezium has processed all changes in it (step 3), the connector fails with ORA-00308: cannot open archived log and shuts down immediately.

Oracle prerequisites

These settings must be in place before starting the connector. Missing any of them causes a startup failure or, worse, silent data loss with no immediate error.

Required

ARCHIVELOG mode

Without ARCHIVELOG mode, Oracle overwrites online redo logs on each log switch. There are no persistent archive files for LogMiner to read.

ALTER DATABASE ARCHIVELOG;

Required

FORCE LOGGING

Without this, direct-path inserts (INSERT /*+ APPEND */) bypass the redo log entirely. Bulk loads disappear from the CDC stream with no error or warning.

ALTER DATABASE FORCE LOGGING;

Required

Supplemental logging

Database-level minimal logging is required for LogMiner to parse redo. Table-level logging determines the completeness of Debezium's before field.

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

ALTER TABLE t ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;  -- recommended

ALTER TABLE t ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;  -- minimum

(ALL) COLUMNS populates the full row pre-image in the before field. (PRIMARY KEY) leaves before incomplete for UPDATE events. (ALL) COLUMNS increases redo volume significantly — size archive storage accordingly.

Required

Connector user privileges

The connector user needs: LOGMINING, CREATE SESSION, EXECUTE_CATALOG_ROLE, SELECT_CATALOG_ROLE, EXECUTE ON DBMS_LOGMNR, and SELECT on the key V$ views: V_$DATABASE, V_$ARCHIVED_LOG, V_$ARCHIVE_DEST_STATUS, V_$LOGMNR_CONTENTS.

A missing grant on a V$ discovery view is a common failure mode. The connector starts, then fails when querying V$ARCHIVED_LOG to locate the log files for its starting SCN.

Related tools

Part of: Debezium Oracle CDC