ODI to Databricks

Architecture

Oracle ODI in. Databricks out.

Deterministic parsers read the estate and emit native Databricks code — not ODI mappings lifted into a hosted ODI.

Oracle ODI → MigryX parser → PySpark + Delta + Workflows

Oracle ODI

MappingsInterfaces

PackagesSerial + loops

Load plansParallel steps

Knowledge modsLKM / IKM / CKM

MigryX Parser

Deterministic parseAI where it helps

Row-level parityBefore cutover

PySpark emitSet-based, not loops

Workflow emitJob DAGs

Databricks

PySpark notebooksSet-based, not loops

Delta LakeACID + time travel

WorkflowsReplaces scheduler

Unity CatalogColumn lineage

Repos + DABsCI/CD promote

MLflowWhere models live

MigryX AI handles the logic parsers cannot resolve alone, and every change it makes goes through the same parity checks. It runs on a model you approve, air-gapped if your estate requires it.

Why Databricks

Knowledge Modules were never meant for lakehouse architectures

KMs generate opaque, Oracle-coupled SQL

IKMs and LKMs produce SQL that is tightly bound to Oracle. When source or target changes, KM-generated code breaks silently. PySpark on Databricks replaces KMs with native Spark read/write strategies that are transparent, testable, and version-controlled.

Package orchestration lacks modern workflow capabilities

ODI packages chain steps linearly with basic error handling. Load plans add parallelism but have limited retry logic and no native integration with cloud services. Databricks Workflows handle DAGs, conditional branching, parameterized runs, and failure recovery natively.

Oracle-specific SQL locks you to a single vendor

ODI mappings routinely use Oracle-specific functions (DECODE, NVL, ROWNUM, CONNECT BY) that have no direct equivalents outside Oracle. Spark SQL on Databricks gives you a vendor-neutral query engine with Delta Lake's ACID guarantees and schema evolution.

Parser output

ODI IKM incremental update to PySpark Delta merge

An ODI mapping using IKM Oracle Incremental Update with merge logic - converted to native PySpark with Delta Lake MERGE INTO. No Knowledge Modules, no generated SQL, no Oracle dependency.

ODI Mapping (IKM Oracle Incremental Update)

<!-- ODI Mapping: Load_Customer_Dim -->
<Mapping name="Load_Customer_Dim">
  <Source schema="STAGING"
         table="STG_CUSTOMERS"/>
  <Target schema="DW"
         table="DIM_CUSTOMER"/>
  <Join condition="SRC.CUST_ID = TGT.CUST_ID"/>
  <Filter>SRC.MODIFIED_DT > :LAST_RUN</Filter>
  <Expression target="FULL_NAME">
    NVL(FIRST_NAME,'')||' '||NVL(LAST_NAME,'')
  </Expression>
  <Expression target="STATUS_DESC">
    DECODE(STATUS,'A','Active','I','Inactive','Unknown')
  </Expression>
  <IKM name="IKM Oracle Incremental Update"
       FLOW_CONTROL="true"
       RECYCLE_ERRORS="true"/>
</Mapping>

→

MigryX
converts

PySpark on Databricks

# ODI IKM Incremental Update → Delta MERGE
from pyspark.sql import functions as F

stg = spark.read.table("staging.stg_customers")
last_run = dbutils.widgets.get("last_run")

src = (
    stg.filter(F.col("modified_dt") > last_run)
    .withColumn("full_name",
        F.concat_ws(" ",
            F.coalesce(F.col("first_name"), F.lit("")),
            F.coalesce(F.col("last_name"), F.lit(""))))
    .withColumn("status_desc",
        F.when(F.col("status") == "A", "Active")
         .when(F.col("status") == "I", "Inactive")
         .otherwise("Unknown"))
)

from delta.tables import DeltaTable
tgt = DeltaTable.forName(spark, "dw.dim_customer")

tgt.alias("tgt").merge(
    src.alias("src"), "tgt.cust_id = src.cust_id"
).whenMatchedUpdateAll() \
 .whenNotMatchedInsertAll() \
 .execute()

IKM Oracle Incremental Update replaced by Delta Lake MERGE INTO. Oracle-specific NVL and DECODE functions converted to PySpark coalesce and when/otherwise. Flow control and error recycling handled by Databricks Workflow retry policies.

Coverage