ODI Smart Export XML parsed structurally. Mappings, Knowledge Modules, packages, and load plans converted to PySpark notebooks and Databricks Workflows. Full lineage, validated parity.
Upload an ODI export, get converted code →IKMs and LKMs produce SQL that is tightly bound to Oracle. When source or target changes, KM-generated code breaks silently. PySpark on Databricks replaces KMs with native Spark read/write strategies that are transparent, testable, and version-controlled.
ODI packages chain steps linearly with basic error handling. Load plans add parallelism but have limited retry logic and no native integration with cloud services. Databricks Workflows handle DAGs, conditional branching, parameterized runs, and failure recovery natively.
ODI mappings routinely use Oracle-specific functions (DECODE, NVL, ROWNUM, CONNECT BY) that have no direct equivalents outside Oracle. Spark SQL on Databricks gives you a vendor-neutral query engine with Delta Lake's ACID guarantees and schema evolution.
An ODI mapping using IKM Oracle Incremental Update with merge logic — converted to native PySpark with Delta Lake MERGE INTO. No Knowledge Modules, no generated SQL, no Oracle dependency.
<!-- ODI Mapping: Load_Customer_Dim -->
<Mapping name="Load_Customer_Dim">
<Source schema="STAGING"
table="STG_CUSTOMERS"/>
<Target schema="DW"
table="DIM_CUSTOMER"/>
<Join condition="SRC.CUST_ID = TGT.CUST_ID"/>
<Filter>SRC.MODIFIED_DT > :LAST_RUN</Filter>
<Expression target="FULL_NAME">
NVL(FIRST_NAME,'')||' '||NVL(LAST_NAME,'')
</Expression>
<Expression target="STATUS_DESC">
DECODE(STATUS,'A','Active','I','Inactive','Unknown')
</Expression>
<IKM name="IKM Oracle Incremental Update"
FLOW_CONTROL="true"
RECYCLE_ERRORS="true"/>
</Mapping>
# ODI IKM Incremental Update → Delta MERGE
from pyspark.sql import functions as F
stg = spark.read.table("staging.stg_customers")
last_run = dbutils.widgets.get("last_run")
src = (
stg.filter(F.col("modified_dt") > last_run)
.withColumn("full_name",
F.concat_ws(" ",
F.coalesce(F.col("first_name"), F.lit("")),
F.coalesce(F.col("last_name"), F.lit(""))))
.withColumn("status_desc",
F.when(F.col("status") == "A", "Active")
.when(F.col("status") == "I", "Inactive")
.otherwise("Unknown"))
)
from delta.tables import DeltaTable
tgt = DeltaTable.forName(spark, "dw.dim_customer")
tgt.alias("tgt").merge(
src.alias("src"), "tgt.cust_id = src.cust_id"
).whenMatchedUpdateAll() \
.whenNotMatchedInsertAll() \
.execute()
IKM Oracle Incremental Update replaced by Delta Lake MERGE INTO. Oracle-specific NVL and DECODE functions converted to PySpark coalesce and when/otherwise. Flow control and error recycling handled by Databricks Workflow retry policies.
| ODI Artifact | Databricks Equivalent | Notes |
|---|---|---|
| Mapping | PySpark notebook | Source-to-target flow with expressions and joins |
| IKM (Integration KM) | Spark write strategy | Append, overwrite, merge via Delta Lake |
| LKM (Loading KM) | spark.read | JDBC, file, cloud storage readers |
| Package | Databricks Workflow | Step sequences with conditional branching |
| Load Plan | Multi-task Job | Parallel/serial execution with restart points |
| Procedure | Python script | Custom logic as notebook cells or utilities |
| Variable | Widget parameter | Runtime parameters passed to notebooks |
| Interface (pre-12c) | DataFrame pipeline | Legacy interfaces parsed and converted |
| CKM (Check KM) | Delta constraints | NOT NULL, CHECK constraints on Delta tables |
| Sequence | Task dependency | Execution order defined in Workflow DAG |
| RKM (Reverse KM) | Unity Catalog schema | Metadata reverse-engineering via catalog APIs |
| JKM (Journalizing KM) | Delta CDF | Change Data Feed replaces journalizing tables |
Data Matching compares ODI mapping output against Databricks notebook output — row by row, column by column. In the case study below, all 1,200 mappings were validated against Oracle source-of-truth data with full production backtesting.
See how Data Matching works →1,200 ODI mappings converted to PySpark notebooks on Databricks. 85 Knowledge Modules — IKMs, LKMs, CKMs — replaced by native Spark read/write strategies and Delta Lake constraints. Load plans restructured as multi-task Databricks Jobs with parallel execution and automatic restart. Oracle-specific SQL (NVL, DECODE, CONNECT BY) transpiled to Spark SQL equivalents. Batch windows reduced from 8 hours to 90 minutes.
Read the full case study →Upload an ODI Smart Export. Get parsed lineage, PySpark code for Databricks, and a validation report.