SSIS packages (.dtsx) and project archives (.ispac) parsed structurally. Data Flows and Control Flows converted to PySpark notebooks on Databricks with Delta Lake. Full lineage, validated parity.
Upload a package, get converted code →OLE DB Source, Derived Column, Lookup, and Conditional Split are single-threaded transforms locked to one server. PySpark distributes the same logic across clusters. Batch jobs that crawled for hours finish in minutes on Databricks.
C# and VB.NET Script Tasks are opaque, difficult to test, and locked to the .NET runtime. Python notebooks on Databricks are version-controlled, unit-testable, and run on Spark's distributed engine natively.
SSIS package chains via Execute Package Task and SQL Agent jobs are fragile, with limited retry and no DAG awareness. Databricks Workflows provide task DAGs, parameterized runs, conditional branching, and built-in failure recovery.
A typical SSIS Data Flow pipeline with OLE DB Source, Derived Column, Lookup, and OLE DB Destination -- converted to native PySpark DataFrames that run on Spark's distributed engine with Delta Lake output.
<!-- Package: Customer_ETL.dtsx -->
<!-- Data Flow Task: Load Customers -->
OLE DB Source: [dbo].[RAW_CUSTOMERS]
SELECT CustomerID, Name, Region,
TotalSpend, LastOrderDate
FROM RAW_CUSTOMERS
WHERE IsActive = 1
Derived Column: [Tier]
(DT_WSTR,10)(
TotalSpend > 50000 ? "Platinum" :
TotalSpend > 10000 ? "Gold" :
"Standard"
)
Lookup: [DIM_REGION]
JOIN ON Region = RegionCode
OUTPUT: RegionName, SalesManager
OLE DB Destination: [dbo].[DIM_CUSTOMERS]
INSERT INTO DIM_CUSTOMERS
(CustomerID, Name, Tier,
RegionName, SalesManager)
# SSIS Data Flow → PySpark on Databricks
from pyspark.sql import functions as F
# OLE DB Source → spark.read.jdbc()
raw = (
spark.read.jdbc(jdbc_url, "RAW_CUSTOMERS")
.filter(F.col("IsActive") == 1)
.select("CustomerID", "Name", "Region",
"TotalSpend", "LastOrderDate")
)
# Derived Column → .withColumn()
tiered = raw.withColumn("Tier",
F.when(F.col("TotalSpend") > 50000, "Platinum")
.when(F.col("TotalSpend") > 10000, "Gold")
.otherwise("Standard"))
# Lookup → .join()
dim_region = spark.read.table("dim_region")
enriched = tiered.join(
dim_region,
tiered.Region == dim_region.RegionCode,
"left"
).select("CustomerID", "Name", "Tier",
"RegionName", "SalesManager")
# OLE DB Destination → Delta Lake write
enriched.write.format("delta") \
.mode("overwrite") \
.saveAsTable("dim_customers")
OLE DB Source, Derived Column, Lookup, and OLE DB Destination replaced by native PySpark. Data Flow components become DataFrame operations. Output writes to Delta Lake with ACID guarantees.
| SSIS Component | Databricks Equivalent | Notes |
|---|---|---|
| OLE DB Source | spark.read.jdbc() | SQL Server, Oracle, and other JDBC sources |
| Derived Column | .withColumn() | SSIS expressions mapped to PySpark column expressions |
| Conditional Split | .filter() | Multiple output paths become filtered DataFrames |
| Lookup | .join() | Full match, partial match, and no-match outputs preserved |
| Merge Join | .join() | Inner, left, full outer join types supported |
| Aggregate | .groupBy().agg() | SUM, COUNT, AVG, MIN, MAX aggregations |
| Sort | .orderBy() | Multi-column sort with ASC/DESC preserved |
| Execute SQL Task | spark.sql() | T-SQL translated to Spark SQL dialect |
| Script Task (C#/VB.NET) | Python notebook | Custom .NET logic rewritten as Python |
| For/Foreach Loop | Python loop / Workflow | Loop containers become Python iterators or multi-task workflows |
| Execute Package Task | Workflow task | Package chains become Databricks Workflow DAGs |
| SSISDB Catalog | Databricks Jobs | Environments, parameters, and scheduling modernized |
Data Matching compares SSIS package output against Databricks output -- row by row, column by column. In the case study below, all ETL pipelines were validated against SQL Server production baselines with full regression testing.
See how Data Matching works →2,200 SSIS packages converted to PySpark on Databricks. 450 Script Tasks rewritten from C#/VB.NET to Python notebooks. Data Flow pipelines with OLE DB, Lookup, and Derived Column transforms replaced by native PySpark DataFrames. SSISDB Catalog scheduling modernized to Databricks Workflows. SQL Agent jobs decommissioned within 90 days.
Read the full case study →Upload an SSIS package (.dtsx). Get parsed lineage, PySpark code for Databricks, and a validation report.