Rewriting everything manually would take years. So we decided to build a better path on Databricks.

Infrastructure migrations get a lot of attention. Notebook migrations tend not to, which is a big reason why they slow teams down.

Our Zeppelin notebooks weren't simple scripts. They contained complex SQL and Python logic, custom interpreters, Oracle and HDFS references, visualizations, widgets and scheduling logic built up over years. Each one reflected institutional knowledge from the business teams who relied on it. The diversity across the entire notebook landscape made a rule-based rewriting engine impractical, since the logic was simply too heterogeneous and too business-specific for automated rules to handle reliably.

That constraint led us to a cleaner design insight: separate structure from logic, and apply the right tool to each. Structural conversion (mapping Zeppelin's paragraph format to Databricks cells, translating interpreter syntax, reformatting metadata) is deterministic and automatable, while logic reconstruction is not. Thankfully, LLMs are great at this structural conversion part..