We are seeking a Senior Databricks Data Engineer to provide technical leadership on a major data platform transformation initiative. This role will lead and execute the migration of a large-scale Microsoft SQL Server data warehouse ecosystem鈥攊ncluding SSIS, SSAS, and SSRS鈥攐nto the Databricks Lakehouse Platform. The ideal candidate combines deep hands-on data engineering expertise with architectural thinking and mentorship capabilities.
Key Responsibilities
- Lead the design and implementation of enterprise-scale data pipelines on Databricks.
- Serve as a technical lead for the migration of a ~20TB SQL Server data warehouse to Databricks.
- Architect migration strategies for hundreds of tables, thousands of SSIS jobs, and complex reporting workloads.
- Redesign legacy ETL and cube-based architectures into modern Lakehouse patterns.
- Define data modeling standards, ingestion frameworks, and transformation best practices.
- Optimize Spark workloads for performance, scalability, and cost efficiency.
- Mentor and guide data engineers across multiple experience levels.
- Partner with BI, analytics, and AI/ML teams to enable advanced analytics use cases.
- Implement data governance, security, lineage, and monitoring using Unity Catalog and platform tooling.
- Translate business requirements into scalable technical solutions.
- Influence platform roadmap, engineering standards, and architectural decisions.