We are looking for a Middle Data Engineer specialized in Azure Databricks to join our data platform team.
The candidate will design and develop modern data pipelines and Lakehouse architectures, leveraging Azure Databricks, Spark, and Azure Data Factory, while integrating with existing SQL Server-based data warehouse environments, also evolving our data platform towards scalable, cloud-based data architectures, enabling advanced analytics and business intelligence.
Key responsibilities:
- Design, develop, and maintain data pipelines using Azure Databricks
- Build and optimize data transformations using PySpark and SQL in Databricks
- Implement and maintain Lakehouse architectures using Delta Lake
- Develop ETL/ELT pipelines orchestrated through Azure Data Factory
- Integrate data from multiple sources into the data platform and analytical layers
- Design and maintain data models and data warehouse structures for analytics
- Ensure data quality, scalability, and performance of large-scale data processing pipelines
- Collaborate with BI teams to support Power BI and reporting platforms
- Support and evolve existing SQL Server data platforms and ETL solutions (SSIS) when required
- Contribute to the design of modern cloud-based data architectures