We are looking for a Senior Developer with strong experience in data engineering and ETL development to join our team. The ideal candidate will have solid expertise in building and maintaining scalable data pipelines, working with big data technologies, and ensuring high-quality data processing within modern data platforms.
You will play a key role in the design, development, and optimization of data solutions, contributing to the development of robust data architectures and supporting business intelligence initiatives.
Key Responsibilities
Design, develop, and maintain ETL pipelines for data ingestion and transformation.
Work with Spark and Scala to build scalable data processing solutions.
Design and implement data models to support analytics and reporting needs.
Develop and maintain data lake solutions using technologies such as Apache Hive, S3, and AWS.
Ensure data quality and validation across data pipelines and datasets.
Write and optimize complex SQL queries for data processing and analysis.
Work with relational database systems and ensure efficient data storage and retrieval.
Contribute to BI and analytics environments, integrating with tools such as Databricks, Power BI, or Starburst.
Implement unit testing and ensure code quality using static code analysis tools such as Sonar or Fortify.
Support and maintain CI/CD pipelines to ensure efficient and reliable software delivery.
Required Experience
5+ years of experience implementing ETL processes.
3+ years of experience with Apache Spark.
3+ years of experience with Scala.
2+ years of experience working with BI tools (e.g., Databricks, Power BI, Starburst).
Strong expertise in SQL.
Experience in data model design.
Hands-on experience with Data Lakes (Apache Hive, S3, AWS).
Experience working with relational database systems.
Experience in data validation and quality assurance.
Experience with unit testing and static code analysis tools (Sonar and/or Fortify).
Strong understanding of CI/CD and modern software delivery practices.
talan