Ever wondered what it feels like to be a Data Engineer (Junior-Mid) in Netcompany ?
Joining us as a Data Engineer (Junior-Mid), you will work with relational databases, SQL, big data tools such as PySpark and Spark SQL, and orchestration tools like Apache Airflow. You will collaborate with other data engineers, analysts, and stakeholders to support the development and optimization of data pipelines.
As a Data Engineer (Junior-Mid) you will:
- Assist in designing, developing, and maintaining scalable data pipelines and ETL/ELT processes.
- Write and maintain SQL queries to extract, transform, and load data into various systems.
- Utilize PySpark and Spark SQL in Notebooks (e.g., Jupyter, Databricks) to process and analyze datasets in distributed computing environments.
- Support the use of orchestration tools like Apache Airflow to schedule and monitor data workflows.
- Collaborate with business analysts and data architects to ensure seamless data integration across systems.
- Ensure data quality and consistency by implementing data validation mechanisms.
- Troubleshoot and resolve basic issues related to data processing and system performance.