Job Title: AWS ETL Cloud Data Engineer
Job Overview:
The AWS Cloud Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines and data infrastructure in the AWS cloud environment. This role requires expertise in AWS services, data modeling, ETL processes, and a keen understanding of best practices for data management and governance.
Key Responsibilities:
- Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Spark/Python on Glue, Redshift, S3, Athena, RDS-PostgreSQL, Airflow, Lambda, DMS, Code Commit, Code Pipeline, Code build, etc.
- Design and build production ETL data pipelines from ingestion to consumption within a big data architecture, using DMS, DataSync & Glue.
- Understand the existing applications(including on-premise Cloudera Datalake) and infrastructure architecture.
- Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
- Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
- Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
- Collaborate with development, infrastructure and data center teams to define Continuous Integration and Continuous Delivery processes in accordance with industry standards.
- Work on hybrid Datalake.
- Work closely with multiple stakeholders to ensure high standards are maintained.
Requirements
Mandatory Skill-set
• Bachelors Degree in Computer Science, Information Technology or other relevant fields
• 5+ years of work experience with ETL, Data Modelling, Data Architecture to build Datalake. Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark.
• 3+ yrs of extensive experience in working on AWS platform using core services like AWS Athena, Glue Pyspark, Redshift, RDS-PostgreSQL, S3 & Airflow(for orchestration).
Good to have Skill-set :
Fundamental of Insurance domain
Functional knowledge on IFRS17
Benefits
14 days AL
Company Insurance Benefits