Responsibilities
Data Solution Design and Development:
- Design, develop, and implement complex and scalable data solutions on the AWS cloud platform.
- Design, build, and optimize data pipelines to ensure efficiency and reliability.
Programming & Scripting:
- Write clean, efficient, and maintainable SQL and Python code.
- Develop and maintain shell scripts to support automation.
DevOps & Automation:
- Utilize Docker containers for containerizing applications and data pipelines.
- Establish and maintain CI/CD pipelines (e.g., GitHub Actions, GitLab CI/CD) for rapid and reliable deployments.
- Automate data pipeline processes.
Cloud Data Services:
- Deliver solutions on AWS data services (e.g., S3, Redshift, Glue, EMR, Lambda).
- Design and implement data modeling and data warehousing solutions.
Project and Team Management:
- Own and manage the technical aspects of deliveries (at least at the component level).
- Participate in project planning, task estimation, testing, and liaising with business stakeholders.
- Provide technical leadership to smaller teams.
- Create technical documentation.