AWS Data Engineer
Experience
Salary
Location
Job Description
JOB DESCRIPTION:
- AWS Data Engineer with minimum of 5 to 7 years of experience.
- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.
- Connect with stakeholders daily to discuss project progress and updates.
- Work within an Agile process to deliver projects in a timely and efficient manner.
- Design and develop Airflow DAGs to schedule and manage ETL workflows.
- Transform SQL queries into Spark SQL code for ETL pipelines.
- Develop custom Python functions to handle data quality and validation.
- Write PySpark scripts to process data and perform transformations.
- Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes.
- Run Spark jobs on AWS EMR cluster using Airflow DAGs.
- Monitor and troubleshoot ETL pipelines to ensure smooth operation.
- Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture.
- Collaborate with other members of the data engineering team to improve processes and implement new technologies.
- Stay up to date with emerging trends and technologies in data engineering and suggest ways to improve the team's efficiency and effectiveness.
Designation
AWS Data Engineer
Key Skills
Industry
IT-ITeS
Functional Area
IT Services
Employment Type
Full Time
Education
Bachelor Degree
Job Type
Work From Office
Related Jobs
Associate Sales Manager
CareEco Technologies Pvt Ltd
Join the CareEco Technologies Pvt Ltd team as a Full Time Associate Sales Manager at Erekrut, specializing in ...
0-3 Lakhs
0-1 Years
Delhi-New ..
Agile Develo...
Inside Sales
Global Talent Manufacturing Private Limited
Join the Global Talent Manufacturing Private Limited team as a Full Time Inside Sales on Erekrut, specializing...
3-6 Lakhs
1-3 Years
Karnataka-..
Agile Develo...
BDM – Events Industry
Rap Events & Xperiences LLP
Join the RAP EVENTS & XPERIENCES LLP team as a Full Time BDM - Events Industry on Erekrut, specializing in BDM...
3-6 Lakhs
1-3 Years
Karnataka-..
Agile Develo...
Billing & Planning Enginee...
Perfect Engineers & Resources Private Limited
Join the Perfect Engineers & Resources Pvt Ltd team as a Full Time Billing & Planning Engineer on Erekrut, spe...
6-10 Lakhs
7-10 Years
Karnataka-..
Agile Develo...
MEP Quantity Surveyor
Perfect Engineers & Resources Pvt. Ltd.
Join the team as a Full Time MEP Billing Engineer on Erekrut, specializing in MEP Quantity Surveyor in the Re...
3-6 Lakhs
1-3 Years
Maharashtr..
Agile Develo...
Business Development Executive
ProSol IT
Join the ProSol IT team as a Full Time Business Development Executive on Erekrut, specializing in Business Dev...
0-3 Lakhs
1-3 Years
Karnataka-..
Agile Develo...
React Native Developer
InQ Technologies
Join the InQ technologies team as a Full Time React native developer at Erekrut, specializing in IT Services a...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Agile Develo...
