AWS Data Engineer
Experience
Salary
Location
Job Description
JOB DESCRIPTION:
- AWS Data Engineer with minimum of 5 to 7 years of experience.
- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.
- Connect with stakeholders daily to discuss project progress and updates.
- Work within an Agile process to deliver projects in a timely and efficient manner.
- Design and develop Airflow DAGs to schedule and manage ETL workflows.
- Transform SQL queries into Spark SQL code for ETL pipelines.
- Develop custom Python functions to handle data quality and validation.
- Write PySpark scripts to process data and perform transformations.
- Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes.
- Run Spark jobs on AWS EMR cluster using Airflow DAGs.
- Monitor and troubleshoot ETL pipelines to ensure smooth operation.
- Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture.
- Collaborate with other members of the data engineering team to improve processes and implement new technologies.
- Stay up to date with emerging trends and technologies in data engineering and suggest ways to improve the team's efficiency and effectiveness.
Designation
AWS Data Engineer
Key Skills
Industry
IT-ITeS
Functional Area
IT Services
Employment Type
Full Time
Education
Bachelor Degree
Job Type
Work From Office
Related Jobs
Area Sales Manager
ARJ Greenearth Building Solutions Pvt. Ltd.
Join the ARJ Greenearth Building Solutions Pvt. Ltd. team as a Full Time Area Sales Manager on Erekrut, specia...
3-6 Lakhs
1-3 Years
Tamil Nadu..
Agile Develo...
Staff Software Engineer-Python
Peak Hire Solutions
Join the Software Technology team as a Full Time Staff Software Engineer-Python at Erekrut, specializing in S...
50-75 Lakhs
11-15 Years
Karnataka-..
Agile Develo...
.JPG)
IT RECRUITERS
Technostrides
Join the Technostrides team as a Full Time IT RECRUITERS at Erekrut, specializing in HR Services industry. We ...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Agile Develo...
Business Development Executive
Intenim Technologies Pvt Ltd
Join the Intenim Technologies Pvt Ltd team as a Full Time Business Development Executive on Erekrut, specializ...
3-6 Lakhs
1-3 Years
Karnataka-..
Agile Develo...
Sales Executive
Intenim Technologies Pvt Ltd
Join the Intenim Technologies Pvt Ltd team as a Full Time Sales Executive on Erekrut, specializing in Sales Ex...
3-6 Lakhs
1-3 Years
Haryana-Gu..
Agile Develo...
Business Development Executive
Skysoft Works
Join the SkySoft Works team as a Full Time Business Development Executiv at Erekrut, specializing in Digital ...
Confidential
0-1 Years
Karnataka-..
Agile Develo...
Jr Architect
Talent Corner Hr Services
Join the talent corner hr services team as a Full Time jr Architect at Erekrut, specializing in Construction i...
3-6 Lakhs
1-3 Years
Tamil Nadu..
Agile Develo...
Sr. Microsoft Dynamics 365 CRM...
Viamagus Technologies Pvt Ltd
Join the Viamagus Technologies Pvt Ltd team as a Full Time Sr. Microsoft Dynamics 365 CRM Developer on Erekrut...
25-50 Lakhs
7-10 Years
Karnataka-..
Agile Develo...