← Back to opportunities
Job Description:
Data engineer
Remote
1. Design, develop, and optimize Spark applications using Scala and Python.
2. Build scalable and high performance ETL/ELT pipelines for batch and streaming data.
3. Work with large datasets in distributed environments.
4. Develop and deploy data pipelines using AWS services such as:
5. S3, EMR, Glue, Lambda, Athena, Redshift, Kinesis
6. Implement CI/CD pipelines for data workflows using AWS-native tools or DevOps frameworks.
7. Manage infrastructure-as-code using Terraform or CloudFormation (preferred).
8. Work with structured, semi structured, and unstructured data.
9. Implement data quality checks, validation frameworks, and metadata management.
10. Optimize Spark jobs for performance, memory usage, and cost efficiency.
11. Partner with da...
About the Role
Job Description:
Data engineer
Remote
1. Design, develop, and optimize Spark applications using Scala and Python.
2. Build scalable and high performance ETL/ELT pipelines for batch and streaming data.
3. Work with large datasets in distributed environments.
4. Develop and deploy data pipelines using AWS services such as:
5. S3, EMR, Glue, Lambda, Athena, Redshift, Kinesis
6. Implement CI/CD pipelines for data workflows using AWS-native tools or DevOps frameworks.
7. Manage infrastructure-as-code using Terraform or CloudFormation (preferred).
8. Work with structured, semi structured, and unstructured data.
9. Implement data quality checks, validation frameworks, and metadata management.
10. Optimize Spark jobs for performance, memory usage, and cost efficiency.
11. Partner with da...
Ready to Join Through a Referral?
Apply now and get connected directly with the hiring team
Apply for this Position