Big Data Engineer

Company: VALUECONNECT CONSULTING CORPORATION
Location: Boston
Posted on: May 1, 2025

Job Description:

(Experience in designing & developing a big data solution using Spark, Scala, AWS Glue, Lambda is a must)

Maximise your chances of a successful application to this job by ensuring your CV and skills are a good match.

Design and modernize
scalable, modular data pipelines from legacy SQL-based workflows.

Build efficient and
reliable ETL processes using Apache Spark and cloud-native tools on AWS.

Create configurable
pipelines that accelerate experimentation and feature selection for data
science teams.

Optimize pipeline
performance, reduce storage costs, and improve compute efficiency.

Ensure compliance with
governance standards while maintaining data integrity and lineage.

Integrate seamlessly with
enterprise data platforms, lakes, and cloud-based storage systems.

Implement and maintain
CI/CD pipelines using Jenkins or similar tools.

Write robust automated
tests, including unit, integration, and end-to-end coverage.

Collaborate within Agile
teams, participating in sprint planning, reviews, and retrospectives.

Keywords: VALUECONNECT CONSULTING CORPORATION, Cambridge , Big Data Engineer, Engineering , Boston, Massachusetts