EDI Data Engineer

Compunnel Software Group

New York, NY

Contract

Senior

7 days ago

Job Description

About the Role

We are seeking a skilled EDI Data Engineer to design and lead scalable data architectures and pipelines that support analytics and business intelligence initiatives. This role focuses on optimizing data workflows, automating processes, and ensuring reliable data operations in a cloud-based environment. The ideal candidate will bring deep technical expertise in big data processing, cloud services, and healthcare data.

Key Responsibilities

Design and implement scalable data architectures and pipelines.
Optimize data workflows and automate data operations.
Ensure data quality and reliability across systems.
Collaborate with cross-functional teams to support analytics and business intelligence needs.
Lead data engineering initiatives in a cloud-based environment.
Apply best practices in Agile development and data governance.

Requirements

8+ years of IT experience, including 5+ years in Python, PySpark, and SQL for big data processing.
Experience with data lakes (Iceberg format), ETL tools (e.g., Informatica), and data quality frameworks.
Proficiency with AWS services: S3, Glue, Redshift, Lambda, EMR, Airflow, and Postgres.
BASH/Shell scripting skills.
Experience working with healthcare data.
Proven experience leading data engineering teams.
Strong problem-solving and communication skills.
Experience with Agile development.