Staff Data Engineer - Data Engineering

About the Role

Key Responsibilities

• Act as the team architect by leading system design reviews, offering recommendations, conducting comprehensive peer reviews, and demonstrating expert-level proficiency in Python and AWS services
• Architecting and implementing scalable data ingestion pipelines handling different file types into Arine platform
• Develop reusable components that can be integrated into data pipelines to enhance efficiency and minimize future implementation time
• Creating configuration-driven, containerized toolsets that can be easily used and maintained by diverse engineering profiles
• Work collaboratively with cross-functional teams to ensure their data requirements are met through ETL components
• Implementing incremental data ingestion strategies for large-scale healthcare datasets
• Building monitoring and alerting systems for data ingestion processes and pipeline health
• Applying software engineering best practices including test-driven development and modular design to data infrastructure
• Refactoring and rebuilding existing data ingestion processes to improve scalability and operational efficiency
• Working with containerization technologies (Docker, Kubernetes) to create portable and maintainable data solutions
• Identify and escalate inefficiencies within and across teams
• Provide technical guidance, mentorship to junior engineers, and promote best practices and coding standards
• Author and support high-quality technical documentation, assisting junior engineers in doing the same

Requirements

• 10+ years of professional experience in data engineering with focus on large-scale data ingestion and infrastructure
• Deep expertise in Python programming and modern data engineering tools
• Experience creating an automated production grade ETL process using Python and SQL
• Strong understanding of ETL/ELT frameworks and distributed data processing
• Experience with data processing, validation, cleaning and debugging data sets
• Experience with API integration for seamless data exchange between systems
• Proven experience handling and processing various file types and formats, including specialized healthcare standards such as HL7, 834, 837, and NCPDP
• Experience integrating and consolidating data from diverse source systems into a unified repository, including data from EHR and claim systems, as well as from file-based and API integrations
• Experience with processing large data sets (over 10GB)
• Experience with incremental data processing and change data capture (CDC) methodologies
• Strong experience designing scalable data architectures in AWS environment
• Deep understanding of software engineering principles including test-driven development, loose coupling, single responsibility, and modular design
• Experience with containerization technologies (Docker, Kubernetes) and building configuration-driven, maintainable systems
• Proven ability to build tools and systems that can be operated by diverse engineering profiles through configuration rather than code changes
• Passion for building new and improving existing data infrastructure with robust, maintainable, and operationally excellent data systems
• Familiarity with healthcare data and regulatory environments (HIPAA compliance) is a plus
• Strong collaboration and communication skills; comfortable working with diverse technical and non-technical stakeholders
• Excellent verbal and written communication skills with ability to explain technical infrastructure concepts to diverse audiences

Nice to Have

• Familiarity with healthcare data and regulatory environments (HIPAA compliance)

Qualifications

• 10+ years of professional experience in data engineering

Benefits & Perks

• Opportunity to contribute to the company's growth and shape its future
• Unparalleled learning and growth prospects
• Collaborating closely with experienced Clinicians, Engineers, Software Architects, Data Scientists, and Digital Health Entrepreneurs
• Base salary range: $165,000-180,000/year