Data Engineer, Financial Data Engineering

About the Role

Key Responsibilities

• Design, develop, test, troubleshoot, and optimize solutions to simplify access to American Express's Big Data Platform.
• Partner with multiple tech and product teams to craft enterprise-scale technical product requirements and deliver public cloud services.
• Analyze, debug, and fix issues in Apache Beam and Flink data pipelines, ensuring quality and performance.
• Implement robust data engineering pipelines for ETL and ELT workflows, ensuring data quality, lineage, and governance.
• Optimize data engineering processes for scalability, reliability, and performance.
• Work within an Agile team environment, contributing to continuous delivery and deployment of data solutions.
• Create engineering best practices and tools that become integral to platform operations.
• Integrate new technologies into existing stacks and drive operational efficiencies.

Requirements

• Up to 4+ years of data engineering software development experience in a professional environment or comparable experience.
• Bachelor's degree in computer science, computer engineering, or equivalent, with hands-on expertise in application design, software development, and automated testing.
• Experience in engineering and deploying enterprise-grade technology products in an Agile environment.
• Large scale software product engineering experience with tools and delivery methods such as DevOps, CD/CI, Agile.
• Hands-on experience with in-memory frameworks and core technologies including Java/J2EE, NodeJS, Python, Big Data Technologies, Apache Flink, Apache Beam, SingleStore, No-SQL Databases, Relational Databases, JSON, XML, REST, Kafka.
• Experience with jobs orchestration tools like Airflow and Automic.
• Expertise in designing and implementing data engineering pipelines for ETL and ELT workflows.
• Proficiency in data engineering best practices for data ingestion, transformation, orchestration, and data quality.
• Strong understanding of data partitioning, sharding, schema evolution, and optimization for scalability and performance.
• Hands-on experience with cloud-based data platforms such as GCP Dataflow, AWS Glue, Azure Data Factory.
• Proficiency in SQL and PL/SQL.
• Experience developing high-performance APIs and microservices using multiple architectural and design patterns.
• Ability to communicate effectively with internal and external business partners.

Nice to Have

• Knowledgeable and experienced with in-memory computation and data virtualization tools & techniques.
• Experience in data engineering of financial data.
• Knowledge of GCP, Kubernetes, and Docker.
• Experience with memory-first databases like SingleStore.
• Experience with DevOps tools, automating engineering and operational tasks.

Qualifications

• Bachelor's degree in computer science, computer engineering, or equivalent

Benefits & Perks

• Competitive base salaries
• Bonus incentives
• 6% Company Match on retirement savings plan
• Free financial coaching and financial well-being support
• Comprehensive medical, dental, vision, life insurance, and disability benefits
• Flexible working model with hybrid, onsite, or virtual arrangements
• 20+ weeks paid parental leave for all parents
• Free access to global on-site wellness centers (depending on location)
• Free and confidential counseling support through Healthy Minds
• Career development and training opportunities