Big Data Engineer (Scala/Java/Python)

Luxoft

Weehawken, NJ

Full Time

Mid Level

15 days ago

Job Description

About the Role

ACQA is built on Microsoft Azure cloud computing technology. It aims to deliver scalable cost-efficient infrastructure using cloud PaaS components, a single core platform with open architecture designed for change, itemised cost metrics, and automated data lineage. The platform is shared across Front Office, Finance, and Risk, improving regulatory compliance. The ACQA platform consists of components providing next-generation valuation and risk management services.

Key Responsibilities

Development of big data technologies like Apache Spark and Azure Databricks
Programming complex production systems in Scala or Java or Python
Experience in a platform engineering role on a major cloud provider (ideally Microsoft Azure)
Development and/or operations of CI/CD pipelines using modern cloud-friendly build systems like GitLab
Experience with containerisation in development, build and runtime environments including experience of Kubernetes

Requirements

Development of big data technologies like Apache Spark and Azure Databricks
Programming complex production systems in Scala or Java or Python
Experience in a platform engineering role on a major cloud provider (ideally Microsoft Azure)
Development and/or operations of CI/CD pipelines using modern cloud-friendly build systems like GitLab
Experience with containerisation in development, build and runtime environments including experience of Kubernetes

Nice to Have

Production experience with Terraform on a major public cloud provider (ideally Microsoft Azure)
Python experience
Testing and instrumentation practices, observability and alerting tooling
Understanding of information modelling, data structures and algorithms
Technical architecture and low-level design
Agile development and planning practices
Ability to collaborate effectively in a large global team and influence key architectural decisions