Senior Data Engineer

About the Role

Key Responsibilities

• Provide day-to-day oversight and technical direction for data engineering initiatives, offering guidance and support
• Lead the design and development of data pipelines using Apache Spark and Databricks, incorporating best practices in performance, scalability and maintainability
• Build and optimize data solutions in Azure, leveraging cloud-native PaaS tools and open-source technologies, such as Delta Lake
• Partner closely with software engineers, the architecture team and product owners to deliver data solutions aligned with business goals
• Mentor engineers on data modeling, performance tuning and platform design while fostering a collaborative and growth-oriented team culture
• Stay current on emerging trends in cloud data platforms, lakehouse architecture and open-source tools and offer recommendations to improve our stack and practice
• Apply strong analytical and problem-solving skills to troubleshoot data pipeline issues across varied systems
• Participate in and promote learning opportunities through the annual training and development plan, team knowledge sharing and mentorship

Requirements

• Experience building data assets, products and pipelines with Apache Spark and Databricks
• Familiarity with Azure cloud services and PaaS offerings (or equivalent experience with other cloud providers)
• Recent data engineering experience with Scala or Java (or willingness to learn) is a plus
• Programming skills in SQL and experience with RDBMS systems
• Familiarity with NoSQL databases
• Experience with open-source data lake storage technologies such as Delta Lake
• Knowledge of data lakehouse architectures
• Familiarity with one statically typed enterprise programming language (i.e., C#, Java) is a plus
• College degree in computer science, software engineering or a related field, or equivalent industrial or technical experience
• Fundamental knowledge of data structures and algorithms
• Ten or more years of data and/or software engineering experience
• Experience working in Agile/SCRUM environments and with modern DevOps practices
• Ability to adapt to rapidly changing project scopes and priorities
• Strong written and verbal communication skills
• A collaborative, hands-on approach to leadership and technical problem-solving

Nice to Have

• Experience with Scala or Java (or willingness to learn)
• Knowledge of data lakehouse architectures
• Experience with open-source data lake storage technologies such as Delta Lake
• Familiarity with one statically typed enterprise programming language (i.e., C#, Java)

Qualifications

• College degree in computer science, software engineering or a related field, or equivalent industrial or technical experience
• Fundamental knowledge of data structures and algorithms
• Ten or more years of data and/or software engineering experience
• Experience working in Agile/SCRUM environments and with modern DevOps practices

Benefits & Perks

• Performance-based triannual bonuses
• Medical benefits paid at 100% for full-time employees and 80% for eligible part-time employees
• Profit-sharing program
• Free lunch every day while onsite
• More than 450 annual personal and professional development courses