Senior Data Engineer

About the Role

Key Responsibilities

• Maintain and enhance Spotify's reporting pipelines ensuring they remain business-critical.
• Build large-scale batch and real-time data pipelines for efficient collection, processing, and storage of measurement-related data.
• Work closely with stakeholders to understand business requirements and translate them into technical solutions.
• Write scalable, testable, maintainable, and high-quality code.
• Collaborate with engineers, product managers, and other stakeholders, embracing learning and leadership opportunities daily.
• Monitor daily execution, diagnose issues, and ensure SLAs are met by maintaining and fixing business-critical pipelines.
• Write distributed, high-volume services and know how to deploy and keep them running in production.

Requirements

• Proven experience working with high-volume, heterogeneous data, ideally within distributed systems such as Hadoop, BigTable, Cassandra, GCP, AWS, or Azure.
• Familiarity with data processing frameworks like Scio, GCP Dataflow, AWS EMR, and Apache Beam.
• Strong programming skills in Java or Scala.
• Commitment to agile software processes, data-driven development, reliability, and responsible experimentation.
• Hands-on experience with orchestration tools like Airflow or similar.
• Excellent problem-solving and communication skills.

Nice to Have

• Experience with cloud platforms such as GCP, AWS, or Azure.
• Knowledge of API and backend system design.
• Experience working in a fast-paced, collaborative environment.

Qualifications

• Experience with distributed backend systems and data processing frameworks.
• Proficiency in Java or Scala.
• Experience with orchestration tools like Airflow.

Benefits & Perks

• Health insurance
• Six month paid parental leave
• 401(k) retirement plan
• Monthly meal allowance
• 23 paid days off
• 13 paid flexible holidays
• Paid sick leave