Senior Spark Data Engineer

About the Role

Key Responsibilities

• Serve as the subject matter expert in technologies used for our data strategy in the cloud.
• Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals.
• Solve complex data problems to deliver insights that help the organization's business achieve their goals.
• Create data products for analytics and data scientist team members to improve their productivity.
• Advise, consult, mentor, and coach other data and analytic professionals on data standards and practices.
• Foster a culture of sharing, re-use, design for scale, stability, and operational efficiency of data and analytical solutions.
• Lead the evaluation, implementation, and deployment of emerging tools and processes for analytic data engineering to improve organizational productivity.
• Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.

Requirements

• Bachelor's degree in a quantitative discipline such as Computer Science, MIS or related field with 10 years of related experience.
• Development and tuning experience on RDBMS technologies (Oracle, SQL Server, Teradata, SingleStore).
• Strong knowledge of using Spark for fast imagery and big data processing.
• Expert knowledge in managing and tuning Data Lake and Lakehouse storage objects with accompanying data engineering pipelines.
• Expert knowledge in Python and SQL development and troubleshooting.
• Willingness to undertake assignments involving data engineering and analytics tools.

Qualifications

• Bachelor's degree in a quantitative discipline such as Computer Science, MIS or related field.
• 10 years of related experience in data engineering or analytics.