Lead Data Engineer

About the Role

Key Responsibilities

• Provide technical leadership in data architecture and engineering, collaborating with solution stakeholders to align data solutions with business objectives.
• Mentor less experienced data engineers and team members, fostering a collaborative and innovative work environment.
• Explore and implement new technologies and practices in data engineering to continually improve data processes and infrastructure.
• Manage and enhance complex SSIS packages for data integration and ETL processes, along with SSRS reports for business intelligence.
• Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory, implementing both batch and real-time data integration processes.
• Develop and optimize Databricks notebooks for big data processing and analytics in Azure Databricks environments.
• Implement and manage data storage solutions using Delta Parquet files in Azure Data Lake Storage, optimizing for query performance and cost-efficiency.
• Integrate and configure Profisee Master Data Management tools within Azure, focusing on data governance and quality.
• Set up and manage Change Data Capture processes for real-time data synchronization in Azure data platforms.
• Implement data cataloging, classification, and lineage using Azure Purview, developing data governance policies, and ensuring compliance.
• Enhance and support Tableau reporting solutions, developing complex dashboards, and integrating with Azure data services.
• Drive automation and process optimization using Azure services and scripting, and implement CI/CD pipelines using Azure DevOps.
• Lead SQL Server database optimization and performance tuning efforts.
• Lead, coach, and mentor junior staff on projects.

Requirements

• Bachelor's degree required; Master's degree preferred in Computer Science, Information Technology, Engineering, or related discipline.
• 12+ years of experience in Data Engineering with a Bachelor's degree, or 8+ years with a Master's degree.
• Subject Matter Expert in SQL Server Management, Azure Data Factory Development, Databricks Engineering, Delta Parquet File Management, and SQL Server Integration Services.
• Proficiency in Profisee MDM Integration, Change Data Capture implementation, and data quality and governance with Azure Purview.
• Proven experience in Tableau and BI reporting, with strong skills in automation and process.
• Advanced Data Analytics Certifications (ADAC), SAS SBAPC, or Cloudera CCP Data Engineer certifications are a plus.

Nice to Have

• Experience with machine learning and predictive analysis.
• Knowledge of programming languages relevant to data engineering.
• Leadership skills and experience managing technical teams.

Qualifications

• Bachelor's degree in a related field; Master's preferred.
• 12+ years of experience in Data Engineering with a Bachelor's degree or 8+ years with a Master's degree.

Benefits & Perks

• Comprehensive and best-in-class benefits package including paid time off for holidays, vacation, sick and personal days.
• Paid parental leave, mental health services, and additional support.
• Target salary range of $140,000 to $175,000 annually, with eligibility for bonuses and benefits.
• Reasonable accommodations for individuals with disabilities.