Senior Data Engineer

About the Role

Key Responsibilities

• Lead the process of identifying, prioritizing, and resolving production issues, ensuring minimal disruption to business operations.
• Investigate and identify the root causes of incidents to prevent recurrence.
• Maintain clear and timely communication with stakeholders, including business users, IT teams, and management, throughout the incident lifecycle.
• Continuously monitor the data warehouse's performance, identifying bottlenecks and areas for improvement.
• Implement performance tuning techniques to optimize query execution times, ETL processes, and overall system performance.
• Collaborate with other teams to ensure the data warehouse has sufficient resources to meet current and future demands.
• Implement and maintain data validation processes to ensure the accuracy and integrity of data in the data warehouse.
• Address data quality issues, including data cleansing and transformation, to ensure data accuracy and consistency.
• Participate in the planning and execution of data warehouse releases, ensuring minimal disruption to production operations.
• Test new releases and changes to the data warehouse environment to ensure stability and functionality.
• Build and maintain strong relationships with business users, IT teams, and other stakeholders.
• Maintain comprehensive documentation of the data warehouse architecture, processes, and procedures.
• Possess a strong understanding of data warehouse technologies, ETL processes, and database management systems.
• Demonstrate strong analytical and problem-solving skills to effectively address technical issues and challenges.
• Lead and mentor team members, providing guidance and support as needed.

Requirements

• Bachelor's degree required in Computer Science, Computer Information Systems, Information Systems, Information Technology or Computer Engineering or equivalent work experience.
• 8+ years of strong understanding of ETL development concepts and tools such as ETL development solutions (Informatica/IICS).
• 5+ years of experience with Data Warehousing and Cloud Technologies like Azure (Synapse, Data bricks, Snowflake).
• 3+ years of experience in scheduling jobs using Autosys (or comparable distributed scheduler).
• 3+ years of experience writing Unix/Linux or Windows Scripts in tools such as PERL, Shell script, Python, etc.
• 3+ years of experience in creating complex technical specifications from business requirements/specifications.
• Demonstrated critical thinking skills and the ability to identify and resolve data quality issues, performance bottlenecks, and other ETL-related problems.
• Excellent communication and interpersonal skills.
• Experience leading a team and providing technical guidance to junior data engineers.

Qualifications

• Bachelor's degree in relevant technical fields or equivalent work experience.