Senior Software Engineer

About the Role

Key Responsibilities

• Design, implement, and manage scalable data platforms, including data lakes and data warehouses, using cloud infrastructure and big data tools.
• Collaborate with the EDW team, Data Architects, DBAs, application team developers and business stakeholders to deliver large-scale Big Data solutions.
• Consistently create optimal design adhering to architectural best practices; consider scalability, reliability and performance of systems/contexts affected when defining technical designs.
• Apply Dev/Ops mindset, take ownership of production success, optimize operation success via automation/active alerting/self-healing, and lead the resolution of production issues to ensure high-availability and performance.
• Develop high-quality code, define best engineering practice, perform peer code reviews to ensure successful deliverables with engineering excellence.
• Document engineering artifacts such as technical design documents, flowcharts, system context diagrams, code comments etc.
• Drive the release planning and execution with end-to-end understanding and insights (effort, risk, priority) of the planned features.
• Work with Azure (or other cloud platforms) to ensure optimal performance, security, and cost-efficiency of cloud-based systems.
• Apply machine learning and new AI technologies to software products, develop and shape our technical strategy to deliver data solutions for customer360 customers.
• Utilize orchestration tools such as Airflow to automate and manage data workflows.
• Implement and manage CI/CD pipelines using GitHub Actions, ensuring smooth deployment processes.
• Collaborate cross-functionally with engineers, data scientists, and business teams to refine data models and improve data systems.

Requirements

• Bachelor's degree, or equivalent work experience.
• Five to six years of relevant experience.
• 7+ years of professional experience in data engineering or related roles.
• Hands-on experience in building large-scale data integration solutions with Spark, Scala and other programming languages.
• Proficient in Python for delivering data processing solutions.
• Proficient with SQL and the ability to optimize complex queries.
• Proficient in data modeling and database design.
• Experience with Azure Synapse, Azure Open AI services & Azure Synapse.
• Experience with orchestration tool Airflow.
• Knowledge of GitHub Actions and general CI/CD practices.
• Experience in system design with a focus on building data lakes or data warehouses from scratch.
• Strong ability to construct data models and work within relational and non-relational databases.

Nice to Have

• Experience with cloud infrastructure and big data tools.
• Knowledge of machine learning and AI technologies.
• Experience with cloud platforms other than Azure.
• Experience with automation and self-healing systems.

Qualifications

• Bachelor's degree or equivalent work experience.
• Five to six years of relevant experience.

Benefits & Perks

• Healthcare (medical, dental, vision).
• Basic term and optional term life insurance.
• Short-term and long-term disability.
• Pregnancy disability and parental leave.
• 401(k) and employer-funded retirement plan.
• Paid vacation (from two to five weeks depending on salary grade and tenure).
• Up to 11 paid holiday opportunities.
• Adoption assistance.
• Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year.