DECAL Analytics Engineer

About the Role

Key Responsibilities

• Build and maintain ETL/ELT pipelines using Azure Databricks and Microsoft Fabric.
• Implement medallion architecture (Bronze, Silver, Gold layers) to support data lifecycle and quality.
• Develop real-time and batch ingestion processes from IES Gateway and other source systems.
• Ensure data quality, validation, and transformation logic is consistently applied.
• Use Python, Spark, and SQL in Databricks and Fabric notebooks for data transformation.
• Implement Delta Lake for data versioning, ACID transactions, and schema enforcement.
• Integrate Databricks with other Azure services like Azure One Lake, Azure ADLS Gen2, and Microsoft Fabric.
• Collaborate with Domain Owners to design dimensional and real-time data models.
• Create analytics-ready datasets for Power BI and other reporting tools.
• Standardize field naming conventions and schema definitions across datasets.
• Apply data classification and tagging based on DECAL's data governance framework.
• Implement row-level security, data masking, and audit logging as per compliance requirements.
• Support integration with Microsoft Purview for lineage and metadata management.
• Partner with BI developers to ensure data models are optimized for Power BI.
• Provide curated datasets that align with reporting requirements and business logic.
• Create BI dashboards and train users.
• Support CI/CD pipelines for data workflows using Azure DevOps.
• Assist in monitoring, logging, and performance tuning of data jobs and clusters.

Requirements

• Bachelor's degree in computer science, data engineering, or a related field.
• 3+ years of experience in data engineering or analytics engineering roles.
• Proficiency in advanced SQL techniques for data transformation, querying, and optimization.
• Hands-on experience with Azure Databricks (Spark, Delta Lake), Microsoft Fabric (Dataflows, Pipelines, OneLake), SQL, and Python (Pandas, PySpark).
• Familiarity with data modeling, data governance, and data security best practices.
• Strong understanding of ETL/ELT processes, data quality, and schema design.

Nice to Have

• Experience with data lineage and metadata management tools.
• Knowledge of data security and compliance standards.
• Experience with Power BI and dashboard creation.

Qualifications

• Educational background in computer science, data engineering, or related fields.
• Minimum 3 years of relevant experience in data or analytics engineering roles.

Benefits & Perks

• Choose from two comprehensive medical plans through Blue Cross Blue Shield, complete with dental and vision coverage.
• Life, critical illness, and accident insurance through Unum.
• Flexible Spending Account and Daycare FSA through Navia.
• Health Gap Insurance through Sunlife.
• 401(k) retirement plan through The Standard, eligible after six months.