Databricks Expert-Developer

About the Role

Key Responsibilities

• Demonstrate hands-on experience with Databricks
• Work in big data frameworks like Apache Spark, Delta Lake, and structured streaming
• Develop ETL/ELT pipelines, data lakehouse architecture, and data modeling
• Program using Python and SQL, and possibly Scala and Java
• Integrate with external data sources, APIs, and on-prem databases
• Demonstrate data governance, security, and performance optimization in Databricks
• Mentor junior developers and foster a strong engineering culture

Requirements

• Bachelor's degree with 6-8 years of experience or commensurate experience
• Experience with machine learning workflows using MLflow or AutoML
• Exposure to BI tools (e.g., Oracle BI EE, Tableau) and data catalog tools (e.g., Purview, Alation)
• Migration of reports from OBIEE/Tableau into Databricks
• Knowledge of cost optimization and governance strategies in cloud environments
• Strong experience in Databricks (including SQL and PySpark notebooks, Jobs, Workflows, Delta Lake, MLflow)
• Proficiency in big data frameworks like Apache Spark, Delta Lake, and structured streaming
• Hands-on experience with cloud platforms—preferably Azure Databricks, but AWS or Google Cloud Platform is also valued
• Deep understanding of ETL/ELT pipelines, data lakehouse architecture, and data modeling
• Strong programming experience in Python, SQL, and optionally Scala or Java
• Solid understanding of DevOps/CI-CD practices for Databricks using tools like Databricks Repos, Terraform, Git, and Azure DevOps
• Experience integrating with external data sources, APIs, and on-prem databases
• Ability to design end-to-end data pipelines and lakehouse architecture using Databricks
• Strong knowledge of data governance, security, and performance optimization in Databricks
• Familiarity with Unity Catalog, RBAC, and cluster policies
• Ability to acquire and maintain federal Public Trust clearance
• Experience in mentoring junior developers and fostering a strong engineering culture

Nice to Have

• Experience with machine learning workflows using MLflow or AutoML
• Exposure to BI tools (e.g., Oracle BI EE, Tableau) and data catalog tools (e.g., Purview, Alation)
• Migration of reports from OBIEE/Tableau into Databricks
• Knowledge of cost optimization and governance strategies in cloud environments

Qualifications

• Bachelor's degree with 6-8 years of experience or equivalent
• Ability to acquire and maintain federal Public Trust clearance

Benefits & Perks

• Employee Assistance Program (EAP)
• Corporate Discounts
• Learning & Development platform, including certification preparation content
• Training, Education and Certification Assistance (available to full-time employees)
• Referral Bonus Program
• Internal Mobility Program
• Pet Insurance
• Flexible Work Environment