Data Engineer

Ryan Specialty Group

Chicago, IL

Full Time

Mid Level

78k-90k

17 days ago

Job Description

About the Role

The Data Engineer is responsible for designing, building, and maintaining data pipelines, models, ETL processes, and other data infrastructure to support the organization’s data needs. The role involves collaborating with data scientists and analysts to develop scalable solutions, integrating data from various sources, ensuring data quality and consistency, and implementing data governance practices.

Key Responsibilities

Design, develop, and optimize scalable ETL pipelines using Databricks (Spark / PySpark).
Integrate data from various structured and unstructured sources into our data lake and warehouse environments.
Implement data quality checks, monitoring, and validation processes.
Collaborate with Data Scientists, Analysts, and other Engineers to understand data needs and deliver solutions.
Tune data jobs for performance, scalability, and reliability.
Develop and maintain documentation around data systems, processes, and best practices.
Assist in migrating legacy ETL jobs to modern cloud-native solutions (e.g., Databricks, Delta Lake).

Requirements

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field, or equivalent experience.
2+ years of experience in data engineering or related field.
Strong hands-on experience with Databricks, Spark (PySpark or Scala), and distributed data processing.
Proficiency in SQL, data modeling, and performance tuning.
Experience working with cloud platforms (e.g., Azure, AWS, GCP) and cloud-native data services.
Solid understanding of ETL best practices and data pipeline architecture.
Strong problem-solving skills and ability to work independently and collaboratively.

Nice to Have

Experience with Delta Lake and structured streaming.
Knowledge of CI/CD processes and DevOps practices for data engineering.
Familiarity with other ETL tools like Informatica, Apache Airflow, or dbt.
Experience with Snowflake, Redshift, or other cloud data warehouses.
Python programming skills beyond Spark (e.g., scripting, automation).

Qualifications

Educational background as specified in requirements (e.g., Bachelor's degree or equivalent).

Benefits & Perks

Competitive salary and benefits package.
Flexible work environment (remote/hybrid options available).
Opportunity to work with modern technologies and help shape our data infrastructure.
Supportive team culture focused on innovation, learning, and collaboration.
Professional development opportunities and technical training.
Comprehensive and best-in-class benefits package including paid time off, parental leave, mental health services, and more.

Working at Ryan Specialty Group

Ryan Specialty is committed to building and sustaining a diverse workforce in an inclusive and equitable workplace where all employees are valued for their performance and contributions. The company fosters a culture focused on innovation, learning, collaboration, and supporting its teammates through comprehensive benefits and professional development opportunities.

Apply Now