Mid Level Data Engineer(Pyspark)

ManpowerGroup

San Francisco, CA

Full Time

Mid Level

110k-110k

6 days ago

Job Description

About the Role

Our client, a leading technology firm, is seeking a Data Engineer to join their team. As a Data Engineer, you will be part of the data engineering department supporting innovative data initiatives. The ideal candidate will have strong problem-solving skills, effective communication abilities, and a collaborative mindset which will align successfully in the organization.

Key Responsibilities

Design, develop, and maintain robust ETL pipelines using PySpark and other data processing tools.
Work with large-scale datasets from various sources, both structured and unstructured.
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver actionable insights.
Optimize data workflows for performance, scalability, and reliability.
Ensure data quality, integrity, and consistency across systems.
Monitor and troubleshoot data pipelines, resolving issues in a timely manner.
Document data processes, workflows, and architecture to support knowledge sharing and compliance.

Requirements

Bachelor's degree in Computer Science, Engineering, or a related field.
3-5 years of experience in data engineering or a similar role.
Strong proficiency in PySpark and Apache Spark for distributed data processing.
Solid experience with ETL tools and frameworks such as Apache Airflow, AWS Glue, or Informatica.
Proficiency in SQL and experience working with relational databases like PostgreSQL or MySQL.
Experience with cloud platforms (AWS, Azure, or GCP), especially cloud-based data services.
Familiarity with data warehousing concepts and tools such as Snowflake, Redshift, or BigQuery.

Nice to Have

Experience working in a fast-paced environment with multiple projects.
Knowledge of data security and compliance standards.

Qualifications

Educational background in Computer Science, Engineering, or related fields.

Benefits & Perks

Dental Plan
Vision Plan
Health Savings Account
Health Flexible Spending Account
Dependent Care Flexible Spending Account
Supplemental Life Insurance
Short Term and Long Term Disability Insurance
Business Travel Insurance
401(k)
Weekly Pay

Working at ManpowerGroup

The company fosters a collaborative and inclusive team culture that values diverse perspectives. It offers professional development opportunities, a hybrid work model with onsite collaboration, and is committed to diversity, inclusion, and ethical employment practices.

Apply Now