Data Engineer

Tata Consultancy Services

Irving, TX

Full Time

Senior

120k-135k

about 2 months ago

Job Description

About the Role

The role involves designing, developing, and maintaining scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services. The position requires expertise in various data engineering tools and technologies to support data processing workflows and ensure efficient data management within the organization.

Key Responsibilities

Design, develop, and maintain scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services.
Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks.
Good analytical skills to debug pipeline failures and resolve issues within SLA.
Design and develop ETL jobs using DataStage tool.
Work with Snowflake database.
Create shell scripts using Unix shell scripting.

Requirements

8+ years of experience as a Data Engineer.
Must have skills in DataStage development, Azure Data Factory, Databricks, Snowflake, and UNIX.
Experience in designing and developing ETL jobs.
Knowledge of Spark jobs, data transformations, and data processing workflows.
Good knowledge of DataStage tool.
Proficiency in Unix shell scripting.
Good analytical skills for debugging pipeline failures.

Benefits & Perks

Discretionary Annual Incentive.
Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support: Maternal & Parental Leaves.
Insurance Options: Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
Time Off: Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.