Lead Data Engineer with experience in Pyspark, Python

Infosys

Charlotte, NC

Full Time

Senior

9 days ago

Job Description

About the Role

Infosys is seeking a Lead Data Engineer with experience in Pyspark and Python. This position's primary responsibility will be to provide technical expertise and coordinate for day-to-day deliverables for the team. The role involves assisting in the technical design of large business systems, building applications, understanding data security, retention, and recovery, and contributing to technology-specific best practices and standards. The candidate should be able to research technologies independently and guide the team during the build and test phases. The company is a global leader in digital services and consulting, enabling clients in over 50 countries to navigate their digital transformation with a focus on AI, agility, and continuous learning.

Key Responsibilities

Provide technical expertise and coordinate day-to-day deliverables for the team.
Assist in the technical design of large business systems.
Build applications and interfaces between applications.
Understand data security, retention, and recovery processes.
Research technologies independently to recommend appropriate solutions.
Contribute to technology-specific best practices and standards.
Contribute to success criteria from design through deployment, including reliability, cost-effectiveness, performance, data integrity, maintainability, and scalability.
Guide and mentor the team during the build and test phases.

Requirements

Bachelor's degree or foreign equivalent from an accredited institution, or three years of progressive experience in the specialty in lieu of every year of education.
At least 4 years of Information Technology experience.
Experience in creating data engineering pipelines using Python with Object-oriented concepts, Pyspark, and AWS cloud Native solutions.
Design, develop, and implement new features to existing frameworks using PySpark and Python.
Write efficient and effective standalone scripts in PySpark with transformations as per business logic.
Experience in CI/CD pipeline using Jenkins, CloudFormation, etc.
Proficient in Python, PySpark, and Object-oriented programming concepts.
Strong problem-solving skills and ability to troubleshoot complex software issues.
Excellent communication skills, both written and verbal.
Ability to work independently with minimal supervision and in a diverse, multi-stakeholder environment.
Ability to learn and adapt to emerging technologies.

Nice to Have

Experience with AWS is preferred but not mandatory.
Experience working in a global delivery environment.
Experience in testing and debugging code to ensure desired results.
Ability to understand requirements and develop solutions collaboratively.
Experience documenting programming tasks and procedures.

Qualifications

Bachelor's degree or foreign equivalent from an accredited institution.

Working at Infosys

Infosys is committed to equal employment opportunities and fostering a diverse, inclusive work environment. The company emphasizes continuous learning, digital innovation, and empowering employees through digital skills development and an innovation ecosystem.

Apply Now