Lead Kafka Middleware Engineer

Request Technology

Chicago, IL

Full Time

Senior

10 days ago

Job Description

About the Role

The Lead Kafka Middleware Engineer role at Request Technology involves designing, implementing, and maintaining scalable and high-availability Middleware solutions with a focus on Kafka, cloud automation, container orchestration, and CI/CD pipelines. The position plays a crucial role in automating infrastructure provisioning, deployments, and operations to enable rapid application scaling and delivery within a prestigious financial institution.

Key Responsibilities

Design, implement and manage highly available and scalable Kafka clusters.
Monitor Kafka performance, troubleshoot issues and optimize configurations.
Develop and maintain Infrastructure as Code (IaC) using Ansible and Terraform for infrastructure provisioning and configuration management.
Create and maintain reusable IaC modules.
Design and implement cloud-based infrastructure solutions on AWS and Azure.
Automate cloud resource provisioning, scaling and management using cloud-native tools and services.
Deploy and manage containerized applications using Kubernetes and Rancher.
Troubleshoot container-related issues and optimize container performance.
Design, implement and maintain CI/CD pipelines using tools like GitHub, Artifactory, Harness and Jenkins.
Automate the build, test and deployment of Middleware components.
Integrate IaC and container technologies into CI/CD pipelines.
Document all processes and procedures.
Work with development teams to ensure smooth deployments.

Requirements

Strong proficiency in IaC tools, specifically Ansible, Terraform and bash Scripting.
Extensive experience with cloud automation and provisioning on AWS and Azure.
Proficiency in CI/CD tools, including GitHub, Artifactory, Harness and Jenkins.
Strong Scripting skills in languages like Python and Bash.
Excellent troubleshooting and problem-solving skills.
Understanding of networking principles.
Experience with monitoring tools like Splunk, Splunk OTEL, Prometheus and Grafana.
In-depth expertise in Kafka administration, including cluster management, performance tuning, and security.
Bachelor's degree in Computer Science, Engineering or a related field (or equivalent experience).
10+ years of experience in infrastructure Middleware administration.

Nice to Have

AWS Solutions Architect certification.
CKAD or CKA certifications.

Qualifications

Bachelor's degree in Computer Science, Engineering or a related field (or equivalent experience).
10+ years of experience in infrastructure Middleware administration.