AI Capacity Engineer

About the Role

Key Responsibilities

• Collaborate with Core AI teams across multiple geographies to analyse, plan and execute on AI initiatives.
• Analyse the AI capacity intake requirements for prioritization and scheduling.
• Seek out and lead execution of performance optimizations of our AI related assets to ensure efficient use.
• Scope includes Nvidia SuperPod, On-Prem Kubernetes, Azure, and GCP based clouds.
• Understand key performance metrics and scaling characteristics of LLM, non-LLM AI models.
• Understand key concepts and sizing metrics related to RAG, Vector Search, Grounding.
• Influence customer choice of AI models to improve ROI and cost efficiency.
• Design and build dashboards to support management of AI workloads and infrastructure.
• Analyze historical data, trends, and growth patterns to develop accurate capacity models such as compute, network, storage, and platform optimization requirements.
• Collaborate with multi-functional teams to gather relevant information on business objectives, technology requirements, and upcoming projects.
• Evaluate and refine existing capacity engineering processes and methodologies to improve accuracy and efficiency.
• Monitor system performance metrics and utilization levels to identify potential bottlenecks or areas of underutilization.
• Take ownership and drive for the realization of gains from improving utilization.
• Collaborate with technology partners to understand future technology trends and initiatives, anticipate resource demands, and develop proactive capacity plans.
• Conduct 'what-if' scenarios to assess the impact of different business scenarios and help guide decision-making processes.
• Manage capacity for large federated Kubernetes environments on primarily private cloud but including some public cloud.
• Communicate capacity insights, recommendations, and performance metrics to key partners.
• Advocate for initiatives that provide clear business value through data.

Requirements

• Bachelor's degree in computer science, information systems, statistics or a related field and 2+ years of experience.
• 2 years of experience in capacity engineering, resource allocation, and forecasting in a technology-intensive environment.
• Specialized experience in AI.
• Strong proficiency with grafana, prometheus, thanos, elastic search and kibana (ELK).
• Proficiency in Kubernetes and understanding of how applications operate in a k8s environment.
• Strong analytical skills with the ability to analyze complex data sets and identify relevant patterns and trends.
• Familiarity with technology infrastructure components, including Ubuntu Linux, servers, databases, networks, storage systems, and cloud platforms.
• Knowledge of recommendation engines concepts and their application in infrastructure management.
• Ability to communicate capacity insights, recommendations, and performance metrics effectively to partners.

Nice to Have

• Experience with Nvidia SuperPod and GPU relevant metrics.
• Experience with workload optimization and performance tuning in AI infrastructure.
• Experience with cloud platforms such as Azure and GCP.

Qualifications

• Bachelor's degree in a relevant field.
• At least 2 years of experience in capacity engineering or related roles.
• Experience with AI and cloud infrastructure.

Benefits & Perks

• Full range of medical, financial, and other benefits including 401(k) eligibility.
• Paid time off benefits such as PTO and parental leave.
• Potential for target bonus and restricted stock units.
• Competitive base pay range of $95,200 - $168,700, depending on location, skills, and experience.