Software Engineer, Machine Learning Infrastructure - Gen AI

About the Role

Key Responsibilities

• Design, build, and optimize LLMOps infrastructure including generative AI gateways, data pipelines for RAG systems, guardrails, batch inference, and model fine-tuning.
• Engineer sophisticated data pipelines to power Retrieval-Augmented Generation (RAG) systems, connecting models to real-time data for improved accuracy and relevance.
• Develop robust AI guardrails to ensure models operate safely, ethically, and responsibly.
• Pioneer development of advanced tool-calling capabilities and AI Agent frameworks to enable models to execute complex tasks as intelligent agents.
• Build and optimize scalable systems for batch inference and model fine-tuning to customize state-of-the-art models for specific use cases.
• Collaborate with ML and Product Engineers to evolve a reliable, scalable, and highly observable ML platform.

Requirements

• Industry experience: 2+ years building ML infrastructure or related disciplines.
• Familiarity with large language models (LLMs) and related technologies including prompt engineering, fine-tuning, retrieval-augmented generation (RAG).
• Problem-solving skills with the ability to tackle complex technical challenges.
• Passion for working at the intersection of machine learning and data engineering.

Nice to Have

• Experience with popular vector databases such as Qdrant, Pinecone, or FAISS.
• Familiarity with modern LLM frameworks and their interaction with vector search.
• Experience with real-time data processing frameworks like Spark, Flink, or Kafka.
• Experience with cloud-based infrastructure (AWS, GCP, Azure) for data and model deployment.
• Knowledge of retrieval optimization techniques for large-scale systems.
• Experience with AI Agent SDKs such as OpenAI Agents, Google ADK, CrewAI, LangGraph.

Qualifications

• Formal educational background not explicitly specified, but industry experience required.

Benefits & Perks

• Competitive salary within the range of $137,100-$201,600 USD, determined by skills, experience, and location.
• Opportunities for equity grants.
• Comprehensive benefits including 401(k) with employer matching, 16 weeks paid parental leave, wellness benefits, commuter benefits, paid time off, and paid sick leave.
• Medical, dental, and vision benefits, 11 paid holidays, disability and life insurance, family-forming assistance, mental health programs.
• Paid time off: flexible vacation, sick leave, and accrual details based on role type.