Senior AI Research Engineer, Model Inference (Remote)

About the Role

Key Responsibilities

• Implement and optimize custom inference and fine-tuning kernels for small and large language models across multiple hardware backends.
• Implement and optimize full and LoRA fine-tuning for small and large language models across multiple hardware backends.
• Design and extend datatype and precision support (int, float, mixed precision, ternary QTypes, etc.).
• Design, customize, and optimize Vulkan compute shaders for quantized operators and fine-tuning workflows.
• Investigate and resolve GPU acceleration issues on Vulkan and integrated/mobile GPUs.
• Architect and prepare support for advanced quantization techniques to improve efficiency and memory usage.
• Debug and optimize GPU operators (e.g., int8, fp16, fp4, ternary).
• Integrate and validate quantization workflows for training and inference.
• Conduct evaluation and benchmarking (e.g., perplexity testing, fine-tuned adapter performance).
• Conduct GPU testing across desktop and mobile devices.
• Collaborate with research and engineering teams to prototype, benchmark, and scale new model optimization methods.
• Deliver production-grade, efficient language model deployment for mobile and edge use cases.
• Work closely with cross-functional teams to integrate optimized serving and inference frameworks into production pipelines designed for edge and on-device applications.
• Define clear success metrics such as improved real-world performance, low error rates, robust scalability, and optimal memory usage, and ensure continuous monitoring and iterative refinements for sustained improvements.

Requirements

• Proficiency in C++ and GPU kernel programming.
• Proven expertise in GPU acceleration with Vulkan framework.
• Strong background in quantization and mixed-precision model optimization.
• Experience and expertise in Vulkan compute shader development and customization.
• Familiarity with LoRA fine-tuning and parameter-efficient training methods.
• Ability to debug GPU-specific performance and stability issues on desktop and mobile devices.
• Hands-on experience with mobile GPU acceleration and model inference.
• Familiarity with large language model architectures (e.g., Qwen, Gemma, LLaMA, Falcon).
• Experience implementing custom backward operators for fine-tuning.
• Experience creating and curating custom datasets for style transfer and domain-specific fine-tuning.

Nice to Have

• Experience with empirical research to overcome challenges in model optimization.
• Knowledge of additional model architectures and training techniques.
• Experience working in remote, collaborative environments.

Qualifications

• Deep expertise in kernel development, model optimization, fine-tuning, and GPU acceleration.
• Hands-on experience with Vulkan backend and mobile GPU debugging.

Benefits & Perks

• Work remotely from anywhere in the world.
• Be part of a pioneering team shaping the future of digital finance and AI.
• Collaborate with some of the brightest minds in fintech and AI.