Research Engineer, Benchmarks

ABOUT THE ROLE Join a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. You will own the design and implementation of domain-specific evaluations that help research teams and customers understand how agents perform on realistic workflows. WHAT YOU'LL DO - Design, implement, and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks. - Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria. - Build reliable infrastructure to run models and agents against benchmark tasks at scale. - Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes. - Validate whether benchmark results align with real-world evaluations and user needs. - Write clear technical documentation and reports explaining benchmark methods and results. WHAT WE'RE LOOKING FOR - Two to four years of experience in software engineering, machine learning engineering, or research engineering, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments. - Technical education or strong fundamentals in a technical field. - Proficiency with Python, Docker, and Linux for research or production infrastructure. - Hands-on experience designing and operating evaluations or environments for AI agents or large language models. - Experience building reliable model or agent execution infrastructure, developing evaluation metrics or statistical analyses, and validating benchmark performance. - Ability to work with subject-matter experts and understand workflows across technical or business domains. - Strong technical writing, attention to detail, independent problem-solving, and comfort in an early-stage environment. - Published research or technical writing on AI benchmarks, evaluation methods, or model failure modes is valued. Reinforcement learning pipelines, data generation, and agent evaluation experience are also relevant. COMPENSATION & BENEFITS Base salary: $100,000 to $170,000 USD annually. Visa sponsorship is available. LOCATION On-site in Singapore, Singapore.