Zum Inhalt springen

Alle Stellen

Machine Learning Engineer, Evaluation

mpathic2

  • Location not stated
  • Veröffentlicht am 21. Nov. 2026
  • ML- und KI-Ingenieur
  • Jobrona kann sich für Sie bewerben

Über die Stelle

About mpathic mpathic builds AI that understands clinical and human conversations: analyzing therapy and clinical-trial sessions, producing expert-labeled data for AI builders, and publishing open safety benchmarks. About the role We’re looking for a Machine Learning Engineer to build and improve ML-powered products, with a particular focus on evaluating their quality and reliability. You’ll work across the ML lifecycle, combining hands-on engineering with rigorous experimentation to understand how systems perform and where they need to improve. This is a generalist ML engineering role with an evaluation focus. You’ll contribute to production systems while helping the team make sound decisions about models, data, and product changes. What you’ll do - Build, deploy, and maintain ML models and services in collaboration with engineering and product teams. - Design evaluation approaches that reflect product needs and real-world operating conditions. - Create and maintain evaluation datasets, benchmarks, and automated regression tests that measure safety, empathy, and risk. - Select appropriate metrics, analyze performance across meaningful data segments, and investigate failure modes. - Design experiments and use statistical methods to judge whether an improvement is real. - Connect offline evaluation results with what customers see in production. - Develop tools that make evaluation repeatable and useful throughout development and release workflows. - Communicate findings, limitations, and recommendations clearly to clinicians, product leads, and customers. What you’ll bring - Practical experience developing and shipping ML systems, typically 2–5 years of relevant work or equivalent. - Strong Python skills and familiarity with common ML frameworks. - Solid ML fundamentals, including training, validation, generalization, and common sources of model error. - Experience with evaluation metrics, benchmark design, and systematic error analysis. - An understanding of statistical inference, sampling, confidence intervals, and experimental design. - Experience identifying data leakage, evaluation bias, and gaps in test coverage. - Good software engineering practices, including testing, version control, and code review. - Skill at turning ambiguous product goals into measurable quality criteria. Helpful additional experience - A/B testing and production performance monitoring. - Human evaluation, annotation guidelines, and inter-rater agreement. - Speech or conversational data: transcripts, diarization and multi-speaker setups. - Evaluating ranking, retrieval, recommendation, or generative AI systems. - Robustness testing, calibration, fairness analysis, or distribution shift detection. - Building evaluation tooling and integrating quality checks into CI/CD workflows. - Clear, direct communication, both written and verbal. - Startups and cross-functional teams.

Nicht nur diese Stelle

Mit demselben Lebenslauf findet Jobrona Stellen wie diese und bewirbt sich für Sie darauf.