Job Title: Data Science Professional
Join us in shaping the future of our corporate wellness platform, where innovative technology meets human wellbeing. As a highly skilled Data Scientist, you will play a pivotal role in evaluating and measuring generative AI systems that improve user experience.
Key Responsibilities:
* Collaborate with cross-functional teams to define metrics for user engagement, satisfaction, and chatbot effectiveness;
* Perform data cleansing, transformation, and quality analysis on product data;
* Design robust evaluation metrics to assess agentic AI performance, combining automated evaluation, adversarial testing, human-in-the-loop evaluations, and custom behavioral metrics;
* Craft reusable templates and clear instructions to measure system behavior, optimizing for challenging edge cases;
* Design monitoring systems that track LLM behavior in production, capturing key metrics around information retrieval, hallucinations, and latency;
* Design, run, and analyze A/B tests to optimize chatbot interactions and user engagement;
* Create and maintain interactive dashboards using BI tools for real-time visualization of performance metrics and insights.
Requirements:
* Bachelor's degree in Computer Science, Data Science, Machine Learning, Statistics, or a related field;
* Proficiency in Python for data handling and analysis;
* Advanced NoSQL and SQL skills for querying large datasets;
* Strong problem-solving abilities, with a focus on experimental design and data analysis;
* Ability to work collaboratively in a team environment and communicate effectively across different departments;
* Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders;
* Fluent in English.
Preferred Qualifications:
* Familiarity with evaluation methodologies for generative AI, including human-in-the-loop assessments, LLM-as-a-judge approaches, and adversarial robustness testing;
* Proven track record of tracking and evaluating generative AI models in production;
* Experience in multi-agent systems and LLMs.
What We Offer:
We are committed to the health and wellbeing of our employees, offering benefits such as digital fitness programs, online wellness resources, flexible work arrangements, paid time off, parental leave, career growth opportunities, and a supportive environment.