We are seeking a skilled Data Scientist to join our team in Portugal.
Main Responsibilities
* Collaborate with product managers and engineers to design and measure metrics for user engagement, satisfaction, and chatbot effectiveness;
* Perform data cleansing, transformation, and quality analysis on product data;
* Evaluate and test generative AI models, using automated evaluation methods, adversarial testing, and human-in-the-loop assessments;
* Craft reusable templates and clear instructions to measure system behavior, optimizing for challenging edge cases;
* Design monitoring systems to track LLM behavior in production, capturing key metrics around information retrieval, hallucinations, and latency;
* Run A/B tests to optimize chatbot interactions and user engagement;
* Create and maintain interactive dashboards for real-time visualization of performance metrics and insights.
About the Role
This is an exciting opportunity to work with a multidisciplinary team to create innovative AI applications that solve real-world problems. As a Data Scientist, you will play a crucial role in evaluating, measuring, and assessing generative AI systems that improve the wellbeing of users.
Requirements
* Bachelor's degree in Computer Science, Data Science, Machine Learning, Statistics, or a related field;
* Proficiency in Python for data handling and analysis;
* Advanced NoSQL and SQL skills for querying large datasets;
* Strong problem-solving abilities, with a focus on experimental design and data analysis;
* Ability to workcollaboratively in a team environment and communicate effectively across different departments;
* Excellentverbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders;
* Fluent inEnglish.
Preferred Qualifications
* Familiarity with evaluation methodologies for generative AI, including human-in-the-loop assessments, LLM-as-a-judge approaches, and adversarial robustness testing;
* Proven track record of tracking and evaluating generative AI models in production;
* Experience in multi-agent systems and LLMs.
Benefits
We offer a range of benefits to support your wellbeing and career growth, including:
* A flexible work environment that fosters collaboration, community, and team building;
* A home office stipend and flexible work allowance to help cover the costs of working from home;
* Paid timeoff to recharge and pursue personal interests;
* Parental leave to support new parents and extended maternity leave;
* Outstanding opportunities for personal and career growth through employee development programs.