As a Staff Data Scientist, you will serve as an architect and technical visionary for our core Generative AI capabilities, specifically focusing on the lifecycle of autonomous agents. You will lead the design and governance of a robust Evaluation Framework鈥攕panning offline "golden dataset" validation, real-time online monitoring, and the implementation of Continuous Learning loops that allow our agents to improve from human feedback and production interactions. Your expertise will be critical in advancing our Information Retrieval strategies, ensuring our RAG (Retrieval-Augmented Generation) pipelines provide the precise context needed for high-stakes decision-making. Beyond retrieval, you will be hands-on in sophisticated prompt tuning and engineering to squeeze maximum performance, reliability, and "agentic" reasoning out of Large Language Models. In this cross-squad leadership position, you won't just build models; you will define the architectural standards for how Freshworks extracts, processes, and evaluates data to turn complex Text2Action visions into a seamless, conversational reality.
Responsibilities:
Strategic Collaboration: Collaborate closely with product and business teams to gain a comprehensive understanding of the challenges and opportunities within the GenAI landscape, aligning data science initiatives with the organization's objectives.
Metric Definition: Define key performance metrics that accurately reflect the value delivered to end-users through GenAI solutions.
Intelligent System Design: Apply deep expertise in machine learning, statistics, and advanced mathematics to conceptualize, experiment, and design intelligent systems powered by GenAI.
Big Data Processing: Develop efficient systems capable of processing vast volumes of data, demonstrating proficiency in distributed programming frameworks like Hadoop and Spark.
ML/AI Architecture: Collaborate closely with ML Engineers to design scalable systems and model architectures that enable real-time ML/AI services.
End-to-End ML Pipelines: Take ownership of ML pipelines from end to end, encompassing data pre-processing, model generation, cross-validation, and feedback sharing.