Retail Subject Matter Expert - Large Language Model Training and Evaluation

Overview

Contribute your deep expertise in retail to enhance the reasoning quality of advanced language models. This role focuses on designing detailed retail tasks and evaluating outputs for accuracy and logic, helping shape cutting-edge AI solutions within the retail domain.

Key Responsibilities

• Create domain-specific retail scenarios covering merchandising, category management, and operations, producing authoritative solutions for training data pipelines.
• Assess large language model (LLM) responses using structured rubrics, providing clear and precise written evaluations regarding their correctness, reasoning quality, and judgment.
• Detect gaps in model understanding and advise research and engineering teams to improve retail-specific content coverage.
• Craft and iteratively enhance scoring rubrics and evaluation criteria that reflect the complexities of retail practice.
• Collaborate with fellow subject matter experts to maintain consistency and correctness across training datasets during rubric creation and peer review processes.

Required Qualifications

• Substantial practical experience in retail, with career progression to roles such as Category Manager, Senior Manager, or Director within areas like merchandising, buying/planning, or retail operations at a recognized leading retailer.
• Demonstrable prior involvement in evaluating LLM or automation model outputs based on structured rubrics or scoring frameworks; applicants must describe this experience.
• Proven skill in translating expert retail judgments into reproducible written evaluation logic fit for structured training workflows.
• Documented advancement within a notable retail organization such as national/global retailers or major e-commerce platforms.
• Strong written communication abilities to clearly and succinctly explain subtle reasoning errors to both technical and non-technical audiences.

Additional Preferred Skills

• Experience with category analytics, assortment strategies, or demand planning, enhancing the depth of task and rubric development.
• Familiarity with annotation tools, data quality procedures, or structured evaluation methodologies is highly beneficial.
• Experience working within extended workforce models, contract roles, or embedded teams indicates readiness for this engagement style.