Python Quality Assurance Lead

Role overview

We are looking for a Python Quality Assurance Lead to support Python AI training initiatives in a remote contractor capacity paid on an hourly basis. In this role, you will help maintain the quality, consistency, and overall performance of trainers and QA contributors working on Python-focused projects. Your work will involve checking AI-produced Python code and reviewer output, measuring it against project rules, and giving clear written feedback that helps raise standards across the team.

You will evaluate content for correctness, execution behavior, debugging quality, readability, maintainability, speed, security awareness, test coverage, formatting, and how well it follows instructions and project rubrics. The role also includes identifying repeated quality problems, sharing updates with trainers and QAs, supporting onboarding, keeping documentation current, and helping bring inactive contributors back into engagement.

This company is an AI data services provider that creates training data for major AI companies and foundation model labs. Your contribution will help improve Python model training by ensuring the output is accurate, runnable, idiomatic, well explained, documented, and aligned with client expectations.

There is no active project available immediately for this position. However, candidates who qualify may be contacted first when matching opportunities arise and may also gain access to future projects through the expert network.

Selection process

The hiring process includes an AI interview, a domain-specific assignment, and a conversation with a recruiter.

Requirements and profile

• A bachelor’s or master’s degree in Computer Science, Software Engineering, Data Science, Information Technology, or comparable professional software engineering experience.
• Strong English communication skills for understanding guidelines, coordinating with teams, and delivering precise technical feedback.
• At least 3 years of professional experience in Python development, backend engineering, automation, scripting, data workflows, code review, QA, teaching, or technical mentoring.
• Solid command of Python fundamentals, including data structures, functions, classes, modules, exceptions, comprehensions, iterators, generators, decorators, context managers, virtual environments, packaging, and testing.
• Ability to assess Python work using rubrics and detect issues such as faulty logic, non-runnable code, weak exception handling, inefficient algorithms, unsafe file or network operations, invented APIs, or incomplete explanations.
• Preferred familiarity with pytest, unittest, typing, mypy, pip, poetry, virtualenv, FastAPI, Flask, Django, requests, asyncio, pandas, SQLAlchemy, GitHub, Docker, and CI/CD.
• Experience leading or supporting remote groups of trainers, engineers, reviewers, annotators, educators, or QAs is strongly preferred.
• Comfort using Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management tools.
• Strong organizational skills to manage style guides, FAQs, trackers, onboarding content, honeypots, calibration tasks, and documentation.
• Background in AI training, data annotation, LLM evaluation, code QA, or rubric-based code review is a valuable advantage.

Key responsibilities

• Review Python submissions, detect quality problems, share feedback directly with contributors, and escalate repeated or serious concerns.
• Assess AI-generated Python code, debugging answers, algorithms, backend examples, automation scripts, tests, and written explanations.
• Keep trainers and QAs informed on Discord about guideline changes, process updates, and Python-specific quality expectations.
• Answer questions related to Python syntax, runtime behavior, exceptions, package usage, testing, typing, security, performance, and rubric interpretation.
• Contact inactive contributors, encourage them to resume work, track follow-ups, and highlight availability problems.
• Build and update Python documentation, style guides, trackers, FAQs, examples, honeypots, and onboarding resources.
• Lead onboarding and training calls for Python contributors.
• Identify misleading, unsafe, inefficient, non-executable, or non-production-ready Python advice and flag it appropriately.
• Find recurring quality weaknesses and help improve Python QA processes and workflows.

Additional information

This is a remote contractor role with hourly compensation. The position is centered on quality leadership for Python data used in AI model training, and the work is expected to be highly detail-oriented, collaborative, and documentation-heavy.