Freelance Python Data Scraping Engineer

About the Position

Mindrift is seeking experienced Python Data Scraping Engineers to join the Tendem project, contributing to specialized data extraction workflows integrated within a hybrid AI and human collaborative system. This part-time freelance position supports remote work and is designed for technology professionals adept in web scraping and data processing.

Key Responsibilities

• Manage comprehensive data scraping workflows across multifaceted websites ensuring thoroughness, precision, and delivery of well-structured data sets.
• Utilize internal technologies such as Apify and OpenRouter along with custom methods to speed up scraping, validation, and execution aligned with project goals.
• Adapt techniques to reliably extract data from dynamic, interactive web sources that include JavaScript-rendered elements and variable site behaviors.
• Maintain stringent data quality by performing extensive validation, cross-referencing sources, meeting format standards, and verifying data before submission.
• Scale data scraping operations efficiently for large volumes by employing batching or parallel processes, continuously monitoring for and resolving failures, and ensuring stability amidst changes in website layouts.

Requirements

• Minimum three years of experience in data engineering, automation, web scraping, or software development.
• Undergraduate or postgraduate degree preferably in Engineering, Applied Mathematics, Computer Science, or allied fields is advantageous.
• Strong proficiency in Python-based web scraping tools such as BeautifulSoup and Selenium, with thorough experience handling dynamic content (including JS, AJAX, infinite scroll) and API extraction through proxies.
• Demonstrated expertise in collecting data from complex web structures such as hierarchical sites, archived content, and irregular HTML frameworks.
• Experience in cleansing, standardizing, and validating data to produce structured outputs in formats like CSV, JSON, and Google Sheets.
• Practical understanding of Large Language Models (LLMs) and AI frameworks to boost automation and problem resolution.
• Exceptional detail orientation and dedication to maintaining high data accuracy standards.
• Self-motivated with strong problem-solving abilities and capacity to work independently.
• Familiarity with GitHub is considered favorable.
• English language proficiency at least upper-intermediate (B2) level is required.

Compensation

Earn up to $37 per hour equivalent based on subject matter expertise and output speed, with pay scales varying across projects depending on complexity and skill demand.

Additional Information

• Enjoy the flexibility of working entirely remotely and managing your own schedule.
• Gain unique experience combining human expertise and AI technology within an innovative operational model.
• Eligible to participate in bonus programs that reward quality work and consistent productivity.