- Deneyim
- 3+ yaş
- Maaş
- USD 37 / hour
- Açılışlar
- 1
- Yayınlandı
- 3 saat önce
- Çalışma modu
- Evden çalışma
- Eğitim
- Bachelor's or Master's in Engineering, Applied Mathematics, Computer Science or related technical field preferred
- Sürdürmek
- Başvuru yapılması gerekmektedir.
İş tanımı
About Mindrift and the Tendem project
Mindrift specializes in connecting domain experts with AI-driven projects from leading technology innovators worldwide. Their goal is to leverage generative AI's capabilities by integrating human expertise into AI workflows.
As part of the Tendem project, you will serve as an "AI Pilot," collaborating alongside automated Tendem Agents to manage the execution of specialized data scraping processes within a combined AI-human system.
Role Overview
This freelance, part-time position involves handling complex data scraping assignments using Python and related technologies. You will extract and process data from diverse and dynamic web sources, utilizing Mindrift's internal tools such as Apify and OpenRouter, as well as your personal methods to achieve precise results.
Key Responsibilities
- Manage full-cycle data extraction processes from multifaceted websites, ensuring comprehensive coverage and delivering accurate, well-structured datasets.
- Integrate internal scraping tools with custom-developed workflows to enhance data acquisition, validation, and task completion per project specifications.
- Adapt methodologies to reliably extract information from dynamic web content including JavaScript-rendered pages, handling complexities such as AJAX and infinite scrolling.
- Implement rigorous data quality controls including validation checks, cross-verification across sources, and adherence to predefined formatting standards before final delivery.
- Scale data scraping efforts effectively by employing batching or parallelization techniques; monitor and address failures while maintaining resilience to site changes.
Requirements
- Minimum of 3 years' experience in data engineering, web scraping, automation, or software development.
- Educational background in Engineering, Computer Science, Applied Mathematics, or equivalent technical disciplines is advantageous.
- Expertise in Python web scraping tools such as BeautifulSoup and Selenium, with proven skills in handling dynamic content and API requests through proxies.
- Demonstrated capability to extract information from complex data sources, including hierarchically structured sites, archived pages, and inconsistent HTML layouts.
- Strong competence in data cleansing, standardization, validation, and the production of structured outputs (CSV, JSON, Google Sheets).
- Experience working with large language models (LLMs) and AI platforms to improve automation processes and solve problems creatively.
- Meticulous attention to detail contributing toward high data accuracy and quality control.
- Ability to work autonomously with effective troubleshooting skills.
- English language proficiency at an upper-intermediate level (B2) or higher.
- Possession of a GitHub profile is considered a plus.
Compensation
Compensation on this project can reach up to 37 USD per hour depending on experience, skill level, and productivity. Rates may vary across different projects based on complexity and expertise requirements.
Why Join?
- Enjoy the flexibility of remote work with your own timing, requiring only a laptop and internet access.
- Engage in a unique environment merging human insights with AI technology, gaining valuable experience in an emerging field.
- Benefit from performance incentives rewarding consistent and high-quality contributions.