Python Scraping Developer

📋 Description

• • As a Python Scraping Developer at Onhires Inc., you will play a critical role in powering the data engine of a Berlin-based, remote-first scale-up delivering market intelligence to the automotive industry. Your work will directly enable accurate, timely, and scalable data collection that fuels strategic decision-making for clients across global markets.
• • You will own the full lifecycle of high-volume web scraping pipelines — from design and development to deployment, monitoring, and optimization — ensuring data integrity, system resilience, and seamless integration with downstream analytics and API services.
• • Day to day, you will design, test, and deploy robust web scraping scripts using Python and advanced tools like Playwright, Selenium, Requests, and BeautifulSoup to extract structured data from complex, dynamic websites.
• • You will architect and maintain asynchronous scraping systems capable of handling massive-scale data extraction, optimizing for speed, reliability, and resource efficiency under high load.
• • You will implement and refine anti-blocking strategies, including intelligent proxy rotation, header management, and behavioral evasion techniques to maintain high uptime and success rates against sophisticated bot defenses.
• • You will manage and automate data ingestion pipelines, integrating scraped outputs with external REST APIs and internal storage systems, ensuring clean, consistent data flow.
• • You will continuously monitor, debug, and improve scraper performance, reliability, and data quality through logging, alerting, and iterative optimization.
• • You will collaborate closely with backend and data engineers to enhance shared scraping infrastructure, tooling, logging frameworks, and monitoring dashboards.
• • You will support DevOps initiatives by containerizing applications with Docker, managing Linux-based environments, and contributing to CI/CD pipelines for automated testing and deployment.
• • The team is a small, international, and highly skilled group operating with a remote-first mindset, fostering autonomy, technical excellence, and continuous learning in a fast-growing scale-up environment.
• • In this role, you will deepen your expertise in large-scale web scraping, distributed systems, and production-grade data engineering — gaining hands-on experience with cutting-edge tools and architectures while making a tangible impact on a product that drives real-world automotive market insights.