Senior Python Scraping Engineer

📋 Description

• • Are you a seasoned Python engineer with a passion for tackling complex web scraping challenges? Do you thrive in environments where you need to outsmart sophisticated anti-bot defenses and adapt to constantly evolving website structures? If so, Adthena is looking for you to join our mission to shape the future of paid search intelligence.
• • Adthena is an award-winning search intelligence platform, recognized for its patented AI technology and its ability to provide unparalleled insights into the paid search market. Our solutions, including Whole Market View, Smart Monitor, and Local View, empower brands, marketers, and agencies to optimize their ad spend, boost ROI, and maintain a competitive edge. Trusted by industry giants like Citibank, L'Oréal, and Volvo, and backed by Updata Partners, we have been at the forefront of search intelligence since 2012. Our continued success is evidenced by our numerous industry awards, including Best AI Search Software Solution and Best Search Software Tool.
• • As a Senior Python Scraping Engineer, you will play a pivotal role in designing, building, and operating large-scale, resilient web scraping systems. Your primary focus will be on extracting data from Google and similar highly defended search environments. This role is a unique blend of data engineering, reverse engineering, and systems reliability, demanding innovative solutions for complex scraping problems. You will tackle challenges such as dynamic rendering, sophisticated anti-bot mechanisms, frequent markup changes, and the need for high-volume, accurate data collection.
• • We are at the cutting edge of AI adoption in coding, and this role is perfect for individuals who embrace agentic coding principles and are eager to enhance their skills in this domain. You will be instrumental in ensuring the accuracy, timeliness, and trustworthiness of the data that fuels our clients' success.
• • Your responsibilities will include:
• • Designing, developing, and maintaining robust, large-scale Python scraping systems engineered for reliability against Google and other heavily protected websites.
• • Building sophisticated scrapers for dynamic, JavaScript-intensive pages, employing browser automation techniques and hybrid approaches to overcome rendering challenges.
• • Continuously adapting and evolving scraping strategies to counter changes in page structure, request flows, and emerging anti-automation technologies.
• • Engineering resilient data extraction pipelines that prioritize data accuracy, consistency, and comprehensive observability, ensuring data integrity from collection to delivery.
• • Implementing advanced proxy management, browser fingerprinting, and intelligent request-routing strategies to maximize system stability, throughput, and efficiency.
• • Proactively monitoring the health of scraping operations, detecting failures at their earliest stages, and rapidly debugging production issues to minimize downtime and data loss.
• • Optimizing the performance, cost-effectiveness, and latency of our scraping infrastructure to ensure scalability and efficiency.
• • Collaborating closely with data engineers, data scientists, and product teams to guarantee that scraped data is not only collected but also readily usable, accurate, and trusted for downstream applications.
• • Creating and maintaining comprehensive documentation for scraping logic, operational procedures, and runbooks to ensure long-term maintainability and knowledge transfer within the team.
• • This is an opportunity to join a company that offers the stability of an established leader with the dynamism of a scaling tech organization. You'll experience less red tape, more autonomy, and accelerated career growth. We foster a strong engineering culture, evidenced by a 93% approval rating and an eNPS of 27. You'll be part of an award-winning product team, with opportunities to share in our success through stock options. We believe in empowering our employees with trust-based vacation, flexible remote-first work arrangements, and dedicated career development through "Sherpa plans." Regular hackdays, training days, and a supportive family care package further enhance the employee experience. We also provide a home office stipend and welcome gifts to ensure you have the best setup for success.
• • If you are a proactive, detail-oriented engineer who excels at problem-solving in challenging technical landscapes and is driven by a desire to deliver high-quality, reliable data solutions, we encourage you to apply. We value curiosity, a bias towards action, and a commitment to continuous improvement. We are an equal opportunity employer and welcome applicants from all backgrounds, especially those who may not meet every single requirement but possess the core skills and a strong desire to learn and contribute.