Python Engineer (Web Scraping)

Role: Python Engineer (Web Scraping & Extraction)

As a Python Engineer at Scrapfly, you will be instrumental in maintaining and improving our industry-leading web scraping capabilities. Your work will involve tackling complex extraction challenges, optimizing HTML parsing techniques, and enhancing our fingerprinting and HTTP handling strategies. You’ll collaborate with the support team to analyze blocked targets and improve our ability to navigate evolving anti-bot protections.

If no messages are included in your application, your CV will not be considered. Explain why you should join Scrapfly and how you can help us.

Key Responsibilities

• Technical Client Support: Our client are developers, and sometimes they discovered bug or require to be guided to adjust their configuration
• Web Scraping & Extraction: Develop and optimize Python-based solutions for HTML parsing, structured data extraction, and handling dynamic content.
• Feature Development & Maintenance: Enhance and support key scraping features, such as screenshot capture, extraction templates, LLM-based processing
• CDP Browser Maintenance: Maintain and improve our proprietary browser automation stack to handle dynamic content extraction.
• Blocked Target Analysis: Work closely with the support team to diagnose and resolve challenges related to access restrictions.

Ideal Candidate

We’re seeking a proactive Python engineer with a passion for web scraping and data extraction. This role offers significant autonomy while ensuring close collaboration with our CTO and other team members.

It’s a plus to know PHP and Golang (dashboard, web application are in PHP and our API gateway is written in Golang)

Already works in a full remote/async company with culture of writing communication and pro actively raise subject.

Qualifications

• Experience: 3+ years in Python development with a strong focus on web scraping and data extraction.
• Skills:
• Expertise in HTML parsing and extraction techniques (e.g., XPath, CSS selectors, regex, structured data handling).
• In-depth understanding of HTTP mechanics, headers, cookies, sessions, and request fingerprinting.
• Understanding and using proxies, managing errors
• Experience handling bot protections, rate limiting, and dynamic content challenges.
• Familiarity with CDP (Chrome DevTools Protocol) and browser automation tools (e.g., Puppeteer, Playwright, Selenium) is a plus.
• Familiarity with Kubernetes (helm, k3d/k3s) as it is used in the development environment.
• Comfortable working in a Linux environment (Ubuntu-based development workspaces).
• Excellent writing communication

Company Policies and Benefits

• Remote-First: Fully remote environment using Slack, Notion, and GitHub for communication and project management.
• No-Meeting Policy: We do not have meetings, so you stay focus on your tasks and works, there is no “agile” methodology, we just have ticket prepared with details to pick, most of them are created based on discussion where you are in the loop, and if needed discussed over slack, Simple.
• Contractor Model: Paid in USD with 20 paid leave days per year (additional unpaid leave available). You require to have a company invoicing us each month accepting international payment in USD (From United States). If our bank support your currency (https://www.brex.com/support/domestic-and-international-wires) we could convert before sending. You can’t operate from excluded countries listed below.
• Developer Resources: Access to a private Linux remote machine (8-core CPU, 32GB RAM, 240GB SSD) via our internal VPN.
• Income Range: Starting from $1,200 USD/month up to $3,500 USD/month, based on expertise and responsibilities. We value autonomy and comprehensive skill coverage. Entry-level roles focused on technical client support start at $1,200 USD/month, while engineers handling full feature development and debugging can earn around $3,000 USD/month. Compensation grows based on delivered value and contributions. Negociation does not need to wait for end-year reviews, it’s purely based on your outcome and capacity to pick ticket.

Excluded countries

As contractor, your legal entity must not be registrer in the following countries due to sanction or bank restrictions.

• Afghanistan
• Antarctica
• Cuba
• Iran (Islamic Republic of)
• North Korea (Democratic People's Republic of Korea)
• South Sudan
• Syria (Syrian Arab Republic)
• Venezuela (Bolivarian Republic of)
• Russia