Software Engineer (.net - Web Scraping - WinForms)

Location

[In-Office, Bangalore]

About PriceEasy

PriceEasy is transforming how fuel and convenience retailers make business decisions through AI, pricing intelligence, analytics, and automation. Trusted by leading retailers across the globe, we're building products that simplify complex operations and help businesses grow smarter.

What You'll Do:

• Design and build scalable, fault-tolerant scraping solutions in C# / .NET — as WinForms apps, console apps, Windows Services, and Web APIs — to extract structured data (pricing, product, location, and market data) from a wide variety of websites.
• Architect scraping solutions for both static HTML and dynamic, JavaScript-rendered sites.
• Build and maintain anti-bot evasion strategies: proxy rotation, IP/user-agent rotation, CAPTCHA handling, browser fingerprint management, and request throttling.
• Implement browser automation using Selenium and Microsoft WebView2 for complex, JS-heavy or embedded-browser targets.
• Consume and integrate external REST APIs as part of scraping and data-pipeline workflows.
• Extract data from mobile applications (Android/iOS) via API traffic interception, app automation, and reverse-engineering of app endpoints where no public API exists.
• Design data pipelines to clean, validate, normalize, and store scraped data reliably in MS SQL Server.
• Monitor scraper health, detect site layout/structure changes, and build self-healing or alerting mechanisms to minimize data downtime.
• Optimize scrapers for speed, reliability, and low detection footprint across thousands of concurrent targets.
• Collaborate with data engineering and product teams to define data requirements and delivery SLAs.
• Ensure scraping practices align with legal, ethical, and site-specific compliance considerations.
• Mentor junior engineers and contribute to scraping architecture/tooling standards across the team.

Required Skills & Experience:

Database

• MS SQL Server — schema design, stored procedures, query optimization for high-volume scraped data.

Languages & Frameworks

• 5+ years of professional development experience with C# / .NET.
• Hands-on experience building WinForms applications, console apps, Windows Services, and Web APIs.

Web Scraping

• Strong hands-on experience with Selenium (WebDriver) for browser automation.
• Experience with Microsoft WebView2 for embedded browser-based scraping/automation.
• Experience consuming external REST APIs for data extraction and integration.
• Practical experience working with proxies (rotation, management, and troubleshooting blocked/rate-limited requests).
• Skilled at identifying and targeting web elements using XPath.
• Experience with mobile app data extraction: traffic interception/MITM tools (Charles Proxy, mitmproxy, Fiddler), SSL pinning bypass, APK decompilation/reverse engineering (APKTool, jadx), and mobile automation frameworks (Appium) to extract data from Android/iOS apps.

General

• Experience handling large-scale, concurrent scraping (multi-threading, async/await, retries/backoff, job scheduling).
• Strong debugging skills — comfortable reverse-engineering site/app behavior when layouts or endpoints change.
• Understanding of data quality practices: deduplication, validation, and normalization at scale.
• Familiarity with version control (Git) and collaborative development workflows.
• Good understanding of exception handling, logging, and building maintainable, production-grade C# code.

Good to Have

• Experience with AWS technologies (EC2, S3, Lambda, or similar) — a plus.
• Familiarity with headless browser tools like Playwright or Puppeteer.
• Experience with scheduling/orchestration tools (Hangfire, Quartz.NET, Windows Task Scheduler).
• Exposure to message queues (RabbitMQ, Kafka, AWS SQS).
• Prior experience in pricing intelligence, e-commerce, or data-aggregation products.
• Understanding of legal/compliance considerations around web scraping (ToS, robots.txt, data privacy).
• Exposure to CI/CD pipelines and basic containerization (Docker).