Web Crawler and Scraper for Files and Links

Web Crawler can be used to get links, emails, images and files from a webpage or site. Web Crawler has a simple and intuitive interface.

Web Crawler and Scraper for Files and Links
Web Crawler and Scraper for Files and Links

About Web Crawler and Scraper

Web Crawler can be used to get links, emails, images and files from a webpage or site.

Web Crawler has a simple and intuitive interface.

The crawler is multithreaded and optimized for performance. It scans the webpage based on MIME types and file extensions, so it can find hidden links.

Two applications are included in the package. A Windows Forms application and a new WPF application with extended functionality. The “Deep crawl” feature allows the crawler to search all the linked pages from the selected website.

After crawling, the Web Crawler will save all links and e-mail addresses to the selected folder, along with all the crawled files.

The WPF crawler/scraper allows the user to input a regular expression to scrape through the webpages. The new application gives the user a greater control over the crawling process.