Netpeak Spider

Netpeak Spider is a professional SEO crawler that you can use to collect data about your websites and figure out how to improve rankings and boost your exposure on search engines. It can calculate the PageRank, find pages with duplicate content, locate missing ALT attributes of images, identify internal and external links, and more.

Installing the app shouldn't be a problem but make sure you have .NET Framework. After setup, you gain access to the modern interface with a neatly structured and elegant appearance, where you can get started by entering a website you want to crawl.

By default, the entire website is analyzed, including subdomains and excluding external links, and the targeted types of content are HTML files and redirects, images and other MIME types besides JavaScript and CSS files. However, you can configure crawl settings and instruct Netpeak Spider to crawl only a list of URLs (with or without the subdomains), an XML sitemap, a directory only, or a Google search engines result page (SERP).

Once the scan is over, you can apply filters, view the incoming and outgoing links, images, h1 headers, and issues, among others. The table can be exported to file in Excel or CSV format. There are also a couple of tools provided by Netpeak Spider for page source and HTTP header analysis as well as internal PageRank calculation.

All parameters involved in the search job can be examined up close to determine whether or not you want to ignore anything. These are general (like #, URL, status code, issues, level, response time), indexation (like Robots.txt, redirects, redirect target URLs, metarobots), head tags (lke title, description, description length, keywords, pagination), content (like images, unique hashes), links (incoming, outgoing external and internal) and the internal PageRank.

Custom rules can be created to include and/or exclude specific URLs containing, exactly matching, matching regex, begins with or ends with a specific string of characters, or matching a specific length of characters. It's also possible to set crawling restrictions when it comes to the maximum depth or number of crawled URLs, the response timeout, maximum number of redirects and canonical chain length.

All aspects considered, Netpeak Spider offers practical and easy-to-use tools to crawl your websites and list problems you need to resolve to boost your ranking on search engines. It's wrapped in a user-friendly interface and worked smoothly on Windows 10 in our evaluation. It can be used alongside Netpeak Checker, made by the same developer.