Spiking

Commonly referred to as spiders, web bots, engine bots, web crawlers are software that "crawl" (the technical term for automatically accessing a website and obtaining data) across the Internet to index content for search engines.

Spiking is one such tool, a minimalist web crawler designed to make it as easy as possible for users to round up online content. The app is wrapped around an Electron-based GUI that doesn't require much know-how in order to be used properly.

Spiking has two modes, concurrent and multithreaded, boasting an efficient crawling engine. Its main features include support for a custom initial link, automatic similarity, progress visualization, a proxy, as well as headless browser support.

To use it, users have to download the archive, unzip it, run the application, choose either Multiple or DynamicMultiple from the top of the main window, enter the URL, a local path for all the content, a concurrency number, and optionally, a proxy.

Once the Start button is clicked, the app displays various progress figures. For instance, it displays the total number of elements that can be crawled, how many images there are, as well as the number of errors.

To conclude, Spiking is a user-friendly web crawler that can be used by just about anybody without much effort. It features a modern GUI and a simple feature set.