Skip to content

Xciyang/spiking

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Spiking

Chinese | English

A common web crawler tool developed using Node.js.

Advantage

  • Graphical Interface (Electron)
  • Custom initial link
  • Headless browser support (puppeteer)
  • Concurrent / multithreaded
  • Efficient crawling mechanism
  • Automatic similarity
  • Progress visualization
  • Using a proxy

Getting started

git clone https://github.com/Xciyang/spiking.git

cd spiking

npm install

npm start

Usage

Download the latest [Release] (https://github.com/Xciyang/spiking/releases) version.

Please read usage: English(en_US)

To-do list

  • Localization
  • Tasks Saving
  • Custom filtering method

More

Electron

puppeteer