Token Crawler will provide users with a command line utility that crawls websites refered in txt file and searches for a specific token / string /pattern in the HMTL and in the js files used by the site.
It will also show an excert of where the token was found. You can use it to find specific scripts, tags, or urls in a set of web sites.
Here are some key features of "TokenCrawler":
· File Specifies the input file that contains the links to crawl (one per line)
· Help Displays this help text
· Output The file to output the results of execution to (defaults to TokenCrawler.txt)
· Token Specifies the token to be searched for in the crawled files
· Verbose Identify verbose level:0 minimum, 1 normal, 2 full
Requirements:
· Internet Connection
What's New in This Release: [ read full changelog ]
· New option to search the pattern in the HTTP response headers of each web request
· New ULR command line option to search only one file and not many as it is with the /File option
· Possibility to insert comments in the file that specifies the URLs to crawl
· Include sample Files and test script
· Changed command line switch from “-“ to “/”
· Change IgnoreCase default to true
Find us on Google+
