Web Slurper application was designed to be a simple but useful tool that will help users download partial or entire websites for offline viewing.
Here are some key features of "Web Slurper":
· Enable the user to download a website of a certain depth for offline browsing which are converted into relative links
· The Web Slurper 2.0 can also stop and resume a particular download task
· Download status are available via a tabbed panel which has a status tree and status table in it.
· Both HTTP and FTP protocol are supported.
· FTP leeching are also supported
· Support scheduled downloads of more than 24 hours.
· Support Multiple running downloads
· Flexibility in user defined settings such as
· number of spider threads per download
· number of retry per site
· connection timeout
· HTML attributes to be parsed by the parser
· Form Login Authentication attribute to be parsed by the parser
· Download Region
· File Format Exclusion
· Direct editing functionality in the download status table
· Error reporting mechanism for download errors
· Support link redirection
· Basic parser for dynamic links like asp, jsp, php extension and automatic conversion to html format for offline browsing.
· Download progress bar is shown in the status table
· Smart downloads of only newer file signature
· Support Cookies
Requirements:
· Java platform installed
What's New in This Release: [ read full changelog ]
· Support for scheduled downloads
· Support for link redirection (META Tag Refresh and HTTP Redirection)
· Improvement for the Graphical User Interface
· Basic parser for other document Types (asp, jsp, php)
· Added progress bar in the download status table
· Smart update for only newer file signature
· File extension exclusion
· Support FTP leeching
· Support cookies
· Fixed Java 1.3 deprecated classes
· Fixed the http:// and ftp:// for URL without protocol prefix
· Fixed website without first document to be downloaded "index.html"
· Fixed download depth issue, where the user cannot download deeper if there is previous download task from the same host
· Fixed a bug where there is a strange index_temp.txt appearing in ftp download directories
· Fixed GUI JSplitPane Bug where the status tree when missing ( blink) for a while when download is completed
· Fixed "user.set" file problem where the Slurper program will hang if the user.set file is not found or has not default values
· Fixed HTML Attribute extraction bug in ParserForHTML where the implementation logic is not correct
· Support for downloads for files with blank spaces in between, where it is appended with and will not return download error
· Additional delmiter support for user-settings dialog boxes for for "space" and "comma" issue
· File path Changes for Slurper system files and default download directory, solved incompatibility with Unix machine issue.
· Enhanced Construction of Best effort Absolute URL from relative link to retrieve more links to be downloaded. Erroneous relative links with depth less than its parent path will be made equal depth with its parent path.
· Enhanced GUI in the status table where user can input download information directly for redownloading.