Softpedia >Windows >Programming >Components & Libraries >jsoup >  Changelog

jsoup Changelog

What's new in jsoup 1.8.2

Apr 16, 2015
  • Performance improvements for parsing HTML on Android, of 1.5x to 1.9x, with larger parses getting a bigger speed increase. For non-Android JREs, around 1.1x to 1.2x.
  • Dramatic performance improvement in HTML serialization on Android (KitKat and later), of 115x. Improvement by working around a character set encoding speed regression in Android.
  • Performance improvement for the class name selector on Android (.class) of 2.5x to 14x. Around 1.2x on non-Android JREs.
  • File upload support. Added the ability to specify input streams for POST data, which will upload content in MIME multipart/form-data encoding.
  • Add a meta-charset element to documents when setting the character set, so that the document's charset is
  • unambiguous.
  • Added ability to disable TLS (SSL) certificate validation. Helpful if you're hitting a host with a bad cert, or your JDK doesn't support SNI.
  • Added ability to further tweak the canned Cleaner Whitelists by removing existing settings
  • Added option in Cleaner Whitelist to allow linking to in-page anchors (#)
  • Use a lowercase doctype tag for HTML5 documents.
  • Add support for 201 Created with redirect, and other status codes. Treats any HTTP status code 2xx or 3xx as an OK response, and follow redirects whenever there is a Location header.
  • Added support for HTTP method verbs PUT, DELETE, and PATCH.
  • Added support for overriding the default POST character of UTF-8
  • W3C DOM support: added ability to convert from a jsoup document to a W3C document, with the W3Dom helper class.
  • In the HtmlToPlainText example program, added the ability to filter using a CSS selector. Also clarified the usage documentation.
  • Fixed validation of cookie names in HttpConnection cookie methods.
  • Fixed an issue where tags would be missed when preparing a form for submission if missing a selected attribute.
  • Fixed an issue where submitting a form would incorrectly include radio and checkbox values without the checked attribute.
  • Fixed an issue where Element.classNames() would return a set containing an empty class; and may have extraneous whitespace.
  • Fixed an issue where attributes selected by value were not correctly space normalized.
  • In head+noscript elements, treat content as character data, instead of jumping out of head parsing.
  • Fixed performance issue when parsing HTML with elements with many children that need re-parenting.
  • Fixed an issue where a server returning an unsupport character set response would cause a runtime
  • UnsupportedCharsetException, instead of falling back to the default UTF-8 charset.
  • Fixed an issue where Jsoup.Connection would throw an IO Exception when reading a page with zero content-length.
  • Improved the equals() and hashcode() methods in Node, to consider all their child content, for DOM tree comparisons.
  • Improved performance in Selector when searching multiple roots.

New in jsoup 1.8.1 (Sep 29, 2014)

  • Introduced the ability to chose between HTML and XML output, and made HTML the default. This means img tags are
  • output as , not . XML is the default when using the XmlTreeBuilder. Control this with the
  • Document.OutputSettings.syntax() method.
  • Improved the performance of Element.text() by 3.2x
  • Improved the performance of Element.html() by 1.7x
  • Improved file read time by 2x, giving around a 10% speed improvement to file parses.
  • Tightened the scope of what characters are escaped in attributes and textnodes, to align with the spec. Also, when
  • using the extended escape entities map, only escape a character if the current output charset does not support it.
  • This produces smaller, more legible HTML, with greater control over the output (by setting charset and escape mode).
  • If pretty-print is disabled, don't trim outer whitespace in Element.html()
  • In the HTML Cleaner, allow span tags in the basic whitelist, and span and div tags in the relaxed whitelist.
  • Added Element.cssSelector(), which returns a unique CSS selector/path for an element.
  • Fixed an issue where was parsed as
  • Fixed an issue where a UTF-8 BOM character was not detected if the HTTP response did not specify a charset, and
  • the HTML body did, leading to the head contents incorrectly being parsed into the body. Changed the behavior so that
  • when the UTF-8 BOM is detected, it will take precedence for determining the charset to decode with.
  • Relaxed doctype validation, allowing doctypes to not specify a name.
  • Fixed an issue in parsing a base URI when loading a URL containing a http-equiv element.
  • Fixed an issue for Java 1.5 / Android 2.2 compatibility, and verify it doesn't regress.
  • Fixed an issue that would throw an NPE when trying to set invalid HTML into a title element.
  • Added support for quoted attribute values in CSS Selectors
  • Fixed support for nth-of-type selectors with unknown tags.
  • Added support for 'application/*+xml' mimetypes.
  • Fixed support for allowing script tags in cleaner whitelists.

New in jsoup 1.7.3 (Sep 29, 2014)

  • Introduced FormElement, providing easy access to form controls and their data, and the ability to submit forms with Jsoup.Connect.
  • Reduced GC impact during HTML parsing, with 17% fewer objects created, and 3% faster parses.
  • Reduced CSS selection time by 26% for common queries.
  • Improved HTTP character set detection.
  • Added Document.location, to get the URL the document was retrieved from. Helpful if connection was redirected.
  • Fixed support for self-closing script tags.
  • Fixed a crash when reading an unterminated CDATA section.
  • Fixed an issue where elements added via the adoption agency algorithm did not preserve their attributes.
  • Fixed an issue when cloning a document with extremely nested elements that could cause a stack-overflow.
  • Fixed an issue when connecting or redirecting to a URL that contains a space.
  • Added support for the HTTP/1.1 Tempory Redirect (307) status code.

New in jsoup 1.7.2 (Jan 30, 2013)

  • Added support for supplementary characters outside of the Basic Multilingual Plane
  • Added support for structural pseudo CSS selectors, including :first-child, :last-child, :nth-child, :nth-last-child
  • first-of-type, :last-of-type, :nth-of-type, :nth-last-of-type, :only-child, :only-of-type, :empty, and :root
  • Added a maximum body response size to Jsoup.Connection, to prevent running out of memory when trying to read extremely large documents. The default is 1MB
  • Refactored the Cleaner to traverse rather than recurse child nodes, to avoid the risk of overflowing the stack
  • Added Element.insertChildren(), to easily insert a list of child nodes at a specific index
  • Added Node.childNodesCopy(), to create an independent copy of a Node's children
  • When parsing in XML mode, preserve XML declarations ()
  • Introduced Parser.parseXmlFragment(), to allow easy parsing of XML fragments
  • Allow Whitelist test methods to be extended
  • Added Document.OutputSettings.outline mode, to aid HTML debugging by printing out in outline mode, similar to browser HTML inspectors
  • When parsing, allow all tags to self-close. Tags that aren't expected to self-close will get an end tag
  • Fixed an issue when parsing /RCData tags containing unescaped closing tags that would drop the traling >
  • Corrected the javadoc for Element#child() to note that it throws IndexOutOfBounds
  • When cloning an Element, reset the classnames set so as not to hold a pointer to the source's
  • Limit how far up the stack the formatting adoption agency algorithm will travel, to prevent the chance of a run-away parse when the HTML stack is hopelessly deep
  • Modified Element.text() to build text by traversing child nodes rather than recursing. This avoids stack-overflow errors when the DOM is very deep and the VM stack-size is low

New in jsoup 1.7.1 (Jan 30, 2013)

  • Improved parse time, now 2.3x faster than previous release, with lower memory consumption.
  • Reduced memory consumption when selecting elements.
  • Introduced finer granularity of exceptions in Jsoup.connect, including HttpStatusException and
  • UnsupportedMimeTypeException.
  • Fixed an issue when determining the Windows-1254 character-set from a meta tag when run in the Turkish locale.
  • Fixed whitespace preservation in tags.
  • In jsoup.connect, fail faster if the return content type is not supported.
  • In jsoup.clean, allow custom OutputSettings, to control pretty printing, character set, and entity escaping.
  • Fixed an issue that prevented frameset documents to be cleaned by the Cleaner.
  • Fixed an issue when normalising whitespace for strings containing high-surrogate characters.
  • If a server doesn't specify a content-type header, treat that as OK.
  • If a server returns an unsupported character-set header, attempt to decode the content with the default charset UTF8), instead of bailing with an unsupported charset exception.
  • Removed an unnecessary synchronisation in Tag.valueOf, allowing multi-threaded parsing to run faster.
  • Made entity decoding less greedy, so that non-entities are less likely to be incorrectly treated as entities.
  • Whitespace normalise document.title() output.
  • In Jsoup.connection, enforce a connection disconnect after every connect. This precludes keep-alive connections to the same host, but in practise many implementations will leak connections, particularly on error.

New in jsoup 1.6.3 (Jan 30, 2013)

  • Fixed parsing of group-or commas in CSS selectors, to correctly handle sub-queries containing commas.
  • If a node has no parent, return null on previousSibling and nextSibling instead of throwing a null pointer exception.
  • Updated Node.siblingNodes() and Element.siblingElements() to exclude the current node (a node is not its own sibling).
  • Fixed HTML entity parser to correctly parse entities like frac14 (letter + number combo).
  • Fixed issue where contents of a script tag within a comment could be incorrectly parsed.
  • Fixed GAE support: load HTML entities from a file on startup, instead of embedding in the class.
  • Fixed NPE when HTML fragment parsing a tag
  • Fixed issue with :all pseudo-tag in HTML sanitizer when cleaning tags previously defined in whitelist
  • Fixed NPE in Parser.parseFragment() when context parameter is null.
  • In HTML whitelists, when defining allowed attributes for a tag, automatically add the tag to the allowed list.

New in jsoup 1.6.2 (Mar 30, 2012)

  • Added a simplified XML parsing mode, which can usefully parse valid and invalid XML, but does not enforce any HTML document structure or special tag behaviour.
  • Added the optional ability to track errors when tokenising and parsing.
  • Added jsoup.connect.cookies(Map) method, to set multiple cookies at once, possibly from a prior request.
  • Added Element.textNodes() and Element.dataNodes(), to easily access an element's children text nodes and data nodes.
  • Added an example program that demonstrates how to format HTML as plain-text, and the use of the NodeVisitor interface.
  • Added Node.traverse() and Elements.traverse() methods, to iterate through a node's descendants.
  • Updated jsoup.connect so that when requests made as POSTs are redirected, the redirect is followed as a GET.
  • Updated the Cleaner and whitelists to optionally preserve related links in elements, instead of converting them to absolute links.
  • Updated the Cleaner to support custom allowed protocols such as "cid:" and "data:".
  • Updated handling of tags, to act on only the first one seen when parsing, to align with modern browsers.
  • Updated Node.setBaseUri(), to recursively set on all the node's descendants.
  • Fixed handling of null characters within comments.
  • Tweaked escaped entity detection in attributes to not treat &entity_... as an entity form.
  • Fixed doctype tokeniser to allow whitespace between name and public identifier.
  • Fixed issue where comments within a table tag would be duplicate-fostered into body.
  • Fixed an issue where a spurious byte-order-mark at the start of a document would cause the parser to miss head contents.
  • Fixed an issue where content after a frameset could cause a NPE crash. Now correctly implements spec and ignores the trailing content.
  • Tweaked whitespace checks to align with HTML spec
  • Tweaked HTML output of closing script and style tags to not add an extraneous newline when pretty-printing.
  • Substantially reduced default memory allocation within Node.outerHtml, to reduce memory pressure when serialising smaller DOMs.

New in jsoup 1.6.1 (Jul 2, 2011)

  • Fixed Java 1.5 compatibility.

New in jsoup 1.6.0 (Jun 13, 2011)

  • HTML5 conformant parser. Complete reimplemenation of HTML tokenisation and parsing. This ensures jsoup parses HTML identically to current modern browsers.
  • When parsing files from disk, files are loaded via memory mapping, to increase parse speed.
  • Reduced memory overhead and lowered garbage collector pressure with Attribute, Node and Entity model optimisations.
  • Improved "abs:" absolute URL handling in Elements.attr("abs:href") and Node.hasAttr("abs:href").
  • Fixed cookie handling issue in jsoup.Connect where empty cookies would cause a validation exception.
  • Added jsoup.Connect configuration options to allow HTTP errors to be ignored, and the content-type to be ignored. Contributed by Jesse Piascik (piascikj)
  • Added Node.before(node) and Node.after(node), to allow existing nodes to be moved, or new nodes to be inserted, into precise DOM positions.
  • Added Node.unwrap() and Elements.unwrap(), to remove a node but keep its contents. Useful for e.g. removing unwanted formatting tags.
  • Now handles unclosed tags in document by breaking out of the title at the next start tag, instead of eating up to the end of the document.
  • Added OSGi bundle support to the jsoup package jar.

New in jsoup 1.5.2 (Feb 27, 2011)

  • Fixed issue with selector parser where some boolean AND + OR combined queries (e.g. "meta[http-equiv], meta[content]") were being parsed incorrectly as OR only queries (e.g. former as "meta, [http-equiv], meta[content]")
  • Fixed issue where a content-type specified in a meta tag may not be reliably detected, due to the above issue.
  • Updated Element.text() and Element.ownText() methods to ensure tags output as whitespace.
  • Tweaked Element.outerHtml() method to not generate initial newline on first output element.

New in jsoup 1.4.1 (Nov 24, 2010)

  • Added ability to load and parse HTML from an input stream.
  • Implemented Node.clone() to create deep, independent copies of Nodes, Elements, and Documents.
  • Added :not() selector, to find elements that do not match the selector. E.g. div:not(.logo) finds divs that do not have the "logo" class name.
  • Added Elements.not(selector) method, to remove undesired results from selector results.
  • Implemented DataNode.setWholeData() to allow updating of script and style data contents.
  • Relaxed parse rules of H1 - H6, to allow nested content. This is against spec, but matches browser and publisher behaviour.
  • Relaxed parse rule of SPAN to treat as block, to allow nested block content.
  • Fixed issue in jsoup.connect when extracting character set from content-type header; now supports quoted charset declaration.
  • Fixed support for jsoup.connect to follow redirects between http & https URLs.
  • Document normalisation now more enthusiastically enforces the correct document structure.
  • Support node.outerHtml() method when node has no parent (e.g. when it has been removed from its DOM tree)
  • Fixed support for HTML entities with numbers in name (e.g. ¾, ¹). Fixed absolute URL generation from relative URLs which are only query strings.

New in jsoup 1.3.3 (Nov 24, 2010)

  • Implemented Elements.empty() and Elements.remove(). This allows easy element removal
  • Fixed issue in Entities when unescaping $
  • Added restricted XHTML output entity option

New in jsoup 1.3.2 (Nov 24, 2010)

  • Treat HTTP headers as case insensitive in Jsoup.Connection. Improves compatibility for HTTP responses.
  • Improved malformed table parsing by implementing ignorable end tags.

New in jsoup 1.3.1 (Nov 24, 2010)

  • Removed dependency on Apache Commons-lang. Jsoup now has no external dependencies.
  • Added new Connection implementation, to enable easier and richer HTTP requests that parse to Documents. This includes support for gzip responses, cookies, headers, data parameters, user-agent, referrer, etc.
  • Added Element.ownText() method, to get only the direct text of an element, not including the text of its children.
  • Added support for selectors :containsOwn(text) and :matchesOwn(regex), to supplement Element.ownText().
  • Added support for non-pretty-printed HTML output, to more closely mirror the input HTML.
  • Further speed optimisations for parsing and output generation.
  • Fixed support for case-sensitive HTML escape entities.
  • Fixed issue when parsing tags with keyless attributes.

New in jsoup 1.2.3 (Nov 24, 2010)

  • Added support for automatic input character set detection and decoding. Jsoup now automatically detects the encoding character set when parsing HTML from a File or URL. The parser checks the content-type header, then the or tag, and finally falls back to UTF-8.
  • Added ability to configure the document's output charset, to control which characters are HTML escaped, and which are kept intact. The output charset defaults to the document's input charset. This simplifies non-ascii output.
  • Added full support for all new HTML5 tags.
  • Added support for HTML5 dataset custom data attributes, with the Element.dataset() map.
  • Added support for the [^attributePrefix] selector query, to find elements with attributes starting with a prefix. Useful for finding elements with datasets: [^data-] matches
  • Added support for namespaced elements () and selectors to find them (fb|name) * Implemented Node.ownerDocument DOM method
  • Improved implicit table element handling (particularly around thead, tbody, and tfoot).
  • Improved HTML output format for empty elements and auto-detected self closing tags
  • Changed DT & DD tags to block-mode tags, to follow practice over spec
  • Added support for tag names with - and _ (, )
  • Handle tags with internal trailing space ()
  • Fixed support for character class regular expressions in [attr=~regex] selector

New in jsoup 1.2.2 (Nov 24, 2010)

  • Performance optimisation:
  • core HTML parser engine now 3.5 times faster
  • HTML generator now 2.5 times faster
  • much lower memory use and garbage collection time
  • Added support for :matches(regex) selector, to find elements containing text matching regular expression
  • Added support for [key~=regex] attribute selector, to find elements with attribute values matching regular expression
  • Upgraded the selector query parser to allow nested selectors like 'div:has(p:matches(regex))'

New in jsoup 1.2.1 (Nov 24, 2010)

  • Added .before(html) and .after(html) methods to Element and Elements, to insert sibling HTML
  • Added :contains(text) selector, to search for elements containing the specified text
  • Added :has(selector) pseudo-selector
  • Added Element#parents and Elements#parents to retrieve an element's ancestor chain
  • Fixes an issue where appending / prepending rows to a table (or to similar implicit element structures) would create a redundant wrapping elements
  • Improved implicit close tag heuristic detection when parsing malformed HTML
  • Fixes an issue where text content after a script (or other data-node) was incorrectly added to the data node.
  • Fixes an issue where text order was incorrect when parsing pre-document HTML.

New in jsoup 1.1.1 (Nov 24, 2010)

  • Added selector support for :eq, :lt, and :gt
  • Added TextNode#text and TextNode#text(String)
  • Throw exception if trying to parse non-text content
  • Added Node#remove and Node#replaceWith
  • Allow _ and - in CSS ID selectors (per CSS spec).
  • Relative links are resolved to absolute when cleaning, to normalize output and to verify safe protocol. (Were previously discarded.)
  • Allow combinators at start of selector query, for query refinements
  • Added Element#val() and #val(String) methods, for form values
  • Changed textarea contents to parse as TextNodes, not DataNodes, so contents visible to text() (and val(), as treated as form input)
  • Fixed support for Java 1.5

New in jsoup 0.3.1 (Nov 24, 2010)

  • New features: supports Elements#html(), html(String), prepend(String), append(String); bulk methods for corresponding methods in Element.
  • New feature: Jsoup.isValid(html, whitelist) method for user input form validation.
  • Improved Elements.attr(String) to find first matching element with attribute. Fixed assertion error when cleaning HTML with empty attribute

New in jsoup 0.2.2 (Nov 24, 2010)

  • jsoup packages are now available in the Maven central repository.
  • New feature: supports Element#addClass, removeClass, toggleClass;also collection class methods on Elements.
  • New feature: supports Element#wrap(html) and Elements#wrap(html).
  • New selector syntax: supports E + F adjacent sibling selector
  • New selector systax: supports E ~ F preceding sibling selector
  • New: supports Element#elementSiblingIndex()
  • Improved document normalisation.
  • Improved HTML string output format (pretty-print)
  • Fixed absolute URL resolution issue when a base tag has no href.

New in jsoup 0.1.2 (Nov 24, 2010)

  • Fixed unrecognised tag handler to be more permissive