Parsing HTML

suggest change

HTML is not XML. Real pages have unclosed <p> and <li> tags, wrongly nested elements, unquoted attributes and stray end tags. XML parsers such as JAXP (see XML Parsing using the JAXP APIs) reject that input, and regular expressions break on it. The JDK has no HTML parser, so the standard choice is jsoup. It implements the WHATWG HTML5 parsing algorithm, the same one browsers use, so it builds the same tree a browser would. It then lets you query that tree with CSS selectors or XPath, change it, and write it back out.

This article is about parsing HTML you already have: a string, a file, or an HTTP response body. Downloading pages, cookies and logins are covered in Web scraping.

Add jsoup to your build:

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.23.2</version>
</dependency>

Gradle: implementation("org.jsoup:jsoup:1.23.2"). jsoup has no dependencies of its own. Every example below is a single file you can run with java -cp jsoup-1.23.2.jar Example.java. The output shown comes from JDK 25.

Parsing a string

Jsoup.parse(String) returns a Document. It never throws on bad markup. Instead it fixes the markup the way a browser would:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;

public class ParseBasics {
    public static void main(String[] args) {
        // Broken HTML: no <html>/<head>/<body>, unclosed <p> and <li>, stray </div>
        String html = """
            <title>Shop</title>
            <h1>Products</h1>
            <p>First paragraph
            <p>Second <b>bold <i>both</b> italic?</i>
            <ul><li>One<li>Two</ul>
            </div>
            """;
        Document doc = Jsoup.parse(html);
        System.out.println("Title: " + doc.title());
        System.out.println(doc.html());
    }
}

Output:

Title: Shop
<html>
 <head>
  <title>Shop</title>
 </head>
 <body>
  <h1>Products</h1>
  <p>First paragraph</p>
  <p>Second <b>bold <i>both</i></b><i> italic?</i></p>
  <ul>
   <li>One</li>
   <li>Two</li>
  </ul>
 </body>
</html>

The parser added the <html>, <head> and <body> elements, moved <title> into <head>, closed the paragraphs and list items, dropped the stray </div>, and split the overlapping <b>/<i> into a properly nested tree.

Other ways to get a Document:

Method Use it for
Jsoup.parse(html) A full page as a string
Jsoup.parse(html, baseUri) Same, with a base URI for resolving relative links
Jsoup.parseBodyFragment(html) A snippet such as a user comment. It goes into <body>
Jsoup.parse(path, charset) A file (Path or File). Pass null as charset to detect it
Jsoup.parse(inputStream, charset, baseUri) An HTTP response or other byte stream
Jsoup.parse(html, baseUri, Parser.xmlParser()) XML, or HTML that must not be fixed

Getting text, HTML and attributes

After parsing, you work with Element objects. These are the methods you will use most:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

public class Navigate {
    public static void main(String[] args) {
        String html = """
            <div id="post" class="article featured" data-id="42">
              <h2>Parsing   HTML</h2>
              <p>Use <a href="/docs/jsoup">jsoup</a> &amp; CSS selectors.<br>It is easy.</p>
              <img src="img/logo.png" alt="Logo">
            </div>
            """;
        // The base URI is used to resolve relative links
        Document doc = Jsoup.parse(html, "https://example.com/blog/");

        Element post = doc.getElementById("post");
        System.out.println("id attr:    " + post.id());
        System.out.println("classes:    " + post.classNames());
        System.out.println("hasClass:   " + post.hasClass("featured"));
        System.out.println("data-id:    " + post.attr("data-id"));
        System.out.println("dataset:    " + post.dataset());
        System.out.println("missing:    '" + post.attr("nope") + "'");

        Element p = post.selectFirst("p");
        System.out.println("text():     " + p.text());
        System.out.println("ownText():  " + p.ownText());
        System.out.println("wholeText():" + p.wholeText());
        System.out.println("html():     " + p.html());
        System.out.println("outerHtml():" + post.selectFirst("h2").outerHtml());

        Element a = p.selectFirst("a");
        System.out.println("href:       " + a.attr("href"));
        System.out.println("abs:href:   " + a.attr("abs:href"));
        System.out.println("absUrl src: " + post.selectFirst("img").absUrl("src"));

        System.out.println("children:   " + post.children().size());
        System.out.println("parent:     " + a.parent().tagName());
        System.out.println("next sib:   " + post.child(0).nextElementSibling().tagName());
    }
}

Output:

id attr:    post
classes:    [article, featured]
hasClass:   true
data-id:    42
dataset:    {id=42}
missing:    ''
text():     Use jsoup & CSS selectors. It is easy.
ownText():  Use & CSS selectors. It is easy.
wholeText():Use jsoup & CSS selectors.
It is easy.
html():     Use <a href="/docs/jsoup">jsoup</a> &amp; CSS selectors.
<br>
It is easy.
outerHtml():<h2>Parsing HTML</h2>
href:       /docs/jsoup
abs:href:   https://example.com/docs/jsoup
absUrl src: https://example.com/blog/img/logo.png
children:   3
parent:     p
next sib:   p

Some things to know:

Selecting elements with CSS selectors

select(css) returns Elements, which extends ArrayList<Element> and adds helper methods. selectFirst(css) returns the first match or null. jsoup supports the usual CSS selectors and adds some of its own, such as :contains and :matches:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class Selectors {
    public static void main(String[] args) {
        String html = """
            <ul id="menu">
              <li><a href="/">Home</a></li>
              <li class="active"><a href="/docs">Docs</a></li>
              <li><a href="https://github.com/jhy/jsoup" rel="external">GitHub</a></li>
              <li><a href="mailto:me@example.com">Contact</a></li>
            </ul>
            <div class="card"><h3>Java</h3><span class="price">$10</span></div>
            <div class="card"><h3>Kotlin</h3></div>
            <div class="card sale"><h3>Go</h3><span class="price">$7</span></div>
            """;
        Document doc = Jsoup.parse(html);

        print(doc, "#menu > li.active a");
        print(doc, "a[href^=http]");
        print(doc, "a[href~=(?i)^mailto:]");
        print(doc, "li:nth-child(odd) a");
        print(doc, "li:last-child");
        print(doc, "a:contains(git)");
        print(doc, "div.card:has(.price) h3");
        print(doc, "div.card:not(:has(.price)) h3");
        print(doc, "h3:matchesOwn(^[JK])");
        print(doc, ".card.sale, li.active");

        // selectFirst returns null when nothing matches
        Element missing = doc.selectFirst("table");
        System.out.println("selectFirst(table) = " + missing);
        // expectFirst throws instead
        try {
            doc.expectFirst("table");
        } catch (IllegalArgumentException e) {
            System.out.println("expectFirst: " + e.getMessage());
        }

        // Elements is a List<Element> with helpers
        Elements links = doc.select("#menu a");
        System.out.println("eachText: " + links.eachText());
        System.out.println("eachAttr: " + links.eachAttr("href"));
    }

    static void print(Document doc, String css) {
        System.out.printf("%-32s -> %s%n", css, doc.select(css).eachText());
    }
}

Output:

#menu > li.active a              -> [Docs]
a[href^=http]                    -> [GitHub]
a[href~=(?i)^mailto:]            -> [Contact]
li:nth-child(odd) a              -> [Home, GitHub]
li:last-child                    -> [Contact]
a:contains(git)                  -> [GitHub]
div.card:has(.price) h3          -> [Java, Go]
div.card:not(:has(.price)) h3    -> [Kotlin]
h3:matchesOwn(^[JK])             -> [Java, Kotlin]
.card.sale, li.active            -> [Docs, Go $7]
selectFirst(table) = null
expectFirst: No elements matched the query 'table' in the document.
eachText: [Home, Docs, GitHub, Contact]
eachAttr: [/, /docs, https://github.com/jhy/jsoup, mailto:me@example.com]

A quick reference:

Selector Matches
tag, #id, .class By tag name, id or class
a b, a > b, a + b, a ~ b Descendant, child, next sibling, any later sibling
[attr], [attr=v] Has the attribute, or has it with an exact value
[attr^=v], [attr$=v], [attr*=v] Value starts with, ends with, or contains v
[attr~=regex] Value matches a regular expression
:nth-child(an+b), :first-child, :last-child, :eq(n) By position (:eq counts from 0)
:has(sel), :not(sel), :is(sel) Contains a match, doesn't match, matches any of
:contains(text), :containsOwn(text) Text contains the string, ignoring case
:matches(regex), :matchesOwn(regex) Text matches a regular expression

Results always come back in document order, even when you combine selectors with a comma.

selectFirst returns null when nothing matches. If a missing element means the page has changed and your code can't go on, use expectFirst. It throws an exception with a clear message, so you don't get a NullPointerException later on.

Extracting a table into records

The most common job is to turn repeating markup into Java objects. Select the rows, map each one to a record, and convert values as you go:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import java.math.BigDecimal;
import java.util.List;

public class Table {
    record Book(String title, String author, BigDecimal price, String url) {}

    public static void main(String[] args) {
        String html = """
            <table id="books">
              <thead><tr><th>Title</th><th>Author</th><th>Price</th></tr></thead>
              <tbody>
                <tr><td><a href="/b/1">Effective Java</a></td><td>Joshua Bloch</td><td>$45.99</td></tr>
                <tr><td><a href="/b/2">Java Concurrency in Practice</a></td><td>Brian Goetz</td><td>$39.50</td></tr>
                <tr class="ad"><td colspan="3">Sponsored</td></tr>
                <tr><td><a href="/b/3">Modern Java in Action</a></td><td>Raoul-Gabriel Urma</td><td>$49.00</td></tr>
              </tbody>
            </table>
            """;
        Document doc = Jsoup.parse(html, "https://shop.example.com/");

        List<Book> books = doc.select("#books tbody tr:not(.ad)").stream()
            .map(Table::toBook)
            .toList();
        books.forEach(System.out::println);

        BigDecimal total = books.stream().map(Book::price).reduce(BigDecimal.ZERO, BigDecimal::add);
        System.out.println("Total: " + total);
    }

    static Book toBook(Element row) {
        Element link = row.expectFirst("td:eq(0) a");
        String author = row.child(1).text();
        String price = row.child(2).text().replace("$", "");
        return new Book(link.text(), author, new BigDecimal(price), link.absUrl("href"));
    }
}

Output:

Book[title=Effective Java, author=Joshua Bloch, price=45.99, url=https://shop.example.com/b/1]
Book[title=Java Concurrency in Practice, author=Brian Goetz, price=39.50, url=https://shop.example.com/b/2]
Book[title=Modern Java in Action, author=Raoul-Gabriel Urma, price=49.00, url=https://shop.example.com/b/3]
Total: 134.49

tr:not(.ad) skips the ad row. Elements is a List, so stream() and the other collection methods work on it. Use BigDecimal for prices instead of double (see BigDecimal).

Selecting with XPath

When a CSS selector can't express what you need, such as "the <dd> right after the <dt> that says License", use selectXpath:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.TextNode;

public class XPath {
    public static void main(String[] args) {
        String html = """
            <dl>
              <dt>Version</dt><dd>1.23.2</dd>
              <dt>License</dt><dd>MIT</dd>
              <dt>Java</dt><dd>8+</dd>
            </dl>
            """;
        Document doc = Jsoup.parse(html);

        // Find the <dd> that follows the <dt> whose text is "License"
        System.out.println(doc.selectXpath("//dt[text()='License']/following-sibling::dd[1]").text());
        System.out.println(doc.selectXpath("//dd[position() > 1]").eachText());

        // XPath can return text nodes when you ask for a node type
        for (TextNode t : doc.selectXpath("//dt/text()", TextNode.class)) {
            System.out.print(t.text() + " ");
        }
        System.out.println();
    }
}

Output:

MIT
[MIT, 8+]
Version License Java 

selectXpath runs on jsoup's own tree. You don't need to convert to a W3C DOM. The overload that takes a class can return other node types, such as TextNode.

Walking the tree

select covers most needs. When you have to visit every node, for example to build a table of contents or collect all text, there are three tools:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.nodes.Node;
import org.jsoup.nodes.TextNode;
import java.util.Map;
import java.util.TreeMap;
import java.util.stream.Collectors;

public class Traverse {
    public static void main(String[] args) {
        String html = """
            <article>
              <h1>Title</h1>
              <p>Intro with <a href="#a">a link</a>.</p>
              <h2>Part one</h2><p>Text <em>here</em>.</p>
              <h2>Part two</h2><p>More <a href="#b">links</a>.</p>
            </article>
            """;
        Document doc = Jsoup.parse(html);
        Element article = doc.expectFirst("article");

        // 1. Element.stream(): this element and all descendants, depth-first
        Map<String, Long> counts = article.stream()
            .collect(Collectors.groupingBy(Element::tagName, TreeMap::new, Collectors.counting()));
        System.out.println(counts);

        // 2. NodeVisitor sees every node, including text nodes, with its depth
        article.traverse((Node node, int depth) -> {
            if (node instanceof Element e && e.tagName().matches("h[1-6]")) {
                int level = e.tagName().charAt(1) - '0';
                System.out.println("  ".repeat(level - 1) + "- " + e.text());
            }
        });

        // 3. textNodes / nodeStream for text only
        String allText = article.nodeStream(TextNode.class)
            .map(TextNode::text)
            .map(String::strip)
            .filter(s -> !s.isEmpty())
            .collect(Collectors.joining("|"));
        System.out.println(allText);
    }
}

Output:

{a=2, article=1, em=1, h1=1, h2=2, p=3}
- Title
  - Part one
  - Part two
Title|Intro with|a link|.|Part one|Text|here|.|Part two|More|links|.

The difference between Node and Element matters here. An Element is a tag. A Node can also be text (TextNode), a comment (Comment) or script/style content (DataNode). children() returns only child elements, while childNodes() returns all child nodes.

Modifying the document

A parsed document can be changed and written back out. You might strip scripts, rewrite links or add attributes:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

public class Modify {
    public static void main(String[] args) {
        String html = """
            <div id="content">
              <p>Read the <a href="/guide">guide</a>.</p>
              <script>track();</script>
              <p class="ad">Buy now!</p>
              <img src="/a.png">
            </div>
            """;
        Document doc = Jsoup.parse(html, "https://example.com/");

        doc.select("script, .ad").remove();                       // delete elements
        doc.select("a[href]").forEach(a -> a.attr("href", a.absUrl("href"))
                                            .attr("rel", "nofollow"));
        doc.select("img:not([alt])").attr("alt", "");             // bulk attribute set
        doc.select("p").first().addClass("lead");

        Element content = doc.expectFirst("#content");
        content.prependElement("h1").text("Welcome & hello");     // text is escaped
        content.appendElement("footer").html("<small>(c) 2026</small>");
        content.select("img").wrap("<figure></figure>");

        System.out.println(content.outerHtml());

        // Compact, single-line output
        doc.outputSettings().prettyPrint(false);
        System.out.println(content.outerHtml());
    }
}

Output:

<div id="content">
 <h1>Welcome &amp; hello</h1>
 <p class="lead">Read the <a href="https://example.com/guide" rel="nofollow">guide</a>.</p> 
 <figure>
  <img src="/a.png" alt="">
 </figure>
 <footer>
  <small>(c) 2026</small>
 </footer>
</div>
<div id="content"><h1>Welcome &amp; hello</h1>
  <p class="lead">Read the <a href="https://example.com/guide" rel="nofollow">guide</a>.</p>
  
  
  <figure><img src="/a.png" alt=""></figure>
<footer><small>(c) 2026</small></footer></div>

Methods on Elements such as remove(), attr(k, v), addClass() and wrap() apply to every matched element. text(s) escapes its argument, while html(s) parses it as markup, so never pass untrusted input to html(). To change how the output looks, use doc.outputSettings(). prettyPrint(false) keeps the original whitespace (the blank lines are where the removed <script> and .ad paragraph were), indentAmount(n) sets the indentation, and charset(...) and escapeMode(...) control how characters are escaped.

Sanitizing untrusted HTML

If you show user-supplied HTML, such as comments or rich-text editor content, you must remove scripts, event handlers and javascript: URLs, or you have an XSS hole. Jsoup.clean parses the input as a body fragment and keeps only what a Safelist allows:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.safety.Safelist;

public class Clean {
    public static void main(String[] args) {
        String comment = """
            <p onclick="steal()">Nice <b>post</b>!<script>alert(1)</script>
            <a href="javascript:alert(2)">click</a> <a href="https://example.com" target="_blank">site</a>
            <img src="x.png" onerror="alert(3)"><iframe src="evil"></iframe>
            """;

        // Body fragment: parsed as the content of <body>, no <html>/<head> added
        Document frag = Jsoup.parseBodyFragment("<li>one<li>two");
        System.out.println(frag.body().html());
        System.out.println("---");

        System.out.println(Jsoup.clean(comment, Safelist.basic()));
        System.out.println("---");
        System.out.println(Jsoup.clean(comment, Safelist.none()));
        System.out.println("---");

        // Customize a Safelist
        Safelist custom = Safelist.basic()
            .addTags("img")
            .addAttributes("img", "src", "alt")
            .addProtocols("img", "src", "https");
        System.out.println(Jsoup.clean(comment, "https://blog.example.com/", custom));
        System.out.println("---");

        System.out.println("valid? " + Jsoup.isValid(comment, Safelist.basic()));
        System.out.println("valid? " + Jsoup.isValid("<p>Hi <b>there</b></p>", Safelist.basic()));
    }
}

Output:

<li>one</li>
<li>two</li>
---
<p>Nice <b>post</b>! <a rel="nofollow">click</a> <a href="https://example.com" rel="nofollow">site</a> </p>
---
Nice post! click site
---
<p>Nice <b>post</b>! <a rel="nofollow">click</a> <a href="https://example.com" rel="nofollow">site</a> <img src="https://blog.example.com/x.png"></p>
---
valid? false
valid? true

The built-in safelists:

Safelist Allows
none() Only text. All tags are removed
simpleText() b, em, i, strong, u
basic() Simple formatting, lists, quotes, and links with http, https, ftp or mailto. Adds rel="nofollow" to links
basicWithImages() basic() plus img with http/https sources
relaxed() Also tables, headings, div, span and more. Doesn't add nofollow

Notes:

Seeing parse errors

jsoup recovers from errors without telling you. For debugging, or to check markup quality, ask the parser to record what it fixed:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.parser.Parser;

public class Errors {
    public static void main(String[] args) {
        String html = "<p>Unclosed <b>bold<p>Next</i><div attr=\"x>";
        Parser parser = Parser.htmlParser().setTrackErrors(10);
        Document doc = Jsoup.parse(html, "", parser);

        parser.getErrors().forEach(System.out::println);
        System.out.println(doc.body().html());
    }
}

Output:

<1:23>: Unexpected StartTag token [<p>] when in state [InBody]
<1:31>: Unexpected EndTag token [</i>] when in state [InBody]
<1:44>: Unexpectedly reached end of file (EOF) in input state [AttributeValue_doubleQuoted]
<1:44>: Unexpected EOF token [] when in state [InBody]
<p>Unclosed <b>bold</b></p>
<p><b>Next</b></p>

Each error has a line:column position. The <div with an unclosed attribute quote ran to the end of input and was dropped. Notice that <b> was reopened inside the second paragraph, which is what browsers do too.

HTML mode vs XML mode

The HTML parser lowercases tag names and applies HTML rules: <title> goes into <head>, and self-closing syntax like <x/> is ignored for tags that aren't void elements. For XML input, such as RSS feeds, or when you need case-sensitive names, use Parser.xmlParser():

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.parser.Parser;

public class XmlMode {
    public static void main(String[] args) {
        String xml = "<feed><Entry id='1'><title>A</title></Entry><Entry id='2'/></feed>";

        Document asHtml = Jsoup.parse(xml);
        System.out.println(asHtml.select("entry").size() + " " + asHtml.body().html().replace("\n", ""));

        Document asXml = Jsoup.parse(xml, "", Parser.xmlParser());
        System.out.println(asXml.select("Entry").eachAttr("id"));
        System.out.println(asXml.html().replace("\n", ""));
    }
}

Output:

2 <feed> <entry id="1">  <title>A</title> </entry> <entry id="2"></entry></feed>
[1, 2]
<feed><Entry id="1"><title>A</title></Entry><Entry id="2" /></feed>

The XML parser doesn't add <html>/<body>, keeps Entry capitalized, and understands <Entry id='2'/> as an empty element. Unlike a real XML parser, it still accepts malformed input. For strict, schema-aware processing use JAXP or XOM.

Parsing files and character encodings

HTML files on disk come in many encodings. Pass null as the charset and jsoup works it out: it checks for a byte order mark, then a <meta charset> or <meta http-equiv="Content-Type"> tag, and falls back to UTF-8:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;

public class FromFile {
    public static void main(String[] args) throws Exception {
        Path file = Files.createTempFile("page", ".html");
        String html = "<html><head><meta charset=\"windows-1250\"><title>Zażółć gęślą jaźń</title></head></html>";
        Files.write(file, html.getBytes(Charset.forName("windows-1250")));

        // null charset: detect from BOM or <meta charset>, falling back to UTF-8
        Document doc = Jsoup.parse(file, null);
        System.out.println(doc.title());
        System.out.println(doc.charset());

        Files.delete(file);
    }
}

Output (run with -Dstdout.encoding=UTF-8 so the Windows console prints Polish characters):

Zażółć gęślą jaźń
windows-1250

If you know the encoding, pass it, for example Jsoup.parse(path, "ISO-8859-1"). Reading the file into a String yourself with the wrong charset corrupts the text before jsoup sees it, so pass jsoup the bytes (Path, File or InputStream) whenever the encoding isn't known.

Parsing an HttpClient response

You don't have to use Jsoup.connect to fetch pages. If your application already uses java.net.http.HttpClient, request the body as an InputStream and give it to Jsoup.parse. Use the response URI as the base URI, because that's the URL after redirects:

import com.sun.net.httpserver.HttpServer;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.InputStream;
import java.net.InetSocketAddress;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;

public class FromHttp {
    public static void main(String[] args) throws Exception {
        // Local server so the example runs offline
        HttpServer server = HttpServer.create(new InetSocketAddress("localhost", 0), 0);
        server.createContext("/news/", ex -> {
            byte[] body = """
                <html><head><meta charset="utf-8"><title>News</title></head>
                <body><a href="today.html">Today</a> <a href="../archive/">Archive</a></body></html>
                """.getBytes(StandardCharsets.UTF_8);
            ex.getResponseHeaders().set("Content-Type", "text/html; charset=utf-8");
            ex.sendResponseHeaders(200, body.length);
            ex.getResponseBody().write(body);
            ex.close();
        });
        server.start();
        URI uri = URI.create("http://localhost:" + server.getAddress().getPort() + "/news/");

        HttpClient client = HttpClient.newHttpClient();
        HttpResponse<InputStream> resp = client.send(
            HttpRequest.newBuilder(uri).build(), HttpResponse.BodyHandlers.ofInputStream());

        // Parse the byte stream; null charset = sniff from BOM / <meta>, default UTF-8.
        // Pass the final URI as base so relative links resolve correctly.
        try (InputStream in = resp.body()) {
            Document doc = Jsoup.parse(in, null, resp.uri().toString());
            System.out.println(doc.title());
            doc.select("a[href]").forEach(a ->
                System.out.println(a.text() + " -> " + a.absUrl("href")));
        }
        server.stop(0);
    }
}

Output (the port changes from run to run):

News
Today -> http://localhost:57996/news/today.html
Archive -> http://localhost:57996/archive/

Streaming large documents

Jsoup.parse builds the whole tree in memory, which is fine for typical pages of a few MB. For very large files, such as data dumps or huge generated reports, StreamParser (jsoup 1.17+) parses incrementally. It hands you each element as soon as it's complete, and you can remove it once you've processed it:

import org.jsoup.nodes.Element;
import org.jsoup.parser.Parser;
import org.jsoup.parser.StreamParser;
import java.io.IOException;
import java.io.StringReader;

public class Streaming {
    public static void main(String[] args) throws IOException {
        // Imagine this is a multi-gigabyte file read with Files.newBufferedReader(path)
        StringBuilder sb = new StringBuilder("<html><body><table>");
        for (int i = 1; i <= 100_000; i++) {
            sb.append("<tr class=row><td>").append(i).append("</td><td>item ").append(i).append("</td></tr>");
        }
        sb.append("</table></body></html>");

        long sum = 0;
        int rows = 0;
        try (StreamParser streamer = new StreamParser(Parser.htmlParser())) {
            streamer.parse(new StringReader(sb.toString()), "");
            Element row;
            while ((row = streamer.selectNext("tr.row")) != null) {
                sum += Long.parseLong(row.child(0).text());
                rows++;
                row.remove(); // drop the processed row so memory stays flat
            }
        }
        System.out.println("rows=" + rows + " sum=" + sum);
    }
}

Output:

rows=100000 sum=5000050000

selectNext(css) returns the next complete element that matches, or null at the end. Calling row.remove() after each row keeps memory use flat no matter how big the input is. StreamParser also works with Parser.xmlParser(). Its stream() and iterator() methods give you every element as it is completed.

Tips

Feedback about page:

Feedback:
Optional: your email if you want me to get back to you:



Table Of Contents