@sdxc/sitemap

Read and write the sitemap protocol: collect URLs into a document, or fetch a published sitemap and read the pages it lists.

Writing one is the well-known half — a handler gathers its routes and serves <urlset> XML for a crawler. Reading one is the half that usually has no home, and it is the more useful of the two: a published sitemap is the cheapest inventory of a site there is. It is one request, it is maintained by the site's own author, and it carries the modification dates that say which of those pages are worth looking at again.

Both directions answer with the same class, so the element names and the value rules have one implementation. What append accepts is what an entry reports back, and a document that is read, filtered and re-serialized comes out as the document type it went in as.

Installation

npm add @sdxc/sitemap

Parsing and retrieval report their outcome as a Result from @sdxc/result, which is where isFailure and isSuccess come from. Sitemap.parse takes a document from @sdxc/xml. Both install alongside this package.

Usage

Build A Sitemap

import { Sitemap } from "@sdxc/sitemap";

let sitemap = new Sitemap();

sitemap.append(new URL("https://example.com/"), { priority: 1, frequency: "weekly" });
sitemap.append(new URL("https://example.com/about"), { priority: 0.5, frequency: "yearly" });
sitemap.append(new URL("https://example.com/blog/hello"), { updatedAt: new Date("2026-09-11") });

sitemap.toString();
// <?xml version="1.0" encoding="UTF-8"?>
// <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">…</urlset>

updatedAt is written as <lastmod> and frequency as <changefreq>, the element names the protocol expects.

Read A Published Sitemap

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let result = await Sitemap.fetch("https://example.com/sitemap.xml");
if (isFailure(result)) throw result.error;

let sitemap = result.data;

sitemap.kind; // "urlset"
sitemap.size; // 128

for (let entry of sitemap.entries) {
	entry.loc; // URL
	entry.updatedAt; // Date | undefined
	entry.frequency; // Sitemap.Frequency | undefined
	entry.priority; // number | undefined
}

Sitemap.fetch reads the body whenever the response is ok, and Content-Type plays no part in the decision: sitemaps arrive as application/xml, as text/xml and as text/plain. The root element is the check that answers whether one arrived.

Parse A Document You Already Hold

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";
import { XML } from "@sdxc/xml";

let xml = XML.parse(source);
if (isFailure(xml)) throw xml.error;

let sitemap = Sitemap.parse(xml.data);

Two steps that each say what they do, and an XMLParseError that arrives from the layer that produced it.

Walk An Index

A large site splits into one sitemap per section and lists them in a <sitemapindex>. That document reads into the same class, under a kind of "index":

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let index = await Sitemap.fetch("https://example.com/sitemap.xml");
if (isFailure(index)) throw index.error;

if (index.data.kind === "index") {
	for (let entry of index.data.entries) {
		let section = await Sitemap.fetch(entry.loc);
		if (isFailure(section)) continue;
		// every entry of section.data is a page
	}
}

One call retrieves one document, so how many children to fetch and how fast stays yours to decide.

Parsing Rules

The root element decides whether a sitemap arrived. A root of <urlset> or <sitemapindex> sets kind, and any other root is a SitemapParseError naming the root it found. That is also the content check: a host that answers an unknown path with its own not-found template under a 200 fails here, with a message saying the root was html.

An entry needs a <loc> that parses as an absolute URL. An entry without one is skipped and the rest of the document is kept, because one malformed row should not cost you the other 49,999.

A value the protocol refuses costs its own field, not the entry. An unreadable <lastmod> leaves updatedAt undefined, a <changefreq> outside the protocol's enum leaves frequency undefined, and a <priority> outside 0.0–1.0 leaves priority undefined. What you read back is what append accepts.

<lastmod> is W3C Datetime, so a date alone (2026-09-11) and a full timestamp (2026-09-11T14:32:00+02:00) both read.

Extension namespaces are dropped. image:, video: and news: children, and xhtml:link alternates, have no vocabulary in Sitemap.Entry. Parse the document with @sdxc/xml and read those elements yourself when you need them.

API

new Sitemap()

An empty document of kind "urlset".

sitemap.append(loc: URL, options?: Sitemap.AppendOptions): void

Adds one entry. loc is a URL because <loc> is absolute. options carries the crawler metadata the protocol allows: updatedAt, frequency and priority.

sitemap.toString(): string

Serializes the entries as the document type this instance carries — <urlset> for one built by appending, and whichever root a parsed instance was read from. An empty document serializes to a self-closing root.

Sitemap.parse(xml: XML): Result<Sitemap, SitemapParseError>

Reads a parsed document as a sitemap, taking an XML instance rather than source text. The root element decides what arrived; see Parsing Rules.

Sitemap.fetch(input, init?): Promise<Result<Sitemap, SitemapFetchError | SitemapParseError | XMLParseError>>

Retrieves one sitemap and parses it. Takes what fetch takes — a URL, a string or a Request, plus request options — and reports a refused request or an error status as a SitemapFetchError naming what came back.

await Sitemap.fetch("https://example.com/sitemap.xml", {
	headers: { "User-Agent": "example-crawler" },
});

A .xml.gz sitemap served as application/gzip carries a gzip member in the body. Pipe the response through a DecompressionStream("gzip") and hand the text to Sitemap.parse.

sitemap.kind: Sitemap.Kind

Which document this instance carries: "urlset" lists pages, "index" lists other sitemaps. A sitemap built by appending is always a "urlset".

sitemap.entries: Set<Sitemap.Entry>

The entries themselves, as the live set, whether they were appended or parsed.

sitemap.size: number

How many entries the document holds.

Errors

SitemapParseError

A document is not a sitemap. The message names the root element that arrived, so an error page served under a 200 reports itself as received <html>.

SitemapFetchError

A sitemap could not be retrieved: the request was refused, or the response carried an error status. The message names what came back.

Types

Types live under the Sitemap namespace.

type Kind = "urlset" | "index";

type Frequency = "always" | "hourly" | "daily" | "weekly" | "monthly" | "yearly" | "never";

interface Entry {
	loc: URL;
	updatedAt?: Date;
	frequency?: Frequency;
	priority?: number;
}

interface AppendOptions {
	updatedAt?: Date;
	frequency?: Frequency;
	/** Priority value between 0.0 and 1.0, default is 0.5 */
	priority?: number;
}

Pattern: Serving A Sitemap

The document is a string, so serving it is a Response with the content type a crawler expects. Resolving every entry against the request's own URL makes one handler serve production and every preview deployment:

import { Sitemap } from "@sdxc/sitemap";

export async function GET(request: Request) {
	let base = new URL(request.url);
	let sitemap = new Sitemap();

	sitemap.append(new URL("/", base), { priority: 1, frequency: "weekly" });

	for (let post of await listPosts()) {
		sitemap.append(new URL(`/blog/${post.slug}`, base), { updatedAt: post.updatedAt });
	}

	return new Response(sitemap.toString(), {
		headers: {
			"Content-Type": "application/xml; charset=utf-8",
			"Cache-Control": "public, max-age=3600",
		},
	});
}

Pattern: Auditing A Site Against Its Own Sitemap

A sitemap is one request for the list of pages a site claims to publish, which makes it the starting point for a crawl that checks those pages actually answer:

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let result = await Sitemap.fetch("https://example.com/sitemap.xml");
if (isFailure(result)) throw result.error;

let missing: URL[] = [];

for (let entry of result.data.entries) {
	let response = await fetch(entry.loc, { method: "HEAD" });
	if (!response.ok) missing.push(entry.loc);
}

Filtering by updatedAt first turns the same loop into a recrawl that visits only what has changed since the last run.

Pattern: Writing A Sitemap Index

A site past 50,000 URLs splits into one sitemap per section, listed by an index document. Building one from scratch means writing the <sitemapindex> root yourself, since a document you construct is a <urlset>:

import { unwrap } from "@sdxc/result";
import { XML } from "@sdxc/xml";

const SECTIONS = ["pages", "blog", "products"];

let source = unwrap(
	XML.stringify({
		declaration: { version: "1.0", encoding: "UTF-8" },
		root: {
			name: "sitemapindex",
			attributes: { xmlns: "http://www.sitemaps.org/schemas/sitemap/0.9" },
			children: SECTIONS.map((section) => ({
				name: "sitemap",
				children: [{ name: "loc", children: [`https://example.com/sitemap-${section}.xml`] }],
			})),
		},
	}),
);

An index that came from Sitemap.parse needs none of this: filter its entries and call toString(), and it is written back as a <sitemapindex>.