Quick overview
This workflow runs weekly (or on demand) to crawl your sitemap, extract and deduplicate links from your pages, check each link’s HTTP response without following redirects, and post a summary of new broken links and redirect chains to Slack while tracking findings in an n8n Data Table.
How it works
- Runs every Monday on a schedule or starts manually, then loads the sitemap URL and crawl limits from the workflow settings.
- Fetches the sitemap XML and parses it as either a sitemap index (child sitemaps) or a flat sitemap (page URLs).
- Selects and caps the page URLs to crawl, then fetches each page’s HTML in batches.
- Extracts all anchor links from the fetched pages, resolves relative URLs, filters unwanted link types, and deduplicates to a capped list of distinct internal and (optionally) external URLs.
- Requests each distinct URL once using HTTP GET with redirects disabled, then classifies results as ok, blocked, broken, or redirected-to-a-different-target.
- Compares broken/redirected findings against previously reported items stored in an n8n Data Table and keeps only newly changed problems.
- Posts a formatted report with counts and new findings to a Slack channel and upserts the newly reported problems back into the Data Table.
Setup
- Add a Slack credential and set the target channel name in the workflow settings.
- Create an n8n Data Table named
link_rot with columns link_key, url, status, verdict, found_on, and last_seen.
- Update the sitemap URL (and optionally
max_pages, max_links, url_must_contain, and check_external) in the workflow settings to match your site and crawl scope.
Requirements
- A Slack credential. Nothing else - no API keys, no model, no scraping service.
Customization
- check_external in Settings decides whether links to other sites are checked. Leaving it on is the point: your own pages rarely rot, the sites you linked to three years ago do.