Quick Overview
This workflow monitors a list of competitor web pages daily using ScraperAPI, detects meaningful content changes, summarizes them with Anthropic Claude, stores the latest snapshots in an n8n Data Table, and sends a daily digest to Slack and email (Gmail or SMTP).
How it works
- Runs on a daily schedule at 8am (and can be run manually to initialize the storage table).
- Builds a list of competitor URLs to monitor, including the ScraperAPI proxy country code for each target.
- Iterates through the URLs, scrapes each page via ScraperAPI, cleans/truncates the markdown content, and computes a hash for change detection.
- Looks up the previous snapshot for each URL in an n8n Data Table and determines whether the content has materially changed.
- For changed pages, sends the previous and current snapshots to Anthropic Claude to generate a short change summary and upserts the updated snapshot and summary back into the Data Table.
- For unchanged pages, upserts the latest “last seen” timestamp and hash into the Data Table.
- After processing, fetches today’s changed summaries from the Data Table, assembles a daily digest, and posts it to Slack and sends it via Gmail and/or SMTP.
Setup
- Add a ScraperAPI credential and ensure it is selected in the ScraperAPI node.
- Add an Anthropic credential for the Claude chat model used by the AI agent.
- Configure the competitor URL list (and optional
countryCode) in the “Prepare URL List” code step.
- Add Slack credentials and set the target channel for posting the digest.
- Configure email delivery by setting up either Gmail credentials and the recipient address, or SMTP settings (from/to addresses and SMTP credential details).