See llms.txt for all machine-readable content.

Back to Templates

Scrape NSR specialist data and log results to Google Sheets

Created by

Created by: Jeammy Rodjas || jeammyrodjas
Jeammy Rodjas

Last update

Last update 9 hours ago

Categories

Quick overview

This workflow pulls website URLs from Google Sheets, scrapes each site via the Scrape.do API, extracts and filters email addresses from the page content, and writes the best matching email back to the same Google Sheet.

How it works

  1. Starts when you manually execute the workflow in n8n.
  2. Reads rows from a Google Sheets spreadsheet that contains a “Websites” column with URLs to process.
  3. Iterates through the URLs in batches and sends each one to the Scrape.do API to fetch the page content.
  4. Parses the returned content to find email addresses, removes duplicates and common junk/platform domains, and prefers emails that match the scraped site’s domain.
  5. Appends or updates the corresponding row in Google Sheets by matching on “Websites” and writing the selected email into the “Emails” column.

Setup

  1. Create a Google Sheets OAuth2 connection in n8n and update the Spreadsheet ID and sheet/tab selection if you want to use a different file.
  2. Ensure your Google Sheet has a “Websites” column with full URLs and an “Emails” column to store the results.
  3. Add your Scrape.do API token to the HTTP request query parameter named “token” in the Scrape.do request.
  4. (Optional) Adjust the blocked email domains list and filtering rules in the JavaScript code if you want to allow common providers like Gmail or Outlook.

Requirements

  • n8n instance (self-hosted or cloud)
  • Google Sheets account with a spreadsheet ready (headers: Name, License, Specialty 1, Specialty 2, Gender, Grad Year)
  • Google Sheets credential connected in n8n
  • Internet access to nsr.org.my (no auth required, public register)

Customization

  • Target specialty (currently Cardiology, code 302) — swap via numeric specialty code from NSR's search form
  • Output spreadsheet/sheet destination
  • Rate-limit delay between requests (default 0.7s)
  • Dedupe key for re-runs (default: License / NSR No)

Additional info

Scrapes Malaysia's National Specialist Register (nsr.org.my) for a chosen medical specialty.
Runs in 2 phases: paginates all result pages to collect the doctor list, then fetches each
doctor's detail page for Gender/Specialty/Grad Year. Handles session-cookie-based filtering
(NSR has no URL-based filter). ~10-15 min runtime for ~460 records.