HTTP Request and uProc integration

n8n lets you integrate HTTP Request with uProc to build powerful workflows. Design automation that extracts, transforms and loads data between your apps and services. You can choose from thousands of ready-made apps or use our universal HTTP connector to sync apps not yet in our library.

Get started

See n8n in action

Build your own HTTP Request and uProc integration

Create custom HTTP Request and uProc workflows by choosing triggers and actions. Nodes come with global operations and settings, as well as app-specific parameters that can be configured. You can also use the HTTP Request node to query data from any app or service with a REST API.

How to connect HTTP Request and uProc

Step 1: Add the first step
Step 2: Add HTTP Request and uProc nodes
Step 3: Connect HTTP Request and uProc
Step 4: Run workflow

Scrape and store data from multiple website pages

This workflow allows extracting data from multiple pages website.

The workflow:

Starts in a country list at https://www.theswiftcodes.com/browse-by-country/.
Loads every country page (https://www.theswiftcodes.com/albania/)
Paginates every page in the country page.
Extracts data from the country page.
Saves data to MongoDB.
Paginates through all pages in all countries.

It uses getWorkflowStaticData('global') method to recover the next page (saved from the previous page), and it goes ahead with all the pages.

There is a first section where the countries list is recovered and extracted.

Later, I try to read if a local cache page is available and I recover the cached page from the disk.

Finally, I save data to MongoDB, and we paginate all the pages in the country and for all the countries.

I have applied a cache system to save a visited page to n8n local disk. If I relaunch workflow, we check if a cache file exists to discard non-required requests to the webpage.

If the data present in the website changes, you can apply a Cron node to check the website once per week.

Finally, before inserting data in MongoDB, the best way to avoid duplicates is to check that swift_code (the primary value of the collection) doesn't exist.

I recommend using a proxy for all requests to avoid IP blocks. A good solution for proxy plus IP rotation is scrapoxy.io.

This workflow is perfect for small data requirements. If you need to scrape dynamic data, you can use a Headless browser or any other service.

If you want to scrape huge lists of URIs, I recommend using Scrapy + Scrapoxy.

Nodes used in this workflow

Get started

Copy workflow

Popular HTTP Request and uProc workflows

Scrape and store data from multiple website pages

Create screenshots with uProc, save to Dropbox and send by email

uProc supported modes

Guess

Only_verify

Verify

HTTP Request and uProc integration details

HTTP Request node docs + examples

See HTTP Request integrations

Related categories

Development

uProc offer businesses data optimization solutions to boost sales, marketing and development. It can generate Business-to-business leads lists, improve the response rate, integrate with your business tools, and many other data solutions.

uProc node docs + examples

uProc credential docs

See uProc integrations

Related categories

Data & Storage