Back to Integrations
integration integration
integration Remove Duplicates node

Integrate Remove Duplicates with 500+ apps and services

Unlock Remove Duplicates’s full potential with n8n, connecting it to similar Core Nodes apps and over 1000 other services. Create adaptable and scalable workflows between Remove Duplicates and your stack. All within a building experience you will love.

Popular ways to use Remove Duplicates integration

HTTP Request node
Notion node
+8

Automate Competitor Research with Exa.ai, Notion and AI Agents

This n8n workflow demonstrates a simple multi-agent setup to perform the task of competitor research. It showcases how using the HTTP request tool could reduce the number of nodes needed to achieve a workflow like this. How it works For this template, a source company is defined by the user which is sent to Exa.ai to find competitors. Each competitor is then funnelled through 3 AI agents that will go out onto the internet and retrieve specific datapoints about the competitor; company overview, product offering and customer reviews. Once the agents are finished, the results are compiled into a report which is then inserted in a notion database. Check out an example output here: https://jimleuk.notion.site/2d1c3c726e8e42f3aecec6338fd24333?v=de020fa196f34cdeb676daaeae44e110&pvs=4 Requirements An OpenAI account for the LLM. Exa.ai account for access to their AI search engine. SerpAPI account for Google search. Firecrawl.dev account for webscraping. Notion.com account for database to save final reports. Customising the workflow Add additional agents to gather more datapoints such as SEO keywords and metrics. Not using notion? Feel free to swap this out for your own database.
jimleuk
Jimleuk
Google Sheets node
HTTP Request node
Google Sheets Trigger node
Remove Duplicates node

Verifying Email deliverability using google sheets and Effibotics API

This workflow helps marketers verify and update data using EffiBotics Email Verifier API. Copy and create a list with emails as on this one https://docs.google.com/spreadsheets/d/1rzuojNGTaBvaUEON6cakQRDva3ueGg5kNu9v12aaSP4/edit#gid=0 The trigger checks for any updates in the number of rows that are present in a sheet and updates the verified emails on Google sheets Once you update a new cell, the new data is read, and the email is checked for its validity before. The results are then updated in real-time on the sheet. Happy Emailing!
imperolq
Imperol
HTTP Request node
Merge node
Supabase node
Markdown node
+11

Autonomous AI crawler

This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: text tool:** to retrieve all the text from the page, URLs tool:** to extract all possible links from the page. 💡 You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. 👉 This workflow uses Supabase as storage (input/output). Feel free to change it to any other database of your choice. 🎬 See this workflow in action in my YouTube video. How it works? The workflow uses the input URL (website) as a starting point to retrieve the data (e.g. example.com). Using the "URLs tool", the agent is able to retrieve all links from the page and navigate to them. For example, if you want to retrieve contact information, agent will try to find a subpage that might contain this information (e.g. example.com/contact) and extract the information using the text tool. Set up steps Connect database with input data (website addresses) or pin sample data to trigger node. Configure the crawling agent to retrieve the desired data (e.g. modify prompt and/or parsing schema). Set credentials for OpenAI. Optionally: split agent tools to separate workflows. If you like this workflow, please subscribe to my YouTube channel and/or my newsletter.
workfloows
Oskar
HTTP Request node
Webhook node
Respond to Webhook node
+3

Extract emails from website HTMLs

How to scrap emails from websites This workflow shows how to quickly build an Email scraping API using n8n. Email marketing is at the core of most marketing strategies, be it content marketing, sales, etc. As such, being able to find contacts in bulk for your business on a large scale is key. There are available tools available in the market that can do this, but most are premium; why not build a custom one with n8n? Usage The workflow gets the data from a website and performs an extraction based on the date around on the website Copy the webhook URL to your browser Add a query parameter eg ?Website=https://mailsafi.com . This should give you a URL like this {{$n8nhostingurl/webhook/ea568868-5770-4b2a-8893-700b344c995e?Website=https://mailsafi.com Click on the URL and wait for the extracted email to be displayed. This will return the email address on the website, or if there is no email, the response will be "workflow successfully executed." Make sure to use HTTP:// for your domains Otherwise, you may get an error.
imperolq
Imperol
HTTP Request node
Merge node
+12

Import workflows and map their credentials using a Multi-Form

Purpose This workflow allows you to import any workflow from a file or another n8n instance and map the credentials easily. How it works A multi-form setup guides you through the entire process At the beginning you have two options: Upload a workflow file (JSON) Copy workflow from a remote n8n instance If you choose the second option, you get to choose one of your predefined (in the Settings node) remote instances first, then it retrieves a list of all the workflows using the n8n API which you then can choose a workflow from. Now both initial options come together - the workflow file is being processed In parallel all credentials of the current instance are being retrieved using the Execute Command node The next form page enables a mapping of all the credentials used in the workflow. The matching happens between the names (because one workflow can contain different credentials of the same type) of the original credentials and the ones available on the current instance. Every option then shows all available credentials of the same type. In addition the user has always the choice to create a new credential on the fly. For every option which was set to create a new credential, an empty credential is being created on the current instance using the n8n API. An emoji is being appended to the name, which indicates that it needs to be populated. Finally the workflow gets updated with the new credential ID’s and created on the current instance using the n8n API. Then the user gets a message, if the process has succeeded or not. Setup Select your credentials in the nodes which require those Configure your remote instance(s) in the Settings node. (You can skip this step, if you only want to use the File Upload feature) Every instance is defined as object with the keys name, apiKey and baseUrl. Those instances are then wrapped inside an array. You can find an example described within a note on the workflow canvas. How to use Grab the (production) URL of the Form from the first node Open the URL and follow the instructions given in the multi-form Disclaimer Security: Beware, that all credentials are being decrypted and processed within the workflow. Also the API keys to other n8n instances are stored within the workflow. This solution is primarily meant for transferring data between testing environments. For production use consider the n8n enterprise edition which provides a reliable way to deploy workflows between different environments without the need of manual credential mapping.
octionic
Mario

Supported modes

Clean Database
Clear all values stored for a key in the database

Supported Actions

Remove Items Repeated Within Current Input
Remove duplicates from incoming items
Remove Items Processed in Previous Executions
Deduplicate items already seen in previous executions
Clear Deduplication History
Wipe the store of previous items

Over 3000 companies switch to n8n every single week

Connect Remove Duplicates with your company’s tech stack and create automation workflows

in other news I installed @n8n_io tonight and holy moly it’s good

it’s compatible with EVERYTHING

Last week I automated much of the back office work for a small design studio in less than 8hrs and I am still mind-blown about it.

n8n is a game-changer and should be known by all SMBs and even enterprise companies.

We're using the @n8n_io cloud for our internal automation tasks since the beta started. It's awesome! Also, support is super fast and always helpful. 🤗

Implement complex processes faster with n8n

red icon yellow icon red icon yellow icon