+4
Scrape ProductHunt using Google Gemini
Published 10 days ago
Template description
Workflow Description: Product Data Extractor
This workflow automates the extraction of product data from Product Hunt by combining webhook interactions, HTML processing, AI-based data analysis, and structured output formatting. It is designed to handle incoming requests dynamically and return detailed JSON responses for further usage.
Overview
The workflow processes a product name submitted through a webhook. It fetches the corresponding Product Hunt page, extracts and analyzes inline scripts, and structures the data into a well-defined JSON format using AI tools. The final JSON response is returned to the client through the webhook.
Workflow Steps
1. Webhook Listener
- Node: Receive Product Request
- Function: Captures incoming requests containing the product name to process.
- Details: Accepts HTTP requests and extracts the
productparameter from the query string, such as<custom_webhook_url>/?product=epigram.
2. Fetch Product HTML
- Node: Fetch Product HTML
- Function: Sends an HTTP request to retrieve the HTML content of the specified Product Hunt page.
- Details: Constructs a dynamic URL using the product name and fetches the page data.
3. Extract Inline Scripts
- Node: Extract Inline Scripts
- Function: Parses the HTML content to extract inline scripts located within the
<head>section. - Details: Excludes scripts containing
srcattributes and validates the presence of inline scripts.
4. Process Data with LLM
- Node: Process Script with LLM
- Function: Analyzes the extracted scripts using a language model to identify key product data.
- Details: Processes the script to derive structured and meaningful insights.
5. Refine Data with Google Gemini
- Node: Analyze Script with Google Gemini
- Function: Leverages Google Gemini AI for enhanced analysis of script data.
- Details: Ensures the extracted data is precise and enriched.
6. Format Product Data to JSON
- Node: Format Product Data to JSON
- Function: Structures the processed data into a clean JSON format.
- Details: Defines a schema to ensure all relevant fields are included in the output.
7. Send JSON Response to Client
- Node: Send JSON Response to Client
- Function: Returns the final structured JSON response to the client.
- Details: Sends the response back via the same webhook that initiated the request. For example,
<custom_webhook_url>.
Key Features
- Versatile Use Cases: This workflow can be used to gather Product Hunt data for creating blog posts or as a tool for AI agents to research products efficiently.
- Dynamic Processing: Adapts to various product names through dynamic URL construction.
- AI Integration: Utilizes the Gemini 1.5 8B AI model, offering reduced latency and minimal or no cost depending on the use case.
- Selector Independence: Functions even if Product Hunt's DOM structure changes, as it does not rely on direct DOM selectors.
- Reliable Data Output: A low temperature setting (0) and a precisely defined JSON schema ensure accurate and real data extraction.
- Dynamic Processing: Adapts to various product names through dynamic URL construction.
- AI Integration: Utilizes advanced language models for data extraction and refinement.
- Structured Output: Ensures the output JSON adheres to a predefined schema for consistency.
- Error Handling: Includes validations to handle missing or malformed data gracefully.
Customization Options
Limitations
-
Dependency on Product Hunt: Significant changes to the way Product Hunt loads data on its pages might require modifications to the workflow.
-
Adaptability: Even if changes occur, the workflow can be updated to maintain functionality due to its reliance on AI and not direct DOM selectors.
-
Modify the webhook path to suit your application.
-
Adjust the prompt for the language model to include additional fields.
-
Extend the JSON schema to capture more data fields as needed.
Expected Output
Performance Metrics
- Response Time: Typically ~6 seconds per product.
- Accuracy: Data extracted with >95% precision due to the pre-defined JSON schema.
A JSON object containing detailed information about the specified product. Below is an example of a complete response for the product Epigram:
{
"id": "861675",
"slug": "epigram",
"followersCount": 181,
"name": "Epigram",
"tagline": "Open-Source, Free, and AI-Powered News in Short",
"reviewsRating": 0,
"logoUuid": "735c2528-554c-467c-9dcf-745ee4b8bbdd.png",
"postsCount": 1,
"websiteUrl": "https://epigram.news",
"websiteDomain": "epigram.news",
"metaTitle": "Epigram - Open-source, free, and ai-powered news in short",
"postName": "Epigram",
"postTagline": "Open-source, free, and ai-powered news in short",
"dailyRank": "3",
"description": "An open-source, AI-powered news app for busy people. Stay updated with bite-sized news, real-time updates, and in-depth analysis. Experience balanced, trustworthy reporting tailored for fast-paced lifestyles in a sleek, user-friendly interface.",
"pricingType": "free",
"userName": "Fazle Rahman",
"userHeadline": "Co-founder & CEO, Hashnode",
"userUsername": "fazlerocks",
"userAvatarUrl": "https://ph-avatars.imgix.net/129147/f84e1796-548b-4d6f-9dcf-745ee4b8bbdd.jpeg",
"makerName1": "Fazle Rahman",
"makerHeadline1": "Co-founder & CEO, Hashnode",
"makerUsername1": "fazlerocks",
"makerAvatarUrl1": "https://ph-avatars.imgix.net/129147/f84e1796-548b-4d6f-9dcf-745ee4b8bbdd.jpeg",
"makerName2": "Sandeep Panda",
"makerHeadline2": "Co-Founder @ Hashnode",
"makerUsername2": "sandeepg33k",
"makerAvatarUrl2": "https://ph-avatars.imgix.net/101872/80b0b618-a540-4110-a6d1-74df39675ad0.jpeg",
"primaryLinkUrl": "https://epigram.news/",
"media1OriginalHeight": 1080,
"media1OriginalWidth": 1440,
"media1ImageUuid": "ac426fd1-3854-4734-b43d-34a5e06347ea.gif",
"media1MediaType": "video",
"media1MetadataUrl": "https://www.loom.com/share/b1a48a9b3cac4ba89ce772a3fbcc2847?sid=75efc771-25fa-4ac0-bb1b-5e38fc447deb",
"media1VideoId": "b1a48a9b3cac4ba89ce772a3fbcc2847",
"media2OriginalHeight": 630,
"media2OriginalWidth": 1200,
"media2ImageUuid": "8521a6bd-7640-487b-abd6-29b9f65fee32",
"media2MediaType": "image",
"media2MetadataUrl": null,
"launchState": "featured",
"thumbnailImageUuid": "735c2528-554c-467c-9dcf-745ee4b8bbdd.png",
"link1StoreName": "Website",
"link1WebsiteName": "epigram.news",
"link2StoreName": "Github",
"link2WebsiteName": "github.com",
"latestScore": 233,
"launchDayScore": 233,
"userId": "129147",
"topic1": "News",
"topic2": "Open Source",
"topic3": "Artificial Intelligence",
"weeklyRank": "24",
"commentsCount": 20,
"postUrl": "https://www.producthunt.com/posts/epigram"
}
Target Audience
This workflow is ideal for developers, marketers, and data analysts seeking to automate the extraction and structuring of product data from Product Hunt for analytics, reporting, or integration with other tools.
Share Template
More Product workflow templates
+5
🚀 Boost your customer service with this WhatsApp Business bot!
This n8n workflow demonstrates how to automate customer interactions and appointment management via WhatsApp Business bot.
After submitting a Google Form, the user receives a notification via WhatsApp. These notifications are sent via a template message.
In case user sends a message to the bot, the text and user data is stored in Google Sheets.
To reply back to the user, fill in the ReplyText column and change the Status to 'Ready'. In a few seconds n8n will fetch the unsent replies and deliver them one by one via WhatsApp Business node.
Customize this workflow to fit your specific needs, connect different online services and enhance your customer communication! 🎉
Setup Instructions
To get this workflow up and running, you'll need to:
👇 Create a WhatsApp template message on the Meta Business portal.
Obtain an Access Token and WhatsApp Business Account ID from the Meta Developers Portal. This is needed for the WhatsApp Business Node to send messages.
Set up a WhatsApp Trigger node with App ID and App Secret from the Meta Developers Portal.
Right after that copy the WhatsApp Trigger URL and add it as a Callback URL in the Meta Developers Portal. This trigger is needed to receive incoming messages and their status updates.
Connect your Google Sheets account for data storage and management. Check out the documentation page.
⚠️ Important Notes
WhatsApp allows automatic custom text messages only within 24 hours of the last user message. Outside with time frame only approved template messages can be sent.
The workflow uses a Google Sheet to manage form submissions, incoming messages and prepare responses. You can replace these nodes and connect the WhatsApp bot with other systems.
Eduard
+6
Notion AI Assistant Generator
This n8n workflow template lets teams easily generate a custom AI chat assistant based on the schema of any Notion database. Simply provide the Notion database URL, and the workflow downloads the schema and creates a tailored AI assistant designed to interact with that specific database structure.
Set Up
Watch this quick set up video 👇
Key Features
Instant Assistant Generation**: Enter a Notion database URL, and the workflow produces an AI assistant configured to the database schema.
Advanced Querying**: The assistant performs flexible queries, filtering records by multiple fields (e.g., tags, names). It can also search inside Notion pages to pull relevant content from specific blocks.
Schema Awareness**: Understands and interacts with various Notion column types like text, dates, and tags for accurate responses.
Reference Links**: Each query returns direct links to the exact Notion pages that inform the assistant’s response, promoting transparency and easy access.
Self-Validation**: The workflow has logic to check the generated assistant, and if any errors are detected, it reruns the agent to fix them.
Ideal for
Product Managers**: Easily access and query product data across Notion databases.
Support Teams**: Quickly search through knowledge bases for precise information to enhance support accuracy.
Operations Teams**: Streamline access to HR, finance, or logistics data for fast, efficient retrieval.
Data Teams**: Automate large dataset queries across multiple properties and records.
How It Works
This AI assistant leverages two HTTP request tools—one for querying the Notion database and another for retrieving data within individual pages. It’s powered by the Anthropic LLM (or can be swapped for GPT-4) and always provides reference links for added transparency.
Max Tkacz
+9
Actioning Your Meeting Next Steps using Transcripts and AI
This n8n workflow demonstrates how you can summarise and automate post-meeting actions from video transcripts fed into an AI Agent.
Save time between meetings by allowing AI handle the chores of organising follow-up meetings and invites.
How it works
This workflow scans for the calendar for client or team meetings which were held online. * Attempts will be made to fetch any recorded transcripts which are then sent to the AI agent.
The AI agent summarises and identifies if any follow-on meetings are required.
If found, the Agent will use its Calendar Tool to to create the event for the time, date and place for the next meeting as well as add known attendees.
Requirements
Google Calendar and the ability to fetch Meeting Transcripts (There is a special OAuth permission for this action!)
OpenAI account for access to the LLM.
Customising the workflow
This example only books follow-on meetings but could be extended to generate reports or send emails.
Jimleuk
+13
AI Agent To Chat With Files In Supabase Storage
Video Guide
I prepared a detailed guide explaining how to set up and implement this scenario, enabling you to chat with your documents stored in Supabase using n8n.
Youtube Link
Who is this for?
This workflow is ideal for researchers, analysts, business owners, or anyone managing a large collection of documents. It's particularly beneficial for those who need quick contextual information retrieval from text-heavy files stored in Supabase, without needing additional services like Google Drive.
What problem does this workflow solve?
Manually retrieving and analyzing specific information from large document repositories is time-consuming and inefficient. This workflow automates the process by vectorizing documents and enabling AI-powered interactions, making it easy to query and retrieve context-based information from uploaded files.
What this workflow does
The workflow integrates Supabase with an AI-powered chatbot to process, store, and query text and PDF files. The steps include:
Fetching and comparing files to avoid duplicate processing.
Handling file downloads and extracting content based on the file type.
Converting documents into vectorized data for contextual information retrieval.
Storing and querying vectorized data from a Supabase vector store.
File Extraction and Processing: Automates handling of multiple file formats (e.g., PDFs, text files), and extracts document content.
Vectorized Embeddings Creation: Generates embeddings for processed data to enable AI-driven interactions.
Dynamic Data Querying: Allows users to query their document repository conversationally using a chatbot.
Setup
N8N Workflow
Fetch File List from Supabase:
Use Supabase to retrieve the stored file list from a specified bucket.
Add logic to manage empty folder placeholders returned by Supabase, avoiding incorrect processing.
Compare and Filter Files:
Aggregate the files retrieved from storage and compare them to the existing list in the Supabase files table.
Exclude duplicates and skip placeholder files to ensure only unprocessed files are handled.
Handle File Downloads:
Download new files using detailed storage configurations for public/private access.
Adjust the storage settings and GET requests to match your Supabase setup.
File Type Processing:
Use a Switch node to target specific file types (e.g., PDFs or text files).
Employ relevant tools to process the content:
For PDFs, extract embedded content.
For text files, directly process the text data.
Content Chunking:
Break large text data into smaller chunks using the Text Splitter node.
Define chunk size (default: 500 tokens) and overlap to retain necessary context across chunks.
Vector Embedding Creation:
Generate vectorized embeddings for the processed content using OpenAI's embedding tools.
Ensure metadata, such as file ID, is included for easy data retrieval.
Store Vectorized Data:
Save the vectorized information into a dedicated Supabase vector store.
Use the default schema and table provided by Supabase for seamless setup.
AI Chatbot Integration:
Add a chatbot node to handle user input and retrieve relevant document chunks.
Use metadata like file ID for targeted queries, especially when multiple documents are involved.
Testing
Upload sample files to your Supabase bucket.
Verify if files are processed and stored successfully in the vector store.
Ask simple conversational questions about your documents using the chatbot (e.g., "What does Chapter 1 say about the Roman Empire?").
Test for accuracy and contextual relevance of retrieved results.
Mark Shcherbakov
+3
Generate SEO Seed Keywords Using AI
What this workflow does:
This flow uses an AI node to generate Seed Keywords to focus SEO efforts on based on your ideal customer profile. You can use these keywords to form part of your SEO strategy.
Outputs:
List of 20 Seed Keywords
Setup
Fill the Set Ideal Customer Profile (ICP)
Connect with your credentials
Replace the Connect to your own database with your own database
Pre-requisites / Dependencies
You know your ideal customer profile (ICP)
An AI API account (either OpenAI or Anthropic recommended)
Made by Simon @ automake.io
simonscrapes
+7
✨ Vision-Based AI Agent Scraper - with Google Sheets, ScrapingBee, and Gemini
Important Notes:
Check Legal Regulations:
This workflow involves scraping, so ensure you comply with the legal regulations in your country before getting started. Better safe than sorry!
Workflow Description:
😮💨 Tired of struggling with XPath, CSS selectors, or DOM specificity when scraping ?
This AI-powered solution is here to simplify your workflow! With a vision-based AI Agent, you can extract data effortlessly without worrying about how the DOM is structured.
This workflow leverages a vision-based AI Agent, integrated with Google Sheets, ScrapingBee, and the Gemini-1.5-Pro model, to extract structured data from webpages. The AI Agent primarily uses screenshots for data extraction but switches to HTML scraping when necessary, ensuring high accuracy.
Key Features:
Google Sheets Integration**: Manage URLs to scrape and store structured results.
ScrapingBee**: Capture full-page screenshots and retrieve HTML data for fallback extraction.
AI-Powered Data Parsing**: Use Gemini-1.5-Pro for vision-based scraping and a Structured Output Parser to format extracted data into JSON.
Token Efficiency**: HTML is converted to Markdown to optimize processing costs.
This template is designed for e-commerce scraping but can be customized for various use cases.
Dataki
More AI workflow templates
AI agent chat
This workflow employs OpenAI's language models and SerpAPI to create a responsive, intelligent conversational agent. It comes equipped with manual chat triggers and memory buffer capabilities to ensure seamless interactions.
To use this template, you need to be on n8n version 1.50.0 or later.
n8n Team
+7
Scrape and summarize webpages with AI
This workflow integrates both web scraping and NLP functionalities. It uses HTML parsing to extract links, HTTP requests to fetch essay content, and AI-based summarization using GPT-4o. It's an excellent example of an end-to-end automated task that is not only efficient but also provides real value by summarizing valuable content.
Note that to use this template, you need to be on n8n version 1.50.0 or later.
n8n Team
+5
AI agent that can scrape webpages
⚙️🛠️🚀🤖🦾
This template is a PoC of a ReAct AI Agent capable of fetching random pages (not only Wikipedia or Google search results).
On the top part there's a manual chat node connected to a LangChain ReAct Agent. The agent has access to a workflow tool for getting page content.
The page content extraction starts with converting query parameters into a JSON object. There are 3 pre-defined parameters:
url** – an address of the page to fetch
method** = full / simplified
maxlimit** - maximum length for the final page. For longer pages an error message is returned back to the agent
Page content fetching is a multistep process:
An HTTP Request mode tries to get the page content.
If the page content was successfuly retrieved, a series of post-processing begin:
Extract HTML BODY; content
Remove all unnecessary tags to recude the page size
Further eliminate external URLs and IMG scr values (based on the method query parameter)
Remaining HTML is converted to Markdown, thus recuding the page lengh even more while preserving the basic page structure
The remaining content is sent back to an Agent if it's not too long (maxlimit = 70000 by default, see CONFIG node).
NB:
You can isolate the HTTP Request part into a separate workflow.
Check the Workflow Tool description, it guides the agent to provide a query string with several parameters instead of a JSON object.
Please reach out to Eduard is you need further assistance with you n8n workflows and automations!
Note that to use this template, you need to be on n8n version 1.19.4 or later.
Eduard
+2
Telegram AI Chatbot
The workflow starts by listening for messages from Telegram users. The message is then processed, and based on its content, different actions are taken. If it's a regular chat message, the workflow generates a response using the OpenAI API and sends it back to the user. If it's a command to create an image, the workflow generates an image using the OpenAI API and sends the image to the user. If the command is unsupported, an error message is sent. Throughout the workflow, there are additional nodes for displaying notes and simulating typing actions.
Eduard
+4
OpenAI GPT-3: Company Enrichment from website content
Enrich your company lists with OpenAI GPT-3 ↓
You’ll get valuable information such as:
Market (B2B or B2C)
Industry
Target Audience
Value Proposition
This will help you to:
add more personalization to your outreach
make informed decisions about which accounts to target
I've made the process easy with an n8n workflow.
Here is what it does:
Retrieve website URLs from Google Sheets
Extract the content for each website
Analyze it with GPT-3
Update Google Sheets with GPT-3 data
Lucas Perret
+5
Ask questions about a PDF using AI
The workflow first populates a Pinecone index with vectors from a Bitcoin whitepaper. Then, it waits for a manual chat message. When received, the chat message is turned into a vector and compared to the vectors in Pinecone. The most similar vectors are retrieved and passed to OpenAI for generating a chat response.
Note that to use this template, you need to be on n8n version 1.19.4 or later.
David Roberts
More Marketing workflow templates
+4
OpenAI GPT-3: Company Enrichment from website content
Enrich your company lists with OpenAI GPT-3 ↓
You’ll get valuable information such as:
Market (B2B or B2C)
Industry
Target Audience
Value Proposition
This will help you to:
add more personalization to your outreach
make informed decisions about which accounts to target
I've made the process easy with an n8n workflow.
Here is what it does:
Retrieve website URLs from Google Sheets
Extract the content for each website
Analyze it with GPT-3
Update Google Sheets with GPT-3 data
Lucas Perret
+5
Automated Web Scraping: email a CSV, save to Google Sheets & Microsoft Excel
How it works:
The workflow starts by sending a request to a website to retrieve its HTML content.
It then parses the HTML extracting the relevant information
The extracted data is storted and converted into a CSV file.
The CSV file is attached to an email and sent to your specified address.
The data is simultaneously saved to both Google Sheets and Microsoft Excel for further analysis or use.
Set-up steps:
Change the website to scrape in the "Fetch website content" node
Configure Microsoft Azure credentials with Microsoft Graph permissions (required for the Save to Microsoft Excel 365 node)
Configure Google Cloud credentials with access to Google Drive, Google Sheets and Gmail APIs (the latter is required for the Send CSV via e-mail node).
Mihai Farcas
+2
Flux AI Image Generator
Easily generate images with Black Forest's Flux Text-to-Image AI models using Hugging Face’s Inference API. This template serves a webform where you can enter prompts and select predefined visual styles that are customizable with no-code. The workflow integrates seamlessly with Hugging Face's free tier, and it’s easy to modify for any Text-to-Image model that supports API access.
Try it
Curious what this template does? Try a public version here: https://devrel.app.n8n.cloud/form/flux
Set Up
Watch this quick set up video 👇
Accounts required
Huggingface.co account (free)
Cloudflare.com account (free - used for storage; but can be swapped easily e.g. GDrive)
Key Features:
Text-to-Image Creation**: Generates unique visuals based on your prompt and style.
Hugging Face Integration**: Utilizes Hugging Face’s Inference API for reliable image generation.
Customizable Visual Styles**: Select from preset styles or easily add your own.
Adaptable**: Swap in any Hugging Face Text-to-Image model that supports API calls.
Ideal for:
Creators**: Rapidly create visuals for projects.
Marketers**: Prototype campaign visuals.
Developers**: Test different AI image models effortlessly.
How It Works:
You submit an image prompt via the webform and select a visual style, which appends style instructions to your prompt. The Hugging Face Inference API then generates and returns the image, which gets hosted on Cloudflare S3. The workflow can be easily adjusted to use other models and styles for complete flexibility.
Max Tkacz
+2
Gmail AI Auto-Responder: Create Draft Replies to incoming emails
This workflow automatically generates draft replies in Gmail.
It's designed for anyone who manages a high volume of emails or often face writer's block when crafting responses.
Since it doesn't send the generated message directly, you're still in charge of editing and approving emails before they go out.
How It Works:
Email Trigger: activates when new emails reach the Gmail inbox
Assessment: uses OpenAI gpt-4o and a JSON parser to determine if a response is necessary.
Reply Generation: crafts a reply with OpenAI GPT-4 Turbo
Draft Integration: after converting the text to html, it places the draft into the Gmail thread as a reply to the first message
Set Up Overview (~10 minutes):
OAuth Configuration (follow n8n instructions here):
Setup Google OAuth in Google Cloud console. Make sure to add Gmail API with the modify scope.
Add Google OAuth credentials in n8n. Make sure to add the n8n redirect URI to the Google Cloud Console consent screen settings.
OpenAI Configuration: add OpenAI API Key in the credentials
Tweaking the prompt: edit the system prompt in the "Generate email reply" node to suit your needs
Detailed Walkthrough
Check out this blog post where I go into more details on how I built this workflow.
Reach out to me here if you need help building automations for your business.
Nicolas Chourrout
+5
Personalize marketing emails using customer data and AI
This workflow uses AI to analyze customer sentiment from product feedback. If the sentiment is negative, AI will determine whether offering a coupon could improve the customer experience.
Upon completing the sentiment analysis, the workflow creates a personalized email templates. This solution streamlines the process of engaging with customers post-purchase, particularly when addressing dissatisfaction, and ensures that outreach is both personalized and automated.
This workflow won the 1st place in our last AI contest.
Note that to use this template, you need to be on n8n version 1.19.4 or later.
n8n Team
+8
Scrape business emails from Google Maps without the use of any third party APIs
Who is this template for?
This workflow template is designed for sales, marketing, and business development professionals who want a cost-effective and efficient way to generate leads. By leveraging n8n core nodes, it scrapes business emails from Google Maps without relying on third-party APIs or paid services, ensuring there are no additional costs involved.
Ideal for small business owners, freelancers, and agencies, this template automates the process of collecting contact information for targeted outreach, making it a powerful tool for anyone looking to scale their lead generation efforts without incurring extra expenses.
How it works
This template streamlines email scraping from Google Maps using only n8n core nodes, ensuring a completely free and self-contained solution. Here’s how it operates:
Input Queries
You provide a list of queries, each consisting of keywords related to the type of business you want to target and the specific region or subregion you’re interested in.
Iterates through Queries
The workflow processes each query one at a time. For each query, it triggers a sub-workflow dedicated to handling the scraping tasks.
Scrapes Google Maps for URLs
Using these queries, the workflow scrapes Google Maps to collect URLs of business listings matching the provided criteria.
Fetches HTML Content
The workflow then fetches the HTML pages of the collected URLs for further processing.
Extracts Emails
Using a Code Node with custom JavaScript, the workflow runs regular expressions on the HTML content to extract business email addresses.
Setup
Add Queries:
Open the first node, "Run Workflow" and input a list of queries, each containing the business keywords and the target region.
Configure the Google Sheets Node:
Open the Google Sheets node and select a document and specific sheet where the scraped results will be saved.
Run the workflow:
Click on "Test workflow" and watch your Google Sheets document gradually receive business email addresses.
Customize as Needed:
You can adjust the regular expressions in the Code Node to refine the email extraction logic or add logic to extract other kinds of information.
Akram Kadri
