This is the first version of a template for a RAG/GenAI App using WordPress content.
As creating, sharing, and improving templates brings me joy 😄, feel free to reach out on LinkedIn if you have any ideas to enhance this template!
This template includes three workflows:
This template can be applied to various use cases:
This workflow retrieves your WordPress pages and posts, generates embeddings from the content, and stores them in Supabase using pgvector
.
Nodes :
Postgres - Create Documents Table
: This table is structured to support OpenAI embedding models with 1536 dimensionsPostgres - Create Workflow Execution History Table
These two nodes create tables in Supabase:
Nodes :
WordPress - Get All Posts
WordPress - Get All Pages
Merge WordPress Posts and Pages
These three nodes retrieve all content and metadata from your posts and pages and merge them.
**Important: ** Apply filters to avoid generating embeddings for all site content.
Nodes :
Set Fields
Filter - Only Published & Unprotected Content
HTML to Markdown
These three nodes prepare the content for embedding by:
protected=false
), ensuring private or unpublished content is excluded from your GenAI application.Nodes:
Supabase Vector Store
Embeddings OpenAI
Default Data Loader
Token Splitter
Aggregate
Supabase - Store Workflow Execution
This step involves generating embeddings for the content and storing it in Supabase, followed by logging the workflow execution details.
Embeddings OpenAI
node generates vector embeddings for the content.Default Data Loader
prepares the content for embedding storage. The metadata stored includes the content title, publication date, modification date, URL, and ID, which is essential for managing upserts.⚠️ Important Note : Be cautious not to store any sensitive information in metadata fields, as this information will be accessible to the AI and may appear in user-facing answers.
Token Splitter
ensures that content is segmented into manageable sizes to comply with token limits.Supabase - Store Workflow Execution
node saves the workflow execution ID and timestamp, enabling tracking of when each content update was processed.This setup ensures that content embeddings are stored in Supabase for use in downstream applications, while workflow execution details are logged for consistency and version tracking.
This workflow should be executed only once for the initial embedding.
Workflow 2, described below, will handle all future upserts, ensuring that new or updated content is embedded as needed.
Content on a website follows a lifecycle—it may be updated, new content might be added, or, at times, content may be deleted.
In this first version of the template, the upsert workflow manages:
Nodes:
CRON - Every 30 Seconds
Postgres - Get Last Workflow Execution
WordPress - Get Posts Modified After Last Workflow Execution
WordPress - Get Pages Modified After Last Workflow Execution
Merge Retrieved WordPress Posts and Pages
A CRON job (set to run every 30 seconds in this template, but you can adjust it as needed) initiates the workflow. A Postgres SQL query on the n8n_website_embedding_histories
table retrieves the timestamp of the latest workflow execution.
Next, the HTTP nodes use the WordPress API (update the example URL in the template with your own website’s URL and add your WordPress credentials) to request all posts and pages modified after the last workflow execution date. This process captures both newly added and recently updated content. The retrieved content is then merged for further processing.
Nodes :
Set fields2
Filter - Only published and unprotected content
The same that Step 2 in Workflow 1, except that HTML To Makrdown is used in further Step.
Here, I initially aimed to use 'update documents' instead of the delete + insert approach, but encountered challenges, especially with updating both content and metadata columns together. Any help or suggestions are welcome! :)
Nodes:
Loop Over Items
Postgres - Filter on Existing Documents
Switch
Route existing_documents
(if documents with matching IDs are found in metadata):
Supabase - Delete Row if Document Exists
: Removes any existing entry for the document, preparing for an update.Aggregate2
: Used to aggregate documents on Supabase with ID to ensure that Set Fields3
is executed only once for each WordPress content to avoid duplicate execution.Set Fields3
: Sets fields required for embedding updates.Route new_documents
(if no matching documents are found with IDs in metadata):
Set Fields4
: Configures fields for embedding newly added content.In this step, a loop processes each item, directing it based on whether the document already exists. The Aggregate2
node acts as a control to ensure Set Fields3
runs only once per WordPress content, effectively avoiding duplicate execution and optimizing the update process.
The HTML to Markdown node mirrors Workflow 1 - Step 2. Refer to that section for a detailed explanation on how HTML content is converted to Markdown for improved embedding performance and relevance.
Following this, the content is stored in the Supabase vector store to manage embeddings efficiently. Lastly, the **workflow execution table is updated. These nodes mirros the Workflow 1 - Step 3 nodes.
Nodes:
When Chat Message Received
Supabase - Retrieve Documents from Chat Input
Embeddings OpenAI1
Aggregate Documents
Set Fields
When a user sends a message to the chat, the prompt (user question) is sent to the Supabase vector store retriever. The RPC function match_documents
(created in Workflow 1 - Step 0) retrieves documents relevant to the user’s question, enabling a more accurate and relevant response.
In this step:
Directly using the AI agent without these nodes would prevent metadata from being sent to the language model (LLM), but metadata is essential for enhancing the context and accuracy of the AI’s response. By including metadata, the AI’s answers can reference relevant document details, making the interaction more informative.
Nodes:
OpenAI Chat Model
Postgres Chat Memories
This step involves calling the AI agent to generate an answer, responding to the user, and storing the conversation history. The model used is gpt4-o-mini, chosen for its cost-efficiency.
Implement complex processes faster with n8n