Embeddings Google Gemini and Google Drive integration

This workflow implements a Retrieval Augmented Generation (RAG) chatbot that answers employee questions based on company documents stored in Google Drive. It automatically indexes new or updated documents in a Pinecone vector database, allowing the chatbot to provide accurate and up-to-date information. The workflow uses Google's Gemini AI for both embeddings and response generation. How it works The workflow uses two Google Drive Trigger nodes: one for detecting new files added to a specified Google Drive folder, and another for detecting file updates in that same folder. Automated Indexing: When a new or updated document is detected The Google Drive node downloads the file. The Default Data Loader node loads the document content. The Recursive Character Text Splitter node breaks the document into smaller text chunks. The Embeddings Google Gemini node generates embeddings for each text chunk using the text-embedding-004 model. The Pinecone Vector Store node indexes the text chunks and their embeddings in a specified Pinecone index. 7.The Chat Trigger node receives user questions through a chat interface. The user's question is passed to an AI Agent node. The AI Agent node uses a Vector Store Tool node, linked to a Pinecone Vector Store node in query mode, to retrieve relevant text chunks from Pinecone based on the user's question. The AI Agent sends the retrieved information and the user's question to the Google Gemini Chat Model (gemini-pro). The Google Gemini Chat Model generates a comprehensive and informative answer based on the retrieved documents. A Window Buffer Memory node connected to the AI Agent provides short-term memory, allowing for more natural and context-aware conversations. Set up steps Google Cloud Project and Vertex AI API: Create a Google Cloud project. Enable the Vertex AI API for your project. Google AI API Key: Obtain a Google AI API key from Google AI Studio. Pinecone Account: Create a free account on the Pinecone website. Obtain your API key from your Pinecone dashboard. Create an index named company-files in your Pinecone project. Google Drive: Create a dedicated folder in your Google Drive where company documents will be stored. Credentials in n8n: Configure credentials in your n8n environment for: Google Drive OAuth2 Google Gemini(PaLM) Api (using your Google AI API key) Pinecone API (using your Pinecone API key) Import the Workflow: Import this workflow into your n8n instance. Configure the Workflow: Update both Google Drive Trigger nodes to watch the specific folder you created in your Google Drive. Configure the Pinecone Vector Store nodes to use your company-files index.

Workflow based on the following article. https://www.anthropic.com/news/contextual-retrieval This n8n automation is designed to extract, process, and store content from documents into a Pinecone vector store using context-based chunking. The workflow enhances retrieval accuracy in RAG (Retrieval-Augmented Generation) setups by ensuring each chunk retains meaningful context. Workflow Breakdown: 🔹 Google Drive - Retrieve Document: The automation starts by fetching a source document from Google Drive. This document contains structured content, with predefined boundary markers for easy segmentation. 🔹 Extract Text Content - Once retrieved, the document’s text is extracted for processing. Special section boundary markers are used to divide the text into logical sections. 🔹 Code Node - Create Context-Based Chunks: A custom code node processes the extracted text, identifying section boundaries and splitting the document into meaningful chunks. Each chunk is structured to retain its context within the entire document. 🔹 Loop Node - Process Each Chunk: The workflow loops through each chunk, ensuring they are processed individually while maintaining a connection to the overall document context. 🔹 Agent Node - Generate Context for Each Chunk: We use an Agent node powered by OpenAI’s GPT-4.0-mini via OpenRouter to generate contextual metadata for each chunk, ensuring better retrieval accuracy. 🔹 Prepend Context to Chunks & Create Embeddings - The generated context is prepended to the original chunk, creating context-rich embeddings that improve searchability. 🔹 Google Gemini - Text Embeddings: The processed text is passed through Google Gemini text-embedding-004, which converts the text into semantic vector representations. 🔹 Pinecone Vector Store - Store Embeddings: The final embeddings, along with the enriched chunk content and metadata, are stored in Pinecone, making them easily retrievable for RAG-based AI applications. Use Case: This automation enhances RAG retrieval by ensuring each chunk is contextually aware of the entire document, leading to more accurate AI responses. It’s perfect for applications that require semantic search, AI-powered knowledge management, or intelligent document retrieval. By implementing context-based chunking, this workflow ensures that LLMs retrieve the most relevant data, improving response quality and accuracy in AI-driven applications.

This n8n workflow creates a financial analysis tool that generates reports on a company's quarterly earnings using the capabilities of OpenAI GPT-4o-mini, Google's Gemini AI and Pinecone's vector search. By analyzing PDFs of any company's earnings reports from their Investor Relations page, this workflow can answer complex financial questions and automatically compile findings into a structured Google Doc. How it works: Data loading and indexing Fetches links to PDF earnings document from a Google Sheet containing a list of file links. Downloads the PDFs from Google Drive. Parses the PDFs, splits the text into chunks, and generates embeddings using the Embeddings Google AI node (text-embedding-004 model). Stores the embeddings and corresponding text chunks in a Pinecone vector database for semantic search. Report generation with AI agent Utilizes an AI Agent node with a specifically crafted system prompt. The agent orchestrates the entire process. The agent uses a Vector Store Tool to access and retrieve information from the Pinecone database. Report delivery Saves the generated report as a Google Doc in a specified Google Drive location. Set up steps Google Cloud Project & Vertex AI API: Create a Google Cloud project. Enable the Vertex AI API for your project. Google AI API key: Obtain a Google AI API key from Google AI Studio. Pinecone account and API key: Create a free account on the Pinecone website. Obtain your API key from your Pinecone dashboard. Create an index named company-earnings in your Pinecone project. Google Drive - download and save financial documents: Go to a company you want to analize and download their quarterly earnings PDFs Save the PDFs in Google Drive Create a Google Sheet that stores a list of file URLs pointing to the PDFs you downloaded and saved to Google Drive Configure credentials in your n8n environment for: Google Sheets OAuth2 Google Drive OAuth2 Google Docs OAuth2 Google Gemini(PaLM) Api (using your Google AI API key) Pinecone API (using your Pinecone API key) Import and configure the workflow: Import this workflow into your n8n instance. Update the List Of Files To Load (Google Sheets) node to point to your Google Sheet. Update the Download File From Google Drive to point to the column where the file URLs are Update the Save Report to Google Docs node to point to your Google Doc where you want the report saved.

This workflow automates the process of discovering and extracting APIs from various services, followed by generating custom schemas. It works in three distinct stages: research, extraction, and schema generation, with each stage tracking progress in a Google Sheet. 🙏 Jim Le deserves major kudos for helping to build this sophisticated three-stage workflow that cleverly automates API documentation processing using a smart combination of web scraping, vector search, and LLM technologies. How it works Stage 1 - Research: Fetches pending services from a Google Sheet Uses Google search to find API documentation Employs Apify for web scraping to filter relevant pages Stores webpage contents and metadata in Qdrant (vector database) Updates progress status in Google Sheet (pending, ok, or error) Stage 2 - Extraction: Processes services that completed research successfully Queries vector store to identify products and offerings Further queries for relevant API documentation Uses Gemini (LLM) to extract API operations Records extracted operations in Google Sheet Updates progress status (pending, ok, or error) Stage 3 - Generation: Takes services with successful extraction Retrieves all API operations from the database Combines and groups operations into a custom schema Uploads final schema to Google Drive Updates final status in sheet with file location Ideal for: Development teams needing to catalog multiple APIs API documentation initiatives Creating standardized API schema collections Automating API discovery and documentation Accounts required: Google account (for Sheets and Drive access) Apify account (for web scraping) Qdrant database Gemini API access Set up instructions: Prepare your Google Sheets document with the services information. Here's an example of a Google Sheet – you can copy it and change or remove the values under the columns. Also, make sure to update Google Sheets nodes with the correct Google Sheet ID. Configure Google Sheets OAuth2 credentials, required third-party services (Apify, Qdrant) and Gemini. Ensure proper permissions for Google Drive access.