Google Sheets node
+4

Exponential Backoff for Google APIs

Published 22 days ago

Created by

alexk1919
Alex Kim

Template description

n8n Workflow: Exponential Backoff for Google APIs

Overview

This n8n workflow implements an Exponential Backoff mechanism to handle retries when interacting with Google APIs. It ensures that failed API requests are retried with increasing delays, up to a specified maximum retry count. This approach helps mitigate transient errors (e.g., rate limits or temporary network issues) while maintaining workflow efficiency.

Key Features:

  • Exponential Backoff Logic: Dynamically increases wait time between retries based on the retry count.
  • Error Handling: Stops the workflow and raises an error after a specified number of retries.
  • Dynamic Waiting: Waits for a calculated duration before each retry.
  • Scalable Design: Modular nodes for easy debugging and customization.

Workflow Details

Nodes in the Workflow:

  1. Trigger (When clicking "Test Workflow"):

    • Manually starts the workflow for testing.
  2. Loop Over Items:

    • Iterates over multiple input items to process Google API requests row by row.
  3. Google API Node (Example: Update Sheet):

    • Sends a request to a Google API endpoint (e.g., updating a row in Google Sheets).
    • On success: Moves to the next item in the loop.
    • On error: Passes the error to the Exponential Backoff node.
  4. Exponential Backoff:

    • Calculates the delay for the next retry based on the retry count.
    • Logic:
      const retryCount = $json["retryCount"] || 0;
      const maxRetries = 5;
      const initialDelay = 1; // in seconds
      
      if (retryCount < maxRetries) {
          const currentDelayInSeconds = initialDelay * Math.pow(2, retryCount);
          return {
              json: {
                  retryCount: retryCount + 1,
                  waitTimeInSeconds: currentDelayInSeconds,
                  status: 'retrying',
              }
          };
      } else {
          return {
              json: {
                  error: 'Max retries exceeded',
                  retryCount: retryCount,
                  status: 'failed'
              }
          };
      }
      
  5. Wait:

    • Dynamically waits for the waitTimeInSeconds value calculated in the Exponential Backoff node.
    • Configuration:
      • Resume: After Time Interval
      • Wait Amount: {{ $json["waitTimeInSeconds"] }}
      • Unit: Seconds
  6. Check Max Retries:

    • Evaluates whether the retry count has exceeded the maximum limit.
    • Routes the workflow:
      • True: Passes to the Stop and Error node.
      • False: Loops back to the Google API node for retry.
  7. Stop and Error:

    • Stops the workflow and logs the error when the maximum retry count is reached.

Parameters

Configurable Settings:

  1. Max Retries:

    • Defined in the Exponential Backoff node (const maxRetries = 5).
    • Adjust this value based on your requirements.
  2. Initial Delay:

    • The starting wait time for retries, defined as 1 second.
  3. Google API Configuration:

    • Ensure your Google API node is properly authenticated and configured with the desired endpoint and parameters.

How to Use

  1. Import the Workflow:

    • Copy the workflow JSON and import it into your n8n instance.
  2. Configure Google API Node:

    • Set up the Google API node with your credentials and target API endpoint (e.g., Google Sheets, Gmail, etc.).
  3. Test the Workflow:

    • Manually trigger the workflow and observe the retry behavior in case of errors.
  4. Monitor Logs:

    • Use the console logs in the Exponential Backoff node to debug retry timings and status.

Example Scenarios

Scenario 1: Successful Execution

  • The Google API processes all requests without errors.
  • Workflow completes without triggering the retry logic.

Scenario 2: Transient API Errors

  • The Google API returns an error (e.g., 429 Too Many Requests).
  • The workflow retries the request with increasing wait times.

Scenario 3: Maximum Retries Exceeded

  • The workflow reaches the maximum retry count (e.g., 5 retries).
  • An error is raised, and the workflow stops.

Considerations

  1. Jitter:

    • This workflow does not implement jitter (randomized delay) since it's not required for low-volume use cases.
    • If needed, jitter can be added to the exponential backoff calculation.
  2. Retry Storms:

    • If multiple workflows run simultaneously, ensure your API quotas can handle potential retries.
  3. Error Handling Beyond Max Retries:

    • Customize the Stop and Error node to notify stakeholders or log errors in a centralized system.

Customization Options

  • Adjust the maximum retry limit and delay calculation to suit your use case.
  • Add additional logic to handle specific error codes differently.
  • Extend the workflow to notify stakeholders when an error occurs (e.g., via Slack or email).

Troubleshooting

  • Retry Not Triggering:

    • Ensure the retryCount variable is passed correctly between nodes.
    • Confirm that the error output from the Google API node flows to the Exponential Backoff node.
  • Incorrect Wait Time:

    • Verify the Wait node is referencing the correct field for waitTimeInSeconds.

Request for Feedback

We are always looking to improve this workflow. If you have suggestions, improvements, or ideas for additional features, please feel free to share them. Your feedback helps us refine and enhance this solution!

Share Template

More Building Blocks workflow templates

Webhook node
Respond to Webhook node

Creating an API endpoint

Task: Create a simple API endpoint using the Webhook and Respond to Webhook nodes Why: You can prototype or replace a backend process with a single workflow Main use cases: Replace backend logic with a workflow
jon-n8n
Jonathan
Customer Datastore (n8n training) node

Very quick quickstart

Want to learn the basics of n8n? Our comprehensive quick quickstart tutorial is here to guide you through the basics of n8n, step by step. Designed with beginners in mind, this tutorial provides a hands-on approach to learning n8n's basic functionalities.
deborah
Deborah
HTTP Request node
Item Lists node

Pulling data from services that n8n doesn’t have a pre-built integration for

You still can use the app in a workflow even if we don’t have a node for that or the existing operation for that. With the HTTP Request node, it is possible to call any API point and use the incoming data in your workflow Main use cases: Connect with apps and services that n8n doesn’t have integration with Web scraping How it works This workflow can be divided into three branches, each serving a distinct purpose: 1.Splitting into Items (HTTP Request - Get Mock Albums): The workflow initiates with a manual trigger (On clicking 'execute'). It performs an HTTP request to retrieve mock albums data from "https://jsonplaceholder.typicode.com/albums." The obtained data is split into items using the Item Lists node, facilitating easier management. 2.Data Scraping (HTTP Request - Get Wikipedia Page and HTML Extract): Another branch of the workflow involves fetching a random Wikipedia page using an HTTP request to "https://en.wikipedia.org/wiki/Special:Random." The HTML Extract node extracts the article title from the fetched Wikipedia page. 3.Handling Pagination (The final branch deals with handling pagination for a GitHub API request): It sends an HTTP request to "https://api.github.com/users/that-one-tom/starred," with parameters like the page number and items per page dynamically set by the Set node. The workflow uses conditions (If - Are we finished?) to check if there are more pages to retrieve and increments the page number accordingly (Set - Increment Page). This process repeats until all pages are fetched, allowing for comprehensive data retrieval.
jon-n8n
Jonathan
Merge node

Joining different datasets

Task: Merge two datasets into one based on matching rules Why: A powerful capability of n8n is to easily branch out the workflow in order to process different datasets. Even more powerful is the ability to join them back together with SQL-like joining logic. Main use cases: Appending data sets Keep only new items Keep only existing items
jon-n8n
Jonathan
GitHub node
HTTP Request node
Merge node
+11

Back Up Your n8n Workflows To Github

This workflow will backup your workflows to Github. It uses the public api to export all of the workflow data using the n8n node. It then loops over the data checks in Github to see if a file exists that uses the workflow name. Once checked it will then update the file on Github if it exists, Create a new file if it doesn't exist and if it's the same it will ignore the file. Config Options repo_owner - Github owner repo_name - Github repository name repo_path - Path within the Github repository >This workflow has been updated to use the n8n node and the code node so requires at least version 0.198.0 of n8n
jon-n8n
Jonathan
Webhook node
Item Lists node
Respond to Webhook node

Convert JSON to an Excel file

Send a simple JSON array via HTTP POST and get an Excel file. The default filename is Export.xlsx. By adding the (optional) request ?filename=xyz you can specify the filename. NOTE: do not forget to change the webhook path!
dickhoning
Dick

Implement complex processes faster with n8n

red icon yellow icon red icon yellow icon