S3 and Shuffler integration

Make sure you have n8n installed and running:

• Install n8n on any platform using npm or Docker;
• Or let n8n host it for you on its cloud infrastructure.

In n8n, click the "Add workflow" button in the Workflows tab to create a new workflow.

Add the starting point – a trigger on when your workflow should run: on an app event, on schedule, on a webhook call, when called by another workflow, or manually. In some cases, the HTTP Request node might already serve as your starting point.

Drag the S3 node from the nodes panel on the left to the workflow canvas. Before dragging, choose the action S3 will perform. You can also swap steps and add the Shuffler node first, as the step 4.

Click on the S3 node, then click on the "Select credential" dropdown option next to the “Credential to connect with” field. Follow the authentication process to grant n8n access to your S3. You can also share credentials you've created, and store sensitive credential information in an external vault using n8n's external secrets feature.

Add the HTTP Request node to your workflow canvas and authenticate it using a predefined credential type. This allows you to perform custom operations, without additional authentication setup.

Click on the HTTP node to configure its settings. Use the Shuffler API docs to construct your request. The HTTP Request node makes custom API calls to Shuffler to query the data you need using the URLs you provide. See the example here.

Connect the output of the S3 node to the input of the Shuffler node or vice versa.

Use n8n's core nodes to transform data the way you need and extend the integration:

• Splitting: IF and Switch
• Merging: Merge, Compare Datasets
• Looping: IF and Loop Over Items
• Waiting: Wait
• Creating sub-workflows: Execute Workflow and Execute Workflow Trigger
• Error handling: Stop And Error and Error Trigger
• Code: write custom JavaScript or Python and run it as a step in your workflow.

Once you've configured everything, make sure to save your workflow and activate it. Activating a workflow means that it will run automatically every time a trigger node receives input or meets a condition. By default, all newly created workflows are deactivated.

Run the workflow to see if everything is working as expected. Based on your configuration, data should flow from S3 to Shuffler or vice versa. Easily debug your workflow: you can check past executions to isolate the mistake and fix it.