See llms.txt for all machine-readable content.

Back to Templates

Tune ROV PPO reward weights with OpenAI, Google Sheets and Gmail

Quick overview

This workflow starts a webhook-driven PPO training loop for an ROV, logs each round’s metrics to Google Sheets, uses OpenAI to propose updated reward weights, clamps changes to safe bounds, and emails a final report (or a failure alert) via Gmail.

How it works

  1. Receives a POST webhook request to start a tuning run and sets the run configuration (round count, episodes per round, initial reward weights, and safety bounds).
  2. Sends the current round’s configuration to your PPO trainer over HTTP to start training and then waits and polls the trainer’s status endpoint until it completes or fails.
  3. If training fails, sends a Gmail alert email with the run and round details to the notification address.
  4. When training completes, fetches round metrics from the trainer (mean reward, success rate, collision rate, and energy use).
  5. Appends the round metrics and the current reward weights to Google Sheets to keep an audit trail.
  6. Sends the latest metrics, current weights, and prior-round history to OpenAI to propose adjusted reward weights and a short rationale.
  7. Validates and clamps the proposed weights (bounded values and limited per-round change), updates the run history and best round, and either starts the next round or emails a Gmail final report with the results table and best-performing weights.

Setup

  1. Deploy and expose a PPO trainer REST API that supports POST /train and GET /status/{job_id} and GET /metrics/{job_id}, and set its base URL in the workflow.
  2. Add OpenAI credentials for the Chat Model node and ensure the model name is available in your account.
  3. Connect Google Sheets credentials and replace YOUR_GOOGLE_SHEET_ID (and sheet name, if needed) with a spreadsheet that has columns matching the appended fields (run_id, round, timestamp, metrics, reward_weights).
  4. Connect Gmail credentials and set a default notification email (or pass notificationEmail in the webhook request).
  5. Copy the webhook URL for the “rov-reward-tuning” endpoint into your caller and send a test POST body (optionally including totalRounds, episodesPerRound, and rewardWeights).

Requirements

  • An n8n instance, a PPO trainer reachable by REST (Python or MATLAB wrapper), an OpenAI API key, and Google accounts for Sheets and Gmail.

Customization

  • Swap OpenAI for Anthropic or NVIDIA models, edit weight bounds, change the ranking score, replace Gmail with Slack, or add human approval before each weight update.

Additional info

Cuts manual tuning effort, logs every round, reduces unsafe reward changes through clamping, and shortens iteration time so you can focus on analysis.