Route AI queries cost‑efficiently with GPT‑4o‑mini, GPT‑4o and confidence scoring

This workflow implements a cost-optimized AI routing system using n8n. It intelligently decides whether a request should be handled by a low-cost model or escalated to a higher-quality model based on response confidence.

The goal is to minimize LLM usage costs while maintaining high answer quality.

A query is first processed by a cheaper model. The response is then evaluated by a confidence-scoring AI agent. If the response quality is insufficient, the workflow automatically escalates the request to a more capable model.

This approach is useful for building scalable AI systems where most queries can be answered cheaply, while complex queries still receive high-quality responses.

How It Works

1. Webhook Trigger

   • Receives a user query from an external application.

2. Workflow Configuration

   • Defines parameters such as:
     • confidence threshold
     • cheap model cost
     • expensive model cost

3. Cheap Model Response

   • The query is first processed using GPT-4o-mini to minimize cost.

4. Confidence Evaluation

   • An AI agent analyzes the response quality.
   • It evaluates accuracy, completeness, clarity, and relevance.

5. Structured Output Parsing

   • The evaluator returns structured data including:
     • confidence score
     • explanation
     • escalation recommendation.

6. Decision Logic

   • If the confidence score is below the configured threshold, the workflow escalates the request.

7. Expensive Model Escalation

   • The query is reprocessed using GPT-4o for a higher-quality answer.

8. Cost Calculation

   • Token usage is analyzed to estimate:
     • total cost
     • cost difference between models.

9. Final Response Formatting

   • The workflow returns:
     • AI response
     • model used
     • confidence score
     • escalation status
     • estimated cost.

Setup Instructions

1. Create an OpenAI credential in n8n.

2. Configure the following nodes:

   • Cheap Model (GPT-4o-mini)
   • Expensive Model (GPT-4o)
   • OpenAI Chat Model used by the confidence evaluator agent.

3. Adjust configuration values in the Workflow Configuration node:

   • confidenceThreshold
   • cheapModelCostPer1kTokens
   • expensiveModelCostPer1kTokens

4. Deploy the workflow and send requests to the Webhook URL.

Example webhook payload:

{
  "query": "Explain how photosynthesis works."
}