See llms.txt for all machine-readable content.

Back to Templates

Match makeup shades from selfies with WhatsApp, GPT-4o-mini and Shopify

Created by

Created by: Rahul Joshi || rahul08
Rahul Joshi

Last update

Last update 6 hours ago

Categories

Quick overview

This workflow handles incoming WhatsApp messages and runs a daily schedule to match shoppers to in-stock makeup shades using OpenAI vision, Shopify inventory, and Google Sheets tracking, sending the top three shade recommendations with a Shopify cart link, escalating to Slack when a human needs to assist.

How it works

  1. Triggers when a WhatsApp message is received or every day at 10 AM (IST) to process order attribution and follow-ups.
  2. For WhatsApp chats, normalizes the incoming message, loads the customer’s session from Google Sheets, deduplicates repeat deliveries, and updates session state based on intent (start, consent, category, selfie, stop, or help).
  3. Sends WhatsApp interactive replies to collect consent and category choice, provides selfie instructions, and handles stop/help requests (including posting help requests to Slack).
  4. After consent, downloads the selfie from the WhatsApp Cloud API and sends it to OpenAI (gpt-4o-mini) to validate photo quality and estimate Monk skin tone and undertone with lighting-aware confidence.
  5. If the photo is unusable, sends retake guidance on WhatsApp and, after too many failed attempts, posts a handoff message to Slack and notifies the customer.
  6. If the photo is usable, reads the shade chart from Google Sheets, checks variant availability via the Shopify GraphQL API, scores in-stock shades against the estimated tone/undertone, and selects three recommended shades.
  7. Sends the recommendations on WhatsApp with a Shopify cart link (including a discount and tracking attributes), saves the match to Google Sheets, and updates the customer’s session.
  8. On the daily run, fetches recent Shopify orders to mark attributed matches as purchased in Google Sheets, then sends one approved WhatsApp template follow-up to eligible non-purchasers and marks them as nudged.

Setup

  1. Add WhatsApp Cloud API access (WhatsApp Trigger plus HTTP Header Auth bearer token) and set your WhatsApp phone number ID and approved follow-up template name in the configuration.
  2. Create a Shopify custom app with access to products and orders, add its Admin API token as HTTP Header Auth, and update the shop domain and cart domain values.
  3. Add OpenAI API credentials for the chat completions endpoint with vision (model set to gpt-4o-mini in the configuration).
  4. Connect Google Sheets credentials and replace the Google Sheet document ID wherever it appears, then create the required tabs: Sessions, Matches, and Shade Chart (with fields like category, variant_id, product_title, shade_name, depth, undertone, and price).
  5. Add Slack credentials and set the target channel ID for help requests and human handoff alerts.
  6. Populate the Shade Chart with your Shopify variant IDs and category mapping, then send a test WhatsApp message and selfie to validate end-to-end matching and follow-up behavior.