Shopify launches Catalog API to cluster billions of products for smarter AI shopping.
Concept 1: The Core Problem — No Shared Schema Across Merchants
What is it?
Shopify hosts billions of product listings across millions of stores, but there is no universal standard for how merchants organize their products.
Why does this matter?
Consider two merchants selling the same protein powder:
Merchant A: Merchant B:
───────────────── ─────────────────────────────
1 listing 12 separate listings
"Protein Powder" "Protein Powder - Chocolate"
├── Chocolate flavor "Protein Powder - Vanilla"
├── Vanilla flavor "Protein Powder - Strawberry"
└── Strawberry flavor ... and so on
Both approaches are valid. But when an AI shopping agent tries to find "the best protein powder across all of Shopify," it must understand that these structurally different listings describe the same product line — without anyone explicitly telling it so.
The analogy
Think of it like two librarians organizing the same books differently. One groups all Shakespeare plays in one folder. The other gives each play its own shelf. A reader looking for Shakespeare needs to find all of them, regardless of how they were filed.
Concept 2: The Solution — Product Clustering and Universal Product Identifiers (UPIs)
What is it?
Product clustering is the process of grouping all related product listings — regardless of how merchants structured them — under a single Universal Product Identifier (UPI).
How it works visually
Merchant A listing ──────────────────────────────┐
└── Chocolate variant │
└── Vanilla variant ▼
┌──────────────┐
Merchant B listings ─────────────────────► UPI #4521 │
└── "Protein Powder - Chocolate" │ (Whey Protein│
└── "Protein Powder - Vanilla" │ Brand X) │
└── "Protein Powder - Strawberry" └──────────────┘
▲
Merchant C listing ──────────────────────────────┘
└── "Brand X Whey - Choc/Van/Straw"
Why it matters
- Merchants' products show up where they should across every AI agent
- Agents can search, compare, and recommend with true catalog awareness
- Opens up new distribution channels for merchants
Concept 3: The Two Ways Clustering Can Fail — Precision vs. Recall
What are they?
These are the two opposing failure modes in any clustering system.
Precision Failure
Different products get incorrectly grouped together
Example:
Men's Nike Air Max ──┐
├──► Same UPI ← WRONG!
Women's Nike Air Max─┘
Result: Buyer orders women's sneakers → receives men's version
Recall Failure
A product that belongs in a cluster gets left out
Example:
Jacket in Red ──┐
Jacket in Blue ──┤
Jacket in Green ──┼──► UPI #8832
Jacket in Yellow ──┤
Jacket in White ──┘
Jacket in Black ──► No UPI (missed!) ← WRONG!
Result: Buyer searching for black jacket can't find it,
or sees it as a confusing duplicate
The tension between them
Tighten thresholds ──► Better precision, worse recall
Loosen thresholds ──► Better recall, worse precision
Shopify's choice: Precision-First
| Scenario | Outcome |
|---|---|
| Precision fails | Wrong product surfaces → buyer gets wrong item |
| Recall fails | Item stays findable, just ungrouped |
"Buyers can forgive missing a variant, but they won't forgive getting the wrong product."
The strategy: set a hard precision threshold, then maximize recall within that constraint.
Concept 4: The "Core Value Proposition" Framework
What is it?
This is the central principle used to decide whether two products should be clustered together or kept separate.
The key question
"What is the buyer primarily purchasing this product for?"
- If an attribute does NOT change the answer → it's a variant (same cluster)
- If an attribute DOES change the answer → it's a different product (separate cluster)
Examples
PROTEIN POWDER
──────────────
Buyer's goal: nutrition, protein content, fitness
Chocolate flavor ──┐
├──► Same cluster ✓
Vanilla flavor ──┘
Flavor doesn't change WHY you buy it → variants
PAINT
─────
Buyer's goal: a specific appearance/color
Midnight Blue ──► Cluster A
Sage Green ──► Cluster B
Color IS what you're buying → separate products
Why this matters
This framework gives the AI model a principled rule to apply across any product category it has never seen before, rather than memorizing category-specific rules.
Concept 5: Why LLMs Are the Right Tool for This Job
The problem with traditional rules
Product data lives in unstructured, inconsistent fields:
- Titles
- Descriptions
- Tags
- Vendor fields
- Option names
- Each merchant's own naming conventions
The same signal might live in different fields across different shops. Rigid rules break when merchants don't follow expected patterns.
What LLMs bring
LLMs can read product data in context the way a human shopper would:
Human reading a catalog sees:
"Geometric Design Black Flatwoven Rug"
"Geometric Design Cream Flatwoven Rug"
→ Instantly understands: same rug, different colors
LLM does the same thing, at scale, across millions of shops
The key insight
Don't ask the model to "just cluster these products" — that's too vague. Instead, teach it the principle (core value proposition) and let it apply that principle to new categories.
Concept 6: Starting Small — Intra-Store Clustering
What is it?
Instead of solving the entire global problem at once (billions of products across all merchants), the team started with a simpler question:
"Which listings inside a single shop are actually variants of the same product?"
Why this is smarter
Global approach:
Compare everything to everything
Billions × Billions = Impossible candidate space
Intra-store approach:
Millions of small, independent shop-level tasks
Each shop solved separately → tractable
The key insight
Many merchants sell unique products (their own brand, designs, formulations). For them, the question is never "does another merchant sell this?" — it's just "are my own listings organized correctly?"
Concept 7: Rules Before LLMs — The Singleton Detector
What is it?
A pre-filtering step that determines whether a shop's products need clustering at all, before any LLM is involved.
How it works
Shop's theme code (Liquid templates)
│
▼
Singleton Detector
│
┌──────┴──────┐
│ │
Products already Products need
standalone? clustering
│ │
▼ ▼
Each product Send to LLM
gets its own pipeline
UPI directly
(no LLM needed)
Why this matters at scale
All Shopify shops
│
├── Large majority → Singletons or deterministic rules
│ (tag matching, title similarity,
│ metafield grouping)
│
└── Small percentage → Need LLM-based clustering
Principle: Use deterministic rules first. Reserve LLMs only for genuinely ambiguous cases.
This dramatically reduces cost while protecting precision.
Concept 8: Pre-Chunking — Giving the LLM the Right Context
The problem
Most merchant catalogs are too large for a single LLM prompt. But clustering requires context — the model needs to see naming patterns to understand a shop's structure.
Too much context: Whole shop → LLM overwhelmed, poor quality
Too little context: Isolated products → LLM loses naming patterns
The solution: Neighborhoods (Chunks)
Give the model coherent bundles of related products — similar enough to reveal naming conventions, diverse enough to show what varies within a product line.
How chunks are built: ANN + Average Linkage
Step 1: Build a nearest-neighbor graph
Every product → converted to embedding (vector)
│
▼
HNSW algorithm (via FAISS)
│
▼
Each product connected to its 100 closest neighbors
by cosine similarity
Step 2: Sparse average linkage merging
Start: every product is its own cluster (singleton)
│
▼
Merge closest cluster pairs
(using UPGMA: average of all member-to-member distances)
│
▼
Stop when:
- Closest pair exceeds distance threshold (0.25), OR
- Cluster hits max size (200 products)
Result: Chunks of up to 200 semantically related products — the LLM's "neighborhood"
Concept 9: The Two-Stage LLM Pipeline
Overview
Pre-chunked products
│
▼
┌───────────────┐
│ Stage 1 │ ← Proposes clusters
│ Brand+Model │ (creative work)
│ Extraction │
└───────┬───────┘
│
▼
┌───────────────┐
│ Stage 2 │ ← Critiques proposals
│ Outlier │ (easier, more reliable)
│ Detection │
└───────┬───────┘
│
▼
Final UPI assignments
Stage 1: Brand + Model Extraction
The LLM receives a chunk and returns a structured JSON object:
{
"patterns": ["color varies in suffix", "size in brackets", ...],
"products": {
"product_1": {"brand": "NikeX", "model": "Air Runner"},
"product_2": {"brand": "NikeX", "model": "Air Runner"},
"product_3": {"brand": "NikeX", "model": "Trail Blazer"}
}
}
Products sharing the same brand:model pair → grouped under one UPI
Key trick: The schema requires the patterns array before the products object, forcing the model to analyze the shop's naming conventions before labeling individual products.
Stage 2: Outlier Detection
Input: Proposed cluster (e.g., 15 sweatshirt blazers + 3 tailored blazers)
│
▼
LLM critiques: "Do all these share the same core value proposition?"
│
▼
Output: {
"reason": "Three tailored blazers differ in construction and buyer intent",
"outliers": ["product_8", "product_12", "product_15"]
}
Why critique is easier than creation:
- Stage 1 must decide: how many groups? what are they? where does each product go? → Global, complex
- Stage 2 only asks: does this item fit here? → Local, simpler
Stage 1 proposes. Stage 2 reviews. Together they catch errors neither would catch alone.
Concept 10: Dynamic Structured Output Schemas — Making Reliability an Engineering Constraint
The problem with free-form LLM output
Early attempts with unstructured prompts produced:
- Skipped product IDs (some products never got a UPI)
- Hallucinated brand names (products incorrectly split)
- Malformed JSON (pipeline breaks)
The breakthrough: Dynamic Schema Generation
Instead of hoping the LLM returns correct output, the schema guarantees it.
For a chunk with products [ID_1, ID_2, ID_3]:
Schema generated dynamically:
{
"required": ["ID_1", "ID_2", "ID_3"], ← Every ID must appear
"properties": {
"ID_1": {"brand": string, "model": string},
"ID_2": {"brand": string, "model": string},
"ID_3": {"brand": string, "model": string}
}
}
LLM literally cannot skip a product → completeness guaranteed
Additional schema design techniques
| Technique | Effect |
|---|---|
patterns array required before products | Forces shop-level reasoning before individual labeling |
| Enum constraint on outlier IDs | LLM can only output real product IDs — no hallucinations |
| Normalized product IDs (1, 2, 3... instead of real IDs) | Increases cache hit rates, reduces cost |
Schema as a hyperparameter
The output schema has as much impact on cluster quality as the prompt text itself.
- Cleaning non-ASCII tokens from schema → +8% recall
- Misaligning title format between prompt and schema → recall drops
- Removing all structure → parsing errors rare (0.25%) but quality didn't improve
Concept 11: Measuring Quality — Ground Truth and LLM Judges
Two complementary measurement approaches
┌─────────────────────────────┐ ┌──────────────────────────────┐
│ Human-Labeled Datasets │ │ LLM-Based Judges │
│ │ │ │
│ • Curated product samples │ │ • Evaluate precision/recall │
│ • Stratified across │ │ at scale │
│ categories & merchant │ │ • Calibrated against human │
│ types │ │ annotations │
│ • Ground truth baseline │ │ • Continuous monitoring │
└─────────────────────────────┘ └──────────────────────────────┘
│ │
└──────────────┬───────────────┘
▼
Track quality continuously as
catalog grows and changes daily
Concept 12: The Road Ahead — Cross-Store Clustering
What intra-store clustering solved
Within one merchant's store:
"Protein Powder - Choc" ──┐
"Protein Powder - Van" ──┼──► UPI #1234 (within Merchant A)
"Protein Powder - Straw"──┘
What cross-store clustering must solve
Merchant A's UPI #1234 ──┐
Merchant B's UPI #5678 ──┼──► Global UPI (same real-world product)
Merchant C's UPI #9012 ──┘
Across millions of merchants, with different naming conventions,
different structures, and much more noise
Why it's harder
- Intra-store: one merchant's consistent naming conventions
- Cross-store: millions of different naming conventions, languages, abbreviations, and catalog structures all describing the same real-world items
Summary: The Full Architecture
Billions of Shopify products
│
▼
┌─────────────────────┐
│ Singleton Detector │ ← Parse theme code
│ (Rules-first) │ Deterministic resolution
└──────────┬──────────┘
│ Ambiguous cases only
▼
┌─────────────────────┐
│ Pre-Chunking │ ← ANN (HNSW/FAISS) +
│ (Neighborhoods) │ Sparse average linkage
└──────────┬──────────┘ Max 200 products/chunk
│
▼
┌─────────────────────┐
│ Stage 1: Propose │ ← Brand + Model extraction
│ (Dynamic Schema) │ Core value proposition
└──────────┬──────────┘ framework
│
▼
┌─────────────────────┐
│ Stage 2: Critique │ ← Outlier detection
│ (Enum constraints) │ Precision protection
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Quality Measurement│ ← Human labels +
│ │ LLM judges
└──────────┬──────────┘
│
▼
UPI assignments
(Precision-first,
recall maximized
within constraint)
The core insight threading through every concept: treat product identity as a human judgment problem, encode that judgment as a principled framework, then engineer the system so the AI reliably applies that framework at billion-product scale.