CASE STUDY
OMNI INTELLIGENCE TOOL
Cox’s chatbot and call transcripts held the richest dataset about customer friction, but nobody could use it. I led the product from pre-POC through Call Pipeline V2: authoring requirements, introducing affinity mapping as the taxonomy method, and designing the human-in-the-loop validation that took unknown classifications from 28% to under 2%.
LAUNCHES AUGUST 2026
CLIENT
Cox Communications, co-built with Cox Data Science
YEAR
2025 to 2026
ROLE
Director of UX, BBDO Chicago · Product lead from pre-POC through Call Pipeline V2
DELIVERABLES
Requirements · OOUX architecture · Taxonomy · HITL validation · Evaluation framework · Phase II roadmap
THE SITUATION
Cox handles enormous volumes of customer interactions across web, app, a virtual agent named Oliver, and call center systems. Buried in those transcripts was the single richest dataset about where customers get stuck: which journeys break, which intents the chatbot fumbles, which issues force escalation to a human. But the data was siloed by channel, unstructured, and unlabeled. Teams dug through conversation snippets by hand and assembled an incomplete picture slowly, while the business pressure to grow self-service kept rising. Decisions about chatbot investment were being made on anecdote.
The question the work set out to answer: can we mine support conversations systematically enough to tell the business exactly where friction lives, and trust the answer?
HOW IT STARTED
In November 2025, the AI analysis pipeline running against a batch of call transcripts failed mid-engagement. I audited the failed pipeline, documented its failure modes, and re-ran the analysis by hand across 1,191 transcripts. That manual pass did two jobs at once: it delivered the findings the client needed on schedule, and it showed precisely where and why the automation had broken.
The audit is where OMNI Intelligence began. I authored the prompt-engineering documentation for the original Gemma-based pipeline, and when the evidence supported it, recommended migrating the classification layer to Claude, a recommendation Cox adopted. From January 2026 I was the sole BBDO resource on the build, partnering directly with Cox’s lead data scientist toward an August 2026 launch. Data science owned the models throughout. I owned everything that made the models useful and trustworthy.
The hard problems were not algorithmic. They were definitional, epistemic, and organizational: what should the categories be, how do we know the output is right, and how do we make stakeholders trust machine classifications enough to act on them.
ARCHITECTURE BEFORE MODELS
Before any tuning, I defined the system’s information architecture using OOUX. The core objects (Journey, Turn, Triplet, Agent, Page, Outcome, Alert) and the data model that relates them were design decisions, made and documented before they became engineering constraints.
The model that holds the product together is Turn to Interaction to Journey. Each level captures different intelligence, and the full multi-channel picture lives at the journey.
TURN
A single utterance from a customer or agent. Captures intent and behavior.
INTERACTION
A complete conversation session spanning web and app browsing, the virtual agent, and live agent chat. Captures containment and escalation.
JOURNEY
Every interaction within a 24-hour contact window, where the full multi-channel picture lives. Captures high-intent signals, conversion patterns, friction points, and churn risk.
FIG 1 · The Turn to Interaction to Journey architecture.
WHY IT MATTERS
The taxonomy is the heart of the tool. If the categories are wrong, every insight downstream is wrong. Treating classification as information architecture, owned with research rigor rather than inherited from a database schema, is the single decision the rest of the project stands on.
A TAXONOMY BUILT LIKE A RESEARCH ARTIFACT
I introduced affinity mapping, a core UX research method, as the way the taxonomy would be designed and revised. Mapping real transcript language exposed everything wrong with the flat first version: overlapping codes, ambiguous boundaries, and a single catch-all absorbing 39% of labels, which is not analytically useful. The revised structure used prefix-based separation (intent_, signal_, response_) and domain-specific granularity, so troubleshoot_internet, troubleshoot_tv, and troubleshoot_equipment stopped collapsing into one bucket. The current taxonomy spans 79 codes across three dimensions covering intent, customer behavior, and agent behavior.
HUMAN-IN-THE-LOOP AS THE CREDIBILITY ENGINE
A proof of concept earns exactly as much trust as its accuracy on day one, so I designed structured human review into every checkpoint rather than relying on confidence scores alone. The protocol: stratified sampling so validation sets represent the full distribution of conversation types, blind review so human labels are made before seeing the model’s, and systematic disagreement analysis so every human-model conflict feeds the next taxonomy revision.
01
Evaluate.
Confusion matrix analysis on the current model version.
02
Identify.
Isolate the top failure modes and worst-performing classes.
03
Refine.
Merge ambiguous classes; adjust taxonomy boundaries.
04
Label.
Active learning: 50 to 150 targeted examples where the model is weakest.
05
Retrain.
Ship the next version and return to step one.
28% → <2%
Unknown classification rate across taxonomy iterations
1,191
Transcripts analyzed by hand in the founding audit
79 / 3
Codes and dimensions in the current taxonomy
PARTNERING ON THE MODEL ITSELF
Cox’s data science team built the models; my job was writing requirements a model could actually be held to, which requires knowing the mechanics. The turn encoder is a multi-task architecture on a SentenceTransformer backbone with three heads (turn intent, customer behavior, agent behavior), trained with class weighting for imbalance, split by conversation ID rather than by turn to prevent leakage, and evaluated on macro F1 so rare classes count.
Two enrichment recommendations came specifically from the UX seat because they connect model output to experience outcomes: integrating Medallia NPS, so a frustration signal at turn 15 can be tied to a detractor score, and capturing the starting page URL, because a customer arriving at the virtual agent from the cancel page is in a different state than one arriving from billing.
UX METHODS, TRANSLATED FOR AI
The larger contribution is a working demonstration that classic UX methods transfer to AI systems with their rigor intact.
TRADITIONAL
APPLIED TO AI
Journey mapping: touchpoints and pain points
Data collection points, inference opportunities, intervention moments
Problem statements: user needs to solve
Prediction framing: what should the model predict, when, at what confidence
Card sorting: users’ mental models for IA
Intent taxonomies and behavior classification built with stakeholders
Usability testing: observe users with prototypes
Validate predictions with domain experts; tune thresholds; test explainability
Affinity mapping: synthesize findings into themes
Cluster model failure modes; surface patterns in outputs
Empathy mapping: thoughts, feelings, pain points
Edge cases, bias risks, failure modes, HITL checkpoints
BEYOND ANALYTICS
The journey data structure is a foundation, not an endpoint. I defined the roadmap from descriptive (patterns identified, manual insight generation) to predictive (RAG-based journey knowledge, natural language queries like ‘show me journeys where competitive pricing led to churn,’ churn prediction) to prescriptive: a contextual multi-armed bandit architecture for real-time next-best-offer personalization. The Phase II proposal gives the client a credible multi-year path instead of a one-off dashboard.
WHERE IT STANDS
OMNI Intelligence launches in August 2026. What is validated to date: the taxonomy’s classification performance through the HITL loop, the methodology’s adoption by the client’s data science organization as its validation standard, and a repeatable four-stage playbook (Foundation, Co-creation, Iteration, Scale) now documented for future AI builds at the agency. Post-launch business results will be added when they exist; this page will not claim them early.
Interactive case study · taxonomy evolution, the HITL loop, and the metrics that matter.
WHAT THIS DEMONSTRATES
AI product leadership grounded in judgment. My contribution was bringing UX research rigor, affinity mapping, structured validation, and requirements grounded in user need into a data science build. That is the human-in-the-loop discipline separating AI products people trust from AI demos people ignore.
NEXT PROJECT
