A build story for Horizon Media · Blu
How we built a
dynamic taxonomy
“Is it a dynamic taxonomy where you type in and say, I’m interested in the US Open? The taxonomy all of a sudden just generates out across Blu.”
This page is the answer. Not slides. The real engine is running behind every section, reading the same shared library it has been educating since day one. Every number below is live from that library right now.
Canonical entities
Aliases learned
Aliases embedded
Sources reconciled
Crosswalk mappings
Decisions logged
Live from the engine’s serving layer at page load. No cached decks.
The whole story, in four lines
Type an event. The taxonomy builds itself across Blu. No hand-mapping between events.
One thing, mapped once. Sprite is Sprite in every dataset, every client, forever. Never remapped.
It gets smarter with every dataset. Each confirmed match is deposited, so the next dataset costs less than the last.
Your data never moves. Raw rows stay in Snowflake. Only what the engine learns crosses over.
That is what was asked for in the room on July 22, and it is what got built. Everything on this page runs against the real engine and the real library, live, while you read. The charts and graphs below are the proof, not the point.
Start here
three things to try, in order · about four minutes
Type “US Open”
Watch the engine surface both the tennis and the golf championship, and refuse to guess between them.
jump there ↓2Onboard a new dataset
Point the engine at a league it has never ingested and watch it derive, score, and report the cost. Run it twice and watch the cost fall.
jump there ↓3Ask the taxonomy a question
Plain language in, a grounded answer out, with the exact entities and sources it used.
jump there ↓Prefer to watch? Press and the page demos itself in about a minute.
First, the ask itself. This whole build answers questions asked out loud in one meeting, so the questions lead.
01 · The ask, in Bob's words
The problem he has already paid to not solve
On July 22, Bob Lord named exactly what he wants and exactly why he does not believe it yet. We are building against his words, not a softened version of them.
“We've mapped Seven Up to Sprite to this to that, not 100 times, and then we're still mapping it again... get to the point where my taxonomy is educated based on what I've learned. That's where I think we're missing the boat.”
“Above Data sold us the whole idea that they could do dynamic taxonomy... I'm only being skeptical, you guys, only because I've seen it.”
“I have not seen that scale from 24 hours across an entire big dataset. That's my skepticism.”
“What can you do in seven days if you take Sunday off?”
Two vendors promised him a dynamic taxonomy and did not deliver. His team still remaps the same brands, client after client, dataset after dataset. The fix is not a bigger mapping project. It is a taxonomy that learns: map an entity once, and every future dataset finds it on its own.
02 · How it works
The resolution ladder
Any incoming name or id descends five rungs, cheapest first, and stops at the first confident answer. Anything below the confidence bar goes to a person, never to a guess. That discipline is what makes it trustworthy.
Watching
A native id the crosswalk knows
Instant. Free. Mapped once, found forever.
T0 · Native idlive
Already in the crosswalk. Seen before, answered instantly, free.
T1 · Known aliaslive
Matches a name the library already knows, checked on word boundaries and vetoed by the rule lists.
T2 · Embeddinglive
Nearest neighbors by meaning, then a scorer that demands surface-form agreement before it auto-accepts.
T3 · Agent adjudicationlanding next
A reasoning agent adjudicates the gray band. Landing next; never auto-publishes.
T4 · Novel proposallanding next
Nothing matched. The engine proposes a new canonical entity for a human to confirm. Landing next.
The education loop
Every confirmed answer deposits a crosswalk row, new aliases, and negative memory back into the library. Dataset ten costs less than dataset one. That is what “educated” means here: the taxonomy remembers what it learned, so nobody maps Seven Up to Sprite a hundred and first time.
Now stop reading and try it. Everything from here down is the live engine, and it is answering you, not a script.
03 · Live · The real engine, not a mockup
Type an entity. Watch it resolve.
This input calls the production resolver against the shared library: 12.6 thousand entities and every alias it has learned. The call you make writes a real row to the append-only decision log, exactly like production traffic.
04 · Live · Generates across Blu
Name a thing. Watch it generate.
Pick an entity and its footprint expands across the Blu sources that really carry crosswalk rows for it, counted live from the library. This is the moment asked for in the room: point at a thing, and the taxonomy generates out across Blu.
Mapped once means never mapped again. Here is the exact Sprite and Seven Up case from the room, run for real.
05 · Live · The educated crosswalk
Mapped once. Never mapped again.
Bob's Sprite and Seven Up example, run for real. We mapped each brand once in a first dataset. Then a second dataset arrived with new ids, variant labels, and a typo. Nobody touched a mapping.
“We've mapped Seven Up to Sprite to this to that, not 100 times, and then we're still mapping it again.”
Dataset B resolutions, from the decision log
| incoming label | tier | confidence | landed on |
|---|---|---|---|
| “seven up soda” | t2 | 82.8% | review queue |
| “Seven-Up” | t1 | 95.0% | brand-7up |
| “sprite zero sgar” | t2 | 94.8% | brand-sprite |
| “Sprite Zero Sugar” | t1 | 95.0% | brand-sprite |
What this proves
The variant label found the brand by alias. The typo found it by embedding. The loosest phrase went to a person instead of being guessed. Zero false accepts, zero re-mapping, and every step above traces to a real decision-log row you can audit.
Type any of these labels into the resolver above and watch the same thing happen live, on your own click.
And here is the test the last vendors failed: bring a brand new dataset and watch the taxonomy derive itself, then watch it get cheaper because the library learned.
06 · Live · Onboard a new dataset
Bring a dataset it has never seen.
This is the test the last vendors failed. Pick a league. The engine pulls its teams from TheSportsDB, a real feed this library has not ingested, and runs every label through the ladder while you watch. It ends in a measured resolution report, not a promise.
Dry run by default: the engine resolves and scores, writes nothing, so this is repeatable forever.
Educate commits the run for real: crosswalk rows, aliases, and human-confirmed new entities land in the shared library, on the record.
07 · Live · Ask the taxonomy
Ask it a question. It answers from what it knows.
A smart system you can talk to, not a table you have to query. Your question is embedded into the same space as the library, the nearest entities are retrieved, and an agent answers from those rows only, citing exactly what it used. When the library has nothing, it says so.
Everything resolved so far lands in one connected library. Open it. Click around. Every panel is a live query.
08 · Live · The library as a graph
One taxonomy, connected. Click anything.
Real rows from the canonical library: the unified sports hierarchy, the brand family, and the typed relationships between them. Click any node and the library opens it: aliases, crosswalk, relationships, and its own decision-log history.
The 12.6 thousand source-derived entities (Affinity brands, PlaceIQ categories, ShareThis interests) hang off this same library through the crosswalk; they are charted in the measurement section below. The dashed edges are real entity_edge rows: home venues, tour membership, who competes where. We draw what exists, nothing more.
09 · The swarm
Grown by agents, answerable to people
The taxonomy is not a nightly batch script. Specialized agents classify and resolve continuously, and every single decision, agent or human, lands in one append-only log. Above the confidence bar it publishes. Below it, a person decides.
Classifier
Reads raw labels and program titles, proposes what they are
live
Matcher
Runs the ladder: crosswalk, aliases, embeddings, the scorer
live
Precision gates
Veto lists, ambiguity checks, margin gate: kills false merges before they publish
live
Scout
Watches for new datasets, profiles them, publishes a measured resolution report per onboarding
landing next
Adjudicator
A reasoning agent for the gray band; capped confidence, never auto-publishes
landing next
Identity resolver
Stitches behavior to one BLU ID across datasets
unlocks with identity access
The append-only decision log · latest entries, live
enforced append-only in the database itself: reversals are new rows
| decision key | tier | action | actor | confidence | outcome |
|---|---|---|---|---|---|
| resolve:live-demo:sprite | t1 | auto_accepted | agent:matcher | 0.950 | brand-sprite |
| resolve:live-demo:us open tennis | t1 | auto_accepted | agent:matcher | 0.950 | event-usopen-tennis |
| resolve:live-demo:us open | t1 | queued | agent:matcher | 0.950 | queued for review |
| resolve:live-demo:sprite zero | t1 | auto_accepted | agent:matcher | 0.950 | brand-sprite |
| resolve:live-demo:us open | t1 | queued | agent:matcher | 0.950 | queued for review |
| propose:utah jazz | t4 | confirmed | human:taxonomy-demo-visitor | team-tsdb-134889 | |
| propose:portland trail blazers | t4 | confirmed | human:taxonomy-demo-visitor | team-tsdb-134888 | |
| propose:oklahoma city thunder | t4 | confirmed | human:taxonomy-demo-visitor | team-tsdb-134887 | |
| propose:minnesota timberwolves | t4 | confirmed | human:taxonomy-demo-visitor | team-tsdb-134886 | |
| propose:denver nuggets | t4 | confirmed | human:taxonomy-demo-visitor | team-tsdb-134885 |
When you used the resolver above, your resolution appeared here. The log is the audit trail and the training data for the next, calibrated scorer. Nothing publishes without a row; if a source cannot support an answer, the answer is that we do not know yet.
10 · Data flow + architecture
What moves, and what never does
Krish asked the right question in the room. Here is the precise answer, drawn.
“You're actually extracting data or insights from Snowflake and then storing it in a separate environment within Databricks.”
raw stays put
Snowflake
- Every Blu source table, read in place.
- Raw and household-level rows never leave.
- Zero-copy is a boundary, not a slogan.
Horizon's data · never moves
ladder + swarm
The engine
- Resolution ladder, T0 through T4.
- Agent swarm proposes; gates and people decide.
- Every decision writes one log row.
Artists & Robots patented IP
derived only
Lakebase
- 12,679 canonical entities.
- 12,855 aliases, 12,855 embedded.
- 12,609 crosswalk mappings, 10,987 logged decisions.
Engine schema: Artists & Robots · client-derived rows: Horizon's
identity stitch
BLU ID
- Classified behavior resolves to one BLU ID across sources.
- Identity access committed by Horizon; the engine side is ready to consume it.
lands when access lands
Horizon's identity graph
Crosses into Lakebase
- 12,679
Canonical entities
one row per real-world thing
- 12,855
Learned aliases + embeddings
the education, made durable
- 12,609
Crosswalk mappings
native id to canonical entity, per source
- 10,987
Decision log rows
the append-only why, for every mapping
Never crosses
Raw source rows
read zero-copy, in place, in Snowflake
Household and individual-level records
the atomic data stays behind Blu's own walls
Client-private derivations
tagged to their owner, never shared across clients
The serving layer holds derived intelligence: what things are and how they connect. It answers in seconds because it is small, structured, and educated. The heavy raw reads stay where the data lives.
“There's no lag, and they're able to go through all our raw data where today, we have to look at summary tables, and it still takes, like, minutes to run a very basic question.”
The simple story holds up under a technical cross-examination. Here is the machinery, precisely, and who owns which part of it.
11 · For the skeptics · How it actually works
Under the story, the machinery.
The plain language above is not hand-waving over a demo script. Here is the same system described precisely: the ladder, the gates, the log, and the boundary. Everything on this list is running behind the surfaces you just used.
The resolution ladder · real method per tier
Exact (source id, native id) lookup in the crosswalk. Confidence 1.0. A duplicate native-id claim is treated as an integrity violation and queued to a person, never coin-flipped.
free · no model calls
live
Deterministic normalization (lowercase, unaccent, collapse), then word-boundary verification against every learned alias, vetoed by rule vocabularies: bare mascots, generic common words, directional prefixes, pro-context guards.
one indexed read
live
pgvector HNSW cosine kNN over voyage-3 1024-dimension alias embeddings. Retrieval only proposes. A hand-weighted feature scorer decides: cosine, jaro-winkler, token-set ratio, type compatibility. Auto-accept requires clearing the confidence bar AND a margin over the runner-up.
one embedding call
live
A reasoning agent adjudicates the gray band: pick from the retrieved candidates or reject them all. Its confidence is capped below the auto-accept bar by construction, so it can never auto-publish.
gray band only
landing next
Nothing matched: the engine proposes a new canonical entity, always human-confirmed before it exists. The onboarding exercise above runs this confirm flow today, with you as the reviewer.
human decision
landing next
Cheapest rung first, stop at the first confident hit. Everything below the auto-accept bar queues to a person. Cosine similarity is a feature the scorer weighs, never a decision on its own.
The log is the learning
Every decision, agent or human, writes one row to a decision log that the database itself enforces as append-only: reversals are new rows, history cannot be edited. Each row carries the full feature vector as it stood at proposal time, before the outcome was known.
That one design choice makes the log two things at once: the complete audit trail behind every published mapping, and the labeled training data for the calibrated scorer that replaces the hand-weighted one. Confirmations, rejections, and redirects are the curriculum. This is the mechanism behind “it gets smarter”: not a claim, a table you can query.
Zero-copy, precisely
Raw and household-level rows are read in place in Snowflake and never move. What materializes to the serving layer is derived intelligence only: canonical entities, crosswalk mappings, learned aliases, and aggregates. The boundary is drawn in full in section 10, and it is auditable: point at any Lakebase table and every row is a derivation, not a copy.
That is the precise answer to “you are extracting our data.” The engine extracts what things are and how they connect. The data itself stays home.
Precision guardrails · what tries to kill a bad merge before it publishes
Exclusion vocabularies
A bare mascot or a generic word can never carry a merge on its own. Vetoed, queued.
The ambiguity rule
More than one entity tied at the strongest match tier is definitionally a human question. AMC the theater chain versus AMC the network queues, always.
Directional prefixes
Central Michigan is not Michigan. A directional prefix on a containment match queues.
The margin gate
A top candidate must clearly beat the runner-up, not just clear the bar. Two strong candidates queue.
The type gate
A declared entity type hard-filters candidates. Absence of evidence is never a bonus.
Adversarial verification
Deterministic integrity verifiers are live today and vote to kill, never to approve. The model-tier adversarial bench lands with T3, same rule: it exists to break a bad merge before it publishes.
You can trip these on purpose in section 03: try “AMC” or a bare mascot and watch the engine queue instead of guess. False-accept resistance is a behavior you can test, not a slide.
Artists & Robots patented IP
licensed, not transferred- The resolution ladder itself: tiered, cheapest-first entity resolution with confidence gates
- The educated crosswalk: map-once, reuse-forever deposit and reuse mechanism
- The word-boundary matcher and its rules vocabularies
- The hand-weighted and calibrated match scorer, features logged at proposal time
- Real-time multi-agent orchestration: Scout, Classifier, Matcher, Entity Proposer, Identity Resolver, and the tiered adversarial verifiers
- The decision-log-as-training-data learning loop
Horizon owns
tenant-isolated, always- Client-derived entities, mappings, aliases, and rules created from Horizon's data
- Every client-private row is tagged to its owner and tenant-isolated; it never crosses clients
- Horizon's raw and household-level data, which never leaves Snowflake at all
This page shows the engine’s behavior, on purpose. The implementation, the weights, the vocabularies, and the orchestration are Artists & Robots patented IP and stay that way. What Horizon derives from Horizon’s data is Horizon’s.
12 · Measured, not asserted
The library, in numbers
Scale is a claim vendors make. These charts are queries. Each one runs against the live library when this page loads.
Entities reconciled, by source
Full Blu source taxonomies, not a hand-tuned demo set: Affinity brands, PlaceIQ place categories, ShareThis interest segments, each mapped to one canonical library through the crosswalk.
Every resolution, by tier and outcome
From the append-only decision log. Auto-accepted means the answer cleared the 92% bar and the gates. Queued means a person decides. There is no third path.
Decisions logged over time, cumulative
The education accumulating. Includes the blind re-derivation benchmark, the deliberate stress run against held-out variants, and every live resolution, including yours from this page.
Source trust weighting
Every source carries a default trust weight in the catalog; resolution confidence and scoring inherit it. Weighting per field, learned from the decision log, is the calibrated-scorer phase.
The falling cost of the next dataset
2
human mappings in dataset A
Sprite and 7UP, each mapped once. That is the entire human spend.
3/4
dataset B labels auto-resolved
New ids, variant labels, one typo. Zero human remapping. The loosest phrase queued to a person instead of guessing.
0
false accepts
A genuinely novel entity will always cost one human decision. The curve falls; it does not lie about reaching zero.
13 · Full disclosure
Every data source, on the record
Glass box means the sources are part of the product. This catalog is generated from the live crosswalk: every source the library actually reconciles today, what it provides, how it is treated, and how it is weighted.
Affinity Solutions (AFS, 7 Tables)
blublu-affinity · Behavioral / Purchase · Blu Spine & Enrichment
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
PlaceIQ
blublu-placeiq · Geolocation · Blu Data Ops
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
ShareThis
blublu-sharethis · Contextual · Blu Data Ops
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
tsdb-nba
tsdb-nba
trust weight: not yet cataloged
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
demo-clientB-loyalty
demo-clientB-loyalty
trust weight: not yet cataloged
Synthetic dataset built for the Sprite walkthrough on this page. Labeled as demo, never counted as client data.
thesportsdb
thesportsdb
trust weight: not yet cataloged
Licensed external reference data: leagues, teams, venues, schedules. Feeds canonical entities and events. No Blu or household data involved.
demo-clientA-pos
demo-clientA-pos
trust weight: not yet cataloged
Synthetic dataset built for the Sprite walkthrough on this page. Labeled as demo, never counted as client data.
TransUnion (Blu ID Spine)
blublu-transunion · Identity Spine · Blu Data Ops
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
Inscape (Vizio ACR + TMS Channel Map)
blublu-inscape · Viewership · Blu Data Ops
Read zero-copy, in place, in Snowflake. Only derived mappings for this source live in the serving layer.
Voyage AI · voyage-3
externalEmbedding model behind tier T2: every alias is embedded at 1,024 dimensions
live
Only alias strings are embedded. No Blu records of any kind are sent to the model.
Dataset registry: 0 measured onboardings published. Automated dataset onboarding (the Scout agent, with a per-dataset resolution report) is the next build phase; until it lands, this line stays honest at its real number.