The Physics of Fashion: Why Semantic Intelligence Matters More Than AI Hype

The Language Problem That's Costing Retailers Sales

Americans call this cuffed. The British call it turned up. In the US, turned up means drunk. But the customer just says “the jeans with the fold at the bottom “. If your search doesn’t know those three phrases point to the same thing, you’ve missed the sale.

That’s one example. Here’s another more “industry ” example. A fashion magazine might call this a color paintbox. A brand might call the same thing “electric periwinkle “. The customer probably is just looking for “a light blue dress “. And even light blue, as you know, comes with a lot of variety.

A human merchandiser holds all of this in their head and still does their best to make the right call. They intuitively know what sounds better. The jeans with the fold at the bottom, the periwinkle dress that the customer is calling blue, the outfit that flatters the body shape the customer didn’t tell you she had. That intuition, along with lots of experience, is what separates a great stylist from a catalog lookup.

Intuitive doesn't work for agents

Agents don’t have that intuition. Agents need the knowledge to be explicit and consistent. They need every rule a stylist carries in their head to be written down somewhere for a machine to read and apply. Documentation of these rules (and intuition) is one of the challenges that our Mapp Fashion stylists have been working on for 10+ years; and the complexity and scale of these rules are some of the things that most retailers underestimate when they try to bolt AI onto a catalog that was built for humans.

Vocabulary isn't enough

Let me start off with some definitions to put us on the same page.

An ontology is a formal representation of what exists in a domain and how the things in it relate. The word semantic means meaning, not just adding labels. Physics, in the sense I’ll use it, is the set of constraints and interactions that make outcomes predictable. Combinatorics is the branch of math that tells you when you put things together, you (often) get far more combinations than you expect.

A knowledge graph is a queryable map of all of that organized into nodes, edges, labels, and crucially, semantic structure. The knowledge has to connect, and the connections have to mean something. If they don’t, you’re just storing labels. And labels aren’t going to be enough to create the physics we want.

The work of making fashion intelligently accessible for agents breaks into four phases: encoding, codifying, decoding, and finally, the agents themselves.

Encoding: Building the Ontology

Encoding is where you describe what things are. At Mapp, our ontological structure is broken into 5 foundational levels:

Two things most people get wrong here. First, semantics only matter in context: a neckline is meaningful for a dress, meaningless for a sneaker. Second, completeness matters as much as accuracy. “Pockets: none” is obvious for shoes, but it’s even more valuable on dresses. A schema that lets you assign “neckline: crew” to a sneaker might look consistent on paper, but it produces nonsense at query time. A schema that lets you say “pockets: none” is obvious for shoes, but even more valuable for dresses. Every feature has to know which categories it applies to otherwise the structure you think you’ve built will end up leading to nonsense.

Predominantly red dresses go fine with black or white jackets. Similar structure, different output. Specifics matter.

A similar logic applies to body shape. The drop waist detail flatters a tall frame because it complements the height. It fights an hourglass figure because it shifts the emphasis away from the natural waist definition. That’s one rule. We have tens of thousands of them.

And they cover more than color and shape. Outfit pairing is a whole category on its own. A biker jacket that has a strong structured collar. It pairs well with necklines that stay clean and contained (like a crew neck). It tends to clash with necklines that add competing geometry.

Each rule is made up of specific taxonomic entities to ensure consistency across the ruleset. And all of these rules sit directly on top of the roughly 25,000 taxonomic elements and 750 context categories. That’s not just seeing how many big numbers I can say in one sentence. They’re what our stylists, merchandisers, and data scientists have encoded after a decade of looking at real products, real outfits, and real customer behavior.

Decoding: turning structure into useful outputs

Once you’ve encoded the garment and codified the rules, you can decode it into really useful day-to-day things. This is where the structure we’ve put in place starts paying off.

Let’s start with something everyone knows, (fashion) trends. “Forest Fairies“, as an aesthetic, celebrates the enchanted-forest vibe: mushrooms, butterflies, moss, gentle animals; generally Earthy kinda things. I know you can picture it. An agent can’t picture it. So we have to translate this into something the agent can “picture”. We call these things Attribute Clauses; and they are something the agent can work with. Some examples:

1. For footwear: boots, specifically hiking or sheepskin boots that are predominantly brown or tan.
2. For tops: cotton or cotton-blend, predominantly yellow, pale yellow, or dark green.

Around twenty clauses in total are required to capture the Forest Fairies aesthetic. Each one of these clauses are built directly on the taxonomy.

Once a trend is decoded, your entire catalog can be classified against it automatically. Imagine something like a Justin Bieber Coachella moment that becomes a two-week trend that is no longer a marketing scramble. The system already knows which items in your catalog fit, and you can spin up a category page with an API call in minutes instead of days.

When you have all this data stored in a systematic and connected manner, you can then directly ask questions of the ontology. These questions can be second or third order steps away from the baseline taxonomy taking advantage of multiple ontological components. For example:

• Brand DNA and Color Rules: Do these colors fit my brand?
• Outfit Rules and Color Rules: What shoes go with red A-line dresses in our catalog?
• Trends, Outfits Rules, and Shape Rules: Which bohemian products cater to an hourglass silhouette?

The answers come from the structure of the data, not from an LLM doing the probabilistic guessing that they are known for. And to save anyone making the mistake: don’t wear boat shoes with an A-line dress. You can have that one for free 🙂

Agents: informed actors with deterministic guardrails

Now you’ve encoded the garments, codified the rules, and decoded it into really useful derivative structures. Here’s where the agentic workflows start to be effectively enabled.

Let’s start with returns as an example. I think it’s safe to say that most fashion retailers have a return problem and it would be great to have the support of an agent.

Let’s start with a goal: you want to reduce avoidable returns without hurting margin. An agent trying to help has to answer (at least) four questions.

1. Is this a size issue or a style issue? You need ontology for that. Fit-based attributes are different from aesthetic attributes, and the agent can only tell them apart if there is a taxonomy that makes the distinction.
2. Is this customer profitable despite the returns? Some serial returners are still net positive, and your analytics platform has to tell the agent which is which.
3. What can we do about it? The intervention has to match the diagnosis: suppress certain products in email, adjust size guidance on specific PDPs, hold back the wrong promotion.
4. Did it work? You need causal measurement tied to the action, not just correlation across a noisy dataset.

Without the semantic layer, the agent is simply left to guess at these answers. With it, the agent can reason about products in the same way a good merchandiser does. That’s the difference between an impressive demo and a production system that has the right context and guardrails and can reduce your return rate.

Brand DNA, encoded

Brand DNA, mentioned earlier, deserves a closer look, because it’s the question every brand team asks first. If agents are making autonomous decisions across your customer experience, how do they stay on brand?

Some brands have brand guidelines that are three sentences. Some have 200-page PDFs. Both can be encoded into rules that map back to the same taxonomy. Once that’s done, the agent has a deterministic way to check itself. Can I make this recommendation? Does this color combination fit this brand? Is this outfit on-brand or off-brand? The answer comes from the structure provided by deterministic rules, not from the mood of the LLM that morning.

This is where predictability becomes a competitive advantage. LLMs will always answer the question. That’s not the hard part. The hard part is answering the question the same way twice. Inconsistency is a risk no brand can take with its equity. A system grounded in deterministic ground truths gives brands the consistency they need to build trust with customers and confidence with their own teams.

Build it yourself, or don't

You can build this. The patterns aren’t secret. All you need is an ontology, a set of constraints, compositional rules, consistent labels across your catalog, and an agent that applies all of it with guardrails. Add some graph theory to help structure the data, and you have the recipe.

But what you’d be building isn’t just architecture, you need a living system.

Fashion doesn’t sit still. Every week brings a new silhouette on a runway, a new color story from a capsule collection, a new trend or aesthetic that gets a name on TikTok before it lands on a PLP. A periwinkle isn’t periwinkle forever. A trend called “Forest Fairies” this season is called “Mori Boys” next season, and the clauses that define it, often shift with it. The taxonomy has to stay stable and complete, the labeling has to stay consistent, and the Brand DNA encoding has to evolve with the brand. Otherwise the agent drifts, and every recommendation it makes drifts with it.

That’s the part most retailers underestimate. Architecture is half the job. The other half is keeping it current. And that’s an R&D function on top of running your actual business.

At Mapp, we have a team whose entire job is to scan the world of fashion and keep the ontology in sync with it. Stylists, taxonomists, and data scientists, all feeding the same graph that retailers’ agents reason from. When something new shows up on a catwalk, it gets codified. When a brand changes the naming of its color palette, the rules get updated. No single retailer can move at this cadence, because no single retailer sees across all the signals at once.

So yes, you can build the foundation yourself. We’ve spent ten years building ours, and we’re still building it every day. If you’d rather not start from scratch, come find me at any of the events we’re at or book a time with us.

Chat with our experts ›