Inside Our Image Matching Algorithms: A Look Behind the Scenes

It’s not magic, even though it often seems like it is. 

If you talk to our customers, they’ll tell you how their conversion rate, AOV, and customer retention suddenly skyrocketed after Syte came into the picture. If you talk to their shoppers, you’ll hear them delight over hyper-personalized customer experiences, endless inspiration and discovery, and innovative, intuitive customer journeys that make buying the products they love easy. 

Sorry for the spoiler, but the “magic” is actually our visual AI software and the image matching algorithms that make it run.

After fielding many questions from customers and prospects on how exactly Syte’s Product Discovery Platform works, I decided to write this piece. And while I can’t share everything about our proprietary technology, I can give you the most important facts. 

What’s Really Going On

Syte’s image matching algorithms power all of our search-based solutions, from image search and augmented text search to “shop similar” or “complete the look” product carousels.

Our visual AI combines the expertise of leading minds in machine learning and deep neural networks. Fortunately for you, I’m not an R&D guy, so I’ll spare you the technical explanation. Here’s a simplified rundown of how it works:

1. Image scanning – When we start working with a new customer, they provide us with a catalogue of all of the SKUs in their inventory. We run every product image through our visual AI, where the matching algorithms identify all possible visual features of each item (e.g., color, pattern, fit, length, texture, shape, and much more). 
2. Vectorizing – Then, the algorithms create a vector for each item, which functions like a unique ID that includes all of the product’s visual descriptors. Later, when a site visitor performs a visual search or engages with a recommendation carousel, the algorithms produce results by identifying relevant vectors within the product catalogue. 
3. Deep tagging – Our visual AI software can also produce a comprehensive text-based list of product attributes called “deep tags.” This automated process enables standardization among product tags, which is essential to activities like text search, merchandising, and product recommendations. Deep tags generated by visual AI also include different ways to describe the same item so that customers can discover all relevant products even if they search using a different keyword.

Once the image matching algorithms have “learned” each product, they can retrieve it at any time to produce highly relevant visual search results and product recommendations, or even to identify complementary items. 

The Matching Algorithms at Work

Here are two examples of what the image matching algorithms look like in action.

• Visual Search

Imagine you’ve been scrolling through Instagram when you see a photo of an influencer you admire wearing an outfit you love. You feel inspired. This is totally your look. You are ready to buy – now you just need to find it online, and in your price range. 

You snap a screenshot of the eye-catching outfit and head to your favorite retailer, Ardene. There, you upload the screenshot to their visual search tool. 

Thanks to Syte’s multi-object detection, each piece of the outfit (in this case, it belongs to fashion influencer Danielle Bernstein) has been identified and the image matching algorithms seamlessly allow you to find visually similar products to the ones Danielle is wearing. That means you can quickly and easily buy a similar sweater and pants from a reputable retailer that you already trust.  

To you, as a shopper, the experience is a little magical. One moment you were scrolling through Instagram and the next you were buying the exact same look – but in your price range and on a trustworthy site. The product search and discovery experience was almost instant, and you didn’t have to do any of the usual scrolling through product landing pages that you normally would. 

Of course, as we know, the matching algorithms aren’t magic. They’re designed precisely to create this kind of effortless and intuitive experience. Behind the scenes, our visual AI technology is hard at work scanning the image you upload, detecting every item within it, and identifying every visually similar result within the company’s product inventory. 

• Discovery Button

The Discovery Button comes into play when a shopper spots a product (either on your PLP or on a PDP), and wants to see additional, visually similar items within your inventory.

Imagine you’re in the market for a new watch in the UK. You head over to H. Samuel’s website and click on the “Men’s Watches” category page. Although you don’t have a specific style in mind, you’ve always liked Casio watches so you select that brand and start scrolling. A few rows down, you see a stainless steel watch that catches your eye — but it has a bit of yellow on the face, and that’s just not you. 

So, you click the Discovery Button to shop for similar options. A new page opens and, without needing to set any filters or scroll through irrelevant items, you can see all of the watches that look similar to the one that initially caught your eye. They all have a stainless steel wristband and a similar face design – but no more yellow accents! Now, choosing the one that best matches your taste is easy. 

Another great use case for the Discovery Button is within PDPs. Say you arrived at a product page for a pair of trousers, and it turns out the website no longer has your size in stock. Instead of needing to return to the PLP and scroll through more items to find another pair that you like, you can simply press the Discovery Button to instantly view similar items that do come in your size. This capability not only prevents disappointment and frustration, it helps shoppers to convert in the moment when they’re poised to purchase.

Again, it seems like magic. Discovering the item that feels the most you, whether it’s a watch, trousers, a sofa, rug, or pair of earrings, can be achieved in the least possible steps (or clicks) with the help of the visual AI matching algorithms.

Key Benefits of Visual AI Technology

For eCommerce brands and retailers, the real magic is not how the image matching algorithms work — it’s the uplifting effect this technology has on their bottom line and customer satisfaction.

Let’s examine how this occurs through a few key examples:

• Continuous product discovery – Capabilities like the Discovery Button, Shop the Look, and Shop Similar allow brands and retailers to virtually eliminate dead ends and achieve continuous product discovery. Whether an item is out of stock, on back-order, or only available in limited sizes, shoppers can seamlessly and instantly navigate to visually similar items in the same price range at the click of a button. This helps prevent friction and disappointing experiences, while seizing shopper inspiration and boosting conversion.

• True personalization – Our personalization engine is 100% session-based. That means a shopper’s age, gender, geographical location, or purchasing history don’t matter. Instead, we deliver product recommendations that match what a shopper has indicated they want now. If you searched for sundresses yesterday, but today you’re searching for a gift for your son’s twentieth birthday, how do recommendations for more sundresses help? By creating recommendations that are informed by in-session actions, we eliminate irrelevant historical data and deliver what shoppers want in the current moment. 

• Business optimization – Our Discovery Console combines the power of the image matching algorithms with merchandising rules, ranking strategies, A/B testing, and data analysis so you can deliver highly personalized customer experiences that support key business goals. For example, when a shopper performs a visual search, you can ensure the top results are SKUs with the highest profit margin. Or, you can prioritize inventory you want to clear out, or dozens of other parameters. The end result is a delightful customer journey that generates more revenue and aligns with your strategy. 

Visual AI is Transformative for eCommerce Businesses

Ultimately, online brands and retailers can improve the entire shopping experience – and make a lot more money – by enhancing the product discovery process and boosting merchandising and marketing efforts with visual AI-enabled data. 

Higher conversion rates, increased AOV, and more repeat shoppers are all direct results of the image matching algorithms.

So there you have it. You now know the secret inner workings of our image matching algorithms (but it’s okay if your shoppers still believe it’s magic.