The Rise and Rise of Visual Search

Svetlana Kordumova

Published on

September 11, 2019

When search engines first appeared, web search was only text-based. No images. Since the late ‘90s, image search was based on typing words that best describe the images you are searching for, and matched with the accompanying text on the web page.

Over the years, search engines were becoming better at understanding components in the images itself. And now not only can you search images with text, but also by uploading an image, feature also known as Visual Search. Although many are still adapting to this new way of search, it’s popularity grows.

There are over 600 million visual searches on Pinterest each month, and images are returned for 26.8 percent of search queries on Google.

People enjoy the advantages of visual search. It’s taking over the way we search. It allows searching the web, social media or e-commerce stores using real-world images. But how does it do that?

Visual search uses AI (artificial intelligence). It’s made possible because of the advances in algorithms from a subfield of AI called Computer Vision. A field that deals with how computers can be made to gain high-level understanding from digital images or videos.

Screenshot from Pixyle.ai live demo

But why is visual search so important? Can’t we just type what we want to see, like in the good old days? Yes, we can. But, will the search engine understand us and correctly return the results we want?

Let’s say you see a photo on Instagram, and you really like the top the girl is wearing. It’s colorful, with an interesting abstract pattern. You want to find a similar one to buy for yourself. Just like for an abstract painting, an abstract print is difficult to explain with words, and is understood subjectively by people.

With the help of visual search, you can upload a photo, and the visual search AI technology will match the pattern with other images and find that cute top for you.

Visual search is perfect for shoppers who face two common dilemmas: “I don’t know what I want, but I’ll know it when I see it”, and “I know what I want, but I don’t know what it’s called”. 
- Amazon statement.

We can make ourselves much happier if we search using pictures, rather than text. This is the part where technology connects with psychology.

  • People remember 10% of what they hear, 20% of what they read, and 80% of what they see and do.
  • Our brain processes visuals 60000x faster than text. 
  • When we see a picture, our mind needs no longer than 13 milliseconds to recognize it.
  • 40% of people respond better to visual info than to text.
  • 90% of the information transmitted to the brain is visual.

When we see an image, it affects us emotionally. That’s why we communicate better with pictures. Visuals are better at untying the knot of today’s hectic lifestyles.

All of the above is why more and more fashion retailers are starting to use AI based visual search for e-commerce.

By 2021, early adopter brands which will support visual and voice search will increase their e-commerce revenue by 30%
- According to Gartner.

With visual search, companies can offer products from their catalog that their customers see and like in the world around them, the easiest way possible. It’s a great fast product discoverability feature. The easier you find the item you’re looking for, the more chances you’ll buy it. And of course, retailers can get more customers and increase their revenue when they improve their customer experience.

What's better customer experience then saving time for your customers and making shopping easy.

But is visual search easy for fashion retailers to develop and integrate themselves in their online stores? The big players on the market have already done so.

  • ASOS’ visual search tool is available to all of its shoppers, allowing purchasing with images. Engadget
  • eBay offers visual search technology for mobile app users. Business Insider
  • Neimann Marcus shows strong quarter by investing in innovation and visual search technology. ZDNet
  • Amazon helps you find the clothes you want by simply analysing a photo. Business Insider

The leaders started the visual search revolution in fashion, and they clearly have shown its potential and benefits. And it’s not an easy technology to develop. It needs to be accurate so that customers can find similar products to the ones they like in images. If done well, it will provide a great customer experience.

Many Computer Vision scientists are working on the Visual Search challenge. It’s AI complexity is way beyond text and voice search.

“In the English language there’s something like 180,000 words, and we only use 3,000 to 5,000 of them. If you’re trying to do voice recognition, there’s a really small set of things you actually need to be able to recognize. Think about how many objects there are in the world, distinct objects, billions, and they all come in different shapes and sizes. So the problem of search in vision is just vastly larger than what we’ve seen with text or even with voice.”
— Clay Bavor, Google

Visual AI works best, when the technology is focused on a single domain. To best recognize fashion items, the AI models should be trained only for fashion. For good accuracy, large quantities of fashion images are needed. Just as when a child sees more, it can better understand the world around.

Images used for teaching fashion to an AI machine, should be specially marked with fashion related keywords like fashion categories, descriptions, attributes like color, pattern, shape, etc. These images help the AI software to interpret images with words, just as we humans do.

The market of e-commerce is huge, and visual search will soon take over many e-commerce stores. It improves the customer experience and helps customers easily find what they already see and like. And those who implement it soon, will be the ones who will keep up with the fast changing fashion e-commerce landscape.

Being able to search the world around you is the next logical step
— Brian Rakowski, VP Product Management, Google.

Svetlana Kordumova