Like that red skirt? Check this one!

Peter KovacsData Science

Have you seen the latest Silicon Valley episode where Jian Yang built hot-dog classifier with convolutional neural network (CNN)? If you do not know what CNN is don’t worry I will explain it.

Intro into convolutional neural nets

The convolutional neural network is an algorithm inspired by the organisation of the animal visual cortex. The typical CNN, as you can see on image, consists of series of modules whose purpose is to extract meaningful information about an image. It is known that early modules learn to capture basic shapes and colours and they specialise as the architecture deepens.

For example if such a network is trained on face detection task, then some modules might activate when they see either nose, ear or an eye. CNNs are currently considered the state-of-the-art models for almost anything where you work with raw pixels. This technology allowed advances in various fields such as medicine (cancer detection, detection of cardiac diseases, …) or transportation (Autopilot from Tesla). Past weeks I spent working on the same technology and experimented with its application for content based image retrieval (CBIR).  If you ever searched for similar images on Google I am pretty sure you can imagine what CBIR looks and works like.

Applications in fashion e-commerce

Imagine you are browsing your favourite fashion e-shop and you find that good looking red skirt which is too long for a Friday disco night. Or lets say you just need something longer for company meeting or perhaps you just don’t like the fabric. How cool would it be if someone offered you similar but little bit different skirts? With CNNs, we can, check results!

query nearest_1 nearest_2 nearest_5 nearest_6

Similar skirts identified by convolutional neural network

You may wonder is it better than if I would only take random skirts from the same category? Or lets make it more elaborate and say we also want it to have the same colour as our original skirt. Well, lets look at how would such algorithms perform.

query 4 3 2 1

query1 2 3 4

Similar products based on simple rules

The first row consist of random images from the skirt category while in the second one we also constrained colour to be “red”. As you can see the imperfections in data labeling can damage algorithms based on rules. I am not saying that our algorithm is flawless (it is not), however I believe it to be superior to simple rule based recommendations.

If you are interested in more results you can Download more comparisons here! Please note that I have not selected/filtered or tempered with results in any way and they are produced only by code. Also in downloadable results, some categories have less then 10 results which is caused by the fact that there is not enough images which have the same colour and category label. This points to another problem of rule based approach.

Would you like to try convolutional neural networks on your fashion e-shop? Drop me a message!

For Exponea data science team,

Peter Kovacs