You can't judge a book by its cover – if you're a robot

Back in September, a report suggested that robots will have eliminated 6% of jobs in the US by 2021. Fortunately for book designers, it doesn’t look as if androids are about to take over cover design any time soon.

MIT Technology Review points us towards a new machine-vision algorithm dreamed up by academics at Kyushu University in Japan. In a new paper, Brian Kenji Iwana and Seiichi Uchida explain how they trained a deep neural network “to predict the genre of a book based on the visual clues provided by its cover”.

It is a great help to everyone involved that once publishers find a good thing for covers – a pair of legs, say, or the back of a woman’s head, or handwritten titles – they tend to stick with it, to the point of cliche.

Using almost 140,000 covers, downloaded from Amazon along with the books’ genres – the academics used a possible 20, from photography to romance – the network was first trained and then tested to see if it could pinpoint the titles’ genres.

“The results make for interesting reading. The algorithm listed the correct genre in its top three choices over 40% of the time and found the exact genre more than 20% of the time. That’s significantly better than chance,” writes MIT Technology Review. “This shows that classification of book cover designs is possible, although a very difficult task,” write the academics.

View image in fullscreen

The algorithm might have done better than chance, but for now at least, it must be significantly underperforming humans (or as publisher Melville House’s blog put it, “Machine built to judge books by their covers reveals pretty much nothing”. Human versus machine face-offs weren’t something the paper considered, but stand in any library or bookshop, spin around, and I bet you’d correctly categorise the majority of the titles you see – you’d definitely do better than one in five.

I wonder how the algorithm would perform on the stunning new collection of jackets from Vintage for Virginia Woolf’s novels – Mrs Dalloway is particularly gorgeous, but might it not end up in arts and photography? Ditto these new looks for Claudio Magris – beautiful, but perhaps a little too abstract for the algorithm to handle.

So for now, when it come to the books world, at least, we’re slightly ahead of the machines. Although they have been trying to write poetry...