[Update: You can read this on the BBC News website, as usual. And the talented Craig Seymour – who was amused to have his many achievements summed up as ‘former gay stripper’ – has an interesting and pretty straightforward suggestion for how Amazon could resolve the issue of ‘adult’ content on his blog.
Further news: Nick Carr quotes my comment that Amazon will take a long time to recover, but notes that ‘for Amazon, a “long time” in realtime is equal to about five minutes in clock time’. I fear he’s absolutely spot on – would be fascinating to see if Amazon’s sales figures even show last weekend as a blip. Ah well…
]
While millions of people tuned in to Doctor Who and Red Dwarf over the Easter weekend my holiday entertainment was provided by typing ‘amazonfail’ into the Twitter search engine and watching the stream of outraged posts about the company that used to be the world’s favourite bookstore flow across my laptop screen.
The PR nightmare started at some point on Sunday when an angry post on the LiveJournal blog site by author and publisher Mark Probst broke through into online consciousness. He had noticed that his book The Filly, though still listed on Amazon’s US website, had lost its sales ranking data and was no longer appearing in relevant searches.
An email from an Amazon representative informed Probst that his book, a romance featuring gay characters, had been classed as ‘adult’ and removed from the ratings system so that the search pages would be more ‘family-friendly.’
Two other high profile gay romance books, Transgressions by Erastes and False Colors by Alex Beecroft, had also lost their sales ranking. This meant it was only possible to find them by looking specifically for the title and author, so people searching for similar books, or similar titles were unlikely to find or buy them.
It emerged that thousands of other books had been similarly delisted, including such radical texts as The Well of Loneliness and John Barrowman’s autobiography, while a little research by interested bloggers found Playboy: The Complete Centerfolds, the Parent’s Guide to Homosexuality and Hitler’s Mein Kampf were all still searchable and proudly displayed.
Two years ago this would have resulted in a collection of angry, interlinked blog postings. A year ago there would have been a Facebook group to join. But this time it was the Twitter microblogging service that led the way, with thousands of tweets linked by the tag ‘amazonfail’.
The timing was perfect. It was a slow news weekend on what is an extended holiday in many parts of the world. Amazon’s ability to respond quickly was limited, while the echo chamber of Twitter, LiveJournal and Facebook meant that the noise of outrage quickly reached a crescendo.
Many of those affected were heavy net users because authors have realised that engaging with the online audience is a good way to boost visibility and sales. In fact I first became aware of the issue after furious tweets from a friend, the sex blogger ‘Abby Lee’, after her book Girl With A One Track Mind dropped out of Amazon search results.
Within hours Amazon’s reputation had plummeted among those – whether gay or straight – who think that being inclusive about homosexuality does not make something ‘adult’ or merit banishment to the ghetto of unranked titles. As I write this the company has claimed that the episode was an unfortunate glitch and does not reflect a new policy, and I’m tempted to believe that it was never their intention to delist or downgrade books that are about gay, lesbian, transsexual or bisexual issues – or those written by authors who are not aggressively heterosexual in their appetites.
Some careful analysis by “Jane” on the Dear Author site indicates that the problem lies in the metadata, the additional details about each book added by Amazon and publishers. Books classed as “gay”, ”lesbian”, ”transgender”, “erotic” or “sex” have been filtered while the Playboy book, whose content is classed as “nude”, not “sex”, remains.
In fact some sort of filtering seems to have been going on since at least early February when former gay stripper Craig Seymour saw the sales ranking on his memoir disappear. He complained at the time and the ranking reappeared.
What this seems to show is that Amazon are trying to make search results more ‘family-friendly’. A children’s author I know has been in discussion with Amazon over this for well over year after clearly adult titles kept cropping up when people searched for books she had written.
There’s nothing wrong with this, and Amazon aren’t violating the First Amendment or even, I suspect, breaking the terms of their agreement with publishers by doing it. However they have clearly broken the bond of trust with a large number of their readers, and it will take a long time to recover.
Filtering in itself is not a bad things, and every time we do a search or turn to a web service we expect some sort of selection to take place, otherwise we would be faced with too much information to cope with. But filtering has to be done in an open, transparent and configurable way, like Google’s ‘safe search’ which offers a simple all or nothing model that can easily be turned off.
There’s a lot of speculation about what actually happened, with some putting forward a view that this was a carefully planned move by US fundamentalists who were gaming Amazon to cause just this sort of public upset and one or two people claiming they were behind it all.
But I’m pretty sure that the error was in the algorithm, in the set of rules that Amazon’s systems apply when selecting books to display, coupled perhaps with some unfortunate choices about how certain books should be tagged in the metadata Amazon holds.
This is not, however, a source of comfort. When a book is misfiled in my local Borders it may result in a few lost sales, but for the whole of Amazon to ‘misplace’ my book may mean nobody in the world buys it. If the filtering had affected people writing about how to keep sheep as domestic pets instead of gay fiction we might not have noticed the error.
The consequences of living by the algorithm do not just affect Amazon, they affect all of us as we increasingly rely on recommendation systems to suggest books to buy, friends to add on social networks, emails to take notice of and places to visit. We have put our faith in Google PageRank and ‘Amazon recommends’, and found them wanting, yet we do not have an alternative.
The only real solution is the one that fixes so many other problems. We need transparency for the algorithms just as we need transparency about MPs expenses, police behaviour, programme interfaces and how the deep packet inspection that will deliver targetted adverts to our web pages actually works.
Of course we then have to trust the companies, agencies and government departments involved to implement the systems in accordance with the published specifications, and it’s unlikely that Amazon or any other e-commerce service will willingly publish its source code for inspection. Perhaps the UN should commission internationally recognised ‘algorithm inspectors’ just as it has ‘weapons inspectors’.
Bill’s Links
Probst post: :
Amazonfail on twitter search:
Dear Author analysis:
Girl with a One Track Mind:
One claim of responsibility:
And a rebuttal:
Algorithms are always going to get some things wrong. What Amazon and Google might need to realise, is that they need to self-police the shortfalls of their web technology rather than giving their users bad results and wait for angry complaints to fix things.
How about a series of options that could be switched on or off?
Also, am I right in thinking Amazon has no “search within results” option? That always bugs me.
Interesting article… someone should create a cyber scrap heap where we could all go looking for exciting things that have been marginalised and rejected by mainstream internet institutions.
I am very disappointed in this article. Why do you think that a private company should be subject to regulation over a policy/algorithm they use?
It may have been foolish of Amazon to tag these incorrectly but they will miss out in loss of business. I cannot understand a need for regulation that you suggest.
If Waterstones choose to not stock your book or if it is in the wrong section by the same thought you should have decision police come around and quiz them. Hopefully you will agree that is completely wrong.
There is no difference in online or offline for this.
Comparing “weapon inspectors” to choosing to display what content an individual/company has on the site is just silly. In fact, it is still on their site just not via the addition recommendations.