Big Data!
I was reading in a magazine an article by the excellent Simon Brett earlier today. It was an obituary for Phyllis James and mentioned that some time ago, while talking about writing, they discussed the worst questions authors are asked.
My own favourite worst is, “Where do your books sell?”
It’s a natural question. It’s the first thing people ask me who are interested in my books and sales. A friend is trying to help me with marketing. She asked me. Another friend is keen to see how to analyse my sales data to increase sales. He asked me. Many people ask me the same question.
How should I know? That kind of data is not given to the poor old authors scribbling away in their garrets. I have asked for it, many times, but apparently the vagaries of the publishing industry mean that data of that sort just isn’t available.
Let’s get down to basics here.
Many of us get really hacked off with the notes from Amazon about what we ought to be interested in. I do, anyway.
You know how it is, you sign into Amazon, and are presented with a series of items that should be just up your street, because other “Customers who viewed this also viewed” etc. Which is brilliant for me. It means I get to see all the teenage/young adult books coming around. That is, after all, what my daughter looks for on my Amazon account.
She doesn’t have her own account.
But big data is very important nowadays. Much more so than most people realise. Your supermarket knows what you eat, it knows what cleaners you use, it knows how much you value your heart and arteries. It knows how many children you have, and even how much petrol you buy. There is so much data about you sloshing around, it’s hardly surprising that a computer-driven firm like Amazon wouldn’t be making as much use of it as possible.
And there are two problems with this.
First, publishers get only a small amount of data. They can see where books are sold, generally, and they can get a feel for whether or not a special promotion has worked. But for the majority of the sales data, Amazon has a clear monopoly and doesn’t share it. Why should it? Amazon wants to publish its own books, and that means they’ll own all production and selling of books in the most efficient manner because they know what we buy, when we buy it, what we highlighted in books, what we bookmark, even which books we never finished. That means publishers have no idea where to market new books. How can they? Meanwhile, Amazon retains control of all book markets. They know what to stock, who to target for specific books, the works.
Painting of Josselin Castle, Britanny
Second, let’s just think about the concept here. Let’s suppose Amazon were to share all their data with publishers. Would that be good? Publishers could then focus on the ever-reducing number of authors who are producing exactly what the buyers want. Brilliant! At a stroke all the wonderful, less-than-bestselling books could be trashed. Instead readers would be presented with a wonderful array of near-identical genre books. The readers’ choice would reduce dramatically, with only the top-level bestsellers winning new contracts, producing the same works with all the names changed.
Okay, perhaps there would be more variety than that, but at the end of the day, where would the flair of the individual editor show itself?
The appalling thing, for me, was to learn that Amazon doesn’t use humans any more. A programmer had a brilliant idea: people who bought one product may often buy a second or third. If you have enough people in a sample, you can predict with some accuracy what the second and third products will be. It’s the same as an actuarial calculation: you may not be able to say when Joe Bloggs is going to die, but if you have a sample of 100,000 or 1,000,000 people, you can predict how many of them are likely to die in any year. You can predict based on age, based on health, based on occupation, or any number of other attributes. All you need is the data, which Amazon has now acquired.
Editors provide an essential function in publishing. They find the oddball books that can fire the imagination of a generation. Take Harry Potter. Who could have predicted that the series would grow to be so popular? I seriously doubt that Amazon would have predicted that. Similarly with Vampire series, or dystopian futures in which children are forced to fight to the death. These books would not have achieved anything if pure data sets were used.
So, from an author’s perspective, I really wish Big Data didn’t exist. Since it does, I’m sure Amazon maintaining a monopoly is not good for anyone.
The other side is, authors cannot be driven towards one type of book or another. That is good. Authors need to write with their own authorial voice, they need to be fired with enthusiasm, anger, love or loathing before they pick up a keyboard. If they are seen as mere word-jugglers, who can be instructed to “write another story like X and Y, with these factors added to the plot”, we shall all be the poorer, but that is the inevitable direction Big Data will point in.
So, as an author, I am glad that my publishers really don’t have much of a clue about where my books sell. The main thing is, they appeal to all ages. That, for me, has to be enough.
Mind you, the next few years are going to be very interesting indeed.
writerlywitterings 
I heard about a further possibility for data collected on kindle: it seems Amazon knows if a reader finished a book, or up to which page he/her kept reading.
Not sure if that would be of any interest for an author or a publisher, but this gives a new perspective on some best sellers; for example ‘Capital in the XXI Century’ which was bought by a lot of peoples but read by very few.
LikeLike
Was that the Piketty book? I think that was finished by only 2.4% of readers – probably putting it into the same bracket as “Universe in a Nutshell” or “A Brief History of Time”!
LikeLike
I hate to break it to Amazon, but I don’t necessarily follow their recommendations. First, I don’t buy from them very often, preferring to go into our great local bookstore. Second, I have some favorite writers whom I follow and whose books I buy based on the fact that I like what they write. Amazon is useful in finding out when the next book by those authors will be published. Sometimes it’s hard to be patient!
LikeLike
Sadly, I don’t think they care at all! Just as they don’t care about paying taxes!
LikeLike
I love what you said about how authors must have passion before they even begin to write. If the author is not passionate about the book, who else will be?
LikeLike
I agree. It’d be impossible to excite someone with your writing if you aren’t passionate yourself already!
LikeLike
Because I write exclusively for Kindle, it will come as no surprise that the US is my main market. As for the those bloody awful adverts Michael, I’ve given up how many times Amazon’s algorithm has suggested I may like to buy and read one of my own books. ;)
LikeLike
If only. When I look, they never suggest me, only other authors’ books!
LikeLike
Reblogged this on Have We Had Help? and commented:
More from Michael :)
LikeLike
Thanks for the reblog, Jack. Happy writing!
LikeLike
The other day Amazon suggested I might like to buy bull pizzles!!! What on earth must I have been looking at? Cat carrying baskets. Amazon fail :)
On a more serious note, I agree with you that big publishers would probably concentrate only on their best sellers. But I don’t think that’s where Amazon is headed. As well as all the mainstream stuff one can of course buy there, I have always had obscure minority interests, like runology or historical dressmaking, and Amazon can still be used to locate such things. In the same way, with their self publishing, a much wider range of authors can potentially find an audience. Not always easily, because they do tend to guide in the direction of the mainstream, but it does not exclude less obviously popular stuff the way big publishers might. You cite Harry Potter, but of course that was rejected very many times before someone finally took a risk.
With the internet providing a global marketplace of digital products or POD, it is easier than ever before for people with minority interests to hook up with providers relating to them, and conversely for those producers to find their audience. Amazon is part of that, but not the only part (and it must never be allowed to become the only part!) I’m not saying I’m a total Amazon fan, like any big business I am very wary of it. It has its uses, but I am far more likely to buy a book from my favourite local bookshop.
LikeLike
I do agree with you. My biggest concern is, Amazon has information, increasingly, on so many aspects of our lives. So do supermarkets, but Amazon has such a massively dominant position, that it now has a near monopoly. My own dislike, though, comes from them taking tax revenues from the UK. They put out of business smaller local shops, so the councils lose their rates, the government loses the employment income, and then has to pay out dole and other benefits to the people Amazon leaves aside. It really is ludicrous that massive companies like Amazon can claim to make a loss in the UK despite their vast sales. If they really are making a loss, they should be allowed to go bust and allow more UK based firms to spring up and compete globally. As things stand a small number of US firms are aspiring to monopolistic control of their markets, and it has to be stopped.
LikeLike
Very true. I think if all governments co-operated to force companies like this to pay fair taxes the world would be a better place. Oh look, a pig just flew past. I try to buy local for everything I can, as money put into the local economy tends to stay there and boost it, whereas money spent in any big company, including supermarkets, tends to end up in rich peopl’e offshore bank accounts. Of course in the real world, most people have to do what is convenient.
Supermarkets only get info on you if you use their card to get offers. If you want to opt out of that, just pay cash and refuse their card.
LikeLike
Sorry to take so long to reply. Blasted work, computers and life all conspired to get in the way! I agree. Companies should pay tax based on where they do their business. If they earn money in the UK, tax on those transactions should be payable in full according to UK tax. I really fail to understand why companies that want to make money out of us should think they can evade tax by setting up in Liechtenstein, licensing UK firms from there, then charging the UK firm a daft amount to carry on business, so that the company in the UK makes a loss and doesn’t pay tax … well, my patience with cowboy outfits ended a long time ago!
LikeLike