Jodie was interviewed recently by Nathan Bransford. Nathan is an author (see How to Write a Novel) and former literary agent who runs a popular blog on the industry. We thought you'd enjoy their discussion.
What are some of the most important elements that correlate with bestellerdom?
The book tries to distill and give narrative to the almost 2000 data points that have an impact on bestsellerdom. The lists are about the sweet spot--a sort of exquisite balance of them all. That's why the lists are exciting and so many authors are enjoying themselves with their own secret sauce of writing, and tweaking the recipes. We work with a lot of authors now, both indies and high profile award-winning people, and over several drafts we find out the strengths and weaknesses of each writer's innate understanding of their craft. Over the past couple of years since the book came out I think I've noticed that most authors who work with Archer Jockers miss out because of flat or bumpy areas in their plot line. With the character agency data and the thematic distribution graphs, I help them see how they can use their characters and themes to even out that all-important plot line in the next draft. So the data points are all a network: when you change one you impact the rest of them. My advice to any author right now would be keep a strong eye on the plot line and use your heart to feel for any updates in the demands of the the market when it comes to the qualities of the hero of the now, and the themes needed to render him (or her) closely. The individual character is currently more important than the themes: that trend comes and goes in cycles.
What books have come out of the blue that your algorithm wouldn't have predicted? What do you think accounted for their success?
Over the past few years I can't say there have been any surprises. We are interested in memoir and the patterns in hits there, and in other areas of non-fiction. We're also thinking about story in film and TV: we aren't predicting that stuff yet but we're looking at it. We only officially still offer fiction analysis.
Is there a risk that as "big data" becomes better at predicting what will resonate with audiences, storytelling will become more predictable and formulaic?
No, not really. We think that that might happen in a small area of the published market, but it would reflect a less exciting use of big data than is possible. The data can get as granular as any individual writer's fingerprints of sentiment and style: there is no reason at all that formulaic fiction will be the result of the big data shift. If it's done right, it should in fact have the opposite effect. Writers can look at all these points of data and go deeper and deeper into what makes their work uniquely rendered rather than lazily dashed onto the page.
How far away are we from an editorial department staffed by robots? Do you think algorithmically-determined publishing program would beat the humans in sales?
I don't think we will see editorial departments in traditional publishers staffed by robots any time in the near future. Two things might happen and it is likely they both will in different areas of the publishing map. We might see editors using big data more and more to develop and refine the careers of their writers. We might see editors go even more hands off with manuscripts if they allow algorithms to choose books to publish. I'd be confident enough with the latter since I don't think our algos would let those editors down, but I'd idealize about the former. We are a few years in now, and the opportunity here really is to see a return in the big houses to that old connoisseurship of word-loving editorial practice. Budgets have pushed on that practice a lot over the past years, so that editors have been pushed into a more marketing heavy role. What I think might happen in the next two or three years with the next impact of big data is a rise of small companies who are willing to be almost all algorithmically-determined (and they will get good funding and do well). Alongside that, the big more slowly moving old publishers might also use the data but go the other way. Their editors might work really closely again with their writers, which is the craft they love, and pull back a bit from that sales and marketing side of the role again. They might get excited about all this extra information on their authors and translate it with them book after book as they build a career with more and more understanding of the fine details of their individual voice. I'm really happy to be in publishing and writing while this shift is happening and it is starting. The sales war will continue, but writers will determine more and more the channels where their books will sell, and there will be more and more choice for them thanks to big data and algorithms. I think it's likely writers will beat both humans (I think you mean editors and agents here), and algo-determined programs because they are getting very savvy and more and more in control.
Given structural issues in publishing for much of the twentieth century, is there a risk of introducing things like racial bias into algorithmic predictors of sales? If the publishing industry falsely believed, for instance, that books by and for people of color "didn't sell" and didn't put their muscle behind these books for a hundred years, wouldn't sales track data be skewed accordingly? Could past poor publishing choices influence how a model predicts what's going to sell?
So long as algorithms are programmed on self-published books as well as the books coming through the traditional publishing channels then I don't think this will be an issue. We are willing to pull from all areas of publishing, and all genres and subgenres when we model, and we don't always have to model on the New York Times list.
Based on what you know now, what are some pieces of writing advice you'd give authors?
Just keep going.