One of the most intractable publishing technology questions in the data-driven content world is a simple one: How do we publish (meaning post, upload, or otherwise distribute) databases of time-sensitive content in a way that allows them to be retrieved and used forever? Shyamali’s post on stone cylinders points out one of the first attempts at this: a durable medium with “time-stamped” data (i.e., Sumerian wheat production for the years X-Y). And, as Shyamali describes, paper-papyrus was adopted over a medium that lasted longer (but weighed too much) despite the fact that it is a clearly imperfect medium (see “Alexandria Library Fire”) and paper dominated for 2,000 years … until pretty much exactly now.

Weirdly enough, we are all personally witnessing one of the great periods of “media migration” as printing on paper disappears and the businesses once reliant on it (think yellow page directories) face an existential threat. And yet there is no new true “platform” that has arisen to replace it. Before you say “XML repositories in the cloud…,” remember that data needs to be searched for and retrieved somehow, and therein lies the rub: the shifting sands of software.

The issue of archival data stored electronically has been a growing concern for at least the last twenty years when CD-ROMs (data welded to software and a device requiring a specific mechanical “reader”) roamed the Earth. Of course the issue predates that–think LP records, reels of film, and other magnetic tape media–so we have even less excuse for sitting around umpteen years later and still having no idea of “what’s next?” The underlying problem is the disconnect between the storage device/systems folks and the content folks. The hardware/software world certainly looks forward, but it likes proprietary technology, which is inherently problematic long-term. The publishing world … well, we all know how far ahead we look. So, it is the archivists (librarian super-users) who know the most about the issue. Do they hold the solution to this problem?

The MediaNews Group chief executive (who got his start in journalism working at a tabloid newspaper based in Toronto) started his talk off with a bang by saying that the newspaper industry and journalism as a whole are more at risk from “crappy newspaper executives” than from any changes that have been wrought by the Internet. Their refusal to try and adapt to the new digital reality, Paton said, is “like aging ingénues… declaring they can still play Juliette. And nobody has the heart to break it to them.” Paton added:

[W]orse still [are] mediocre journalists, wrapping themselves in the flag of long-form journalism, to deride the value of social media as a reporting tool… and then having to watch them use that ignorance to dismiss the phenomenon of participatory journalism. And while these false, zero-sum arguments play themselves out, Rome burns.

Since 2005, Paton reminded his audience, the U.S. newspaper industry has lost “more than 60 percent of its advertising revenue and so many jobs no one can accurately count them,” and new platforms are targeting customers in such precise ways that “we print folks are effectively taking a knife to a gunfight.” Instead of building paywalls around their content and criticizing the rise of social-media tools, the MediaNews CEO said that newspapers should spend more time trying to understand “how professional journalists can come together with the people we used to call the audience.”

Traditional media needs to shed its assumptions, Paton says

Paton also referred to media analyst and journalism professor Clay Shirky’s essay on the future of newspapers from last year, in which Shirky argued that the old-media model these entities are based on is fundamentally flawed in a digital age, and that “there is no general model for newspapers to replace the one the Internet just broke.” So what are media companies to do? Paton says they must rethink some long-held views about what a newspaper is supposed to do, and get rid of many of these assumptions, including:

• “We are the gatekeepers of information.”
• “That we are the agenda setters and that we decide what news is and what is not.”
• “That we keep the Outside world outside and only let in the chosen few – people like us.”

Paton said the uncomfortable fact is that “we have entered a new era where what we know and what we traditionally do has finally found its value in the marketplace and that value is about zero.” And what does a newspaper chain do in that kind of environment? The MediaNews Group CEO says he plans to roll out the same changes he introduced as part of a digital-first model at the Journal-Register Co. chain — including a “community newsroom” that builds engagement with readers, and a focus on using social tools to distribute content in as many different ways as possible.

Sharing equals influence, and influence is what advertisers want

Those efforts, which Paton has said in the past are having a positive effect on the Journal-Register chain’s bottom line, are designed to take advantage of the web instead of trying to hide behind a paywall, the Media News CEO said:

One of the reasons I am so stern on paywalls and other walled gardens is because I firmly believe that in the future content will go to the audience and not the other way around. Smart, original content, tagged with advertising will gain value by being shared through networks. Shared content equals influence. And influence in the new ecosystem equals engagement. And engagement equals value to those advertisers and others trying to reach that engaged audience.

Paton’s criticisms of the traditional newspaper business may be uncomfortable for industry executives to hear, but I think his focus is the right one. Paywalls like the one the New York Times installed last year may produce some extra revenue, but they will not be enough to make a dent in the bottom line of most newspapers as advertising continues to fall — and in that sense, as I’ve argued before, they are more like a wall of sandbags designed to try and stem the rising tide than they are a true digital strategy. The New York Times needs a lot more than just a paywall if it is to remain relevant and survive in a digital world.

The distributed model that Paton describes is much like the approach The Guardian has taken with its “open platform,” which offers its content to developers and services via an open API, and is designed to be able to monetize that content wherever it appears, instead of trying to hide it behind a paywall. Former Washington Post managing editor Raju Narisetti and journalism professor Jeff Jarvis have both talked about a “reverse paywall” that could reward regular readers who engage with a newspaper, instead of penalizing them by charging them money.

The bottom line, Paton says, is that media executives of all kinds have to start learning from and being open to the potential of digital and social media, instead of always seeing it as a threat that needs to be defended against. As the MediaNews CEO put it, telling readers “you’re gonna miss us when we’re gone” isn’t much of a business model.

In the following interview, Collingridge talks about how real-time data and analytics can help publishers and he shares insights from the beta period of Bookseer, a market intelligence service for books his company is developing.

Peter Collingridge: I think despite the increasing awareness of data as being a critical tool for publishers to compete, it's genuinely hard for people to look at data as a natural addition to the work they are doing, whether that's in PR, marketing, acquisition, or pricing.

Publishing has operated in a well-defined way for a long time, where experience and intuition have dominated decision making and change is hard. What has been really exciting is that when people have the data in front of them, clearly showing the immediate impact of something they did — a link between cause and effect that they couldn't see before — they get really excited. We've had people talking about being "obsessed" and "addicted" to the data.

Some of the most surprising findings: That on some titles, big price changes aren't as relevant to volume as everyone thinks; that big-name glowing reviews of literary fiction don't have anywhere near the impact on sales to merit the effort; and that social media buzz almost never translates into sales.

For me, the key observations so far are around marketing. First, big budget media spending and ostentatious banner ads might impress authors and bookshops, but they deliver very poor return on investment (ROI) for sales. Secondly, the super-smart publishers are behaving like startups and doing tiny little pieces of very focused and cheap marketing — and watching the results like hawks before iterating in direct response to the data. Bookseer is designed to disclose the former and to aid the latter — and that is probably our biggest finding: it works!

Peter Collingridge: Before we built Bookseer, we spoke with 25 people across the industry, including authors big, small and unpublished; editors and publishers; managing directors; digital directors; sales, marketing and PR directors; and literary agents. We asked exactly that question.

For most people, the data they had was pretty basic: Nielsen (which obviously only goes to the granularity of one week) plus the F5 button to manically refresh an Amazon web page for changes in sales rank. Neither of these is particularly helpful in determining the impact of an activity.

Of course, there are loads of data points, but we began with the lowest-hanging fruit. Aggregated sales (print and digital) across multiple sources; Amazon sales rank; price; best-seller charts; social media mentions; buzz; review coverage in mainstream and new media, and on social reading sites; and other factors such as promotion (advertising and other) and merchandising.

We think the most important thing to do is aggregate activity and data points across as many sources as possible, building a picture of what's going on for one title or across a whole retailer, and allowing publishers to draw their own conclusions.

Peter Collingridge: Publishing has been B2B, about supplying books into bookshops, for forever — combined with working with media to support that. And for that world, weekly aggregated retail sales work, I guess. But when you're in a much faster-paced world, with the industry moving toward being consumer- rather than trade-facing, and with a fragmented retail and media landscape, you need to make decisions based on fact: What is the ROI on a £50,000 marketing campaign? Where do my banner ads have the best CTR? Who are the key influencers here — are they bloggers, mainstream media, or somewhere else? How many of our Twitter followers actually engage? When should we publish, in what format, and at what price?

Data should absolutely inform the answers to these questions. Furthermore, with a disciplined approach to promotion, where activities are separated from each other by a day or a few hours, real-time measurement can identify what works and what doesn't. We can identify the difference between Al Gore tweeting about a book and Tim O'Reilly doing the same; the difference between a Time review and a piece on CNN; the impact of a price drop against an email sent to 200,000 subscribers; and measure the exact ROI on a £300 campaign against a £30,000 one.

Over time, you build up a picture of which tactics work best and which don't. And immediate feedback allows you to hone your activities in real-time to what works best (particularly if you are A/B testing different approaches), or from a more strategic perspective, to plan out campaigns that have historically worked best for comparable titles.

Peter Collingridge: Right now, sales drives social — not the other way round. However, I believe there will come a point when that's not the case, and we will be able to identify that.

This interview was edited and condensed.

by Shyamali Ghosh

We tend to look at change as a feature of modern life, but I was reminded last week that publishing has always been a magnet for new technology. I had the opportunity to visit the Morgan Library, a must-see for all who love books. Exhibits included papers belonging to Charles Dickens, Virginia Woolf, Antoine de Saint-Exupéry, and one of three Gutenberg bibles owned by the Morgan. There was also a little room full of cylinder seals. According to the descriptive tags, they were “among the earliest known objects to use pictorial symbols to communicate ideas.” These finely carved little cylinders were once rolled over clay to officially seal containers and doors, and later to authenticate record documents. They came into use in western Asia around 3500 BC and were common for thousands of years, until 330 BC. That year, Alexander the Great defeated the last Persian king and moved his new territory’s center of government to Greece, where official records were kept on parchment or papyrus.

The description ended there, but it occurred to me that the switch might have had wide-reaching effects on ancient western Asia. Were the streets suddenly filled with newly unemployed clay vendors? Did donkey cart owners have to scramble for work, now that there were no more heavy official tablets to transport? What did the highly skilled seal-carvers find to carve? Did citizens worry that parchment was too impermanent a medium for important public records? The impact of the shift from clay to parchment, a step vital to the development of the publishing industry as we know it, was likely unsettling at the time.

I think we’ve come to a similar moment right now. Publishing processes have already changed dramatically in the last 25 years. We keep our content current year-round instead of running cumbersome annual updates. We publish instantly instead of waiting weeks for our words to see the light of day. We sort or recut data to meet our clients’ needs almost at the drop of a hat. New software and devices appear every day, each with the potential to permanently change the way we work. That’s why I’m delighted to be attending the O’Reilly Tools of Change for Publishing conference this week in New York City. It’s an exciting time to be in publishing. As the conference name indicates, our industry is evolving every day, and the opportunity to get a glimpse of the next big change is one I would not miss.

As the “big data” juggernaut* continues to build momentum, it is about time to pause and reflect on those things that are holding us back from realizing the true potential of data integration, overlays, and analysis. Mark Miller posted a great piece on MediaPost (“Three Steps To Dealing With Data Paralysis,” Jan 31, 2012) that points out a few of those issues and how to circumvent them.

• Favor smart data over big data: “Data issues like missing values, missing linkages and data anomalies can impede your ability to harvest data and move the business forward…. have a plan to deal with quality issues that will otherwise make your data dumb.” Hallelujah.
• Use analytics to mine your smart data: “…identify the 10-15 most important things you need to know… Focus on the information that helps you determine…how you can create value.” Setting the right data priorities is, of course, key.
• Create a roadmap using data and analytics: “…conduct an audit of the processes, systems, tools, and talent within your organization. Identify the gaps. [This audit] should be connected to harder metrics like sales, revenue and profit.” There a couple of good points here: 1) If you don’t have the talent or bandwidth in-house for the data work then go get it on the open market; and, 2) Always focus on “real” numbers — confusing “reporting” with “analysis” is a fatal mistake and focusing on hard metrics (not forgetting that cost reduction is among those) is essential.

Miller comes from the CRM world, but what he is saying is relevant to many others in the world of creating and managing data. We may all be awash in data and have decades of experience under our belt managing it, but it’s never to late to go back to basics and make sure that the foundation for our databases is well laid, our cost of ownership is low, and we are getting every dime of value out of what we have built.

*Derived from various Indian words for a very large, portable, wooden temple.

Over 4,000 researchers have signed a petition against Elsevier practices including charging libraries for bundles of journals rather than individual titles.

They also object to Elsevier’s apparent stance on the U.S. bills SOPA, PIPA and the Research Works Act, which critics say would restrict access to taxpayer-funded academic research.

The protest is currently small in the context of the world’s large researcher community, but is causing Elsevier’s business model to come under the knife.

“We think that investors should ask management of Reed Elsevier (NYSE: RUK) how a PR incident of this kind could happen, why crisis management has been so tentative and what other steps management intends to take the handle the protest,” says Bernstein Research European media analyst Claudio Aspesi, in a note titled “Occupy Elsevier”.

“Dropping prices, abandoning the subscription model or the bundles would all impact the economics substantially, at least for some years. So the company can only hope that the controversy will die down in time”

Aspesi thinks Elsevier’s Anglo-Dutch owner Reed Elsevier, whose Reed Business Information is still shedding titles after Reed failed to sell RBI during the downturn, should be broken up, and is pessimistic about the company.

“Our scepticism is based on the expectation that academic libraries will increasingly push back and request lower price increases than in the past, threatening to abandon ‘Big
Deal’ contracts if the company does not lower its expectations for revenue growth,” Aspesi writes.

Although the paywall and related print-subscription deals helped boost circulation revenue by almost 5 percent in the NYT’s media group — which includes the New York Times, the Boston Globe and the International Herald Tribune — and digital advertising revenue was also up by about 5 percent for the quarter, neither of those things were able to compensate for the continued drop-off in print advertising. Print ad revenue fell by almost 8 percent, which helped push the NYT’s fourth-quarter profit down by more than 12 percent, and for the full year the company reported a loss of $40 million.

Paywall revenue isn’t even close to making up the gap

The New York Times didn’t provide any helpful charts that would make the reality of this situation more obvious, so one blogger decided to come up with his own. Paul McMorrow, an editor at CommonWealth magazine, put together a chart that shows the contrast between the NYT’s advertising revenue, circulation revenue and its total revenue.

According to newspaper-industry analyst Ken Doctor, the NYT is probably pulling in about $86 million or so from its digital paywall — or “metered access,” as the paper likes to call it, since you get to read 20 articles for free before you get hit with a request for your credit card. But that’s not even close to being enough to make up for the decline in ad revenue, both print and digital, which dropped by 7 percent in the quarter.

One of the biggest problems for the Times is that its former online star About.com, which the company bought in 2005 for $410 million, has seen both its profitability and revenue-generating ability implode in the wake of an update to Google’s search algorithm — a change that was designed to penalize what the company called “low quality” content sites, or what some call “content farms.” In the most recent quarter, the NYT said About’s revenue fell by 26 percent, and profit fell by a staggering 67 percent.

As McMorrow’s chart shows, the Times is still far under water in terms of revenue, despite the benefit of its paywall. As I’ve argued before, there’s nothing wrong with having a paywall — although in many cases it amounts to building a wall of sandbags around the print newspaper edition, which provides most of the ad revenue — but if a paywall is your only strategy for responding to digital disruption of the media business, then you are almost certainly doomed, whether you are the New York Times or not.

Which way will the new CEO go — towards the past or the future?

So what should a new CEO be looking at to revitalize the NYT for a digital age? Ken Doctor suggests that the paper needs to look beyond just subcription revenue and focus on how it can target those 325,000 digital subscribers — since it knows who they are and where they live, and it already has their credit-card numbers.

I would take it one step further, however, and suggest that the new CEO think about some of the suggestions about “reverse paywalls” that have been made by journalism professor Jeff Jarvis, and also by former Washington Post managing editor Raju Narisetti (who is now at the Wall Street Journal in a digital role). The main principle behind this idea is that regular readers should get more than just a sales rep hitting them up for a monthly payment — the fact that they are a devoted fan should entitle them to earn rewards, whether it’s money off their subscription for interacting with the paper, or offers that others don’t get.

The NYT has taken a few steps towards trying to build relationships with its readers through what I’ve called the “levelling up” process that it recently added to its comment section, where readers can achieve preferred status for good behavior. Those are the building blocks of a relationship that the paper could use to its own benefit in all kinds of ways, many of which could generate new sources of revenue — real-life events, for example, which has been one of the things that has helped turn The Atlantic around, or a line of e-books based on the newspaper’s original reporting.

Another thing the NYT could — and should — be thinking about is what the role of an information provider is in the digital age. Is it to act as a gatekeeper for certain kinds of data and try to reimpose the scarcity that used to exist in the print era? Or is it to find partners to distribute that information in as many ways as possible, and to think of the paper as a data platform, as The Guardian has with its open-platform project? One way looks to the past, and the other to the future. Which way will the NYT go?

Last week, Amazon teamed up with Houghton Mifflin Harcourt to print and distribute the Amazon Publishing East Coast's adult titles under a new imprint, New Harvest. Some speculated the move might get Amazon through the brick-and-mortar doors of B&N. This week, B&N made it clear that not only would HMH's New Harvest imprint not make it in the door, but that no Amazon Publishing title would. In a post for the New York Times, Julie Bosman quoted from a statement made by Jaime Carey, B&N's chief merchandising officer:

"Our decision is based on Amazon's continued push for exclusivity with publishers, agents and the authors they represent. These exclusives have prohibited us from offering certain e-books to our customers. Their actions have undermined the industry as a whole and have prevented millions of customers from having access to content. It's clear to us that Amazon has proven they would not be a good publishing partner to Barnes & Noble as they continue to pull content off the market for their own self interest."

O'Reilly's general manager and publisher Joe Wikert called on B&N this week to disrupt the industry — maybe this is its first move. Bosman also took a look at B&N's position in the industry and its importance to the publishing ecosystem, especially in the face of a competitor like Amazon. Jordan Weissmann at The Atlantic mulled the prospects of Amazon killing publishing and argued: "In a financial arms race, publishers simply can't beat Amazon's arsenal."

Breaking up is hard to do

Amazon had issues with a social networking partner this week as well. As of Monday, Goodreads no longer displayed book data from the Amazon Product Advertising API, opting instead to move its data partnership to the Ingram Book Company. A Goodread's representative told Laura Hazard Owen that "the [API license agreement] terms now required by Amazon have become so restrictive that it makes better business sense to work with other data sources." Owen outlined some of the specifics on the restrictions:

"Amazon requires sites that use its API to link that content back to the Amazon site exclusively — so a book page on Goodreads would have to link only to its product page on Amazon and not to any other source or retailer ... Amazon also does not allow any content from its API to be used on mobile sites and apps."

Jon Mitchell at ReadWriteWeb took a deeper look into the situation — and explained why Goodreads will survive its breakup with Amazon.

The news caused some readers to worry about their cultivated Goodreads bookshelves. GalleyCat detailed potential data issues and offered up a Goodreads link that allows users to check on the state of their shelves to see if any tidying up is necessary.

Jonathan Franzen waxes absurd on ebooks

"I think, for serious readers, a sense of permanence has always been part of the experience. Everything else in your life is fluid, but here is this text that doesn't change. Will there still be readers 50 years from now who feel that way? Who have that hunger for something permanent and unalterable? I don’t have a crystal ball. But I do fear that it's going to be very hard to make the world work if there's no permanence like that."

Chenda Ngak at CBS's techt@lk took offense at Franzen's remarks, stating: "Even if I agree with him, as a book lover, his statements are too condescending to take seriously." Jonathan Segura at NPR chimed in as well, calling Franzen's comments "absurd" and pleading that we "get past the e-books versus print books thing." Segura's final comment pretty much summed up the overarching sentiment:

"We should worry less about how people get their books and — say it with me now! — just be glad that people are reading."

Read full story...

These guys have used ideas from statistical mechanics to model the behaviour of humans influenced by word-of-mouth interactions and advertisements. In this paper, Ishii and co derive a bunch of equations that they use to model the number of people who'll turn up to see a movie or visit an art show.

Inspired by this work, Nicklas Lundblad has written an interesting speculative piece about what the rise of predictability through the analysis of huge data sets might mean for society and openness. He notes that one of the "theorems" of psychohistory is that for it to be effective the data sets and the predictions derived from them must be kept secret from the populations involved – the idea being that if they were able to analyze that same data themselves, they might change their actions and thus nullify the predictions.

He points out that this creates a tension between predictability and openness:

There is an assumption here that is worth highlighting. And that is that for a democracy to remain open it can not be predictable by only a few. That is a complex and perhaps provocative assumption that I think we should examine. I believe this to be true, but others will say that our democracy already is predictable, in some sections and instances, only to a few and that they build their power base on that information asymmetry, but that it is reasonably open still. Maybe. But I think that those asymmetries are not systematic to our democracy, but confined to those phenomena, like stock markets, where they are certain to be important, but where they also do not threat the nature of democracy as such.

In summary, if we share the data and allow everyone to use it, then predictability goes down.

That, in its turn, is an argument for openness. If data held by a government, say, is released freely, anyone can explore its implications and then be able to modify their actions based on them, and thus escape being a statistical part of the predictability that would otherwise be implied by remaining in ignorance. As Lundblad writes:

If there is a conclusion here it seems to be to explore the amazing value of data under the imperative of openness to the the full extent possible to ensure that our societies gain from this new, fantastic age of data innovation, discovery and exploration that we are entering into, but never compromise on that openness in the pursuit of macro-social predictability.