Using Google to Predict the Future

Elegantly simple proposal to measure economic confidence in The Economist’s search for other quirky indicators: searches in the U.S. on Google for “gold price” in the piece Alternative indicators: Behind the bald figures

But the hottest tip came from Edward Ritchie, an investment analyst in London. He tracks Google searches for the “gold price” as an indicator of economic confidence. This does not follow the gold price itself. For example, during most of 2008 when the world’s financial system was melting down, the gold price tumbled yet the number of searches soared. The number of gold-price searches shoots up when American consumer confidence dives and subsides when households perk up again (see chart). That makes it a handy device for spotting turning-points in economic confidence, with the added advantage that the data are available earlier than for conventional survey-based figures. Worryingly, the number of searches has recently vaulted above its 2008 peak, signalling the possibility of a double dip.”

Here’s the graph:

I’m a big fan of using Google search to measure, track and predict things. A few of my previous posts on the matter. And no, I’ve not made any money so far out of this crystal ball.

How To Use Google To Get Round Super Injunctions

Technoratis Decline, Death of Blogging?

Googling the Tsunami

Googles Suicide Watch: where I googled the word “suicide”

Has Quora Peaked?

Fail, Seinfeld and Tina Fey: A Zeitgeist

The Financial Crisis in Charts

Hoodiephobia, Or We Don’t Lie to Google

And this one from 2006: Mapping Trends With Google

2011: Year of The Media App

This is my weekly Loose Wire Service column.

By Jeremy Wagstaff

I predict this year that we’ll settle on a way to make people pay for stuff they so far have proven reluctant to pay for—namely information. This won’t be done by pay walls, exactly, but by what we’re now calling apps. Apps are applications that people seem very willing to pay for when they’re doing it from a device that isn’t a desktop computer.

So people are buying these things because what’s a buck when you know you can get to hurl Angry Birds onto flimsy structures sheltering evil pigs on your device in a couple of seconds? Or listen to Yesterday on your iPod Touch a few seconds after buying it?

Compare this with the laborious process of signing up for an online subscription, or having to download, install and pay for some software and then have to enter a serial number longer than most emails you’ve written.

Others are now trying this route. Google has the Android Marketplace, which lets you do more or less the same thing. In fact, it’s even easier—you don’t get prompted for your password when you buy something. And now they’re trying something on your computer: their own browser, Chrome, now have apps which you can buy or get for free. (Google’s own operating system, Chrome OS, will revolve around these apps.)

In fact these aren’t really anything new—they’re what we might call web-services which are accessible via a website, rather than by downloading software. But by packaging them up as apps Google make it easier for us to get at them and, crucially, break down our resistance to buying something online.

This is how we’ll pay for news in the future. Smart companies like The Economist will give the print edition away free with the iPad version, or vice versa, since we’ll start resisting the idea that we have to pay twice for the same information, whether it’s all glitzy and interactive or not. We will expect to be rewarded for paying for something we know we can get from somewhere else if we tried hard enough. If you’re a news organization use whatever lure you can think of to get the reader back into the paying habit again.

This is the point of the payment process. It has to be easier than getting the information/music/entertainment/book through another means. If I find a book for my Kindle ereader on Amazon I’ll check to see whether there’s a cheaper version—which there quite often is. If it’s under ten bucks I’ll buy it. If not, I’ll read the reviews below to see whether there is a free version somewhere—which is sometimes possible. If there isn’t, I’ll check out Google books to see whether the chapters I’m interested in are there.

OK, I’m a cheapskate. But my thinking is basically this: $10 is my threshold for an eBook. It might be more if I got access to a physical version, or was able to clip bits from it and store it somewhere else. But I’m not, so I won’t pay more than that. Moreover, I don’t want to be the mug who pays for something others get for free.

Everyone else has their own logic, but they’re probably not dissimilar to mine. We pay for things if we think the price is right for the convenience, and if we think that we’re not being suckered—which means that other people aren’t shelling out for it.

This is basically micropayments. It’s what we’d been hoping would happen for some time, and it took Apple’s megalomania and micromanagement to get us there. Now we’re nearly there, but we could still mess up. Some newspapers try to charge us for single articles, for example, misunderstanding that micropayment doesn’t mean microproduct. I don’t want to pay every time I visit your site: I want to pay for something that gets me seamless access to your product.

In other words, we’re paying for not having to pay (or register, or download, or enter codes, or any of that kind of nonsense.) This is why the term pay wall is so revealing—and why it’s doomed as a concept. We’re not buying information with our iPhone or Android app, we’re buying frictionless access to something—an icon on our display that may be a shortcut to a web page, or open an application,  we don’t care. All we care about is that it gets us to where we want to go, when we want to go there.

We’ve some ways to go before this works well. I can’t stand the idea that my Kindle book doesn’t belong to me in the way a real book does, and I refuse to buy any music that I can’t move around as I wish. I succumbed to buying some apps for an iPad I borrowed but Steve Jobs will rue the day if I can’t easily move them onto another iDevice if I ever end up getting one.

But the good thing is that we’ve found a way to make this palatable to people, and I am optimistic that the media, booksellers, music sellers and web developers can turn this into revenue streams that keep them going.

The Economist’s Secret: Its Limits

Interesting piece by Rafat Ali on quoting Michael Hirschorn of The Atlantic as to why The Economist is doing OK, while Newsweek and TIME are in free-fall:

“By repositioning themselves as repositories of commentary and long-form reporting—much like this magazine, it’s worth noting, which has never delivered impressive profit margins—the American newsweeklies are going away from precisely the thing that has propelled The Economist’s rise: its status as a humble digest, with a consistent authorial voice, that covers absolutely everything that you need to be informed about…The Economist has reached its current level of influence and importance because it is, in every sense of the word, a true global digest for an age when the amount of undigested, undigestible information online continues to metastasize. And that’s a very good place to be in 2009.”

Apart from the obvious reasons–the global thing (rather than Newsweek etc, who catered only for Americans Abroad)—I think the key thing here is that The Economist is digestible.

This means it’s finite. It’s a promise of a definitive digest in return for a commitment of time. It’s an odd equation: Give us some money and then we’ll give you back your time. (I’ve whittered on about attention being the scare commodity these days elsewhere.)

The other thing they point to in their critique is the web. The Economist folks aren’t link whores—linking in. They’re not link journalists—linking out.  (This doesn’t mean The Economist shouldn’t be online; it’s just that it shouldn’t try to be just another part of the big wide-web.)

There’s a lesson in here for all mainstream media. Well, several, actually:

  • Don’t focus on eyeballs. Concentrate on attention. Your readers won’t thank you for wasting their time with more stuff to read. They want the digest.
  • Don’t try to be trendy. The Economist looks little different than it did in the 1970s. That, actually, is the selling point.
  • Online has lots of different opportunities. I don’t think they’ve made full use of them yet, but at least they haven’t thrown out the baby with the bathwater. That may prove the smartest thing they’ve done so far. As Hirschorn says in the TV clip, when you can get a subscription to a magazine for virtually nothing, what kind of commitment does that demonstrate (on either side?)

Hirschorn: The Economist Benefited From Being Semi Competent About the Web | paidContent

The Death of DRM, the Rise of Patrons

Forget being a big old mass music consumer. Become a Patron of the Arts.

The IHT’s Victoria Shannon chronicles the last few gasps of life in Digital Rights Management (DRM) for music, saying that “With the falloff in CD sales persisting and even digital revenue growth now faltering in the face of rampant music sharing by consumers, the major record labels appear to be closer than ever to releasing music on the Internet with no copying restrictions.” This has the inevitability of death about it (this morning I tried again to rip my DRM-crippled Coldplay CD of X&Y, unsuccessfully) which makes me wonder: What will follow?

Most thoughts seem to be on the free music, supported by advertising, and largely distributed as promotion for expensive live concerts:

Jacques Attali, the French economist … who forecast in his newest book that all recorded music would be free in the next several decades — consumers will instead pay for live performances, he predicted — said the business model of digital music should reflect the old radio model: free online music supported by advertising.

“A lot of people will still make money out of it,” he said during an interview at Midem.

I think this shows a lack of imagination and understanding of how music has fractured. My sense is that while Britney Spears will continue to exist in the Celebrity for Celebrity’s Sake World, music has already spread via MySpace etc into much smaller, more diverse niches. I’m not saying anything sparkingly new here, but given that most articles about the majors and DRM and online file sharing focus on the big numbers, I would have thought a much more interesting model to look at is that on places like eMusic, of which I’ve been a subscriber since 2002.

What happens for me is this: I find an artist I like by searching through what neighbors are selecting for me, like this balloon on my login page:

And then I’ll follow my nose until I find something I like. Or I’ll listen to until I hear something I really like and then see if it’s up on eMusic. This is all pretty obvious, and I’m sure lots of people do this, and probably more, already. But what I think this leads to is a kind of artistic patronage where we consumers see it in our interests to support those musicians we love.

In my case, for example, if I really like the stuff of one artist I’ll try to contact them and tell them so: No one so far has refused to write back and hasn’t sounded appreciative to hear from a fan. Examples of this are Thom Brennan and Tim Story, whose music I find a suitable accompaniment to anything, from jogging to taking night bus rides to Chiangmai in the rain. I’m summoning up the courage to contact my long time hero, David Sylvian, who doesn’t have a direct email address.

Of course, nowadays one can view their MySpace page, or join an email newletter, and build links up there. But my point is this: My relationship with these musicians is much more along traditional lines of someone who will support their artistic output through financial support — buying their music in their hope that it will help them produce more.

Surely the Internet has taught us one very useful lesson in the past year: That it’s well-suited to help us find what we want, even if can’t define well what it is. First step was Google, which helped us find what we wanted if we knew some keywords about it. Next step: a less specific wander, a browse in the old sense, that helps us stumble upon that which we know we’ll want when we find it.

What Probably Won’t Happen in 2007

The BBC has asked me to make some predictions about the coming year, something I’m always loath to do because I seem to get it wrong. Anyway, here are my notes. They’re based in part on my own bath-time musings, and partly inspired by the thoughts of others.

1999 just took longer than we thought, that’s all

Web 2.0 is not just about AJAX, mashups, blogs and all that. It’s about building a platform. That’s now been done. All we need to do now is let people use it. That is already happening, but it will REALLY happen in 2007:

Delivery will get better

RSS will stop being something we have to keep explaining to people, because they’ll be using it. It will be seamless — a way for anyone to join something, whether it’s a newsletter, a service, a MySpace group. It will stop being known as Rich Site Syndication or Really Simple Syndication and be Really Simple, Stupid.

Content will get better

The real improvement in computers will be the rise of the dual- and four-core processor, i.e. one that uses more than one chip. Think of it as the computer having more than one brain. This will speed up, and make easier, the editing of video and other multimedia content. Our computer, in a word, will no longer be an expensive typewriter. With faster connection speeds, too, video will be the thing that really makes the Internet compelling to people who were previously uninterested. What we watch on YouTube will get better. Individuals will have their 15 megabytes of fame. But this will couple with a rise of content generated specifically for the Internet, further blurring the lines between TV and computer viewing. Silicon Valley is no longer a tech center, but a media one, as Andreas of the Economist told the .

The demise of big software

The rise of online applications will in turn blur the distinction between what is going on in your computer and what is going on at the other end of the line — the server. Vista will seem more like a farewell than a big hello, as big software from big companies locking in users to specific ways of doing things will give way to open source alternatives like Ubuntu. Microsoft will fight this tooth and nail, but I’m sure they already know it.

The mainstreaming of social media

 Web 2.0 is really all about breaking down barriers by making it easier to do stuff, and to mix it up with other people doing stuff. In a way what the Internet has always been about. Expect the influence of blogs to further pervade those last few citadels that have been resisting it, breaking down walls within organizations — internal blogs that flatten hierarchies and build up feedback mechanisms for employees to talk back to their bosses. Think government departments. Think universities, schools and anywhere else where hierarchies exist. This won’t be a one way street: anonymous bloggers in places like Microsoft and China may find themselves outed and lynched.

The rise of the maven

As the Web gets bigger, Google will need to reinvent itself to keep up. Web 2.0 offers some great ways to find stuff through other means, leveraging the knowledge of others who have gone before. We will acknowledge the contribution, and marketers will acknowledge the power, of the maven: the person who seems to somehow know stuff, and is ready to share it. Tagging, blogging, and other social tools will be recognized as extremely powerful ways to do this.

The demise of the big computer

The cellphone will get better at what it does, and we’ll grow to trust it more. We’ll stop calling it a cellphone and just call it a wearable device, or something snazzier I can’t think of right now. One day we’ll think it quaint that we had to sit in one place to do stuff, or near an outlet, or within range of a WiFi signal. Cellphones don’t have those limitations and this will start to hit home in 2007:

Teenagers will show us the way. Again

They’re already sharing everything via Bluetooth, creating networks on the fly (that, incidentally, fly under the radars of commercial networks and marketers). They share videos, ringtones, songs, games, either by exchanging content or playing against each other.


The cellphone has already redefined what space is, and that will continue. Sexual liaisons involving public figures will be recorded by one party as insurance against future hard times. Cellphone television will become more popular, not just because it’s mobile but because it’s personal, a time to be alone under the sheets, on a bus, waiting for a friend, stuck in traffic. Maybe not this year, but soon they’ll be pluggable into the hotel TV. This is tied into the idea of personal space being something you control, either through presence, or through intermediary services where you only ever hand out personal details of your virtual self.

The End of the iPod

The iPod will decline in importance as the music-phone takes center stage. I didn’t think this would happen because cellphone manufacturers mess up the software on the phone, but they’re getting better at it. Even Nokia. So expect most people, starting with teenagers who don’t want more than one gadget and probably can’t afford them, to switch to one device. This will again throw open the mobile music/MP3/DRM debate, and iTunes will start to look a bit wobbly until Apple gets something sorted out so non-iPod users can download from the site easily and cheaply.

The downsides

It’s not all fun and games. Bad things are going to continue to happen, and there’s not much we can do about them. It’s partly just the normal process of utopians being pushed aside by realists, but it’s also about an ongoing debate about how to, or whether to, police a space that seems largely unpoliceable.

A dual identity crisis

Mainstream media’s identity crisis will be compounded by an identity crisis among bloggers. The rise of pay-me blogging, where bloggers get paid for writing about specific companies or products, will lead to some scandals and make people explore more deeply the ethics of blogging, and how they’re not that much different to the ethics developed by journalists over several hundred years. This won’t however, lead to the demise of blogging, but the rise of a sort of online press corps, with its own associations and rules, both written and unwritten. Many bloggers will end up being journalists, and the best journalists will move effortlessly and happily through the blogosphere. Many already do.

Keep up, grandma

Things are moving so fast the slow will look like they’re running backwards. If 2004-6 were anything to go by, 2007 will move quite quickly. Some folk I spoke to said that not much has popped up this year that’s exciting, and that’s true, in a boiling frog type way. It’s the earth shifting that is changing, and we need to change with it. Young people just get it, but us digital immigrants need to not just learn the lingo but keep up with the fast-changing slang. Oh, and MySpace won’t be the place to hang out in 2007; it’ll begin to look like a sad school hall dance arranged by the teachers.

The Rise of the Snoop

We tend to make a distinction between these things, but they’re actually all part of the same thing. Spam is getting worse, and it’s not just an invasion of privacy but an invasion of our productivity (91% of email is spam.) Music and video files will also rise as vectors of trojans, malware and other PUPs. GPS devices married to phones will enable people to track their employees, spouses or offspring, and further empower stalkers. Cellphone monitoring devices like FlexiSpy will get better at distributing themselves, and will be powerful not just in the hands of eavesdropping acquaintances but identity thieves. The rise of virtual worlds will also lead to the rise of virtual identities and virtual identity theft, along the lines of CopyBot. Expect to see cellphone eavesdropping and data theft from cellphones to surge. And we’ll start to realize that Google isn’t as cuddly as it looks; it’s a snoop, too.

An Unlikely Blogger Expelled

Although it’s not good for Sudan, I think it’s good for blogging: CNN reports that 

The government of Sudan on Sunday gave the top U.N. official in the country three days to leave, marking the latest hurdle in international efforts to bring peace to the nation torn apart by civil war.

Sudan expelled Jan Pronk, the top U.N. envoy to Sudan, who has openly criticized Khartoum as well as rebel groups on his Web log.

Pronk has been running a blog for nearly a year and while it doesn’t look like your average blog (really long posts, no external links, no comments, blogs numbered as if they were official UN documents) it’s an impressively direct account of the Sudanese conflict. His third post started as follows: 

This week the seventh round of the Abuja talks between the Government of Sudan and the rebel movements will start. Will it be the last one, producing a peace agreement before the end of the year? The chances are diminishing.

Not the sort of mealy-mouthed stuff we’re used to from senior UN officials. And it’s probably upset the UN as much as it’s upset the Sudanese government. But if so why had the UN not closed him down earlier? Pronk, according to UPI, did not offer any disclaimers, but the UN has since made clear he was writing in a personal capacity. The UN has “no rules barring blogging specifically, though employees face restrictions when publishing articles and participating in interviews.” It seems Pronk was probably senior enough, and his comments uncontroversial enough, for no one to mind too much. Until last week.

What I like about it is that reporters tend to meet these kind of people in the field, and it’s great to hear them sounding off about the situation, but rarely are their words captured in sufficient quantity for their great background knowledge and high level involvement in such diplomatic processes to be read by a wider audience. I’ve not followed the tragedy in Darfur much beyond what I read in the papers, but Pronk’s year-long posts are a diary of immense and satisfying detail about the process, peppered by great photos, that are worthy of more than the word blog. 

Take this one, for example, from June 28

There is a significant risk that the Darfur Peace Agreement will collapse. The agreement does not resonate with the people of Darfur. On the contrary, on the ground, especially amongst the displaced persons, it meets more and more resistance. In my view it is a good text, an honest compromise between the extreme positions taken by the parties during the negotiations in Abuja. That is why the UN, like all international partners, has endorsed the agreement. However, in politics objective rational calculations will always be confuted by subjective emotional perceptions and aspirations. And those perceptions are that the agreement does not meet the expectations of the people in Darfur, has been forced upon them and, rather than meeting the interests of all parties somewhere halfway, only strengthens the position of the government and a minority tribe, the Zaghawa.

That too me is very clear writing, reflecting his knowledge of the situation on many levels. Not every situation could allow a senior figure involved deeply in the political process to write so frankly and openly, but wouldn’t it be great if they could? This to me is the real potential of blogs and citizen reporting. Someone who really knows what is going on telling us about it.

PS: Jan Pronk has a reputation of sorts in Indonesia, my current abode. He earned the lasting enmity of then president Suharto by

Read more