Open Information, Data Viz, & The New York Times

The New York Times will not go quietly into the dark knight of new media. Amidst constant rumors of the death of traditional news, the much-respected industry stalwart is moving quickly to build a compelling and forward-looking solution that redefines “the newspaper as platform”, as ReadWriteWeb’s Marshall Kirkpatrick notes.

Today the NYT announced that 2.8 million articles will be exposed to the digital world through the site’s API, allowing anyone to link, annotate, mashup, and crawl the data for meaning. This opportunity to construct data visualizations that abstract patterns & trends from within the articles is perhaps the most interesting element that immediately adds human value to what is otherwise an overwhelming amount of information (2.8 articles).

The recent Twitter Superbowl visualization, as well as other visualization experiments at NYT.com, are indicators of how the company is gathering data and parsing it in meaningful ways. A list of Twitter posts related to the Superbowl is just a long index table. Even reading the Summize Search feed for such a huge event is dizzying. But a geo-located, timeline mashup of tweets & key terms with a map of the US is immediately valuable to anyone trying to get a bead on trends. Their implementation is simple & entertaining, and you can derive substantial meaning at a glance.

These experiments are proofs of concept that point the way towards more advanced viz mashups now further enabled by opening the NYT information archives and building a coherent API on top. Imagine, for example, sections of NYT.com dedicated to serving all outgoing comm from a particular region, say Gaza & Jerusalem. Imagine seeing seeing real-time visualizations of the thoughts and feelings of average citizens free from the carefully structured statements of the vested power interests hurling rockets and armor at each other. Or imagine crawling the news reports of the last 8 years looking for instances of the words “Bush”, “Abramoff”, and “Florida flight school”…

Of course, this is another big win for the information transparency movement – information wants to be free, after all – and you can expect many others to get the message and follow suit. But it also wraps the current events of our world as reported by NYT in a searchable and re-configurable layer establishing a protocol for interfacing with these vast data stores. This open approach certainly cries out for some sort of semantic layer and I suspect the Reuters/Calais folks are paying very close attention to this announcement.

This is the prevailing trend of this current phase of digitizing culture and communication. Data is accumulating at an ever-increasing rate requiring open standards for archiving, interrogating, & visualizing the meaning held within. The tools are evolving to sort the signal from a vast sea of noise. More and more information archives will be exposed and more and more tech will be created to interface with it and draw out meaning from the morass. The global sharing of information and communication is feeding the pool of innovation that continues to radically alter the face of our world with each new discovery.

Whether or not information wants to be free… We certainly need it to be.

MMOGs and CCTVs: 3D Games For Spectators

Killer feature: Immersive gaming companies like Rockstar, Blizzard, ID, Epic, NCsoft, Valve et al should implement a way for game server admins to tap live in-scene camera feeds that can be bussed out to HTML. Liberty City should have public CCTV’s that viewers can watch from their browser or mobile client. Like a virtual Adam’s Block people could tune in to an alleyway or freeway underpass and let the action unfold in the streets below. World builders should enable first/third-person game cameras to send their viewport to Flash embeds.

Imagine tuning into a South Korean Quake deathmatch tournament as a spectator and being able to view the world through the game eyes of the champion or to switch across various fixed camera feeds in corridors, over walkways, and above central arenas to witness the gameplay from alternate angles. It might seem odd in sports-addled America and the UK but in Asia hundreds of millions are online and they love 3D gaming. The gameworld becomes a performance space. Second Life could finally entertain more than just the local cliques by broadcasting the actions of art collectives and protest groups out to the world.

It doesn’t take much effort to see how this multiplies the available media advertising real-estate considerably. Fixed camera views become hot advert property rented to savvy marketers who know they can reach both the local gamers and the viewing masses. Attention property goes meta. This is a two-way street as external feeds begin to pipe into the game worlds. The walls really start coming down when you can take a cellphone call in-game and then wave to the camera for your friends to see. They take a screenshot and then send it to the clan web site and the GTA Flickr feed that gets displayed on the side of a building in downtown Liberty City. Sponsored by Verizon of course.

Blizzard claims something like 10 million regular users of World of Warcraft. Their WoW wikia page is the largest collection of data on anything in the world. You think they might be interested in being able to view their clan members remotely and communicate with them at any time? Or tune into video feeds showing the night elves in Darnassus, or watch the pass at Chillwind’s Point? (Truth: I googled these.) And if Blizzard wins the fight to keep the gameworld free from advertising, marketers can accrete around the popular viewing channels and their Flash embeds.

These worlds are dark clouds, opaque from all but those that pay the playing fee. Yet there’s so much entertainment and interest and real human nature playing out in these worlds. People will watch. Agents will sift through the user data feeds streaming out along with the A/V feeds. And think of the up-sell as viewers convert to players. Someday I’m certain some of the reality tv channels on my mobile device will be looking into such immersive worlds and showing me where in my friends are and who they’re battling and what type of car they just jacked for a joyride through Liberty City. (I *really* want the GTA CCTV’s!)

Tagging & Findability

I’ve been thinking more about the value of metadata and the challenges in tagging. Clearly we want more data traveling with files. Hard data like source and profile, but also soft associations like folksonomies and tags. I think most people are probably very loose with their tagging and become quickly overwhelmed by the giant tag clouds they create. For people with a lot of content it becomes very important to limit the scope of their taxonomy/dictionary. Likewise, as the audience for content expands it becomes more important to have matadata that is useful and meaningful to the widest range of people. So, highly subjective tagging actually creates more noise around the content and limits its findability.

Which leads to the real value and goal of tagging: findability. Search and sort. So much data is overwhelming. We need simple and intuitive ways to filter the sample. The old way of arranging files in hierarchical folders is fading. And really, for most people the file hierarchy was a necessary structure imposed by the OS but which is generally just a framework for associative cataloging. Most people filing their photos name their folders by association. Vacations. Family. Pets. Tokyo 2008. The value of these terms is not in where they live but what they are.

In an increasingly networked cloud, the physical location of content is unimportant but findability is critical.