Virtual Autopsy Table

Incredible!

The Virtual Autopsy Table from NorrköpingsVisualiseringscenter on Vimeo.

Subcycle Multi-Touch Instrument Experiment

multi-touch the storm – interactive sound visuals – subcycle labs from christian bannister on Vimeo.

Christian Bannister, Subcycle Labs: “Things are starting to sound more song-like and I can really appreciate that. In previous builds everything sounded more like an experiment or a demo. Now I have something more akin to an experimental song. “

The Co-Evolution of Neuroscience & Computation

Image from Wired Magazine.

[Cross-posted from Signtific Lab.]

Researchers at VU University Medical Center in Amsterdam have applied the analytic methods of graph theory to analyze the neural networks of patients suffering from dementia. Their findings reveal that brain activity networks in dementia sufferers are much more randomized and disconnected than in typical brains. "The underlying idea is that cognitive dysfunction can be illustrated by, and perhaps even explained by, a disturbed functional organization of the whole brain network", said lead researcher Willem de Haan.

Of perhaps deeper significance, this work shows the application of network analysis algorithms to the understanding of neurophysiology and mind, suggesting a similarity in functioning between computational networks and neural networks. Indeed, the research highlights the increasing feedback between computational models and neural models. As we learn more about brain structure & functioning, these understandings translate into better computational models. As computation is increasingly able to model brain systems, we come to understand their physiology more completely. The two modalities are co-evolving.

The interdependence of the two fields has been most recently illustrated with the announcement of the Blue Brain Project which aims to simulate a human brain within 10 years. This ambitious project will inevitably drive advanced research & development in imaging technologies to reveal the structural complexities of the brain which will, in turn, yield roadmaps towards designing better computational structures. This convergence of computer science and neuroscience is laying the foundation for an integrative language of brain computer interface. As the two sciences get closer and closer to each other, they will inevitably interact more directly and powerfully, as each domain adds value to the other and the barriers to integration erode.

This feedback loop between computation and cognition is ultimately bringing the power of programming to our brains and bodies. The ability to create programmatic objects capable of executing tasks on our behalf has radically altered the way we extend our functionality by dematerializing technologies into more efficient, flexible, & powerful virtual domains. This shift  has brought an unprecedented ability to iterate information and construct hyper-technical objects. The sheer adaptive power of these technologies underwrites the imperative towards programming our bodies, enabling us to excercies unprecedented levels of control and augmnetation over our physical form, and further reveal the fabric of mind.

Apple Threatens Another Revolution With iTouch Tablet

“We are witnessing a transition in the way the Internet is used. Mobile content requires a tailor made user experience that is not efficiently delivered by the traditional website model.” – Jason Schwarz, Why Apple’s iTouch Tablet Will Become Its Flagship Product

Brain-Computer Interface

In my present tenure as a Visiting Researcher at the Institute for the Future I’ve been posting a lot of Signals pertinent to Brain-Computer Interface over at the Signtific open source research site. My Signals are listed under the tag “ProgrammableEverything”.

Check ‘em out if you’re interested in the fascinating & accelerating field of BCI. Also feel free to add your own Signals you see in the world or are engaging in your professional research.

Cheers!

Companies to Watch: IBM & SAP

In a time of monumental change it’s important to look at how the big player’s are adapting. Their moves are typically the most heavily researched and financed attempts at divining the underlying currents and capitalizing on the shifting technological marketplace. It’s especially interesting when conservative tech stalwarts like IBM & SAP suddenly start looking cool.

Both IBM & SAP are moving quickly into 3 of the most powerful trends in computing, each of which are driven by the enormous amounts of data being captured across all domains: business intelligence & modeling, stream computing, and sustainable systems analysis.

IBM’s new initiative A Smarter Planet states succinctly, “the planet will be instrumented, interconnected, intelligent.” This is a powerful statement from one of the largest and most technologically advanced companies in the world. They’re not just talking about business. IBM CEO Sam Palmisano speaks to the really large-scale planetary challenges in creating smart infrastructures for energy, water, transport, and data.

A key component is the recently-announced System S project for supporting so-called Stream Computing.

System S is designed to perform real-time analytics using high-throughput data streams… to host applications that turn heterogeneous data streams into actionable intelligence… System S applications are able to take unstructured raw data and process it in real time.

“This is about what’s going to happen,” explains [director of high performance stream computing at IBM] Nagui Halim. “The thesis is that there are many signals that foreshadow what will occur if we have a system that is smart enough to pick them up and understand them. We tend to think it’s impossible to predict what’s going to happen; and in many cases it is. But in other cases there is a lot of antecedent information in the environment that strongly indicates what’s likely to be occurring in the future.”

With enough data you can start to create connections and patterns. With patterns you can derive meaning and ultimately be better enabled to make more accurate predictions. Since humans aren’t very well-adapted to processing large data sets, we build tools to handle the heavy lifting. Whether Wall Street indexes, ERP scenarios, government accounting, energy grid analysis, or dynamic climate models, serious hardware & software is required to process operational data into meaningful determinations and prescriptions.

SAP has introduced the Clear New World initiative built on their Business Objects service architecture. Again, the notion is that businesses, enterprises, and even governments can run more efficiently when there is a free-flow of data and a suite of integrated services to crunch and render the info into meaningful contexts.

It’s time to build greater visibility, transparency, and accountability into the way your organization works. Because being clear allows timely and relevant information to be available when and where it is needed. Clarity demonstrates that your company is willing and able to stay accountable to key stakeholders. Clarity helps call out inefficiencies, reveal your best customers, create credible sustainability, and give your business the flexibility needed to anticipate and respond to a complex, ever-changing, global environment.

[See James Governor's recent post for more on how SAP & IBM are tackling enterprise sustainability.]

Note the statements about accountability to stakeholders & creating credible sustainability. Clear data & clear reporting. Now consider the latest announcement about SAP for Public Sector “to support the management and reporting of economic stimulus funds”. As a plugin to their Business Objects suite, this utility drafts on the trends towards open accountability and government transparency, often termed Gov 2.0, to provide support for determining just how stimulus money is being spent.

Both IBM and SAP have the power to execute effectively on these strategies, though it remains to be seen how enterprise spending will move to implement these services or if the companies will offer flexible licensing to LLC’s working on the really challenging non-profit global issues. Likewise, SAP has suffered usability problems for years and their core object architecture is old and slow. They will need more than just branding and plugins to make a more transparent world.

Finally, it’s worth noting the branding for these projects. “A Smarter Planet” is a global posture indicating agency and identity on a planetary scale. This hints at the real deep trend across the human species towards a global sense of purpose and strategy. “Clear New World” acknowledges both the occlusions under which human endeavor has marched thus far and the great clarity of visibility we’re now gaining across all domains & enterprises, while admitting that indeed everything is changing and we are moving into a New World. The technology is stepping forward to help us more effectively manage the present and navigate into the unknown future. But of course like all foresight, it remains to be seen whether individuals will choose to act appropriately with the knowledge they come to possess…

Modeling & Superstructing

A core human competency is the capacity to model outcomes. This predictive ability has contributed to our successful growth as a species and provided the stage from which we extrude our technologies. We observe our world, log our experiences, and use this information to envision & plan our future possibilities. In the rush into tomorrow we’ve deputized machines to assist in our scenario modeling as our plans grow ever greater in scope.

Today we have tremendous amounts of data available about any system we wish to model. Drive platters are bulging into the terabytes just to store all of the information gathered by sensors, services, and empowered humans. Whether we study business networks, financial models, or natural systems, our awareness of their complexity has grown exponentially. Things are far wider and more interconnected than we could have imagined even 20 years ago.

All systems are sets of nodes with properties & variables that govern their behavior, coupled together by relational rules governing their interaction. The more complex a system, the more unique nodes and the more interconnections between nodes. Given the human constraint of being able to hold only 6 or 7 unique objects in mind at any given time it’s clear that we’re overwhelmed by even the relatively simple tasks of understanding, for example, a mid-size business structure enough to predict its future, especially when you consider the business system itself as a single node embedded in a much larger global socio-economic system. Imagine the difficulties climate modelers face trying to document global circulatory systems…

One emerging strategy for modeling complex systems looks to software and the floating-point wonders enabled by Moore’s Law. Computers are phenomenally capable of managing the inconceivable amounts of operations necessary to begin modeling dynamic systems. Yet, until very recently one needed to book time on a supercomputer cluster to run weather models or robust behavioral analysis. Even today’s bleeding hardware strains under the weight of such complexity. Research institutions have pursued natural systems modeling for some time and the business world has been paying attention. SAP now offers modeling capabilities with its business intelligence ERP solutions, enabling executives to run scenarios and envision possible outcomes of strategic decisions. Oracle recently acquired Hyperion, adding “performance management” to their suite of BI tools. You can bet these technologies will work their way into government & geopolitical protocols, as well as social & personal behavioral engineering as we increasingly track & model our lives.

Effectively, this pattern emulates the deeper shift from individual enterprise to collective collaborations. You can only model a complex system with another sufficiently complex system. However, even the most interesting algorithms are encumbered by the impositions of their logic: they can only be as creative as they were written. A second emerging strategy for modeling complex systems looks to deputize humans as processing nodes, crowdsourcing future possibilities across infinitely creative sets of minds. The Institute for the Future has taken this approach with its Signtific Lab and the Superstruct platform, leveraging the principles of gameplay to engage massive participation in envisioning scenarios.

The Superstruct games have drawn in thousands of players offering their thoughts & dreams of the future. Players become processing nodes for the chosen subject (eg. “when augmented reality is everywhere”, or “when personal satellites are as easy to deploy as websites”) iterating across large sets of potential outcomes. From these inputs, patterns emerge showing trends with greater frequency & momentum among the collective. Perhaps even more interesting – and where the Superstruct method is more flexible than computational modeling – are the outliers that emerge from players. Many of the most compelling signals of the future are those that completely break from current patterns. Indeed, one of the most fundamental prevailing shifts in the global paradigm is that change is accelerating in ways we cannot even imagine.

These two approaches both consider complex systems & scenario modeling from architectures that themselves are complex, object-oriented systems. The programmatic approach brings heavy-weight numeric bit-crunching to dynamic data streams, while the Superstructing approach offers wide-reaching creativity and human sensing. Augmenting one approach with the other will mark the next phase of predictive analysis necessary to safely navigate civilization through the future. Envisioning these scenarios and building compelling narratives around them will inevitably draw them into becoming.

Our lives are more & more complex and our enterprises & collaborations are commonly reaching global scales. The need to effectively model & predict is a fundamental human trait, reinforced in the face of escalating complexity in a hyper-connected, Read-Write world.

Patterns: Global Systems & Human Adaptation

Overview: The top-level context for the next 10-20 years will be characterized by growing environmental challenges across the planet, notably more irregular weather patterns with increasingly severe storms, a rise in temperatures and a reduction of rainfall leading to shifting distributions of agriculture and farming. Regions that are heated but retain humidity will face rising bacterial & viral outbreaks, especially if these regions see further economic declines due to declining food production. These changes will challenge many populations, adding pressure to invest in more climate-controlled (and energy-intensive) infrastructure and/or migrate towards more wet & fertile lands. The great dependence on rainfall and water delivery infrastructure coupled to its widely distributed nature will impact drought-stricken regions considerably, as well as neighboring water-rich regions (eg. Los Angeles and Northern California) that may see growing tensions across resource inequities.

Within this global system the primary drivers remain materials technologies, energy capture & generation, health care (freemium & premium), cleaning & streamlining industrial processes, managing supply chains (particularly with respect to resource/energy overhead, social & environmental impacts), remediating toxic environments, and coping with persistent disruptions to all of these. In communities, trends are moving towards group empowerment through emerging technologies for computation, communication, collaboration, design, and fabrication. This empowerment enables both resilience & resistance, aiding some to design better civic structures & local production capacity, for example, while others design and execute disruptive events and attacks on high-value targets.

Across the species, though in no way homogeneous, lifespans are extending, health care is more reliable, mobile computing is more powerful & ubiquitous, screens and media are proliferating, and more people, objects, plants, and animals are creating digital identities and communicating across the cloud. There is a rapid movement to digitize human information and expose it to massive computational structures, iterating exponentially across literally billions of logical nodes. This movement into the cloud has a huge energetic overhead only recently being considered – not to mention the social and economic impacts rapidly rewriting much of the first world.

Computational systems are evolving to model and predict larger living systems. We now model natural systems, business enterprise, financial variables, and human behavior deriving greater ability to predict future probabilities. All in order for the species to continue its adaptive success while willfully managing our resource requirements & impacts while effectively supporting a global virtualization of human endeavor, expression, and creativity.

In a nutshell, the patterns and processes we’ve relied upon are moving into a time of great flux with all systems facing regular perturbations. Change is the only constant. Survival, as it always ultimately has, depends on flexibility, resilience, collaboration, and adaptation.

Disruptive Civil Technologies
Six Technologies with Potential Impacts on US Interests out to 2025 (National Intelligence Council):

Key trends, “most likely to enhance or degrade US national power out to 2025″
- Biogerontechnology
- Energy Storage Materials
- Biofuels and Bio-Based Chemicals
- Clean Coal Technologies
- Service Robotics
- The Internet of Things.

[The NIC report offers some interesting signals but I personally disagree with their sense of trending towards biofuels. Turning human energy sources (food) into industrial energy sources (biofuel) is exceptionally short-sighted and dangerous and has already incurred a large backlash in common sense. I don't know enough about so-called "clean coal" to comment... but I'm highly dubious.]

From the IFTF Winter 2009 Overview and Jane McGonigal’s initial Superstructing results.

Top Signals 2009
- Geolocation
- Biometrics & accelerometers
- Handheld augmented reality
- Simulation engines
- Lifecasting platforms
- Social networks for every living thing
- Avatars everywhere
- Virtual worlds based on real worlds

Critical Factors
- Evolvability
- Extreme scale
- Ambient collaboration
- Reverse scarcity
- Adaptive emotions
- Amplified optimism
- Playtests

DRAFT 2009 Climate Action Team Biennial Report to the Governor and Legislature (California Climate Change portal):

All simulations indicate that extremely hot daytime and nighttime temperatures (heat waves) increase in frequency, magnitude, and duration from the historical period. Within a given heat wave, there is an increasing tendency for multiple hot days in succession—i.e., heat waves last longer. Furthermore, the number of days with simultaneously hot daytime temperatures in multiple regions in the state increases markedly; this has important implications for emergency response and satisfying electricity demand in the state.

…In the northern part of California, the tendency for drying fades and even reverses but in Southern California the amount of drying becomes greater, with decreases in some simulations exceeding 15% drier. became significantly wetter by the end of the century.

…The results suggest that climate change will decrease annual crop yields in the long- term, particularly for cotton, unless future climate change is minimized and/or adaptation of management practices and improved cultivars becomes widespread.

…In summary, without changes in operating rules for the water system in California the reliability of water supply will be severely affected. On the other hand, it seems that California could afford the implementation of adaptation measures that could significantly reduce the system’s vulnerability.

Human Identity & Evolutionary Biology

Some rough notes from the weekend on the Northern California coast… I’m trying to get at the core of my general orientation towards the world. It’s coming into focus at the nexus of evolutionary biology & technology. Or…

How does evolutionary biology express through culture & technology?

Requirements of human biosurvivial & social identity (compare to Maslow’s Hierarchy of Needs):

water, food shelter, fecundity, mortality, socialization, cognition, communication, migration, lineage, history, myth, aspiration, discovery, expression, emotion, time, transcendence.

Global comm networks are rapidly bringing the world closer and changing human cognition in ways we cannot yet fully see. What are the impacts and consequences of the emerging self-identification of the human species? How will we manage the human agency? Do we have a global strategy yet, or just a Balkanized polyculture of mostly-competing sub-identities? (Obv. the latter.) Compare to the Greek & Roman consciousness that embodied emotional states & psychological constructs in the mythic drama of deities & demigods. The western religious myth of Earth as resource and Earth as purgatory elevated us above the natural world. The planet is now urgently reminding us that we are within the natural world – a subset embedded in a much larger and ultimately self-interested system.

The assertion of the natural world compels us towards alignment with biomimetic solutions & protocols. Or towards oblivion as we are corrected by the planetary system. We cannot destroy the world before it limits our ability to do it damage. The compulsion towards environmental protection is a species-wide awareness rising from our very cells and fueled by our growing awareness of our impact on the planetary ecology. Adapt or perish.

Socio-economic & ecological adaptation is not on a uniform schedule. Diverse states & peoples have their own schedules to work out as they march up the pyramid of civilization. Does this demand caretakers & parent states? Globalization is a normalizing force, but inequities between self-appointed parents and emerging economies will grow, as will the ability of smaller networks to inflict their will on states, NGO’s, & global systems. This democratization of technological empowerment is yet another major current working through our species. We’re getting stronger yet the morality(?) & responsibility expected to wield this power is not uniform across cultures & peoples. Core biosurvival needs remain the primary driver, exposed to shifting climates and diminishing conventional energy sources. There will be (more) blood.

The race is whether the technologies of liberation & salvation will outpace the technologies of destruction & exploitation. Of course, the real technology underneath both is the human brain – a much more subtle & powerful tool, highly malleable but stubbornly resistant to overt change.

E-Tech 2009 Twitter Round-up

Here’s a selection of my tweets from the O’Reilly Emerging Technology Conference this past week. These are the ones I think grab the juicy nuggets from the speaker’s presentations. [In temporal order with the earliest (ie Monday eve) listed first.]

Tim O’Reilly: “We have greatness but have wasted it on so much. ”
We have an unprecedented opportunity to build a digital commonwealth. #etech
Work on something that matters to you more than money. This is a robust strategy. #etech
Niall Kennedy: Energy Star rating for web apps? Thinking of clouds & programming like tuning a car for better gas mileage. #etech
Cloud computing: no reasonable expectation of privacy when data is not in your hands. Not protected by 4th amendment. #etech
Alex Steffen: Problems with water supply are based in part on our lack of beavers. #etech
Social media for human rights. http://hub.witness.org #etech
Gavin Starks – Your Energy Identity & Why You Should Care. see http://amee.com #etech
Maureen Mclugh – Consider that technology may be evolving in ways that are not particularly interested in us. #etech
Becker, Muller: We have under-estimated the costs and over-estimated the value of our economy. #etech
Becker, Muller: We assume economic trade must be the primary framing of value in our lives. Why? #etech
Design Patterns for PostConsumerism: Free; Repair Culture; Reputation Scaled; Loanership Society; Virtual Production. #etech
NYT: emerging platforms, text reflow, multitouch, flexy displays, smart content, sms story updates, sensors, GPS localized content. #etech
Jeremy Faludi: Buildings & transport have the largest impact on climate change. Biggest bang for the buck in re-design. #etech
Jeremy Faludi – Biggest contributor to species extinction & habitat loss is encroachment & byproducts from agriculture. #etech
Jeremy Faludi – Best strategies to vastly reduce overpopulation: access to birth control & family planning, empowerment of women. #etech
Tom Raftery: Grid 1.0 can’t manage excess power from renewables. Solution: electric cars as distributed storage. #etech
Considering the impact of pluging AMEE (@agentGav) data in ERP systems for feedback to biz about supply chain impacts. BI meets NRG ID.
Mike Mathieu: Data becoming more important than code. Civic data is plentiful and largely untapped. Make civic apps! #etech
Mike Mathieu: Take 10 minutes today and pick your crisis. Figure out how to create software to help. #etech
What is #SantaCruz doing to make civic data available to service builders? We want to help SC be healthier & more productive.
Mark Fraunfelder: “I haven’t heard of anybody having great success with automatic chicken doors.” #etech [re-emerging technology]
Realities of energy efficiency: 1gallon of gasoline = ~1000hrs of human labor. #etech
Kevin Lynch: Adobe is saving over $1M annually just by managing energy. #etech
Designing backwards: Think about the destiny of the item before thinking about he initial use. (via Brian Dougherty) #etech
RealTimeCity: physical & digital space merges, people incorporate intelligent systems, cities react in accord w/needs of pub welfare. #etech
Oh my we’re being LIDAR’d while Zoe Keating plays live cello n loops. ZOMG!!!
zoe keating & live lidar is blowing my mind at #etech 1.3M points per sec!
Julian Bleeker cites David A. Kirby: “Diegetic prototypes have a major rhetorical advantage over true prototypes” #etech
Julian Bleeker: Stories matter when designing the future, eg. Minority Report. #etech
Julian Bleeker: “Think of Philip K. Dick as a System Administrator. #etech
Rebecca MacKinnon: Which side are we helping, River Crabs or Grass Mud Horses? #etech
Kati London: How can we use games to game The System and how can they be used to solve civic problems? #etech
Nathan Wolfe: Trying to fight pandemics only at the viral human level ignores deep socioeconomic causes of animal-human transmission. #etech
Nathan Wolfe, re: viral jump from animal to human populations: “What happens in central Africa doesn’t stay in central Africa.”
Nathan Wolfe: need to work with % of population w/ hi freq of direct contact with animals for early detection of viral transmission.
Nathan Wolfe: Vast majority of biosphere is microscopic, mostly bacterial & viral. Humans: very small piece of life on Earth. #etech

Another Rant: On the Cloud, Augmented Reality, & the Networked World

[This is a reply I left recently to a Global Futures question about the near-future of the web. It goes a little off-topic at the end but such is the risk of systems analysis. Everything's connected.]

Within 10-15 years mobile devices will constantly interact with the world around us, analyzing objects, faces, signage, locations, and anything else their sensors can engage. Camera viewfinders will identify visual sources using algorithms to match them up with cloud data repositories. Bluetooth and GPS will interact on sub-channels silently exchanging relationships with embedded sensors across devices and objects. A user’s mobile device will become their IP address hosting much of their profile information and mediating relationships across social nets, commercial transactions, security clearances, and the array of increasingly smart objects and devices.

Cloud access and screen presence will be nearly ubiquitous further blurring the line between desktop, laptop, server, mobile devices, and the objects in our world. It will all be screens interfacing between data, objects, and humans. Amidst the overwhelming data/content glut we will outsource mathematical chores to cloud agents dedicated to scraping data and filtering the bits that are pertinent to our personalized affinities and needs. These data streams will be highly dynamic and cloud agents will send them to rich media layers that will render the results in comprehensible and meaningful displays.

The human sensorium and its interaction with reality will be highly augmented through mobile devices that layer rich information over the world around us. The digital world will move heavily into the natural analog world as the boundaries between the two further erode. This will be readily apparent in the increasing amount of communication we will receive from appliances, vehicles, storefronts, other people, animals, and even plants all wired to the cloud. Meanwhile, cloud agents will sort through vast amounts of human behavioral information creating smart profiles and socioeconomic and environmental systems models with incredible complexity and increasing predictive ability. The cloud itself will be made more intelligible to agents by the standardization of semantic web protocols implemented into most new sites and services. Agents will concatenate to tie services together into meta-functions, just as human collectives will be much more common as we move into increasingly multicellular functional bodies.

The sense of self and our philosophical paradigms will be iterating and revising on an almost weekly basis as we spread out across the cloud and innumerable virtual spaces connected through instantaneous communication. Virtual worlds themselves will be increasingly common but will break out of the walled-garden models of the present, allowing comm channels and video streams to move freely between them and the social web. World of Warcraft will have live video feeds from in-world out to device displays. Mobile GPS will report a user’s real-world location as well as their virtual location, mashing both into Google Maps and the SketchUp-enabled virtual map of the planet.

All of this abstraction will press back on the world and create even greater value for real face-to-face interactions. Familial bonds will be more and more cherished and local communities will take greater and greater control of their lives away from unreliable global supply chains and profit-driven corporate bodies. Most families will engage in some form of gardening to supplement their food supply. The state itself will be hollowed out through over-extended conflicts and insurgencies coupled with ongoing failures to manage domestic civic instabilities. Power outages and water failures will be common in large cities. This will of course further invigorate alternative energy technologies and shift civic responsibilities to local communities. US manufacturing will have partially shifted towards alternative energy capture and storage but much of the real successes will be in small progressive towns rallying around local resources, small-scale fab, and pre-existing economic successes.

All in all, the future will be a rich collage. Totally new and much the same as it has been.

Enabling Rich Content Rendering for Dynamic Ambient Displays

Now more than ever, screens are competing for our attention in daily life. The distinctions between desktop PC, laptop PC, mobile, etc… are being replaced by the simple abundance and omnipresence of digital screens conveying information and mediating interaction. As LCD newsfeeds, airport terminal displays, set-top video box menus, billboards, advertisements, multi-touch screens, and augmented mobile devices, data and content are everywhere. All of it requires a lightweight and dynamic graphic layer in which to render content. Dynamic render plugins like Flash and Silverlight are very well engineered to address the requirements of these displays but it should be understood that much of the technology represents a shift away from the 2D ad/interactive paradigm primarily addressed by the Flash runtime.

Mobile augmented reality solutions require heads-up-display, alpha-channel rendering of text and graphics. Dynamic data visualization requires strong integration with back-end databases as well as messaging protocols like JSON, SOAP, and SMS. To enable rendering large datastreams in plugin runtimes like Flash & Silverlight, companies should aggressively pursue runtime adoption across mobile devices and smart phones, while defining prototypes for active camera overlays. They should optimize rendering and expand into all rich-content displays (set-top, automotive, kiosks, smart objects, and embedded systems built on the Android/Linux platform). They should optimize for dynamic rendering of large data streams, like the Nasdaq AIR application. The runtimes should be increasingly exposed to SEO and analytics retrieval so that secondary services and agents can easily be built on user workflows and data collection. And they should not neglect the 3D gaming market, but should consider how it can play a role in immersive worlds as a dynamic data layer (eg billboard adverts that can be remotely updated).

Competitive landscape: Adobe Flash (dominant market share, full turn-key solution), Microsoft (Silverlight remains the strongest competitor to Flash though it continues to play catch-up with our tech), Java (JavaFX just released but has little traction and is too late to the party), W3C (HTML5 has perhaps the largest share of hearts but is also the slowest to move), Google (might be cutting around Flash & Silverlight by pushing its interests into HTML5, Mozilla, and through Android & Chrome).

Related: Augmented reality solutions will require semantic architecture and image recognition
algorithms (identification, recognition, relation).

MMOGs and CCTVs: 3D Games For Spectators

Killer feature: Immersive gaming companies like Rockstar, Blizzard, ID, Epic, NCsoft, Valve et al should implement a way for game server admins to tap live in-scene camera feeds that can be bussed out to HTML. Liberty City should have public CCTV’s that viewers can watch from their browser or mobile client. Like a virtual Adam’s Block people could tune in to an alleyway or freeway underpass and let the action unfold in the streets below. World builders should enable first/third-person game cameras to send their viewport to Flash embeds.

Imagine tuning into a South Korean Quake deathmatch tournament as a spectator and being able to view the world through the game eyes of the champion or to switch across various fixed camera feeds in corridors, over walkways, and above central arenas to witness the gameplay from alternate angles. It might seem odd in sports-addled America and the UK but in Asia hundreds of millions are online and they love 3D gaming. The gameworld becomes a performance space. Second Life could finally entertain more than just the local cliques by broadcasting the actions of art collectives and protest groups out to the world.

It doesn’t take much effort to see how this multiplies the available media advertising real-estate considerably. Fixed camera views become hot advert property rented to savvy marketers who know they can reach both the local gamers and the viewing masses. Attention property goes meta. This is a two-way street as external feeds begin to pipe into the game worlds. The walls really start coming down when you can take a cellphone call in-game and then wave to the camera for your friends to see. They take a screenshot and then send it to the clan web site and the GTA Flickr feed that gets displayed on the side of a building in downtown Liberty City. Sponsored by Verizon of course.

Blizzard claims something like 10 million regular users of World of Warcraft. Their WoW wikia page is the largest collection of data on anything in the world. You think they might be interested in being able to view their clan members remotely and communicate with them at any time? Or tune into video feeds showing the night elves in Darnassus, or watch the pass at Chillwind’s Point? (Truth: I googled these.) And if Blizzard wins the fight to keep the gameworld free from advertising, marketers can accrete around the popular viewing channels and their Flash embeds.

These worlds are dark clouds, opaque from all but those that pay the playing fee. Yet there’s so much entertainment and interest and real human nature playing out in these worlds. People will watch. Agents will sift through the user data feeds streaming out along with the A/V feeds. And think of the up-sell as viewers convert to players. Someday I’m certain some of the reality tv channels on my mobile device will be looking into such immersive worlds and showing me where in my friends are and who they’re battling and what type of car they just jacked for a joyride through Liberty City. (I *really* want the GTA CCTV’s!)

MetaTagging on Adam’s Block!

This is kinda blowing my mind. Adam’s Block has gotten a ton of local and national (international?) press in the last week. Adam Jackson’s pair of webcams capturing the daily & nightly life of the San Francisco Tenderloin is becoming the latest booming media channel.

I just tuned in and now, in the middle of the intersection, framed in optimal view of the camera, is a white spray paint tag: WKT. A tagger has meta-tagged Adam’s Block. Whoever did it was painting the ground but is advertising to the web. There are far more people watching that intersection than the locals. With viewers come advertisers of all stripes. Everyone wants to be seen and heard. This totally up’s the ante of post-reality tv and redefines the very notion of space.

This is just the start. Expect more street art, performance theater, protests, and outright corporate advertising to invade the short stretch of Taylor St. The value of this dirty chunk of SF has just gone up exponentially. Adam’s Block is a hole in the wall between The Cloud and The Street, helping forge that new land where the two are indistinguishable.

There’s something really significant happening here…

G-Speak MIT Gestural Interface –> Minority Report