On Human Networks & Living Biosystems

Increasingly, we live in a world defined by flat networks. Folks like Clay Shirky, Ben Hammersley, and others have observed in great detail how the design patterns of the internet are challenging and changing the landscape of human civilization. So many of our institutions have been built as hierarchical pyramids designed to exert the maximum degree of control over their domains. These top-down management structures have come to define business, government, the military, medicine, education, the family, and knowledge itself. Leaders rise to the top as centralized governors dictating down the chain how things should be, while workers march in step towards execution of their appointed tasks. Such structures were modeled after the clockworks & steam engines of classical mechanics, designed to be precise, rigid, and durable, capable of lasting hundreds of years. These structures informed the defining metaphors of our entire industrialized society.

Computer architecture recapitulated the mechanical metaphor by designating a central processor that assigned & managed tasks bussed out to sub-processors and specialized functional components. In this way the computer became more of a powerful extension of the industrial age rather than a stake in the ground of a new paradigm. While the mechanical metaphor gradually evolved into the computational metaphor which has defined the last two decades, it wasn’t until computers began to follow the model of telecom and began connecting with each other across flat networks that the seed of a biological metaphor began to take hold.

Nature, it seems, does not create very many rigid, top-down control systems. Those are too stiff and inflexible for the dynamics of life. Rather, nature evolves vast horizontal networks that assemble into specialized functions within their environment. For example, the messiest, most distributed organizational structure known – the human brain – does not have a top-tier manager or CPU. There is no executive function within the brain or its mind, though we typically like to think there is. Instead, the brain is a vast & mostly flat hierarchy that is bundled into loosely vertical functional bodies. These functional bodies are themselves existing across a mostly flat horizontal network of interactions. The thalamus receives all inputs and routes them up to higher cortical processing and lower hindbrain autonomic structures, into the amygdala for emotional content and across the hippocampus for memory, then down throughout the body. The processing chain is massively parallel, interconnected, and marked by complex feedback pathways. Mind arises off of these processes in a very ad hoc manner, always shifting, always flexible, and always derived from a mass summation across the network.

Mycelial networks offer another example. When we see mushrooms scattered across a forest floor we’re not seeing individuals. Each mushroom growing from the soil is a fruiting body rising from the underground web-work of mycelia – the skeletal framework of the colony. Some mycelial colonies have been found to have areas extending over 2000 acres making them some of the largest superorganisms on the planet. The pattern suggests mushrooms as terminal nodes and mycelia as the network backbone.

In ecosystems, large predators constitute a form of top-down management but they themselves are part of the predator-prey relationship – a dynamic that must always seek relative equilibrium with the broader network in which it is embedded. Predators do not have a choice to over-consume prey or stockpile & re-sell it to others. Large ocean gyres also suggest a high degree of top-down control by seasonally establishing the engines of hemispheric weather. The North Pacific gyre becomes more active in the Winter of the northern hemisphere, driving the scale & frequency of storms hitting the pacific northwest of the United States. But the North Pacific gyre is an emergent structure that is itself built upon the properties of a nearly-infinite set of factors. It is not a regulatory structure or a governor by intent or design and there is no top-level group of components that determine its next move. It is a super-system derived from innumerable sub-systems.

Most importantly, all biological systems are guided not by top-down governors or control mechanisms but by feedback from the networks in which they are embedded. This is how nature regulates, preserves, and evolves itself towards greater adaptability. There is no fallible ruler driven to resource over-reach and myopic certainty. There is only the ongoing trial & error of embedded growth tempered by continuous communication between & within organisms.

As computers began to connect across the ARPANET, and with the dawning of the visual internet, the CPU evolved away from being specifically a central control system to become a node within a distributed network. This initial shift quickly challenged the established domains of publishing, content creation, intellectual property, and knowledge management while inviting the crowd into a shared virtual space of increasingly global identity & transaction. The advent of social networks established an organizational structure for connecting the human capital of virtuality, making it easier for like-minded people to connect & share & collaborate non-locally, subtly undermining the very notions of borders, statehood, family, and allegiance. Soon after, the mobile revolution has tipped everything on its side and bundled it into a portable device bringing instantaneous global communication & information access to most people on the planet.

The framework was laid for new forms of emergent, non-hierarchical, distributed collaboration & innovation, to both productive & destructive ends. Groups could now form and coordinate around affiliations, interest, and goals in ways that directly challenged the institutional structures monetizing our production & consumption and regulating our behaviors. It has become vastly easier for small organizations to take on multinational interests, whether in business & innovation or in power & politics. The conflicts we see across the world today are, in large part, a symptom of the younger generations leveraging flat network technologies to rise up against the older generations who long ago settled into their legacy hierarchical power structures. To paraphrase Ben Hammersley, the people who are running the world, who are entrusted with our future, are not able to understand the present. They lack the cognitive tools that are a basic part of the Generation C toolkit – the digital natives who grew up with a mobile in their hands and the internet at their fingertips, embedded in specialized networks that span borders and extend identity into the virtual.

The global disruptions that seem to characterize modernity constitute a civilizational correction driven by natural law. The DotCom bubble went through a correction, shedding excess value and pruning the garden of exuberant innovation to favor only the most fit. It was a good thing, if not painful. We witnessed the correction in the housing bubble and will likely see similar corrections in credit & commodities, as well as a painfully positive correction in energy, subsidized and under-valued for so long. The impacts of climate change are a correction imposed upon the legacy model of industrialization & growth by nature itself – the super-system in which all human endeavor is embedded and to which we are ultimately accountable.

The civilization correction is an emergent regulatory mechanism embedded within natural systems forcing our legacy human systems to progressively modify the unsustainable design patterns of our past. The mechanical metaphor & the computational metaphor are necessarily opening to include the biological metaphor. We can see this in every aspect of technology and it is equally emergent across human behavior & social systems. Nanosystems emulate biosystems. Computation & robotics are integrating with neurology & physiology. Individuals are finding agency & empowerment in leaderless multi-cellular collaborations. The built environment is becoming sensory-aware, communicating with itself through discrete feedback mechanisms. It can be argued that the emergence of the internet and of ubiquitous mobile communication & computation is an expression of our natural instincts to move into closer alignment with our environment; to follow the adaptive design patterns of nature in order to find a more sustainable & equitable posture for our species; a thermodynamic need to seek maximum efficiency in energy expenses. And to express a direct intervention programmed by nature itself to nudge the Anthropocene back towards equilibrium.

Such lofty ponderings aside, our world is undoubtedly approaching an inflection point. Everything appears to be upending and it’s all spread out in glorious detail for everyone to see. The feedback loop between humanity and it’s creations – the biological & cybernetic communication among individuals & groups & cultures & organisms & ecosystems – is tightening and getting more & more dense every day, feeding on itself and forcing exceptional degrees of novelty into becoming. It’s frightening & awesome and the Old Guard can barely see it happening right in front of their eyes. The shift may be apocalyptic, a sudden phase change, or an accelerated-but-managed transition… Probably it will be all of these things in differing degrees & locales. However it happens, the emerging paradigm is much more about networks, messaging, feedback, and biology rather than hierarchy, control, power, and mechanization. Nature is the super-system, the ultimate controller enforcing the laws of physics and prescribing the design templates for fitness & adaptation. If we are, as Kevin Kelley suggests, the sex organs of technology, then our technology is born from the natural imperatives coded deeply into our DNA.

[Justin Pickard notes: Biology PhD friend had issues w/ @chris23's latest (http://bit.ly/e0tJSS), citing hierarchies in social insect colonies, meercats & wolves... Furthermore, some biologists now consider social insect colonies to be superorganisms in their own right; akin to @cascio's ecology of mind?

Me: Yes! I considered diving into ants - lot's of research there. Interesting social structures emerge in higher critters/hives... I'd love to read a rebuttal/extension.]

My IFTF Tech Horizons Perspective on Neuroprogramming

IFTF has published the 2010 research for their Technology Horizons program – When Everything is Programmable: Life in a Computational Age. This arc explored how the computational metaphor is permeating almost every aspect of our lives. I contributed the perspective on Neuroprogramming [PDF], looking at the ways technology & computation is directly interfacing with our brains & minds.

From the overview for the Neuroprogramming perspective:

Advances in neuroscience, genetic engineering, imaging, and nanotechnology are converging with ubiquitous computing to give us the ability to exert greater and greater control over the functioning of our brain, leading us toward a future in which we can program our minds. these technologies are increasing our ability to modify behavior, treat disorders, interface with machines, integrate intelligent neuroprosthetics, design more capable artificial intelligence, and illuminate the mysteries of consciousness. With new technologies for modulating and controlling the mind, this feedback loop in our co-evolution with technology is getting tighter and faster, rapidly changing who and what we are.

I also contributed to the Combinatorial Manufacturing perspective with Jake Dunagan. This perspective explores advances in nano-assembly & programmable matter. From the overview:

humans have always been makers, but the way humans manufacture is undergoing a radical transformation. tools for computational programming are converging with material science and synthetic biology to give us the ability to actually program matter—that is, to design matter that can change its physical properties based on user input or autonomous sensing. nanotechnology is allowing us to manipulate the atomic world with greater precision toward the construction of molecular assemblers. Researchers are designing “claytronics”: intelligent robots that will self-assemble, reconfigure, and respond to programmatic commands. And synthetic biologists are creating artificial organic machines to perform functions not seen in nature.

Excerpts From WEF Global Risks 2011 Report

In the lead up to it’s big annual event in Davos, the World Economic Forum’s Risk Response Network has published its Global Risks 2011 report. Here are some of the top-level highlights, taken verbatim from the report. I encourage people to read the entire report as each section is broken out into considerable detail including multiple scenarios. There’s also an overview at Business 21C.

“The world is in no position to face major new shocks.”

2 Cross-Cutting Risks:
1. Economic disparity: Wealth and income disparities, both within countries and between countries, threaten social and political stability as well as economic development.
2. Global governance failures: Weak or inadequate global institutions, agreements or networks, combined with competing national and political interests, impede attempts to cooperate on addressing global risks.

3 Important Risks in Focus:
1. The macroeconomic imbalances nexus: This cluster of three economic risks – global imbalances and currency volatility, fiscal crises and asset price collapse – is characterized by both internal imbalances (within countries) and external imbalances (between countries).
2. The illegal economy nexus: Illicit trade, organized crime and corruption are chronic risks that are perceived as highly likely to occur and of medium impact. As a highly interconnected nexus representing the illegal economy, however, experts see these risks as of central importance to the global risk landscape.
3. The water-food-energy nexus: Water security, food security and energy security are chronic impediments to economic growth and social stability. Food production requires water and energy; water extraction and distribution requires energy; and energy production requires water. Food prices are also highly sensitive to the cost of energy inputs through fertilizers, irrigation, transport and processing.

5 risks to watch:
1. Cyber-security: cyber theft, cyber espionage, cyber war, and cyber terrorism.
2. Demographic challenges: population “cluster bombs”, global graying and demographic dividends.
3. Resource security: extreme commodity price volatility and extreme energy price volatility.
4. Retrenchment from globalization: In many advanced economies strengthening political forces either directly or indirectly advocate retrenchment from globalization.
5. Weapons of mass destruction: the key WMD risk is felt by most experts to be that of nuclear proliferation, both among states and non-state actors, closely followed by the potential use of biological weapons.

3 ways for leaders to improve their response to complex and interdependent risks:
1. Proactively address the causes, rather than the symptoms, of global risk, identifying effective points of intervention in underlying structures and systems.
2. Devise coordinated response strategies to address the existence of difficult trade-offs and the threat of unintended consequences caused in part by increased interconnectedness.
3. Take a longer-term approach to assessment and response, particularly when seeking to manage global risks that emerge over decades rather than months or years.

2011 – Looking Forwards, Looking Backwards

On this the first day of the new year the state of my thoughts feels like a reflection of the world at large. Tumultuous, hopeful-yet-fearful, seeing innumerable strands & details yet struggling to hang on to them, much less to weave them into some coherent story about the future. Change has become so constant and accelerated that the event horizon seems nearer and nearer: the crystal ball has clouded and tomorrow could be radically different from today.

This is what the Singularity folks are tugging at, and what the Mayans supposedly alluded to almost 1500 years ago. That the feedback loops between culture, technology, and our very selves would become so tight and quickened that we’d begin to lose our ability to keep up with it all. The system would accelerate to such an incomprehensible pace that it would all seem to be slipping from our grasp. Then, like a boiling point, everything would undergo a state change, a phase shift into something born of the past but wholly new.

A singularity in physics is a point of absolute density most notably observed within a black hole. It’s so dense that it’s gravitational pull becomes massive, inescapable. Around the singularity is a spherical area physicists refer to as the event horizon. Within this area even light cannot escape, so nothing can be seen within the event horizon from the outside. Cross the line and you are doomed to collapse into singularity.

In contemporary usage, the Singularity alludes to, among other things, the notion that there is some point in our near-future where our lives have shifted so dramatically that the world, our technologies, and perhaps even the nature of our selves are unrecognizable and unforseeable from our current timeframe. Like the event horizon around a black hole, it’s impossible to see what’s on the other side; impossible to predict the next phase state. There are examples throughout natural systems that illustrate such phase shifts, from the formation of ice crystals in water to the emergence of a hurricane or tornado from an otherwise incoherent weather system. But it’s a much more challenging task to imagine such a sudden shift at the scale of global human systems. We have models for complexity & emergence but we have no real ability to model what the next global economic system might be or how human consciousness might shift when billions of minds are wired together in instantaneous communication.

Singularity or not, the world has become arguably post-historical. For millenia history was a book written by a few to record what was seen and known within their domain. Often bounded by geography, language, and political persuasion, the writing of the historical record has been a fairly narrow niche, curated by a relative few. As such, history has retained a reasonable amount of coherency. The post-modern world is post-historical because now virtually everyone is contributing to the historical record. There is no central curation or organizing principle. Every blog, Flickr account, Tweet, Facebook status, HuffPo & RedState post – all of these are writing the historical record. In a world where everyone can publish, who’s account is the most accurate? Is there even such a thing as historical accuracy anymore when so many witnesses chime in with their take? Even Wikipedia is a dynamic, ever-shifting crowdsourced story, constantly amended and updated around the ongoing dialectic of our unfolding. Without a single history we become post-historical, atemporal, and, arguably, much further from the shores of truth.

Indeed, our modern age seems to have little use for truth, tending much more towards the subtleties of persuasion and the blunt cudgel of opinion. In a choppy sea of philosophical relativity it’s more about how you can make your story the most influential among competitors. Pundits and politicians, marketers & evangelists all know this well. We’re all learning this in the internet age, each of us curating our personal brand in the social web. But this din of persuasion and memetic mudslinging has gotten so amplified that it’s drowning out sensibility and rational discussion, much less actual strategic coordination around the very real and threatening issues of our time. Having grokked the power of global broadcast everyone is yelling through their own personal megaphone. And the din is deafening (and oh so entertaining!). Not only do we struggle in a sea of data overload but we’re constantly tossed about by the onrushing waves of opinion. So much data, so much opinion, and so little ability to rationally parse and comprehend it all.

This too may be symptomatic of our transit towards some higher order. Sometimes the best way to remodel a house is to tear it down and rebuild from the ground up. Surely the din makes it considerably more difficult to forecast the future. Is North Korea really going to start a nuclear war in Asia, or is it just rhetorical sabre rattling? Nothing new for seasoned diplomats but now it’s not just a matter for war-room scenarios and diplomatic cables. Now it’s fodder for the 24hr newsfeed, the blogosphere, and the tweetstream. We all share the uncertainty, the fear, the speculation… the net shares as much emotion as it does data. And we’re all open to being manipulated by those who would use such information to persuade us towards some unseen ends. As everything seems to become increasingly disordered so too does our own mental ability to organize the internal maps we keep of the world. As the world comes undone, so too do our minds. Herein lies both the opportunity for emergence of new orders, as well as the very real potential to be subsumed by the chaos and pass into void.

In a post-historical age of persuasion, forecasting the future becomes less predictive and more strategic. It’s less about how we expect the world to unfold and more about how we construct the world to become. Forecasting becomes an activist pursuit. Within increasing chaos there is increasing opportunity to apply new forms of order. The plenum is infinite potential. The slate of civilization appears to be wiping itself of many of the legacy systems we’ve been running on – industrial, economic, religious – now crashing into the finite resource limits of planet and population and fed crazy pills by the sudden massive outlier of the internet.

Transition is always marked by change & uncertainty, passing out of one phase into another as of yet unknown. Expectations reveal the psyche of dreamers more than the machinations of reality. Yet the two may converge if we will it to be. And of course, these thoughts reflect my own biases – western, American, Anglo, middle class, male. My world appears to be fading, threatened while Asia and the African continent perhaps see the shifting sands with greater hope. Asia, the return of the Dragon. Africa, the sandbox for the next global operating system. Not to belittle the very real and endemic suffering that wracks both continents but their indicators are heading upwards while ours appear to be on the decline. Of course the west is seized by apoplectic spasms. Its hegemony and exceptionalism is fading like the tarnished capital rotundas of the Byzantium. State competency is in decline. Deficits run rampant. Internal bickering, moneyed elections, and a bought, scandal-hungry media destroy any hope of democratic collaboration. The corporation is on the rise eager to displace democracy. The gap between Wall Street and Main Street is widening into a nihilistic chasm big enough to swallow the entire American Dream. Super-empowered individuals have more sway over our futures than do rulers. Insurgencies, criminal syndicates, and terror networks all have risen with the empowering tide of technology. The international order is increasingly looking like a great, heaving disorder.

Yet each of these admittedly-fearsome trends is also an enabler for new solutions unencumbered by the Old Ways. In the decline of the state grows the rise of the city-state, localism, and community building. In the economic gap is born innovation and self-empowerment, local resiliency & durable investments. Many super-empowered elites are not actually evil and are spreading their billions across the world for good causes. And every insurgency is in some ways just an angry local makers collective. The design patterns are often agnostic across good & bad. It is intention & compassion, hope & courage, that make the difference.

Here in the impossible post-historical year of 2011 we’re not quite as far along as we thought we’d be but, damn, look at all we’ve accomplished! Look at the crazy majesty of it all unfolding from that beautiful lotus singularity! Look at the dreams and imaginations given life! Look at the subtleties of humanity revealed and shared across the globe! Look at the deep currents of evolution, physics, and biology, seemingly lost and abandoned in the moment but so obviously masterful and commanding in the long-term. We are, each and every one of us, in a dance with nature, spinning away at times but always compelled to return and dance in step.

As the Great Wheel turns the future remains as it always has: a dream waiting to awaken.

5 Dark Scenarios of Transhumanity

[Cross-posted from Humanity + Magazine.]

Emergent technologies often inspire great excitement attended by utopic visions of how they will transform our lives for the better. Yet all innovations introduce risk and the likelihood of unforeseen consequences. The transhumanity stack of technologies – life extension, medical & genetic modification, brain-computer & brain-machine interface, and virtual & augmented realities – offer great opportunities for human enhancement but pose profound risks for all aspects of humanity & civilization. It is critical to confront these dangers and temper the enthusiasm of tranhumanism with diligent risk assessment and thorough scenario modeling for possible outcomes.

To wit, here are 5 scenarios that explore the possible dangers embedded within transhumanism. This is, of course, by no means an exhaustive list but is simply intended to encourage further risk analysis. Most or all have probably been addressed by others elsewhere, and this list is not intended as a criticism of those presently active in the transhumanity community.

1. Population growth from longevity & senescence studies
Life extension looks great from an individual or group perspective but it’s a resource nightmare from a national and global angle. Current human population is about 6.8 billion with most linear estimates projecting somewhere around 9 billion by 2050. If life extension is designed to be readily available to anyone & everyone, we can expect two outcomes: considerable population growth as longevity outpaces mortality, and a rise in global GDP and its commensurate resource consumption as working age extends towards the centenarian. People living longer means people will consume more in the course of their lifetimes. Consider the competition for resources & ecological carrying capacity we currently face in 2010 and roll that forward 40 years with a massive global population and members of the workforce that can potentially stay employed for 70 years….

2. Inequity of technology distribution — the Transhuman Gap
The flip-side of the resource consumption issue arises if we admit that transhuman technologies will not be evenly available to all; that socio-economic factors will gate who has access to technologies that extend human capabilities. In this context, population dynamics will not be appreciably influenced by human life extension as only a small subset of the populace will have access to such enhancement. Indeed, genetic modification, brain-computer interface, advanced prosthesis, and access to virtual & augmented realities are all presently gated by economic barriers to entry that are not likely to diminish any time soon. AR & VW’s may become ubiquitous & cheap but real human enhancement through interventionary technologies will mostly fall along class lines, giving rise to a wealthy tier of augmented & enhanced individuals. If only the wealthy are most able to afford enhancement, the socioeconomic divide will be reinforced by the Transhuman Gap, further disenfranchising those already at a competitive disadvantage by their class circumstances. From such economic disparity, reinforced by the inevitable moralizing and judgments from both sides of the gap, social cohesion will be further challenged and class distinctions will begin to take on a biomechanical & genetic aspect with the threat of technology-enabled superiority.

3. Techno-elitism, civil discord, and eugenics
Throughout history elite classes have used their status & abilities to influence the control systems that govern those beneath them. Likewise, the underclass has looked at elites with both admiration & disdain, occasionally rising to join their ranks but, more often, rising up to knock them down. Civil strife is a common outcome of disparity, driven by inequities in access to resources, opportunities, and power. A class of techno-elite transhumans would pose a profound existential threat to the underclass who might very well perceive themselves as being forever cut-out from the Democratic ideal that “all humans are created equal”, no longer able to compete in any capacity without transhuman enhancements. The anger and victimization from such an outlook would very quickly translate into moralizing against the crimes of human augmentation and stigmatizing those who pursue such “un-natural” and “un-holy” enhancement. In turn, the techno-elite may feel inclined to judge the underclass as “unfit” or “un-evolved” – two distinctions that have historically led to great atrocities.

4. Co-option of transhumanity by fascists, oligarchs, and super-empowered individuals
The slippery slope of this scenario posits the rise of a transhuman ruling class who, when challenged by the underclass, recede into their own sense of authority & enhanced intelligence to determine that the only appropriate course of action is to subjugate the masses and shepherd the rise of transhuman governance. If transhuman enhancement is truly advantageous, yet remains available only to an elite class, then in all likelihood those elites will embrace the technology to their competitive advantage. Since it would be folly to assume that human technological enhancement will remediate our most basest evolutionary program of survival of the fittest, the likelihood of enhanced predatory elites seizing global power is not so small. The darkest scenario might see transhuman governance requiring control & tracking implants in all newborns – perhaps a bit hyperbolic but not inconceivable if the type of global predators that currently traverse societies gained access to advanced transhuman technologies.

5. Fractured reality
Virtual worlds and augmented reality offer many compelling experiences across the spectrum of entertainment, socialization, marketing & advertising, collaboration, and modern knowledge work. At their core, these technologies intermediate our experience of the world, giving third parties access to program our sensorium. Brain-computer interface technologies are working to extend this access to the core structures of our brain, kicking off a wave of neurotechnologies able to more specifically & accurately influencing the mind-brain interface. The opt-in path through designer reality gives us the ability to modify the way we interface with the phenomenal world, electing to commit more of our selves to virtual experiences & relationships, or to overlay our environments with the images of our choosing rather than confront the physical world solely on its terms. While affinity groups will accrete around specific worlds & layers the barriers between differing experiences of objective reality will multiply when the world I experience is markedly different than yours. As the Transhuman Gap threatens social cohesion through class, reality design threatens cohesion across all classes by erecting virtual constructions between adjacent-but-unrelated digital worlds. While we may feel a sense of agency in creating such personalized experiences we do so in digital layers most likely owned by 3rd parties or accessible through public APIs. We may inadvertently wall ourselves off from each other but we’ll become even richer targets for profilers, influencers, and governors. The slippery slope in this scenario suggests that governance might enforce realities onto subjects or that dangerous identity groups might create monstrous, all-encompassing layers as indoctrination tools & neuro-propaganda towards the engineering of social movements. Considering how supremely the television has been used to influence the masses with only basic access to eyes and ears, it’s not unlikely that greater access into the transhuman will yield a greater ability to influence and manipulate.

Again, these scenarios are not meant as accusations or designed to arouse a fear of transhumanism but, rather, to encourage critical thinking along the dystopic possibilities of the future transhuman phase space, as it were, in order to better control for such outcomes. As the saying goes, all technology is inherently neutral. But this glib statement does not acknowledge that all technology is born of humanity and wielded by our hands alone. To paraphrase a great modern philosopher, all of the animals are capably murderous.

Platforms for Growth and Points of Control for Augmented Reality

Tish Shute over at UgoTrade was kind enough to post a conversation we recently had about augmented reality, the Gartner Hype Cycle, and the O’Reilly Web 2.0 Points of Control map.

Chris Arkenberg: There will be much more of a blended reality experience in the living room for sure, and with interactive billboards. Digital mirrors are another area. So I mean if we kind of extend AR to include just blended reality in general, you know, this is moving into our culture through a number of different points. As you mentioned, it will be in the living room, it will be in our department stores where you can preview different outfits in their mirror. We’re already seeing these giant interactive digital billboards in Times Square and other areas.

It’s funny. I mean for me, the sort of blended reality aside, the augmented reality, to me, is actually a very simple proposition in some respects. When I look at this map, augmented reality is just an interface layer to this map in my mind, just as it’s an interface layer to the cloud and it’s an interface layer to the instrumented world. It’s a way to get information out of our devices and onto the world.

Humanity+ Conference in Los Angeles, Dec. 4 & 5 at Caltech

If you like to live on the edge and gaze deeply into the future, a great collection of like-minded folks will be gathering to discuss the possible futures of humanity – and how to make them. From their site:

The Humanity+ at Caltech program will be divided into four main themed sessions. These sessions are:

* Re-Imagining Humans: Mind, Media and Methods
* Radically Increasing the Human Healthspan
* Redefining Intelligence: Artificial Intelligence, Intelligence Enhancement and Substrate-Independent Minds
* Business and Economy in the Era of Radical Technomorphosis

A wide range of interesting, professional speakers, from both the for-profit and non-profit worlds, will address each of the four themes.

Click through for the full program.

IFTF Publishes 2010 Map of the Decade

The Institute for the Future has published it’s 2010 Map of the Decade from the Ten Year Forecast program. I contributed research, analysis, and forecasts for carbon markets, global energy resource disposition, and new models of adaptive power.

We presented the program to many Fortune 500, NGO, and government subscribers this past April in an amazing 2-day conference at the magical Cavallo Point at the northern foot of San Francisco’s Golden Gate Bridge. Thanks again to IFTF for the chance to work with such great folks digging into the major currents & challenges of our times!

From their site:

The future is a high-resolution game. Never before has humanity been
able to explore the emerging landscape in such detail, to measure the
forces of change at such vast scales, and to fill in the details with
such fine grain. But this high-resolution grid is not complete. It
challenges us to envision and build the future we want. As both gamers
and creators of the game, we will fill in the grid over the coming
decade.

Back-Casting From 2043

[This article has become the most popular item I've ever posted on this blog. Thanks so much to everyone who has read it and passed it along!]

When it’s busy like this the viz sometimes shifts like the color bleed you used to see on those old Sunday comics, way back in the day. Ubiquitous fiber pipes & wide-band wireless still can’t give enough bandwidth to the teeming multitudes downtown. The viz starts to lag, gets offset and even orphaned from the hard world it’s trying to be a part of. Hyperclear Ray Ban augments, lenses ground down by hand-sequenced rock algaes to such an impossibly smooth uniformity, run through with transparent circuity & bloodied rare-earth elements, scanning the world in multiple dimensions, pinging the cloud at 10GHz and pushing articulated data forms through massive OLED clusters just to show me where I can find an open null shield and the best possible cup of coffee this side of Ethiopia. Then the pipes clog and those ridiculously expensive glasses turn into cheap 3D specs from 2010 pretending to make 2D look like real life but instead here they’re doing the print offset thing, flattening my world into color shifts and mismatched registers.

Marks are flickering in & out, overlapping & losing their z-order. A public note on a park bench glows green – something about the local chemwash schedule – then loses integrity to one of my own annotations left there, like, a year ago. A poem I cranked out on a late night bender but it’s unreadable with all the other layers clashing. Even the filters get confused when the pipes clog. If you look around fast enough, marks start to trail & stutter in a wash of data echoes like when screens used to have refresh errors. Only now our eyes are the screens and the whole world gets caught in recursive copy loops.

The Ray Bans correct it pretty quickly, attenuating the rendered view and pushing up the hard view as the dominant layer. But for a moment it feels like you’re tripping. It used to be physically nauseating, a sudden vertigo brought on by that weird disconnect of self & place. Like so much of life these days, you spend a lot of time adapting to disconnects between layers. Between real and rendered. Between self & other, human & machine. Between expectations & outcomes.

The arc of glorious progress that opened the 21st century seemed to have found it’s apogee around 2006 or so and then came hurtling back towards Earth. And it wasn’t like earlier “corrections”. This one was big. It was a fundamental stock-taking of the entirety of the industrial age to date and things were suddenly, shockingly, terribly mis-matched from the realities of the world. Planetary-scale disconnects. The carrying capacity of economies, nations, ecosystems, and humanity itself came into clear & violent resolution by the 2020’s when everything started to radically shift under the twin engines of hyper-connectivity and ecological chaos. These two previously unexpected titans directly challenged and usurped the entire paradigm of the developed and developing worlds, setting us all into choppy and uncertain seas.

Sure, we still get to play with the crazy cool tech. Or at least some of us do. What the early cyberpunks showed us, and what the real systems geeks always knew, is that the world is not uniform or binary. It’s not utopia vs. dystopia, win vs. lose, us vs. them, iGlasses or collapse. It’s a complex, dynamic blend of an unfathomable number of inputs, governors, and feedback loops constantly, endlessly iterating across inconceivable scales to weave this crazy web of life. So we have climate refugees from Kansas getting tips from re-settled Ukrainians about resilience farming. We have insurgencies in North America and social collectives across South America. The biggest brands in the world are coming out of Seoul & Johannesburg while virtually-anonymous distributed collaboratives provide skills & services across the globe. And we have Macroviz design teams from Jakarta & Kerala directing fab teams in Bangkok to make Ray Bans to sell to anybody with enough will & credit to purchase. Globalization & it’s discontents has proven to offer a surprising amount of resilience. Heading into the Great Shift it looked like the developed world was headed for 3rd world-style poverty & collapse. But it hasn’t been quite that bad. More of a radical leveling of the entire global macro-economic playing field with the majority settling somewhere on the upper end of lower class. Some rose, many fell. It was… disturbing, to say the least. It simply didn’t fit the models. Everyone expected collapse or transcendence.

We humans want things to be as simple as possible. It’s just natural. Makes it easier to service the needs of biosurvival. But we’ve not created a simple world. Indeed, the world of our making looks about as orderly as the mess of 100 billion brain cells knotted up in our heads or the fragmented holographic complexes of memories & emotions, aspiration & fears, that clog it all up. We built living systems as complex as anything the planet could dish out. Not in the billions of years nature uses to refine and optimize but in a matter of a few millennia. We raced out of the gate, got on top of the resource game, took a look around, and realized the whole thing needed to be torn down and completely redesigned for the realities of the world. The outcomes no longer fit the expectations. In some strange fractal paradox, the maps got so accurate that the territory suddenly looked very different from what we thought.

The null shield was created as a black spot. A cone of silence for the information age. They’re like little international zones offering e-sylum in select coffee shops, parlors, dining establishments, and the finer brick-and-mortar lifestyle shops. And in conflict zones, narco-corridors, favelas, gang tenements, and the many other long-tail alleyways of the ad hoc shadow state. The null shield is a fully encrypted, anonymized, opt-in hotspot that deflects everything and anything the global service/intel/pr industry tries to throw at you or copy from you. What’s better is you don’t even show up as a black spot like the early implementations that would hide you but basically tell the world where you were hidden. You’re invisible and only connected to the exact channels you want.

These were originally created for civ lib types and the militarized criminal underclass as a counter-measure to the encroaching security state. But as traditional states universally weakened under the weight of bureaucracies and insurmountable budgets (and the growing power of cities and their Corp/NGO alignments), the state’s ability to surveil the citizenry declined. All the money they needed to keep paying IT staff, policy researchers, infrastructure operators, emergency responders, and the security apparatus – all that money was siphoned up by the cunning multinationals who used their financial wit & weight to undermine the states ability to regulate them. Now states – even relatively large ones like the U.S. government – are borrowing money from the multinationals just to stay afloat. The iron fist of surveillance & security has been mostly replaced by the annoying finger of marketing & advertising, always poking you in the eye wherever you go.

Keeping on top of the viz means keeping your filters up to date and fully functional. Bugs & viruses are still a problem, sure, but we’ve had near-50 years to develop a healthy immunity to most data infections. We still get the occasional viz jammer swapping all your english mark txt with kanji, and riders that sit in your stream just grabbing it all and bussing it to some server in Bucharest. But it’s the marketing vads and shell scanners that drive the new arms race of personal security. Used to be the FBI were the ones who would scan your browsing history to figure out if you’re an Islamic terrorist or right wing nut, then black-out the Burger Trough and grab you with a shock team right in the middle of your Friendly Meal. Even if they had the money to do it now, the Feds understand that the real threats are in the dark nets not the shopping malls. So the marketers have stepped in. They want your reading list so they can scan-and-spam you wherever you go, whenever, then sell the data to an ad agency. They want access to your viz to track your attention in real-time. They want to fold your every move into a demographic profile to help them pin-point their markets, anticipate trends, and catch you around every corner with ads for the Next Little Thing. And they use their access to rent cog cycles for whatever mechanical turk market research projects they have running in the background.

Google gave us the most complete map of the world. They gave us a repository of the greatest written works of our species. And a legacy of ubiquitous smart advertising that now approaches near-sentience in it’s human-like capacity to find you and push your buttons. In some ways the viz is just a cheap universal billboard. Who knew that all those billions of embedded chips covering the planet would be running subroutines pushing advertising and special interest blurbs to every corner of the globe? There are tales of foot travelers ranging deep into the ancient back-country forests of New Guinea, off-grid and viz-free, only to be confronted by flocks of parrots squawking out the latest tagline from some Bangalore soap opera. Seems the trees were instrumented with Google smart motes a few decades ago for a study in heavy metal bio-accumulation. Something about impedance shielding and sub-frequency fields affecting the parrots…

So while the people colonized the cloud so they could share themselves and embrace the world, the spammers, advert jocks, and marketing hacks pushed in just as quickly because wherever people are, wherever they gather and talk and measure themselves against each other & the world… in those places they can be watched and studied and readily persuaded to part with their hard-earned currency.

Or credits or karma points or whatever. Just like the rest of the big paradigms, value has shifted beyond anybody’s understanding. Gold and currency at least attempted to normalize value into some tangible form. But the markets got too big & complex and too deeply connected to the subtleties of human behavior and the cunning of human predators. While money, the thing, was a tangible piece of value, the marketplace of credit & derivatives undermined it’s solidity and abstracted value out into the cold frontiers of economics philosophers and automated high-frequency trading bots. So much of the money got sucked up into so few hands that the world was left to figure out just how the hell all those unemployed people were going to work again. Instead of signing up for indentured servitude on the big banking farms, folks got all DIY while value fled the cash & credit markets and transfigured into service exchanges, reputation currencies, local scrip, barter markets, shadow economies, and a seemingly endless cornucopia of adaptive strategies for trading your work & talent for goods & services.

Sure, there’s still stock markets, central banks, and big box corps but they operate in a world kind of like celebrities did in the 20th century, though more infamous than famous. They exist as the loa in a web of voodoo economics: you petition them for the trickle-down. Or just ignore them. They’re a special class that mostly sticks among their kind, sustaining a B2B layer that drives the e-teams & design shops, fab plants & supply chains to keep churning out those Ray Ban iGlasses. Lucky for them, materials science has seen a big acceleration since the 2010’s with considerable gains in miniaturization and efficiency so it’s a lot easier to be a multinational when much of your work is dematerialized and the stuff that is hard goods is mostly vat-grown or micro-assembled by bacterial hybrids. Once the massive inflationary spike of the Big Correction passed, it actually got a lot cheaper to do business.

Good news for the rest of us, too, as we were all very sorely in need of a serious local manufacturing capacity with a sustainable footprint and DIY extensibility. Really, this was the thing that moved so many people off the legacy economy. Powerful desktop CAD coupled to lo-intensity, high-fidelity 3d printers opened up hard goods innovation to millions. The mad rush of inventors and their collaborations brought solar conversion efficiency up to 85% within 3 years, allowing the majority of the world to secure their energy needs with minimal overhead. Even now, garage biotech shops in Sao Paulo are developing hybrid chloroplasts that can be vat-grown and painted on just about anything. This will pretty much eliminate the materials costs of hard solar and make just about anything into a photosynthetic energy generator, slurping up atmospheric carbon and exhaling oxygen in the process. Sometimes things align and register just right…

So here we are in 2043 and, like all of our history, so many things have changed and so many things have stayed the same. But this time it’s the really big things that have changed, and while all change is difficult we’re arguably much stronger and much more independent for it all. Sure, not everybody can afford these sweet Ray Bans. And the federated state bodies that kept us mostly safe and mostly employed are no longer the reliable parents they once were. We live in a complex world of great wealth and great disparity, as always, but security & social welfare is slowly rising with the tide of human technological adaptation. Things are generally much cheaper, lighter, and designed to reside & decay within ecosystems. Product becomes waste becomes food becomes new life. Our machines are more like natural creatures, seeking equilibrium and optimization, hybridized by the ceaseless blurring of organic & inorganic, by the innate animal disposition towards biomimicry, and by the insistence of the natural world to dictate the rules of human evolution, as always. After all, we are animals, deep down inside, compelled to work it out and adapt.

Time’s up on the null shield. Coffee is down. And the viz is doing it’s thing now that the evening rush has thinned. Out into the moody streets of the city core, the same streets trod for a thousand years here, viz or no. The same motivations, the same dreams. It always comes back to how our feet fall on the ground, how the food reaches our mouth, and how we share our lives with those we care for.

On Augmented Realities

Image from robinmochi.

[Cross-posted from my post at Boing Boing.]

Augmented Reality is definitely trending up the Hype Cycle in a big way. The past year has seen explosive growth in this nascent field buoyed by the rise of gps-enabled, cloud-aware smart phones. The marketing hype has, of course, been even more resounding, like a wailing chorus of virtual vuvuzelas trumpeting the next great wave of advertising (I couldn’t resist). But beneath the hype and the fluff is a thriving community of innovators & designers working to weave this technology into the very fabric of our lives.

As a quick review, augmented reality is a context-aware UI layer rendered over a camera stream or other transparent interface. This is typically mediated by geo-location, orientation, physical markers (those funky UPC-like symbols), and visual recognition. In this manner AR is able to reveal visually the hidden data shadow of our world, like showing you the nearest coffee shops or details about the air quality in your city. The mobile device gets info about where you are and what direction you’re facing, goes to the cloud to look up data appropriate for the vicinity, then renders it over the camera stream in a way that updates as you move.

A whole industry has been born around this premise, dragging in images, annotations, and data to overlay on the camera stream of our mobiles. But the really interesting stuff is yet to come. As standardization issues, hardware issues, and numerous UI design challenges sort out in the next couple of years, concurrent with the development of AR-specific devices, our interaction with visualized data will become more and more specialized and appropriate to our individual needs. The clutter of markups that currently plagues many AR apps will be attenuated by algorithms that know our interests and affinities and block out the elements we wish to avoid. Just like Amazon makes recommendations based on your click & purchase history, AR apps will screen out the noise and provide us only with the data we need.

When paired with the massive deployment of embedded sensors AR becomes a lightweight visualization layer for interfacing with the instrumented world. Civic workers could see underground cables and pipelines. Homeowners could see real-time energy & network use. Police and early responders could post visual warnings cordoning streets and alerting to hazards. Ecologists could determine water & air quality at-a-glance. Ecosystems begin to have a voice, communicating soil contamination to observers. Public facilities like park benches, utility poles, and street signs could hold annotations & links created by community members, made public or gated by in-group permissions. Geographic social annotations could mark up our cities with tags and content. Virtual worlds might break out of the box and overlay on the physical plane. The environment suddenly becomes much richer – and potentially much nosier – with a flood of information. Augmented reality promises to exteriorize the cloud, drawing it out across the world canvas and making visible our social fabric. But it doesn’t promise to mediate or regulate that content.

We risk myopia, disconnection, visual occlusion, fragmented realities, reinforced tribalism. Consider the seemingly-inevitable future where eyewear mediates a cloud-aware augmented interface with the world. Perhaps you opt to obscure ethnicities or anyone not connected to the net. Ghettos look much nicer when painted over with high-res colors and dancing sprites. The world you experience is really only shared by the other people running your default layer set. Maybe you see paycheck information or health records or political affinities of those you pass, measuring up the once-private lives of your community. Perhaps the most popular layers are hacked to display swastikas or porn or spam swarms or simply to black out your view in the middle of the morning commute. How does the layered world enable crime, gang affinities, and political or religious extremism? What inevitable inequities might arise between those able to purchase such access and those condemned to the dark poverty of quiet disconnection? Do the wealthy become even more enhanced & capable compared to the underclass? And what are the risks of getting lost in the virtual glitz? Are there considerations for how these augmented realities will bring us closer to the natural world in which we’re embedded? And just what is “real” or “natural” anymore?

As connected social computing devices get smaller & smaller and nearer & nearer to us, the weight of the cloud gets lighter. We carry around immense computational power and almost immediate access to the global repository of information. The mobile phone will eventually pair with head’s-up eyewear displays just as more and more people avoid catastrophic disease & injury through the aid of embedded brain-computer interfaces. As computation moves next to and into our bodies, the cloud is breaking out of the screen and washing onto our world. We grow more augmented with computation while our environment is getting smarter and more aware and increasingly able to communicate with us. It may very well be that in 5, 10, 20 years the world is a much more visual, dynamic, and communicative place than we can even imagine.

[For more of my explorations of this subject check out my articles
Breaking Open the Cloud: Heads in an Augmented World and Cognition & Computation: Augmented Reality Meets Brain-Computer Interface.]

KTLS: Emerging Cityscapes

Over at KedgeForward I’ve contributed a piece exploring my sense of what cities might look like in the coming years based on current trends and emerging constraints. The question posed by Kedge founder, Frank Spencer, is:

“In what ways will the concept and landscape of the city change over the next decade, and will this change bring about positive or negative impact in terms of global resilience, transformational development, and human evolution?”

My answer begins:

“All human systems and technologies are ultimately embedded within the larger natural ecosystem of the planet. As we’re now beginning to witness across all such domains, nature is applying more and more pressure on civilization to force it into better alignment with the principles of conservation and homeostasis critical to balanced living systems. As massive aggregations of society, technology, commerce, industry, resource consumption, and waste production, cities will feel tremendous impact from the corrections imposed by the natural world. Megacities in the developing world like Lagos, Jakarta, Delhi, and Mexico City already exhibit enormous stress due to rapid urbanization, rising populations, and the energetic consumption and waste production that attends their growth. With aging populations and over-burdened consumer economies, first world cities like London, Los Angeles, and Tokyo will find it more & more difficult to support their resource demands. Indeed, given projections for energy prices, food stocks, and clean water & sanitation, cities across the world are trending towards a lower common standard of living.

Continued at KedgeForward…

Notes From the IFTF 2010 Ten Year Forecast

Last month I attended & participated in the Ten Year Forecast conference presented by the Institute For The Future. This event at Cavallo Point was the culmination of several months of research looking at the signals, trends, and possible futures of five global domains: the carbon economy, the water ecology, adaptive power, cities in transition, and molecular identity. I contributed research for the carbon economy & adaptive power, looking at carbon markets and the distribution of energy resources for the former and investigating insurgency, narcoterror, and the emerging shadow economy for the latter.

Over two days we presented very challenging content, both in scope & complexity, as well as tone. These are major foundational systems that intersect with every aspect of civilization. Most of the forecasts & scenarios were undercut with a tone of constraint and great challenge given the turbulent nature of these modern transitional times. In attendance were many high-level representatives from some of the largest corporate entities on the planet, as well as from NGO’s, government, and private research. The scenarios presented them with a near-future significantly constrained by resource shortages, rising costs of production, and the growing urgency of climate change. All of these constraints were very clearly articulated to highlight the need to reduce consumption, engineer positive behavioral change, and identify new measures of prosperity & wellness unhinged from growth & GDP.

I spoke directly with several VP’s, some responsible for guiding multi-billion dollar corporations, and all expressed a surprising awareness & understanding of the deeply challenging realities we face. I was met again & again with the sentiment that energy constraints will corral growth and compel companies to both modify their operations to reduce energy use and evolve their products and services to be more sustainable. Indeed, everyone acknowledged the impact of sustainability on their business, admitting that nature has now entered the boardroom. To be clear, some of these companies are the largest transporters on the planet – major keystone energy consumers. So when they start admitting that business-as-usual has to change, it’s hard not to feel the gravity of our times.

The first day was especially powerful. There was a distinct thickness to the large ballroom by the time Jane McGonigal was giving her after-dinner keynote on the Epic Win. We had thrown so much really overwhelming information at the attendees, all of which heralded significant changes that will likely impact all human systems in the next ten years. We painted pictures of a civilization that will either adapt quickly & effectively or spiral into a malaise of constraint, decline, & chaos. Yet the tone of the room and the comments & conversations that emerged were radically optimistic, embracing the dire news and ready to press on into the cold night for a better tomorrow.

Undeniably, we live in interesting times. Things seem increasingly out of control. Or at least, we now see so much of the world in such minute detail that our historic models of what order should look like are failing against the vast interconnected global systems laid bare before us. What we know for sure is that inevitable growth is a cancer and cannot be sustained. We know resources are finite and expensive and their industrial use is poisoning the planet. And we know that the planet itself is the ultimate Invisible Hand that will easily wipe us clean if we don’t acknowledge it’s centrality and honor the necessity of it’s health. Perhaps in more pragmatic terms these realizations are now reaching into the boardrooms and staff rooms of our global institutions. Economics, humanism, and ecology – the triple-bottom line – is making it’s way into the machines of commerce. And more and more people are looking for a meaningful future in their own triple-bottom-line of happiness, resilience, and legacy.

A Map for the Programmable World

IFTF recently published the map for When Everything is Programmable. I did the research & forecast for Neuroprogramming and contributed to Combinatorial Manufacturing. For Neuroprogramming, I focused on brain-computer interface technology in medical, military, and futuretainment. I was, frankly, amazed at just how much rapid development is happening in the field (and how much money is moving through it, as well). Perhaps surprisingly, Neuroprogramming looks much closer than the molecular construction I researched for Combinatorial Manufacturing. The promise of Drexler et al still seems to be a ways off but Claytronics offers a really compelling path towards programmable matter.

[I hope to write something soon about the IFTF Ten Year Forecast retreat this past weekend at Cavallo Point. It was fantastic!]

Breaking Open the Cloud: Heads in an Augmented World

This past Saturday I worked with Mike Liebhold, Gene Becker, Anselm Hook, and Damon Hernandez to present the West Coast Augmented Reality Development Camp at the Hacker Dojo in Mountain View, Ca. By all accounts it was a stunning success with a huge turn-out of companies, engineers, designers, makers, artists, geo-hackers, scientists, techies and thinkers. The planning was mostly done virtually via email and phone meetings with only a couple visits to the venue. On Saturday, the virtual planing phase collapsed into reality and bloomed on site into AR Dev Camp.

As an un-conference, the event itself was a study in grassroots, crowd-sourced, participatory organization with everyone proposing sessions which were then voted on and placed into the schedule. To me, it was a wonderfully organic and emergent process that almost magically gave life and spirit to the skeleton we had constructed. So before I launch into my thoughts I want to give a hearty “Thank You!” to everyone that joined us and helped make AR DevCamp such a great experience. I also want to give a big shout-out to Tish Shute, Ori Inbar, and Sophia for coordinating the AR DevCamp in New York City, as well as Dave Mee & Julian Tate who ran the Manchester, UK event. And, of course, we couldn’t have done it without the help of our sponsors, Layar, Metaio, Qualcomm, Google, IFTF, Lightning Laboratories, Web3D Consortium, IDEAbuilder, MakerLab, and Waze (and URBEINGRECORDED with Cage Free Consulting contributed the flood of afternoon cookies).

So first, just what is Augmented Reality? There’s a tremendous amount of buzz around the term, weighing it down with connotations and expectations. Often, those investing in it’s future invoke the haunting specter of Virtual Reality, doomed by it’s inability to live up to the hype: ahead of it’s time, lost mostly to the realm of military budgets and skunkworks. Yet, the AR buzz has driven a marketing rush throwing gobs of money at haphazard and questionable advertising implementations that quickly reach millions and cement in their minds a narrow association with flashy magazine covers and car ads. Not to diminish these efforts, but there’s a lot more – and a lot less – going on here.

In it’s most distilled form, augmented reality is an interface layer between the cloud and the material world. The term describes a set of methods to superimpose and blend rendered digital interface elements with a camera stream, most commonly in the form of annotations such as text, links, and other 2 & 3-dimensional objects that appear to float over the camera view of the live world. Very importantly, AR includes at it’s core the concept of location mediated through GPS coordinates, orientation, physical markers, point-clouds, and, increasingly, image recognition. This combination of location and superimposition of annotations over a live camera feed is the foundation of AR. As we’re seeing with smart phones, the device knows where you are, what direction you’re facing, what your looking at, who & what is near you, and what data annotations & links are available in the view. In this definition, the cloud is the platform, the AR browser is the interface, and annotation layers are content that blend with the world.

So the augmented reality experience is mediated through a camera view that identifies a location-based anchor or marker and reveals any annotations present in the annotation layer (think of a layer as a channel). Currently, each of these components is uniquely bound to the AR browser in which they were authored so you must use, for example, the Layar browser to experience Layar-authored annotation layers. While many AR browsers are grabbing common public data streams from sources like Flickr & Wikipedia, their display and function will vary from browser to browser as each renders this data uniquely. And just because you can see a Flicker annotation in one browser doesn’t mean you will see it in another. For now, content is mostly bound to the browser and authoring is mostly done by third-parties building canned info layers. There doesn’t seem to be much consideration for the durability and longevity of these core components, and there is a real risk that content experiences may become fractured and ephemeral.

Indeed, content wants to be an inclusive, social experience. One of the core propositions underlying our motivation for AR DevCamp is the idea that the platforms being built around augmented reality should be architected as openly as possible to encourage the greatest degree of interoperability and extensibility. In the nascent but massively-hyped AR domain, there’s a growing rush to plant flags and grab territory, as happens in all emergent opportunity spaces. The concern is that we might recapitulate the Browser Wars – not intentionally but by lack of concerted efforts to coordinate implementations. While I maintain that coordination & open standardization is of necessity, I question my own assumption that without it we’ll end up with a bunch of walled gardens. This may be under-estimating the impact of the web.

Through the lessons and resultant standardization of the Browser Wars, it’s become a best practice (and indeed, a necessity) to design specifically to the most common standards. Arguably, the move from Web 1.0 (essentially a collection of static billboards) to the social interactions that characterize Web 2.0 established and deeply reinforced the fundamental requirement that we’re all able to share information & experiences in the cloud. This social commons necessarily requires an architectural commonality. Thus, we all agree that HTML, JavaScript, PHP, JASON, MySQL, and now RDF, OWL, and SPARQL are the core components of our data service models. Since we understand that AR is primarily a location-aware interface layer for the cloud, it’s very likely that independent implementations will all speak the same language. However, the point of AR DevCamp and similar gatherings is to challenge this assumption and to reinforce commonality by bringing everyone together to press flesh & exchange notes. The social dynamic in the natural world will determine the level of cooperation in the virtual.

Yet, this cooperation and normalization is by no means a given. Just about every chunk of legacy code that the Information Age is built upon retains vestiges of the git-er-done, rush to market start-up midset. Short-sighted but well-meaing implementations based upon limited resources, embryonic design, and first-pass architectures bog down the most advance and expensive software suites. As these code bases swell to address the needs of a growing user base, the gap between core architecture and usability widens. Experience designers struggle against architectures that were not able to make such design considerations. Historically, code architecture has proceeded ahead of user experience design, though this is shifting to some degree in the era of Agile and hosted services. Nevertheless, the emerging platforms of AR have the opportunity – and, I’d argue, the requirement – to include user research, design, & usability as core components of implementation. The open, standardized web has fostered a continuous and known experience across it’s vast reaches. Artsy Flash sites aside, you always know how to navigate and interact with the content. The fundamentals of AR need to be identified and agreed upon before the mosaic of emerging code bases become too mature to adjust to the needs of a growing user base.

Given the highly social aspect of the web, place-based annotations and objects will suffer greatly if there’s not early coordination around a shared standard for anchors. This is where the Browser Wars may inadvertently re-emerge. The anchor is basically the address/location of an annotation layer. When you look through an augmented view It’s the bit of data that says “I’m here, check out my annotations”. Currently there is no shared standard for this object, nor for annotations & layers. You need the Layar browser in order to see annotation layers made in it’s platform. If you only have a Junaio browser, you won’t see it. If you annotate a forest, tagging each tree with a marker linked to it’s own data registry, and then the browser app you used to author goes out of business, all those pointers are gone. The historical analog would be coding your website for IE but anyone with Mosaic can’t see it. This is where early design and usability considerations are critical to ensure a reasonable commonality and longevity of content. Anchors, annotations, & layers are new territory that ought to be regarded as strongly as URL’s and markup. Continuing to regard these as independent platform IP will balkanize the user experience of continuity across content layers. There must be standards in authoring and viewing. Content and services are where the business models should innovate.

So if we’re moving towards an augmented world of anchors and annotations and layers, what considerations should be given to the data structure underlying these objects? An anchor will have an addressable location but should it contain information about who authored it and when? Should an annotation contain similar data, time-stamped and signed with an RDF structure underlying the annotation content? How will layers describe their contents, set permissions, and ensure security? And what of the physical location of the data? An anchor should be a distributed and redundant object, not bound to the durability and security of any single server. A secure and resilient backbone of real-world anchor points is critical as the scaffolding of this new domain.

Earthmine is a company I’ve been watching for a number of months since they presented at the IFTF. They joined us at AR DevCamp to present their platform. While many AR developers are using GPS & compass or markers to draw annotations over the real world, Earthmine is busy building a massive dataset that maps Lat/Long/Alt coordinates to hi-rez images of cities. They have a small fleet of vehicles equipped with stereoscopic camera arrays that drive around cities, capturing images of every inch they see. But they’re also grabbing precise geolocation coordinates that, when combined with the image sets, yields a dense point cloud of addressable pixels. When you look at one of these point clouds on a screen it looks like a finely-rendered pointillistic painting of a downtown. They massage this data set, mash the images and location, and stream it through their API as a navigable street view. You can then place objects in the view with very high accuracy – like a proposed bus stop you’d like to prototype, or a virtual billboard. Earthmine even indicated that making annotations in their 2d map layer could add a link to the augmented real-world view. So you can see a convergence and emerging correlation between location & annotation in the real world, in an augmented overlay, on a flat digital map, and on a Google Earth or Virtual World interface. This is an unprecedented coherency of virtual and real space.

The Earthmine demo is cool and the Flash API offers interesting ways to customize the street view with 2d & 3d annotations but the really killer thing is their dataset. As alluded to, they’re building an address space for the real world. So if you’re in San Francisco and you have an AR browser that uses the Earthmine API (rumors that Metaio are working on something here…) you can add an annotation to every STOP sign in The Mission so that a flashing text of “WAR” appears underneath. With the current GPS location strategy this would be impossible due to it’s relatively poor resolution (~3-5 meters at best). You could use markers but you’d need to stick one on every STOP sign. With Earthmine you can know almost exactly where in the real world you’re anchoring the annotation… and they can know whenever you click on one. Sound familiar?

Augmented reality suggests the most significant shift in computation since the internet. As we craft our computers into smaller and smaller mobile devices, exponentially more powerful and connected, we’re now on the verge of beginning the visual and locational integration of the digital world with the analog world. We’ve digitized much of human culture, pasted it onto screens and given ourselves mirror identities to navigate, communicate, and share in this virtual space. Now we’re breaking open the box and drawing the cloud across the phenomenal world, teaching our machines to see what we see and inviting the world to be listed in the digital Yellow Pages.

So, yeah, now your AR experience of the world is covered in billboards, sloganeering, propaganda, and dancing dinosaurs all competing for your click-through AdSense rating. A big consideration, and a topic that came up again & again at AR DevCamp, is the overwhelming amount of data and the need to filter it to some meaningful subset, particularly with respect to spam and advertising. A glance across the current crop of iPhone AR apps reveals many design interface challenges, with piles of annotations all occluding themselves and your view of the world. Now imagine a world covered in layers each with any number of annotations. UI becomes very important. Andrea Mangini & Julie Meridian led a session on design & usability considerations in AR that could easily be a conference of it’s own. How do you manage occlusion & sorting? Level of detail? What does simple & effective authoring of annotations on a mobile device look like? How do you design a small but visible environmental cue that an annotation exists? If the URL convention is an underlined text, what is the AR convention for gently indicating that the fire hydrant you’re looking at has available layers & annotations? Discoverability of the digital links within the augmented world will be at a tension with overwhelming the view of the world itself.

When we consider the seemingly-inevitable development of eyewear with digital heads-up display, occlusion can quickly move from helpful to annoying to dangerous. No matter how compelling the augmented world is you still need to see when that truck is coming down the street. Again, proper design for human usability is perhaps even more critical in the augmented interface than in a typical screen interface. Marketing and business plans aside, we have to assume that the emergence of truly compelling and valuable technologies are ultimately in line with the deep evolutionary needs of the human animal. We’re certainly augmenting for fun and art and engagement and communication but my sense is that, underneath all these we’re building this new augmented reality because the power & adaptive advantage mediated through the digital domain is so great that we need it to integrate seamlessly with our mobile, multi-tasking lives. It’s been noted by others – Kevin Kelly comes to mind – that we’re teaching machines to do many of things we do, but better. And in the process we’re making them smaller and more natural and bringing them closer and closer to our bodies. Ponderings of transhumanity and cyborgian futures aside, our lives are being increasingly augmented and mediated by many such smart machines.

DARPA wasn’t at AR Dev Camp. Or at least if they were, they didn’t say so. There was a guy from NASA showing a really cool air traffic control system that watched aircraft in the sky, tagged them with data annotations, and tracked their movements. We were shown the challenges to effectively register the virtual layer – the annotation – with the real object – a helicopter – when it’s moving rapidly. In other words, the virtual layer, mediated through a camera & a software layer, tended to lag behind the 80+ mph heli. But in lieu of DARPA’s actual attendance, it’s worth considering their Urban Leader Tactical Response, Awareness & Visualization (ULTRA-Vis) program to develop a multimodal mobile computational system for coordinating tactical movements of patrol units. This program sees the near-future soldier as outfitted with a specialized AR comm system with a CPU worn on a belt, a HUD lens over one eye, a voice recognition mic, and a system to capture gestures. Military patrols rely heavily on intel coming from command and on coordinating movements through back-channel talk and line-of-sight gestures. AR HUDs offer simple wayfinding and identification of team mates. Voice commands can execute distributed programs and open or close comm channels. Gestures will be captured to communicate to units both in an out of line-of-sight and to initiate or capture datastreams. Cameras and GPS will track patrol movements and offer remote viewing through other soldier’s cameras. But most importantly, this degree of interface will be simple, fluid, and effortless. It won’t get in your way. For better or for worse, maximizing pack hunting behaviors with technology will set the stage for the future of human-computer interaction.

After lunch provided by Qualcomm, Anselm Hook led an afternoon session at AR DevCamp titled simply “Hiking”. We convened in a dark and hot room, somewhat ironically called the “Sun Room” for it’s eastern exposure, to discuss nature and what, if any, role AR should play in our interface with the Great Outdoors. We quickly decided to move the meeting out into the parking lot where we shared our interests in both built and natural outdoor environments. A common theme that emerged in words and sentiment was the tension between experience & distraction. We all felt that the natural world is so rich and special in large part due to it’s increasing contrast to an urbanized and mechanized life. It’s remote and wild and utterly disconnected, inherently at peace in it’s unscripted and chaotic way. How is this value and uniqueness challenged by ubicomp and GPS and cellular networks? GPS & cellphone coverage can save lives but do we really need to Twitter from a mountain top? I make no judgement calls here and am plenty guilty myself but it’s worth acknowledging that augmented reality may challenge the direct experience of nature in unexpected ways and bring the capacity to overwrite even the remote corners of the world with human digital graffiti.

But remember that grove of trees I mentioned before, tagged with data annotations? Imagine the researchers viewing those trees through AR lenses able to see a glance-able color index for each one showing CO2, O2, heavy metals, turgidity, growth, and age. Sensors, mesh nets, and AR can give voice to ecosystems, cities, communities, vehicles, and objects. Imagine that grove is one of thousands in the Brazilian rainforest reporting on it’s status regularly, contributing data to policy debates and regulatory bodies. What types of augmented experiences can reinforce our connection to nature and our role as caretakers?

On the other hand, what happens when you and the people around you are each having very different experiences of “reality”? What happens to the commons when there are 500 different augmented versions? What happens to community and society when the common reference point for everything – the very environment in which we exist – is malleable and fluid and gated by permissions and access layers or overwrought with annotations competing for our attention? What social gaps could arise? What psychological ailments? Or perhaps more realistically, what happens when a small class of wealthy westerners begin to redraw the world around them? Don’t want to see other people? No problem! Just turn on the obfuscation layer. Ugly tenements ruining your morning commute? Turn on some happy music and set your iGlasses to the favela paintshop filter! Augmentation and enhancement with technology will inevitably proceed along economic lines. What is the proper balance between enjoying our technological luxuries and responsibly curating the world for those less fortunate? Technology often makes the symptoms look different but doesn’t usually eradicate the cause. In the rush to colonize the augmented reality, in the shadow of a wavering global economic system and deep revision of value and product, now is the best time and the most important time to put solutions ahead of products; to collaborate and cooperate on designing open, robust, and extensible systems; and, in the words of Tim O’Reilly, to “work on stuff that matters”.

At the end of the day, pizza’s arrived (Thanks MakerLab!), beers were opened (Thanks Layar & Lighting Labs), and the buzzing brains of AR DevCamp mingled and shared their thoughts. Hearts alit, I’ll be forgiven some sentimentality to suggest that the Hacker Dojo had a soft, warm glow emanating from all the fine folks in attendance. Maybe it was like this around the Acid Tests in the 60′s (with more paisley). Or the heady days of PARC Xerox in the 80′s (with more ties). That growing inertia and sense of destiny at being at the right place at the right time just at the start of something exceptional…

Special thanks to Andrea Mangini for deep and ranging discussions about all this stuff, among many other things.

Local, Regional, National, Global: A View of Current World Trends

[The following is an unedited, stream-of-thought for a Sunday afternoon, summing up in fairly broad strokes many of the trends in socioeconomics and geopolitics that I'm currently tracking. This is a rough forecast for the next 3-5 years.]

The present arc of instability & change is driven by massive shifts in economy, ideology, connectivity, and trust. Many deeply entrenched behaviors and the institutions built on them are grinding up against social & environmental feedback, bloated bureaucracy, unsustainable growth models, and the interface of global transparency and closed power structures. On the global stage the myth of America is fading, it’s dematerialized information economies eroding it’s own industry, having financed the rise of it’s greatest competitors. The center of economic power is shifting to the East, into the EU and China, though it’s reasonable to suggest that the center itself is distributing across the primary northern actors. The perceived decline of the US may simply be the effects of capital normalization in a globalized world.

American governance continues to lose it’s ability to effectively manage the needs of it’s citizenry. This will shift governance to local and regional bodies mixing public sector legislators, corporate leaders, and empowered citizen groups & lobbies. Loyalties will trend towards those organizations most capable of meeting the needs of people, eg. stable employers, effective community groups, capable local civic leadership, familial & tribal affiliations, and gangs & insurgencies. As top-down national management weakens, corporate NGOs will play an increasing role in managing civil and socio-economic development.

For better or for worse, Google is a good example of an emerging NGO world leader. Google is financially strong, employs thousands, and is actively engineering & underwriting the development of a more prosperous and sustainable future in both it’s information and energy operations. For it’s employees, it’s business partners, it’s ecology of developers, and the many CSR organizations Google works with, they are a more reliable and trustworthy patron than the State of California or the US Gov. Likewise for Microsoft or, more specifically, the Bill & Melinda Gates Foundation that has sent billions of dollars to help African poverty and fight AIDS (as with Google, questionable strategic intentions and pragmatic implementation failures aside). In an age lit with corrupt politicians and bankers, it’s easy for corporations & NGOs to look good by spending money on non-profit causes. In some ways, the market is more effective at getting things done than government these days. Expect to even see investment firms making similar overtures to social and environmental responsibility as capital begins to redistribute in emerging opportunistic investments. The new titans of the information, green tech, and biotech industries will continue to be the powers that are most capable of managing change effectively, though this posture will be both at tension with and an outcome of their need to manage growth and profit for their shareholders. Local and regional investments will inevitably be balanced against the global workforce. Corporate loyalty and integrity will only go so far as outsourcing draws jobs and money away from the locals.

As states risk bankruptcy through bloated bureaucracy, mismanagement, and endless partisan deadlocks, regions & local communities will organize & collaborate to pick up the slack or else decline into economic and social turmoil. This pattern is reflected at the level of national governance, increasingly beholden to global financial centers, extended across the globe in multiple unfavorable & expensive conflicts, and crippled domestically by oppositional bickering in Congress. All these trends reinforce the gap between State & Citizen, Wall St. and Main St., further de-legitimizing traditional governance. With spiking unemployment and the growing commercial credit crunch, the street-level effects of the Great Recession will continue to deepen for some time in spite of Wall St. indicators trying to stoke a rally back to an outmoded growth model. This effect will not be universal but unevenly distributed, reflecting the distribution of innovation, resources, capital, and climate.

For California, some regions, such as the San Francisco Bay Area, are poised to maintain strength and innovation, particularly in the new information economy but also, perhaps more importantly, in the coming biotech revolution. Meanwhile, Fresno, Riverside, Bakersfield, and Los Angeles counties will deteriorate further. Fresno’s fate as it resets against failed housing speculation will be determined primarily by shifting weather patterns. If the region dries, expect even greater flight. Riverside and Bakersfield will likely follow the lead of Los Angeles which seems to be destined for greater chaos, particularly if water supplies are threatened. Expect continued regional migration into LA as extended families draw together to fight unemployment, and job seekers are drawn on hopes of legitimate and criminal financial opportunities. Gang violence will rise as illicit drug and weapons networks stake out territory, increasingly mixing with Mexican and Salvadorean criminal networks. LA and San Diego demographics will continue into a Hispanic majority as people flee a Mexico torn apart by it’s own narco-insurgency. But again, great opportunities exist in the San Francisco region provided the high cost of living and salary requirements do not scare away the most able and potentially benevolent corporate kingpins. The California Bay Area is one of the most innovative and productive centers of US business and will continue to build the information age for the foreseeable future. If not for deeply intractable water issues, one could imagine a Northern California statehood.

Nevertheless, Asia is leading the global economic recovery, not America, underscored most recently by China’s founding of its own NASDAQ-styled ChiNext market, as well as its recent ASEAN attendance. Increasingly, Asian economic bodies are calling for lessened reliance on Western markets, supplementing their currency investments with Euros, Yen, and RMBs. These two features of marked Asian economic growth and increased global currency diversification will inevitably soften the national power of the US, itself holding on by it’s handful of primary innovation centers (increasingly isolated from and at odds with middle America and the south), as well as its absolutely indomitable military force. While it is not in China or anyone’s interest to dump US debt and cause a dollar collapse, it will be interesting to see how an economically weakened and politically fractured America will rely on its military to save face as the global economy becomes more evenly distributed towards Asia & the EU. Similarly, India’s contribution to the region remains unclear as it tryies to resolve it’s hunger for modernization with a huge population mired in poverty and caste culture. Caught between the US & China, India’s primary role in the foreseeable future is to put pressure on Pakistan, forcibly enough to encourage the Pakistani military to put down the Taliban, but subtle enough not to provoke an all-out Pak-India conflict.

However one regards India’s contributions, the rise of Asia will not be easy. There are huge populations but their level of industrial & technological education still lags behind many western nations. Likewise, their financial centers (and political focus) will continue to be challenged by the much larger depressed rural populations still struggling out of feudal poverty. Any attempt to form an Asian counterpart to the EU will be challenged by such vast inequity across many proud and nationalistic cultures that have been traditionally very averse to cooperation. Asia has been at war with itself in some form perhaps as long as the Middle East. Indeed, the rise of the EU was only possible in a post-WW2 reunification towards cooperative economics (and against a rising USSR), and this cooperation remains tenuous at best. Russia itself is a major wildcard sitting between Asia & the EU & the Middle East, economically and politically crippled but still heavily armed with ICBMs & natural resources and making regular overtures to get the Soviet band back together. While the US is only slightly concerned about its southern neighbor, Asia is beset on all sides by nuclear powers, militant Islamists, and strategically-critical but highly chaotic regions. America travels to its conflicts while Asia need only look in its own backyard. Thus, the rise of Asia will be significant, primarily due to the inertia of China (itself in large part buoyed by unsustainable business trickery) and the flocking multinational business interests attracted to its present shininess, but any coherent China-JPN-ASEAN strategy will be beset by many challenges. Indeed, Japan itself will remain heavily dependent on the West, both economically and militarily, with everyone in the region hoping the US will keep North Korea in check. (As an aside, the US military is the *only* military capable of deploying 100,000+ troops anywhere in the world. No other country can move forces globally, control the world seas, and reach any nation with a nuclear strike.)

Either way, there will continue to be a fractured US polity, a weakening dollar, and a lackluster system of state and national governance. Some regions will descend into chaos and become truly feral (eg Detroit, parts of LA), while others will route around bureaucratic impotence with community co-ops, participatory local governance, regional corporate sponsorship, and virtualized global information & content cooperatives. The tensions between neighboring regions will be amplified by ideological and socio-economic differences arising from their respective abilities (or inabilities) to provide for their populations, as well as the presently unforeseeable impacts of climate change and pandemics. One neighbor’s fecundity is another’s reason to migrate or invade.

None of these shifts will radically alter business as usual, in that trade of goods and services will persist. The current global financial paradigm will not likely come crashing down though it will be forced to evolve towards greater transparency & liability. Wall St. can post great numbers but if communities are wracked by unemployment, crime, and dying lands, so-called economic recovery won’t make much difference, especially as the anticipated uptick in business sees more global outsourcing and contract work with diminishing quality of life for the expensive regions. In the near-term, these trends will put pressure on many local communities to fend for themselves (or align with corporate, government, and/or military interests in the region) though some will be uniquely positioned to build local resiliency and self-reliance while remaining highly competitive in a dematerialized global information marketplace.

The next stabilization will likely come on the back of green industry, dematerialized technology, and new energy abundance, all of which will emerge from a deepening understanding and engagement with natural systems & the biochemical world. The present instability is a direct result of an unsustainable resource model kicked off by the industrial revolution. The shift we’re in is driven by the necessity to bring human systems in line with living, natural systems. This is a deep, deep evolutionary priority working it’s way through the human species in ways we don’t even see. But in short, things are shifting quickly. The idealistic paradigms of How The World Is are changing before us on the tide of globalization and instantaneous communication connecting all corners of the Earth. Yet, change is typically not monolithic but is scattered and unevenly distributed. There are many more players on the board now and the myth of US dominance and the ideal of Democracy is challenged on many sides. The system is unlikely to crash but it’s certainly entering a phase of high instability. But don’t under-estimate the collective effort to keep things moving forward. The interconnected web of globalization may actually be the saving grace against system collapse, tying all nations together in mutual reliance.