Scott Smith has a nice article about Our Complicated Love-Hate Relationship With Robots, exploring how robotics have been seeping into the public dialog of late. A couple of the links he cites were good reminders of previous work looking at the aesthetics of machine perception, notably Sensor-Vernacular from the fine folks at BERG and The New Aesthetic Tumblr by James Bridle.
If humanity is a reflection on the experience of perceiving and interacting with the world, what role does machine perception play in this experience? And if nature acts through our hands, to what ends are flocking drones and herds of autonomous machines? A taxonomy of machine perception seems necessary to understand the many ways in which the world can be experienced.
I’ve just returned from a very interesting workshop in Washington, D.C. about fast-moving change, asymmetric threats to security, and finding signals within the wall of noise thrown up by big data. These are tremendous challenges to governance, policy makers, and the intelligence community. I’ll have more to say on these topics in later posts but for now, here’s a round-up of the most popular posts on URBEINGRECORDED in order of popularity:
Outliers & Complexity – On non-linearity, outliers, and the challenges of using the past to anticipate the future…
Thanks to all my readers for taking the time to think about my various rantings & pre-occupations. As always, your time, your participation, and your sharing is greatly appreciated!
I was driving through the Tenderloin the other night – one of the most socio-economically depressed areas of San Francisco. Across a long wall someone tagged “Occupy Wall Street” in big letters with a clean font and preceded by the Twitter “#” hashtag notation. It was a big, funky chorus bridging the grimy street with a shimmering virtuality beckoning from the other side. A shiny enticement to both residents and passers-by, yet it instilled in me that there are some hard reasons why Occupy is still a bit pale, demographically. The Tenderloin is where people fall to the bottom of the American heap, struggling every day just to try and get by. I can’t speak to their cellphone use but I’m guessing most aren’t on Twitter.
In America, poverty & homelessness are specters stalking the nightmares of the middle class. The stigma is crushing and many studies show how hard it is to fall out of society and fail normative expectations, forced to walk as a ghost the rest of us don’t want to acknowledge. We’re all “temporarily embarrassed millionaires”, to quote Steinbeck, but most are scared to death we’ll wake up from the American Dream wearing dirty rags and begging for pittance.
Amidst the looming failure of governance and the siphoning of capital into the hands of elite gamers the system starts to reveal interesting and exploitable gaps. The gaps opening up between the ruling elite, the body politik, the business world, the towers of old-world power, global supply chains and international demand structures, and the organic messiness of the street lashing together its own ad hoc infrastructure, battening down against the hard approach of a faceless Winter. A lot of innovation happens in the gaps.
When a control system releases it’s organizational capacity, the system tends towards a period of turbulence. Turbulence can be thought of as a widening of constraints on energized systems, ie things start getting wonky & unusual. Institutions are challenged. Stability & confidence are shaken. Calcified bureaucracy cedes power to fast, open-source iterations. Hierarchies flatten, though riddled with super-empowered outliers, revealing design patterns more akin to fuzzy biology than the mechanized Taylorism of the Industrial Age. A mycelial hypermedia of distributed, tech-enabled, self-empowered collectives emerges. The landscape is shifting so quickly that even the rules of the game are being forced to adapt. And not in any particularly easy way, mind.
The United States government is failing to adapt or effectively shepherd its populace into the 21st century. Many western nations share a similar sentiment. They’re falling left & right to the slipperiness of the behavioral economy and to top-tier predators drawing capital out of weakened states and widening the gaps between people & power. Meanwhile, gangs & cartels and urban collectives (oh, and the estimated $10T – trillion! – informal economy) are all pulling the weave apart further and staking their territorial claims. The landscape is ragged and hungry and a bit unhinged. Many of us are growing nervous feeling the hot breath of the meathook future on the back of our necks.
“You will not hear me, you will not listen to me, so I will stand in your face and you will be forced to see me.”
Occupy Wall Street is an expression of this sweaty fear & creeping nihilism in a world that looks decidedly different than the one we were raised to expect. It’s an empowered disenfranchisement: the realization and acceptance that the American people no longer have a say in the conversation about our country. “You will not hear me, you will not listen to me, so I will stand in your face and you will be forced to see me.” This is what Occupy says. And it says it encamped in front of your hallowed institutions, deploying local food & health services, brewing ad hoc energy supplies, coordinating collective actions, surveilling the local PD and running mobile counter-ops, holding signs to the media cameras and managing international PR campaigns. This is a new model of power emerging across technologically-savvy collectives, economically detached on the ground but coordinated with well-healed and influential sympathizers among the extended technorati. You get amplification, charitable donations, shout-outs, drop-ins from mayoral candidates, and as-needed mobilization of supporters who still have to hold down their day jobs and take the kids to school. Of course, the PD knows all this & knows how to exploit mobile social media as well.
The Short Message Service (SMS) was implemented in 1992 and is now ubiquitous and coupled to an insanely sophisticated global supply chain. A large driver for cellphone adoption, these discrete packets of information passed almost immediately across non-local nodes have proven extremely powerful. With very lightweight protocols and minimal hardware demands, SMS is fast becoming one of our primary signalling pathways. Witness the simple observation that mobile-enabled teens are constantly texting, rarely speak on the phone, and disregard email almost entirely. More info, less work. Now make sure every one of the somewhat feral and vaguely radical protestors occupying the park across from your ridiculously powerful and possibly sociopathic local tax base, eg The Federal Reserve… make sure they all have SMS mobiles. And make sure all the other urban clans have them too so they can share updates & anticipation, coordinate a distributed response, propagate the sticky phrases and hashtags, and rapidly pass counter intelligence to every single global node. Oh, and there’s this thing called Twitter that will take your SMS and push it out to a broadcast subscriber list that’s being crawled by every journalist, intel org, and revolutionary sympathizer across the modern world.
Of course the NYPD is scared and twitchy. Of course the DHS is yelling at all jurisdictions to get this under control. The true sign of fear will be revealed if they send in the National Guard – a tacit admission that the police are more sympathetic to the protesters than the economic cartels. And if you wanna get really meathook, peep the vid of the armed, self-appointed border guards standing against the Arizona Police Department to defend Occupy. “Using our 2nd amendment rights to defend our 1st amendment rights” was the money quote from that one.
Pundits and old-century analysts can’t get past its slipperiness. It doesn’t look like how protests were supposed to look. It won’t fit into a neat soundbite or flashy statement of demands. This gets really annoying for a mainstream press corps empowered by semantic containment.
Occupy Wall Street is an exceptional sociocultural hack. Grabbing eyes & hearts, they’re making it OK to protest again in America. After 911 the normative pressure around dissent & protest shifted, making it very un-American to disagree with and or show criticism of The U S of A. Occupy is quickly becoming view-fodder for the mainstream media. Spin it any way you like but OWS is grabbing the spotlight globally. Expect the election cycle to raise it as a common talking point – a good reason Occupy can safely find heat indoors for the Winter, come back swinging in Spring. This normative shift allows the many many folks who aren’t yet willing or simply can’t come sleep in the streets to be active & connected sympathizers helping spread the word, defend the narrative, and get downtown at 2am on a Thursday to stand against an expected police action. Social media invites participation at all scales.
People talk of so-called “new models of work”. Remote specialists coming together around a shared task, doing the work with a minimum of resources, taking value, and collaborating with adjacent like-minded ad hoc clusters. All enabled by information technologies and responding to shifting economic realities. BTW, capital is leaving the West and moving eastward and into Africa. Brazil is doing OK as well, I hear. But these new models of work are the same 21st century design patterns iterated on by Anonymous, WikiLeaks, and the Mexican cartels (with varying degrees of flamboyant and or enigmatic leadership). Another eye-opening newsblip from the past week is word that Anonymous is going after Los Zetas cartel (or possibly not). Thing about the Zetas… they don’t just hang out in cyberspace. You don’t wanna be trolled by Zetas.
So yeah, DHS is nervous. They see all these small fires and worry that one will flame up into a real conflict, or that their ranks will be taken over by legitimate militants/gangs/cartels/etc… Meanwhile, China is quietly laughing at us, gently suggesting to the world that maybe Democracy isn’t really all that necessary for a decent house, a reliable job, and good prospects for your kids. It’s worth considering what this means for U.S. diplomacy and the project of Democracy.
The sympathy that boomers have with OWS is rooted in this emigration of prosperity away from our shores. They did fine, my generation is fighting to hang on, and the younger generation can’t get a job. Of course, the Boomers think OWS should be using the tried-and-true models of the 1960′s, not this crazy post-modern artwar stuff. But they lived in a very different world and, ironically, it’s the protest movement of the 60′s that hardened the economic jungle and trained it against the Left.
The front of the emerging cultural war is everywhere & nowhere.
Occupy is a new creature bred to adapt to a markedly different environment. It uses similar design patterns from Tahrir Square and Tehran. It’s the new hyperpolitics enabling virtualized ideologies & coordinated actions by distributed collectives. Like everything else being spread across the real & virtual, the front of the emerging cultural war is everywhere & nowhere. The focus now is on prosperity and equality but it’s tugging at the sweater threads of our entire industrialized economy, already well-frayed & tattered from the wear & tear of the new millenium. Occupy is a statement about failure & fear and a realization that the people who have been entrusted to keep it all together for us are no longer acting on our behalf.
It’s a scary place to find yourself, falling into the gaps. But there’s tremendous potential there as well. And it’s likely a manifestation of far deeper and longer evolutionary imperatives brought to bear on the aging and deprecated foundations of industrial civilization. We are due for a major upgrade. New features & workflows are direly needed, and please patch some of those nasty bugs we’ve been complaining about for centuries. It would be really nice if we could all get back to work helping the world get a little better, day by day.
Leviathan worked with frequent collaborator and renowned VJ Vello Virkhaus on groundbreaking performance visuals for electronic musician Amon Tobin, creating ethereal CG narratives and engineering the geometry maps for an entire stage of stacked cube-like structures. Taking the performance further, the Leviathan team also developed a proprietary projection alignment tool to ensure quick and accurate setup for the show, along with custom Kinect control & visualization utilities for Amon to command.
I’ve been thinking about how identity passes through ubicomp environments and the types of experiences that could occur in such a relationship. We each carry a digital ID in our smartphone. This ID is a key that grants access to voice, data, location, acceleration, and other information both in the net and in our devices. These handshakes occur almost continuously in some form, the most common being the regular polling our mobiles make of our surroundings to determine if we’re in range of a cell tower. Not only do our mobiles contain our digital identification, they also hold rich profiles of our interests, our habits, our journeys, our transactions, and our networks. These elements are forming the core foundation upon which our experience of the networked world is constructed.
Smartphone manufacturers are integrating near-field communication (NFC) chips that enable our devices to manage transactions. At the check-out counter in the corner market (ok, more like Safeway) you wave your phone to make payment. Your mobile knows who you are, it has access to your checking account, and it makes the handshake on your behalf with the trusted vendor. Whether or not NFC becomes the de facto coretech underneath this mechanism, the usability is very sticky. All sorts of lock-and-key relationships like home & vehicle entry, gym membership, library or lab entrance, and network access become a natural characteristic of your presence. Just as your face & voice provision you with access to your parent’s home and induce birthday parties in your name, mobile identity confers digital membership and can initiate personalized experiences around you.
One of the light bulbs that really went off in my head was lit by Ben Cerveny‘s talk at ARE2011. In the course of discussing his data exploration instrument, Bloom, Ben illustrated an example of this type of personal digital provisioning by considering the modern, networked home entertainment system. Imagine you have a dinner party and as your friends arrive their mobiles make the handshake with the local network. The system queries their devices for music likes, recent social network sentiment, and checks their calendars to see how hurried they may be (if the data is shared). It then constructs a playlist on-the-fly that’s tailored to fit the mood. If they like, they can engage the system from their mobiles by sharing media and driving the mood. It’s a simple example that illustrates how we’re sharing a lot more information about ourselves with the computational networks in which we swim, and how those networks can become more aware of us and tailor experiences to fit the context.
Greg Tran has a really great concept video that explores these ideas of local networks and provisioned experiences by looking at augmented reality as a mediating layer. Here’s the video:
Tran postulates a near-future where some form of native augmented reality is ubiquitous, then considers the kinds of experiences that might be possible. He explores how local networks could push such experiences out to provisioned individuals based on profile & location. For example, as you enter a building it reads your digital ID and passes an access profile to your device. This local profile invokes experiences as you pass through different areas of the structure. Perhaps virtual walls are rendered to offer smaller meeting rooms within a larger space. Planar blinds covered in motion graphics rise up to obscure areas or to convey pertinent information. Real walls are rendered transparent to reveal HVAC systems (for ID:HVACRepair), or network lines (ID:CablingContractor), or the floor below you (ID:Bankrobber). The concept video is slick & compelling and suggests a sort of techno-magic that feels only just beyond our fingertips.
The concept work of Keiichi Matsuda serves to illustrate the inevitable tensions likely to rise in such a data-saturated and dynamic media landscape. He explores the somewhat-uneasy co-mingling of our traditional needs & expectations as humans with the growing presence of push media bombarding our every waking moment. There is a suggestion that perhaps traditions will eventually fall as the older generations withdraw from influence. The new young are better fit to parse & move amongst what we might consider a slightly-terrifying visual information overload. It is said that Descartes considered the pineal gland as the reducing valve of the Soul, keeping us from seeing the whole of Creation so we can focus on the more pressing biosurvival tasks at hand. Keiichi’s work, particularly Domestic Robocop, imparts the sense that we’re steadily opening that valve back up.
These examples are really just frameworks within which we can explore the relationship between digital identity and ubiquitous computing. More specifically, they show how we are deputizing our mobile device as legitimate cognitive prostheses and proxy selves. The social landscape is increasingly occupied by machines and so we need machine ambassadors to manage these relationships on our behalf. And as we move en masse into social networks we get closer to the machines and share more and more minutia about our lives. Social networks are incredibly fertile ground for getting to know complete strangers better than they even know themselves. Especially if you’re a data crawler crunching billions of analytics per cycle. Pretty quickly this becomes a surreal sort of digital intimacy that for most people never even registers.
This relationship will become more visceral as we hire a growing array of scripts & cloud agents to do our bidding, initialized and left to run, watching and learning about us, and mediating our needs & expectations to innumerable and often-invisible third-parties. There is a reasonably convincing argument that considers the Greek & Roman pantheon to be the early psychological complexes of the awakening human mind. The young ego wasn’t quite able to recognize the emotions & voices as being local and instead ascribed to them an external embodiment in the form of anthropomorphic deities. We seem to be at a similar junction where we’ve yet to fully internalize and integrate our digital pieces. But it’s our mobile devices that bring them closer and invite them to join us.
Rolling forward with personalized ubicomp we can see a possible world where cloud agents flit about enacting our will, communicating with us, transacting with other agents, invoking local experiences & remote actions. We can imagine a more responsive and amorphous physical world that shifts to meet our needs, to persuade us, and to contain us. How does the individual understand itself when embedded in such a fluid & personalized world? How does cognition and psychology change as it distributes and becomes more & more disembodied? What are the powers of crowds as machine intelligences scan & summate them, customizing group experiences to the common denominator? Will distributed intelligence relate to crowds better than individuals? We’re getting a bit scifi here, I know, but ranters gotta rant.
Suffice it to say that the near-future will really get interesting once digital identity is fully integrated as the core component of the ubicomp landscape. The current effort to move payments into the mobile phone is a major step in this direction (and should serve as a hint when looking at the present identity challenges & goals of Facebook, Twitter, and especially Google Plus). Your social networking is painting a rich profile about who you are. Your credit card is arguably a stronger & more universal ID than your driver’s license or passport. And though we may resist sharing so much of ourselves in such a broad way, it won’t matter. Our devices will identify us and our digital ghosts will betray us to their friends.
I recently participated in the Future of Facebook project that Venessa Miemis & Alvis Brigis have launched as part of their efforts towards an Open Foresight platform. The full collection of out-takes from all forecasters is available on the Future of Facebook You Tube channel.
Below are some of the out-takes from my interview. These are general thoughts I have about what Facebook is, how it seems to be impacting our lives & institutions, and where it might be headed. I should note that these are broad observations and many caveats apply. Also: default YT framegrabs seem to seek out the worst possible pics of me. Enjoy.
Increasingly, we live in a world defined by flat networks. Folks like Clay Shirky, Ben Hammersley, and others have observed in great detail how the design patterns of the internet are challenging and changing the landscape of human civilization. So many of our institutions have been built as hierarchical pyramids designed to exert the maximum degree of control over their domains. These top-down management structures have come to define business, government, the military, medicine, education, the family, and knowledge itself. Leaders rise to the top as centralized governors dictating down the chain how things should be, while workers march in step towards execution of their appointed tasks. Such structures were modeled after the clockworks & steam engines of classical mechanics, designed to be precise, rigid, and durable, capable of lasting hundreds of years. These structures informed the defining metaphors of our entire industrialized society.
Computer architecture recapitulated the mechanical metaphor by designating a central processor that assigned & managed tasks bussed out to sub-processors and specialized functional components. In this way the computer became more of a powerful extension of the industrial age rather than a stake in the ground of a new paradigm. While the mechanical metaphor gradually evolved into the computational metaphor which has defined the last two decades, it wasn’t until computers began to follow the model of telecom and began connecting with each other across flat networks that the seed of a biological metaphor began to take hold.
Nature, it seems, does not create very many rigid, top-down control systems. Those are too stiff and inflexible for the dynamics of life. Rather, nature evolves vast horizontal networks that assemble into specialized functions within their environment. For example, the messiest, most distributed organizational structure known – the human brain – does not have a top-tier manager or CPU. There is no executive function within the brain or its mind, though we typically like to think there is. Instead, the brain is a vast & mostly flat hierarchy that is bundled into loosely vertical functional bodies. These functional bodies are themselves existing across a mostly flat horizontal network of interactions. The thalamus receives all inputs and routes them up to higher cortical processing and lower hindbrain autonomic structures, into the amygdala for emotional content and across the hippocampus for memory, then down throughout the body. The processing chain is massively parallel, interconnected, and marked by complex feedback pathways. Mind arises off of these processes in a very ad hoc manner, always shifting, always flexible, and always derived from a mass summation across the network.
Mycelial networks offer another example. When we see mushrooms scattered across a forest floor we’re not seeing individuals. Each mushroom growing from the soil is a fruiting body rising from the underground web-work of mycelia – the skeletal framework of the colony. Some mycelial colonies have been found to have areas extending over 2000 acres making them some of the largest superorganisms on the planet. The pattern suggests mushrooms as terminal nodes and mycelia as the network backbone.
In ecosystems, large predators constitute a form of top-down management but they themselves are part of the predator-prey relationship – a dynamic that must always seek relative equilibrium with the broader network in which it is embedded. Predators do not have a choice to over-consume prey or stockpile & re-sell it to others. Large ocean gyres also suggest a high degree of top-down control by seasonally establishing the engines of hemispheric weather. The North Pacific gyre becomes more active in the Winter of the northern hemisphere, driving the scale & frequency of storms hitting the pacific northwest of the United States. But the North Pacific gyre is an emergent structure that is itself built upon the properties of a nearly-infinite set of factors. It is not a regulatory structure or a governor by intent or design and there is no top-level group of components that determine its next move. It is a super-system derived from innumerable sub-systems.
Most importantly, all biological systems are guided not by top-down governors or control mechanisms but by feedback from the networks in which they are embedded. This is how nature regulates, preserves, and evolves itself towards greater adaptability. There is no fallible ruler driven to resource over-reach and myopic certainty. There is only the ongoing trial & error of embedded growth tempered by continuous communication between & within organisms.
As computers began to connect across the ARPANET, and with the dawning of the visual internet, the CPU evolved away from being specifically a central control system to become a node within a distributed network. This initial shift quickly challenged the established domains of publishing, content creation, intellectual property, and knowledge management while inviting the crowd into a shared virtual space of increasingly global identity & transaction. The advent of social networks established an organizational structure for connecting the human capital of virtuality, making it easier for like-minded people to connect & share & collaborate non-locally, subtly undermining the very notions of borders, statehood, family, and allegiance. Soon after, the mobile revolution has tipped everything on its side and bundled it into a portable device bringing instantaneous global communication & information access to most people on the planet.
The framework was laid for new forms of emergent, non-hierarchical, distributed collaboration & innovation, to both productive & destructive ends. Groups could now form and coordinate around affiliations, interest, and goals in ways that directly challenged the institutional structures monetizing our production & consumption and regulating our behaviors. It has become vastly easier for small organizations to take on multinational interests, whether in business & innovation or in power & politics. The conflicts we see across the world today are, in large part, a symptom of the younger generations leveraging flat network technologies to rise up against the older generations who long ago settled into their legacy hierarchical power structures. To paraphrase Ben Hammersley, the people who are running the world, who are entrusted with our future, are not able to understand the present. They lack the cognitive tools that are a basic part of the Generation C toolkit – the digital natives who grew up with a mobile in their hands and the internet at their fingertips, embedded in specialized networks that span borders and extend identity into the virtual.
The global disruptions that seem to characterize modernity constitute a civilizational correction driven by natural law. The DotCom bubble went through a correction, shedding excess value and pruning the garden of exuberant innovation to favor only the most fit. It was a good thing, if not painful. We witnessed the correction in the housing bubble and will likely see similar corrections in credit & commodities, as well as a painfully positive correction in energy, subsidized and under-valued for so long. The impacts of climate change are a correction imposed upon the legacy model of industrialization & growth by nature itself – the super-system in which all human endeavor is embedded and to which we are ultimately accountable.
The civilization correction is an emergent regulatory mechanism embedded within natural systems forcing our legacy human systems to progressively modify the unsustainable design patterns of our past. The mechanical metaphor & the computational metaphor are necessarily opening to include the biological metaphor. We can see this in every aspect of technology and it is equally emergent across human behavior & social systems. Nanosystems emulate biosystems. Computation & robotics are integrating with neurology & physiology. Individuals are finding agency & empowerment in leaderless multi-cellular collaborations. The built environment is becoming sensory-aware, communicating with itself through discrete feedback mechanisms. It can be argued that the emergence of the internet and of ubiquitous mobile communication & computation is an expression of our natural instincts to move into closer alignment with our environment; to follow the adaptive design patterns of nature in order to find a more sustainable & equitable posture for our species; a thermodynamic need to seek maximum efficiency in energy expenses. And to express a direct intervention programmed by nature itself to nudge the Anthropocene back towards equilibrium.
Such lofty ponderings aside, our world is undoubtedly approaching an inflection point. Everything appears to be upending and it’s all spread out in glorious detail for everyone to see. The feedback loop between humanity and it’s creations – the biological & cybernetic communication among individuals & groups & cultures & organisms & ecosystems – is tightening and getting more & more dense every day, feeding on itself and forcing exceptional degrees of novelty into becoming. It’s frightening & awesome and the Old Guard can barely see it happening right in front of their eyes. The shift may be apocalyptic, a sudden phase change, or an accelerated-but-managed transition… Probably it will be all of these things in differing degrees & locales. However it happens, the emerging paradigm is much more about networks, messaging, feedback, and biology rather than hierarchy, control, power, and mechanization. Nature is the super-system, the ultimate controller enforcing the laws of physics and prescribing the design templates for fitness & adaptation. If we are, as Kevin Kelley suggests, the sex organs of technology, then our technology is born from the natural imperatives coded deeply into our DNA.
[Justin Pickard notes: Biology PhD friend had issues w/ @chris23's latest (http://bit.ly/e0tJSS), citing hierarchies in social insect colonies, meercats & wolves... Furthermore, some biologists now consider social insect colonies to be superorganisms in their own right; akin to @cascio's ecology of mind?
Me: Yes! I considered diving into ants - lot's of research there. Interesting social structures emerge in higher critters/hives... I'd love to read a rebuttal/extension.]
I’m excited to announce that the second annual Bay Area Augmented Reality Developer’s Camp will be held on Saturday, December 4th, 2010, at the Gray Area Foundation For The Arts in San Francisco! This will be a free, open format, all-day unconference looking at the many aspects of Augmented Reality. We welcome hackers, developers, designers, product folks, biz devs, intellectuals, philosophers, tinkerers & futurists – anyone interested in this fascinating and revolutionary new technology.
This is one of 50 posts about cyborgs – a project to commemorate the 50th anniversary of the coining of the term. Thanks to Tim Maly of Quiet Babalon for running such a great project!
“He would see faces in movies, on T.V., in magazines, and in books. He thought that some of these faces might be right for him…”
The word “cybernetic” derives from a Greek word, kybernetes, meaning “rudder” or “governor”. A cybernetic process is a control system that uses feedback about it’s actions in an environment to better adapt it’s behavior. The cybernetic organism, or “cyborg”, is a class of cybernetic systems that have converged with biological organisms. In this increasingly mythologized form, the cyborg embodies the ongoing dialectic between humanity & technology, and is an aspirational figure onto which we project our superhuman fantasies. While it offers security, enhancement, and corporeal salvation the cyborg also presents an existential threat to the self and to the cherished notions of being uniquely human.
It’s a gamble but we don’t seem able to leave the table. As we offload more of our tasks into technology we enhance our adaptability while undermining our own innate resilience as animals. We wrap ourselves in extended suits of shelter, mobility, health, and communications. We distribute our senses through a global network of hypermedia, augmenting our brains with satellites & server farms & smart phones. Increasingly, our minds & bodies are becoming the convergence point for both the real & the virtual, mediated through miniaturization, dematerialization, and nano-scale hybridization. Our ability to craft the world around us is quickly advancing to give us the ability to craft our bodies & our selves.
“And through the years, by keeping an ideal facial structure fixed in his mind… Or somewhere in the back of his mind… That he might, by force of will, cause his face to approach those of his ideals…”
Computation is miniaturizing, distributing, and becoming more powerful & efficient. It’s moving closer & closer to our bodies while ubiquitizing & dematerializing all around us. The cybernetic process has refined this most adaptive capacity in little more than 50 years to be right at hand, with us constantly, connected to a global web of people, places, things, information, and knowledge. We are co-evolving with our tools, or what Kevin Kelly refers to as the Technium – the seemingly-intentional kingdom of technology. As Terence McKenna suggested, we are like coral animals embedded in a technological reef of extruded psychic objects. By directly illustrating how our own fitness & bio-survival becomes bound to the survival of our technology, the cyborg is a fitting icon for this relationship.
Technology has historically been regarded as something we cast into the world separate from ourselves but it’s worth considering the symbiosis at play and how this relationship is changing the very nature of humanity. As we venture deeper & deeper into the Technium, we lend ourselves to it’s design. By embracing technology as part of our lives, as something we rely upon and depend on, we humanize it and wrap it in affection. We routinely fetishize & sexualize cool, flashy tech. In doing so we impart emotional value to the soul-less tools of our construction. We give them both life & meaning. By tying our lives to theirs, we agree to guarantee their survival. This arrangement is a sort of alchemical wedding between human & machine, seeking to yield gold from this mixture of blood & metal, uncertain of the outcome but almost religiously compelled to consummate.
“The change would be very subtle. It might take ten years or so. Gradually his face would change it’s shape. A more hooked nose. Wider, thinner lips. Beady eyes. A larger forehead…”
In the modern world, our identities include the social networks & affinity groups in which we participate, the digital media we capture & create & upload, the avatars we wear, and the myriad other fragments of ourselves we leave around the web. Who we are as individuals reflects the unique array of technologies through which we engage the world, at times instantiated through multiple masks of diverse utility, at other times fractured & dis-integrated – too many selves with too many virtual fingers picking at them. Our experience of life is increasingly composed of data & virtual events, cloudy & intangible yet remote-wired into our brains through re-targeted reward systems. A Twitter re-tweet makes us happy, a hostile blog comment makes us angry, the real-time web feeds our addiction to novelty. Memories are offloaded to digital storage mediums. Pictures, travel videos, art, calendars, phone numbers, thoughts & treatises… So much of who we are and who we have been is already virtualized & invested in cybernetic systems. All those tweets & blog posts cast into the cloud as digital moments captured & recorded. Every time I share a part of me with the digital world I become copied, distributed, more than myself yet… in pieces.
It can be said that while we augment & extend our abilities through machines, machines learn more about the world through us. The web 2.0 social media revolution and the semantic web of structured data that is presently intercalating into it has brought machine algorithms into direct relationship with human behavior, watching our habits and tracking our paths through the digital landscape. These sophisticated marketing and research tools are learning more and more about what it means to be human, and the extended sensorium of the instrumented world is giving them deep insight into the run-time processes of civilization & nature. The spark of self-awareness has not yet animated these systems but there is an uneasy agreement that we will continue to assist in their cybernetic development, modifying their instructions to become more and more capable & efficient, perhaps to the point of being indistinguishable from, or surpassing, their human creators.
“He imagined that this was an ability he shared with most other people. They had also molded their faces according to some ideal. Maybe they imagined that their new face would better suit their personality. Or maybe they imagined that their personality would be forced to change to fit the new appearance…”
In Ridley Scott’s Blade Runner, the young Tyrell Corporation assistant, Rachel, reflects on her childhood memories while leafing through photographs of her youth. These images are evidence of her past she uses to construct her sense of self. Memories provide us with continuity and frame the present & future by reminding us of our history – critical for a species so capable of stepping out of time. Rachel’s realization that she is a replicant, that her memories are false implants deliberately created to make her believe she’s human, precipitates an existential crises that even threatens Harrison Ford’s character, Rick Deckard, surrounded as he is by photos of his own supposed past. This subtle narrative trick suggests that replicants will be more human-like if they don’t know they’re replicants. But it also invokes another query: If memories are (re-)writable, can we still trust our own past?
Yet both characters do appear quite human. They laugh and cry and love and seem driven by the same hopes and fears we all have. Ridley Scott’s brilliance – and by extension, Philip K. Dick’s – is to obscure the nature of the self and of humanity by challenging our notions of both. Is Rachel simply another mannequin animated by advanced cybernetics or is she more than that? Is she human enough? When the Tyrell bio-engineer J.F. Sebastian sees the Nexus 6 replicants, Pris and Roy Batty, he observes “you’re perfect”, underlining again the aspirational notion that through technology we can be made even better, becoming perhaps “more human than human”. This notion of intelligent artificial beings raises deep challenges to our cherished notions of humanity, as many have noted. But the casual fetishization of technology, as it gets nearer & friendlier & more magical, is perhaps just as threatening to our deified specialness in it’s subtle insinuation into our hands & hearts & minds.
In Mamoru Oshii’s anime classic, Ghost in the Shell, the female protagonist – a fully-engineered and functional robotic human named Kusanagi – at once decries those who resist augmentation, suggesting that “your effort to remain as you are is what limits you”, while simultaneously becoming engaged in a quest to determine if there might be more to her than just what has been programmed. She celebrates her artifice as a supreme achievement in overcoming the constraints of biological evolution while also seeking to find evidence that she is possessed of that most mysterious spark: the god-like ingression of being that enters and animates the human shell. Oshii’s narrative suggests that robots that achieve a sufficient level of complexity and self-awareness will, just like their human creators, seek to see themselves as somehow divinely animated. Perhaps it’s a method to defend the belief in human uniqueness but those writing the modern myths of cybernetics seem to imply that while humans aspire to the abilities of machines, machines aspire to the soulfulness of humans.
“This is why first impressions are often correct…”
Chalk it up to curiosity, the power of design fictions, and an innate need to realize our visions, but if we can see it with enough resolution in our mind’s eye, we’ll try to bring it to life. The Ghost in the Shell & the Ghost in the Machine both intuit the ongoing merger between humanity & technology, and the hopes & fears that attend this arranged and seemingly-unavoidable alchemical wedding. As animals we are driven to adapt. As humans, we are compelled to create.
“Although some people might have made mistakes. They may have arrived at an appearance that bears no relationship to them. They may have picked an ideal appearance based on some childish whim or momentary impulse. Some may have gotten half-way there, and then changed their minds…”
Humans are brilliant & visionary but also impetuous, easily distracted, fascinated by shiny things, and typically ill-equipped to divine the downstream consequences of our actions. We extrude technologies at a pace that far outruns our ability to understand their impacts on the world, much less how they change who we are. As we reach towards AI, the cyborg, the singularity, and beyond, our cybernetic fantasies may necessarily pass through the dark night of the soul on the way to denouement. What is birthed from the alchemical marriage often necessitates the destruction of the wedding party.
“He wonders if he too might have made a similar mistake.” – David Byrne, Seen & Not Seen
Are we working up some Faustian bargain promising the heights of technological superiority only for the meager sacrifice of our Souls? Or is this fear a reflection of our Cartesian inability to see ourselves as an evolving process, holding onto whatever continuity we can but always inevitably changing with the world in which we are embedded? As we offload more and more of our selves to our digital tools, we change what it means to be human. As we evolve & integrate more machine functionality we modify our relationship to the cybernetic process and re-frame our self-identity to accommodate our new capacities.
Like the replicants in Blade Runner and the animated cyborgs of Ghost in the Shell we will very likely continue to aspire to be more human than human, no matter how hard it may be to defend our ideals of what this may mean to the very spark of humanity. What form of cyborg we shall become, what degree of humanity we retain in the transaction, what unforeseen repercussions may be set in motion… The answers are as slippery as the continuum of the self and the ever-changing world in which we live. Confrontation with the existential Other – the global mind mediated through ubiquitous bio-machinery – and the resulting annihilation of the Self that will necessarily attend such knowledge, may very well yield a vastly different type of humanity than what we expect.
This past Saturday I worked with Mike Liebhold, Gene Becker, Anselm Hook, and Damon Hernandez to present the West Coast Augmented Reality Development Camp at the Hacker Dojo in Mountain View, Ca. By all accounts it was a stunning success with a huge turn-out of companies, engineers, designers, makers, artists, geo-hackers, scientists, techies and thinkers. The planning was mostly done virtually via email and phone meetings with only a couple visits to the venue. On Saturday, the virtual planing phase collapsed into reality and bloomed on site into AR Dev Camp.
As an un-conference, the event itself was a study in grassroots, crowd-sourced, participatory organization with everyone proposing sessions which were then voted on and placed into the schedule. To me, it was a wonderfully organic and emergent process that almost magically gave life and spirit to the skeleton we had constructed. So before I launch into my thoughts I want to give a hearty “Thank You!” to everyone that joined us and helped make AR DevCamp such a great experience. I also want to give a big shout-out to Tish Shute, Ori Inbar, and Sophia for coordinating the AR DevCamp in New York City, as well as Dave Mee & Julian Tate who ran the Manchester, UK event. And, of course, we couldn’t have done it without the help of our sponsors, Layar, Metaio, Qualcomm, Google, IFTF, Lightning Laboratories, Web3D Consortium, IDEAbuilder, MakerLab, and Waze (and URBEINGRECORDED with Cage Free Consulting contributed the flood of afternoon cookies).
So first, just what is Augmented Reality? There’s a tremendous amount of buzz around the term, weighing it down with connotations and expectations. Often, those investing in it’s future invoke the haunting specter of Virtual Reality, doomed by it’s inability to live up to the hype: ahead of it’s time, lost mostly to the realm of military budgets and skunkworks. Yet, the AR buzz has driven a marketing rush throwing gobs of money at haphazard and questionable advertising implementations that quickly reach millions and cement in their minds a narrow association with flashy magazine covers and car ads. Not to diminish these efforts, but there’s a lot more – and a lot less – going on here.
In it’s most distilled form, augmented reality is an interface layer between the cloud and the material world. The term describes a set of methods to superimpose and blend rendered digital interface elements with a camera stream, most commonly in the form of annotations such as text, links, and other 2 & 3-dimensional objects that appear to float over the camera view of the live world. Very importantly, AR includes at it’s core the concept of location mediated through GPS coordinates, orientation, physical markers, point-clouds, and, increasingly, image recognition. This combination of location and superimposition of annotations over a live camera feed is the foundation of AR. As we’re seeing with smart phones, the device knows where you are, what direction you’re facing, what your looking at, who & what is near you, and what data annotations & links are available in the view. In this definition, the cloud is the platform, the AR browser is the interface, and annotation layers are content that blend with the world.
So the augmented reality experience is mediated through a camera view that identifies a location-based anchor or marker and reveals any annotations present in the annotation layer (think of a layer as a channel). Currently, each of these components is uniquely bound to the AR browser in which they were authored so you must use, for example, the Layar browser to experience Layar-authored annotation layers. While many AR browsers are grabbing common public data streams from sources like Flickr & Wikipedia, their display and function will vary from browser to browser as each renders this data uniquely. And just because you can see a Flicker annotation in one browser doesn’t mean you will see it in another. For now, content is mostly bound to the browser and authoring is mostly done by third-parties building canned info layers. There doesn’t seem to be much consideration for the durability and longevity of these core components, and there is a real risk that content experiences may become fractured and ephemeral.
Indeed, content wants to be an inclusive, social experience. One of the core propositions underlying our motivation for AR DevCamp is the idea that the platforms being built around augmented reality should be architected as openly as possible to encourage the greatest degree of interoperability and extensibility. In the nascent but massively-hyped AR domain, there’s a growing rush to plant flags and grab territory, as happens in all emergent opportunity spaces. The concern is that we might recapitulate the Browser Wars – not intentionally but by lack of concerted efforts to coordinate implementations. While I maintain that coordination & open standardization is of necessity, I question my own assumption that without it we’ll end up with a bunch of walled gardens. This may be under-estimating the impact of the web.
Through the lessons and resultant standardization of the Browser Wars, it’s become a best practice (and indeed, a necessity) to design specifically to the most common standards. Arguably, the move from Web 1.0 (essentially a collection of static billboards) to the social interactions that characterize Web 2.0 established and deeply reinforced the fundamental requirement that we’re all able to share information & experiences in the cloud. This social commons necessarily requires an architectural commonality. Thus, we all agree that HTML, JavaScript, PHP, JASON, MySQL, and now RDF, OWL, and SPARQL are the core components of our data service models. Since we understand that AR is primarily a location-aware interface layer for the cloud, it’s very likely that independent implementations will all speak the same language. However, the point of AR DevCamp and similar gatherings is to challenge this assumption and to reinforce commonality by bringing everyone together to press flesh & exchange notes. The social dynamic in the natural world will determine the level of cooperation in the virtual.
Yet, this cooperation and normalization is by no means a given. Just about every chunk of legacy code that the Information Age is built upon retains vestiges of the git-er-done, rush to market start-up midset. Short-sighted but well-meaing implementations based upon limited resources, embryonic design, and first-pass architectures bog down the most advance and expensive software suites. As these code bases swell to address the needs of a growing user base, the gap between core architecture and usability widens. Experience designers struggle against architectures that were not able to make such design considerations. Historically, code architecture has proceeded ahead of user experience design, though this is shifting to some degree in the era of Agile and hosted services. Nevertheless, the emerging platforms of AR have the opportunity – and, I’d argue, the requirement – to include user research, design, & usability as core components of implementation. The open, standardized web has fostered a continuous and known experience across it’s vast reaches. Artsy Flash sites aside, you always know how to navigate and interact with the content. The fundamentals of AR need to be identified and agreed upon before the mosaic of emerging code bases become too mature to adjust to the needs of a growing user base.
Given the highly social aspect of the web, place-based annotations and objects will suffer greatly if there’s not early coordination around a shared standard for anchors. This is where the Browser Wars may inadvertently re-emerge. The anchor is basically the address/location of an annotation layer. When you look through an augmented view It’s the bit of data that says “I’m here, check out my annotations”. Currently there is no shared standard for this object, nor for annotations & layers. You need the Layar browser in order to see annotation layers made in it’s platform. If you only have a Junaio browser, you won’t see it. If you annotate a forest, tagging each tree with a marker linked to it’s own data registry, and then the browser app you used to author goes out of business, all those pointers are gone. The historical analog would be coding your website for IE but anyone with Mosaic can’t see it. This is where early design and usability considerations are critical to ensure a reasonable commonality and longevity of content. Anchors, annotations, & layers are new territory that ought to be regarded as strongly as URL’s and markup. Continuing to regard these as independent platform IP will balkanize the user experience of continuity across content layers. There must be standards in authoring and viewing. Content and services are where the business models should innovate.
So if we’re moving towards an augmented world of anchors and annotations and layers, what considerations should be given to the data structure underlying these objects? An anchor will have an addressable location but should it contain information about who authored it and when? Should an annotation contain similar data, time-stamped and signed with an RDF structure underlying the annotation content? How will layers describe their contents, set permissions, and ensure security? And what of the physical location of the data? An anchor should be a distributed and redundant object, not bound to the durability and security of any single server. A secure and resilient backbone of real-world anchor points is critical as the scaffolding of this new domain.
Earthmine is a company I’ve been watching for a number of months since they presented at the IFTF. They joined us at AR DevCamp to present their platform. While many AR developers are using GPS & compass or markers to draw annotations over the real world, Earthmine is busy building a massive dataset that maps Lat/Long/Alt coordinates to hi-rez images of cities. They have a small fleet of vehicles equipped with stereoscopic camera arrays that drive around cities, capturing images of every inch they see. But they’re also grabbing precise geolocation coordinates that, when combined with the image sets, yields a dense point cloud of addressable pixels. When you look at one of these point clouds on a screen it looks like a finely-rendered pointillistic painting of a downtown. They massage this data set, mash the images and location, and stream it through their API as a navigable street view. You can then place objects in the view with very high accuracy – like a proposed bus stop you’d like to prototype, or a virtual billboard. Earthmine even indicated that making annotations in their 2d map layer could add a link to the augmented real-world view. So you can see a convergence and emerging correlation between location & annotation in the real world, in an augmented overlay, on a flat digital map, and on a Google Earth or Virtual World interface. This is an unprecedented coherency of virtual and real space.
The Earthmine demo is cool and the Flash API offers interesting ways to customize the street view with 2d & 3d annotations but the really killer thing is their dataset. As alluded to, they’re building an address space for the real world. So if you’re in San Francisco and you have an AR browser that uses the Earthmine API (rumors that Metaio are working on something here…) you can add an annotation to every STOP sign in The Mission so that a flashing text of “WAR” appears underneath. With the current GPS location strategy this would be impossible due to it’s relatively poor resolution (~3-5 meters at best). You could use markers but you’d need to stick one on every STOP sign. With Earthmine you can know almost exactly where in the real world you’re anchoring the annotation… and they can know whenever you click on one. Sound familiar?
Augmented reality suggests the most significant shift in computation since the internet. As we craft our computers into smaller and smaller mobile devices, exponentially more powerful and connected, we’re now on the verge of beginning the visual and locational integration of the digital world with the analog world. We’ve digitized much of human culture, pasted it onto screens and given ourselves mirror identities to navigate, communicate, and share in this virtual space. Now we’re breaking open the box and drawing the cloud across the phenomenal world, teaching our machines to see what we see and inviting the world to be listed in the digital Yellow Pages.
So, yeah, now your AR experience of the world is covered in billboards, sloganeering, propaganda, and dancing dinosaurs all competing for your click-through AdSense rating. A big consideration, and a topic that came up again & again at AR DevCamp, is the overwhelming amount of data and the need to filter it to some meaningful subset, particularly with respect to spam and advertising. A glance across the current crop of iPhone AR apps reveals many design interface challenges, with piles of annotations all occluding themselves and your view of the world. Now imagine a world covered in layers each with any number of annotations. UI becomes very important. Andrea Mangini & Julie Meridian led a session on design & usability considerations in AR that could easily be a conference of it’s own. How do you manage occlusion & sorting? Level of detail? What does simple & effective authoring of annotations on a mobile device look like? How do you design a small but visible environmental cue that an annotation exists? If the URL convention is an underlined text, what is the AR convention for gently indicating that the fire hydrant you’re looking at has available layers & annotations? Discoverability of the digital links within the augmented world will be at a tension with overwhelming the view of the world itself.
When we consider the seemingly-inevitable development of eyewear with digital heads-up display, occlusion can quickly move from helpful to annoying to dangerous. No matter how compelling the augmented world is you still need to see when that truck is coming down the street. Again, proper design for human usability is perhaps even more critical in the augmented interface than in a typical screen interface. Marketing and business plans aside, we have to assume that the emergence of truly compelling and valuable technologies are ultimately in line with the deep evolutionary needs of the human animal. We’re certainly augmenting for fun and art and engagement and communication but my sense is that, underneath all these we’re building this new augmented reality because the power & adaptive advantage mediated through the digital domain is so great that we need it to integrate seamlessly with our mobile, multi-tasking lives. It’s been noted by others – Kevin Kelly comes to mind – that we’re teaching machines to do many of things we do, but better. And in the process we’re making them smaller and more natural and bringing them closer and closer to our bodies. Ponderings of transhumanity and cyborgian futures aside, our lives are being increasingly augmented and mediated by many such smart machines.
DARPA wasn’t at AR Dev Camp. Or at least if they were, they didn’t say so. There was a guy from NASA showing a really cool air traffic control system that watched aircraft in the sky, tagged them with data annotations, and tracked their movements. We were shown the challenges to effectively register the virtual layer – the annotation – with the real object – a helicopter – when it’s moving rapidly. In other words, the virtual layer, mediated through a camera & a software layer, tended to lag behind the 80+ mph heli. But in lieu of DARPA’s actual attendance, it’s worth considering their Urban Leader Tactical Response, Awareness & Visualization (ULTRA-Vis) program to develop a multimodal mobile computational system for coordinating tactical movements of patrol units. This program sees the near-future soldier as outfitted with a specialized AR comm system with a CPU worn on a belt, a HUD lens over one eye, a voice recognition mic, and a system to capture gestures. Military patrols rely heavily on intel coming from command and on coordinating movements through back-channel talk and line-of-sight gestures. AR HUDs offer simple wayfinding and identification of team mates. Voice commands can execute distributed programs and open or close comm channels. Gestures will be captured to communicate to units both in an out of line-of-sight and to initiate or capture datastreams. Cameras and GPS will track patrol movements and offer remote viewing through other soldier’s cameras. But most importantly, this degree of interface will be simple, fluid, and effortless. It won’t get in your way. For better or for worse, maximizing pack hunting behaviors with technology will set the stage for the future of human-computer interaction.
After lunch provided by Qualcomm, Anselm Hook led an afternoon session at AR DevCamp titled simply “Hiking”. We convened in a dark and hot room, somewhat ironically called the “Sun Room” for it’s eastern exposure, to discuss nature and what, if any, role AR should play in our interface with the Great Outdoors. We quickly decided to move the meeting out into the parking lot where we shared our interests in both built and natural outdoor environments. A common theme that emerged in words and sentiment was the tension between experience & distraction. We all felt that the natural world is so rich and special in large part due to it’s increasing contrast to an urbanized and mechanized life. It’s remote and wild and utterly disconnected, inherently at peace in it’s unscripted and chaotic way. How is this value and uniqueness challenged by ubicomp and GPS and cellular networks? GPS & cellphone coverage can save lives but do we really need to Twitter from a mountain top? I make no judgement calls here and am plenty guilty myself but it’s worth acknowledging that augmented reality may challenge the direct experience of nature in unexpected ways and bring the capacity to overwrite even the remote corners of the world with human digital graffiti.
But remember that grove of trees I mentioned before, tagged with data annotations? Imagine the researchers viewing those trees through AR lenses able to see a glance-able color index for each one showing CO2, O2, heavy metals, turgidity, growth, and age. Sensors, mesh nets, and AR can give voice to ecosystems, cities, communities, vehicles, and objects. Imagine that grove is one of thousands in the Brazilian rainforest reporting on it’s status regularly, contributing data to policy debates and regulatory bodies. What types of augmented experiences can reinforce our connection to nature and our role as caretakers?
On the other hand, what happens when you and the people around you are each having very different experiences of “reality”? What happens to the commons when there are 500 different augmented versions? What happens to community and society when the common reference point for everything – the very environment in which we exist – is malleable and fluid and gated by permissions and access layers or overwrought with annotations competing for our attention? What social gaps could arise? What psychological ailments? Or perhaps more realistically, what happens when a small class of wealthy westerners begin to redraw the world around them? Don’t want to see other people? No problem! Just turn on the obfuscation layer. Ugly tenements ruining your morning commute? Turn on some happy music and set your iGlasses to the favela paintshop filter! Augmentation and enhancement with technology will inevitably proceed along economic lines. What is the proper balance between enjoying our technological luxuries and responsibly curating the world for those less fortunate? Technology often makes the symptoms look different but doesn’t usually eradicate the cause. In the rush to colonize the augmented reality, in the shadow of a wavering global economic system and deep revision of value and product, now is the best time and the most important time to put solutions ahead of products; to collaborate and cooperate on designing open, robust, and extensible systems; and, in the words of Tim O’Reilly, to “work on stuff that matters”.
At the end of the day, pizza’s arrived (Thanks MakerLab!), beers were opened (Thanks Layar & Lighting Labs), and the buzzing brains of AR DevCamp mingled and shared their thoughts. Hearts alit, I’ll be forgiven some sentimentality to suggest that the Hacker Dojo had a soft, warm glow emanating from all the fine folks in attendance. Maybe it was like this around the Acid Tests in the 60′s (with more paisley). Or the heady days of PARC Xerox in the 80′s (with more ties). That growing inertia and sense of destiny at being at the right place at the right time just at the start of something exceptional…
Special thanks to Andrea Mangini for deep and ranging discussions about all this stuff, among many other things.
“Imagine an environment where most physical objects know where they are, what they are, and can, (in principle) network with any other object. With this infrastructure, reality becomes its own database. Multiple consensual virtual environments are possible, each oriented to the needs of its constituency. If we also have open standards, then bottom-up social networks and even bottom up advertising become possible. Now imagine that in addition to sensors, many of these itsy-bitsy processors are equipped with effectors. Then the physical world becomes much more like a software construct. The possibilities are both scary and wondrous.” (Vernor Vinge – intro to ISMAR 2009)
[This is a narrative exploration of an idea @jingleyfish & I had walking around the Westside of Santa Cruz late at night...]
Imagine walking around a town wearing your stylish Ray Ban augmented reality glasses (because hand-held mobile devices will become a significant limiting factor to experiencing the annotated world). You see small transparent white dots glowing on people and objects indicating that they contain accessible cloud content. Maybe you “select” (by whatever mechanism constitutes selection through a pair of eyeglasses) a bench on the sidewalk then view a flyout markup indicating that the bench was commissioned by the Bruce family in memorium of Aldis Bruce, manufactured by the Taiwanese Seating Concern. You click through the family link to see a brief bio of Aldis with a set of links to his life story, works, etc…
In the upper corner of your view a light begins to blink indicating a new feed is available in your subscription list. You select and expand, showing a menu item for Bob’s Neighborhood Chat. Initializing this feed draws a new green dot over the bench, indicating that Bob has published information tagged to it. You click and Bob’s markup flies out with text stating, “Tuesday, March 11, 2010: Don & Charise Ludemeyer celebrated their 25th wedding anniversary by returning to the place where they first kissed in 1985.” A link below this offers the couple’s personal website, a photo gallery & playlist of their wedding, and then a link to more public markups about the bench.
Clicking through the “more” link offers a list of other public comments. You choose the Sur 13 layer just to see what the local hoods are up to. Flyout: “Hernandez Bros. shot down by Westside Brownshirts, Sept. 23, 2009. RIP, locos.” Then, drawn over, a bit-crushed graffiti logo “WSB” animates across the view, hacked into the Sur 13 layer by Brownshirts. A click through would open the full Brownshirt regional layer but you already feel like a trespasser on suddenly dangerous turf.
Unsettled, you call up the local Police layer. A trailing list of crimes in a 5mi radius begins scrolling. You narrow the search to your current location with a 2 week time horizon. 3 yellow car break-ins glow indicators along the road, followed by a red assault marker 10 feet down the walk, and then 2 more blinking red markers at the bench. You hover over the bench markers and learn of two shootings here within the last 4 days.
You open up the iCabNow utility, send up your beacon, and wait nervously for Yellow Cab to find you. You thumb back to the Ludemeyer markup and click through to find the playlist from their wedding. As you hop into the cab a few moments later, the theme from Miami Vice swells up in your earbuds, sending you off from this time-twisted place. You call up the WordTweet micromarker app and make a traveler’s note: “This is a dangerous bench with an old heart.” Click “Publish” and a new feed indicator appears, offering your own layer update to subscribers.
The post title is in quotes because, although the effect is quite clean & nice, the tech is projection and not really digital wallpaper. What the demo suggests is a programmable surface material, perhaps made of some sort of liquid OLED or biolumin. Wallpaper or paint that could cover a room and represent the full RGB spectrum, dynamically. Nevertheless, the video below suggests the end result of such a future tech, much like the 555Kubik video facade projection.
With all the hype flying around Augmented Reality lately, it’s easy to assume the nascent tech is just another flash-in-the-pan destined to burn out in a fury of marketing gimmickry & sensational posturing. Yet, it’s informative to consider the drivers pushing this trend and to tease out the truly adaptive value percolating beneath the hype. As we survey the last 40 years of computation we see vast rooms of tube & tape mainframes consolidating into single stacks & dense supercomputers. These, in turn, rode manufacturing advances into smaller components and faster processors bringing computing to the desktop. In the last 10 years we’ve seen computation un-encumber from the location-bound desktop to powerful, free-roaming mobile platforms. These devices have allowed us to carry the advantages of instant communication, collaboration, and computation with us wherever we go. The trends in computation continue towards power, portability, and access.
Specific implementations aside, augmented reality in it’s purest, most dilute form, is about drawing the experience of computation across the real world. It’s about point-and-click access to the data shadows of everything in our environment. It’s about realizing social networks, content markups, and digital remix culture as truly tangible layers of human behavior. Augmented reality represents another fundamentally adaptive technology to empower individuals & collectives with instant access to knowledge about the world in which we’re embedded. It breaks open both the digital & mental box and dumps the contents out on the floor.
There is a fascinating convergence at play here that, at a glance, seems almost paradoxical. While the contents of our minds are moving beyond the digital containers we’ve used to such creative & collaborative advantage, out into the phenomenal world of things & critters, the physical hardware through which this expression is constructed & mediated is miniaturizing and moving closer & closer towards our physical bodies. DARPA is funding research to push AR beyond current device limitations, envisioning transparent HUDs, eye-trackers, speech recognition, and gestural interfaces that release soldiers from the physical dependencies of our current devices. Today’s mobiles (and the limited AR tech built on them) compete directly with the other most adaptive human feature: our hands. Truly functional mobile comm/collab/comp must be hands-free… and this is the promise taking form in the emerging field of neurotechnology.
Nanomaterials, optogenetics, SPASERs, advanced robotics, neurocomputation, and artificial intelligence are merely a handful of the modalities shaping up to enable tighter integration between humans, machines, and the digital sphere. Advances in understanding the communication protocols and deep brain structures that mediate the human interface between our sensorium and the perceived world are presenting opportunities to capture & program our minds, to more accurately modulate the complexities of human emotion, creativity, trust, & cognition, and to build more expressive interfaces between mind and machine. Augmented reality is co-evolving with augmented physiology.
In it’s current and most-visualized form, augmented reality is clunky and awkward, merely suggesting a future of seamless integration between computation & cognition. Yet the visions being painted by the pioneers are deeply compelling and illustrate a near-future of a more malleable world richly overlaid with information & interface. As AR begins to render more ubiquitously across the landscape, as more & more phones & objects become smart and connected, the requirements for advancing human-computer interface will create exceptional challenges & astonishing results. Indeed, imagine the interface elements of a fully-augmented and interactive merging between analog & digital, between mind & machine… How do you use your mind to “click” on an object? How will the object communicate & interact with you? How do you filter data & interactions out from simple social transactions? How do you obfuscate the layers of data rising off your activities & thoughts? And what are the challenges of having many different opt-in or opt-out realities running in parallel?
Humans have just crossed the threshold into the Information Age. The sheer speed of the uptake is mind-bending as our world is morphing everyday into the science fictional future we spent the last century dreaming of. We may not really need the latest advances in creative advertising (similarly driven to get closer and closer to us) but it’s inarguable that both humans & the planetary ecology would benefit from a glance at a stream that instantly reveals a profile of the pollutants contained within, tagged by call-outs showing the top ten contributing upstream sources and the profiles of their CEOs – with email, Facebook, Twitter, and newsburst links at the ready. Examples and opportunities abound, perhaps best left to the authors and innovators of the future to sort out in a flurry of sensemods, augs, and biosims.
There are, of course, many challenges and unforeseen contingencies. The rapid re-wiring of the fundamental interface that such “capably murderous” creatures as us have with the natural world, and the attendant blurring of the lines between real & fabricated, should give pause to the most fevered anticipatory optimists. In a very near future, perhaps 10 or 15 years ahead, amidst an age of inconceivable change, we’ll have broken open the box, painted the walls with our minds, and wired the species and the planet to instantaneous collaboration and expression, with massively constructive and destructive tools at our fingertips. What dreams and nightmares may be realized when the apes attain such godhood? When technology evolves at a lightning pace, yet the human psyche remains at best adolescent, will we pull it off without going nuclear? Will the adaptive expressions of our age save us in time? I think they will, if we design them right and fairly acknowledge the deeply biological drivers working through the technologies we extrude.
recent comments