This past Saturday I worked with Mike Liebhold, Gene Becker, Anselm Hook, and Damon Hernandez to present the West Coast Augmented Reality Development Camp at the Hacker Dojo in Mountain View, Ca. By all accounts it was a stunning success with a huge turn-out of companies, engineers, designers, makers, artists, geo-hackers, scientists, techies and thinkers. The planning was mostly done virtually via email and phone meetings with only a couple visits to the venue. On Saturday, the virtual planing phase collapsed into reality and bloomed on site into AR Dev Camp.

As an un-conference, the event itself was a study in grassroots, crowd-sourced, participatory organization with everyone proposing sessions which were then voted on and placed into the schedule. To me, it was a wonderfully organic and emergent process that almost magically gave life and spirit to the skeleton we had constructed. So before I launch into my thoughts I want to give a hearty “Thank You!” to everyone that joined us and helped make AR DevCamp such a great experience. I also want to give a big shout-out to Tish Shute, Ori Inbar, and Sophia for coordinating the AR DevCamp in New York City, as well as Dave Mee & Julian Tate who ran the Manchester, UK event. And, of course, we couldn’t have done it without the help of our sponsors, Layar, Metaio, Qualcomm, Google, IFTF, Lightning Laboratories, Web3D Consortium, IDEAbuilder, MakerLab, and Waze (and URBEINGRECORDED with Cage Free Consulting contributed the flood of afternoon cookies).

So first, just what is Augmented Reality? There’s a tremendous amount of buzz around the term, weighing it down with connotations and expectations. Often, those investing in it’s future invoke the haunting specter of Virtual Reality, doomed by it’s inability to live up to the hype: ahead of it’s time, lost mostly to the realm of military budgets and skunkworks. Yet, the AR buzz has driven a marketing rush throwing gobs of money at haphazard and questionable advertising implementations that quickly reach millions and cement in their minds a narrow association with flashy magazine covers and car ads. Not to diminish these efforts, but there’s a lot more – and a lot less – going on here.

In it’s most distilled form, augmented reality is an interface layer between the cloud and the material world. The term describes a set of methods to superimpose and blend rendered digital interface elements with a camera stream, most commonly in the form of annotations such as text, links, and other 2 & 3-dimensional objects that appear to float over the camera view of the live world. Very importantly, AR includes at it’s core the concept of location mediated through GPS coordinates, orientation, physical markers, point-clouds, and, increasingly, image recognition. This combination of location and superimposition of annotations over a live camera feed is the foundation of AR. As we’re seeing with smart phones, the device knows where you are, what direction you’re facing, what your looking at, who & what is near you, and what data annotations & links are available in the view. In this definition, the cloud is the platform, the AR browser is the interface, and annotation layers are content that blend with the world.

So the augmented reality experience is mediated through a camera view that identifies a location-based anchor or marker and reveals any annotations present in the annotation layer (think of a layer as a channel). Currently, each of these components is uniquely bound to the AR browser in which they were authored so you must use, for example, the Layar browser to experience Layar-authored annotation layers. While many AR browsers are grabbing common public data streams from sources like Flickr & Wikipedia, their display and function will vary from browser to browser as each renders this data uniquely. And just because you can see a Flicker annotation in one browser doesn’t mean you will see it in another. For now, content is mostly bound to the browser and authoring is mostly done by third-parties building canned info layers. There doesn’t seem to be much consideration for the durability and longevity of these core components, and there is a real risk that content experiences may become fractured and ephemeral.

Indeed, content wants to be an inclusive, social experience. One of the core propositions underlying our motivation for AR DevCamp is the idea that the platforms being built around augmented reality should be architected as openly as possible to encourage the greatest degree of interoperability and extensibility. In the nascent but massively-hyped AR domain, there’s a growing rush to plant flags and grab territory, as happens in all emergent opportunity spaces. The concern is that we might recapitulate the Browser Wars – not intentionally but by lack of concerted efforts to coordinate implementations. While I maintain that coordination & open standardization is of necessity, I question my own assumption that without it we’ll end up with a bunch of walled gardens. This may be under-estimating the impact of the web.

Through the lessons and resultant standardization of the Browser Wars, it’s become a best practice (and indeed, a necessity) to design specifically to the most common standards. Arguably, the move from Web 1.0 (essentially a collection of static billboards) to the social interactions that characterize Web 2.0 established and deeply reinforced the fundamental requirement that we’re all able to share information & experiences in the cloud. This social commons necessarily requires an architectural commonality. Thus, we all agree that HTML, JavaScript, PHP, JASON, MySQL, and now RDF, OWL, and SPARQL are the core components of our data service models. Since we understand that AR is primarily a location-aware interface layer for the cloud, it’s very likely that independent implementations will all speak the same language. However, the point of AR DevCamp and similar gatherings is to challenge this assumption and to reinforce commonality by bringing everyone together to press flesh & exchange notes. The social dynamic in the natural world will determine the level of cooperation in the virtual.

Yet, this cooperation and normalization is by no means a given. Just about every chunk of legacy code that the Information Age is built upon retains vestiges of the git-er-done, rush to market start-up midset. Short-sighted but well-meaing implementations based upon limited resources, embryonic design, and first-pass architectures bog down the most advance and expensive software suites. As these code bases swell to address the needs of a growing user base, the gap between core architecture and usability widens. Experience designers struggle against architectures that were not able to make such design considerations. Historically, code architecture has proceeded ahead of user experience design, though this is shifting to some degree in the era of Agile and hosted services. Nevertheless, the emerging platforms of AR have the opportunity – and, I’d argue, the requirement – to include user research, design, & usability as core components of implementation. The open, standardized web has fostered a continuous and known experience across it’s vast reaches. Artsy Flash sites aside, you always know how to navigate and interact with the content. The fundamentals of AR need to be identified and agreed upon before the mosaic of emerging code bases become too mature to adjust to the needs of a growing user base.

Given the highly social aspect of the web, place-based annotations and objects will suffer greatly if there’s not early coordination around a shared standard for anchors. This is where the Browser Wars may inadvertently re-emerge. The anchor is basically the address/location of an annotation layer. When you look through an augmented view It’s the bit of data that says “I’m here, check out my annotations”. Currently there is no shared standard for this object, nor for annotations & layers. You need the Layar browser in order to see annotation layers made in it’s platform. If you only have a Junaio browser, you won’t see it. If you annotate a forest, tagging each tree with a marker linked to it’s own data registry, and then the browser app you used to author goes out of business, all those pointers are gone. The historical analog would be coding your website for IE but anyone with Mosaic can’t see it. This is where early design and usability considerations are critical to ensure a reasonable commonality and longevity of content. Anchors, annotations, & layers are new territory that ought to be regarded as strongly as URL’s and markup. Continuing to regard these as independent platform IP will balkanize the user experience of continuity across content layers. There must be standards in authoring and viewing. Content and services are where the business models should innovate.

So if we’re moving towards an augmented world of anchors and annotations and layers, what considerations should be given to the data structure underlying these objects? An anchor will have an addressable location but should it contain information about who authored it and when? Should an annotation contain similar data, time-stamped and signed with an RDF structure underlying the annotation content? How will layers describe their contents, set permissions, and ensure security? And what of the physical location of the data? An anchor should be a distributed and redundant object, not bound to the durability and security of any single server. A secure and resilient backbone of real-world anchor points is critical as the scaffolding of this new domain.

Earthmine is a company I’ve been watching for a number of months since they presented at the IFTF. They joined us at AR DevCamp to present their platform. While many AR developers are using GPS & compass or markers to draw annotations over the real world, Earthmine is busy building a massive dataset that maps Lat/Long/Alt coordinates to hi-rez images of cities. They have a small fleet of vehicles equipped with stereoscopic camera arrays that drive around cities, capturing images of every inch they see. But they’re also grabbing precise geolocation coordinates that, when combined with the image sets, yields a dense point cloud of addressable pixels. When you look at one of these point clouds on a screen it looks like a finely-rendered pointillistic painting of a downtown. They massage this data set, mash the images and location, and stream it through their API as a navigable street view. You can then place objects in the view with very high accuracy – like a proposed bus stop you’d like to prototype, or a virtual billboard. Earthmine even indicated that making annotations in their 2d map layer could add a link to the augmented real-world view. So you can see a convergence and emerging correlation between location & annotation in the real world, in an augmented overlay, on a flat digital map, and on a Google Earth or Virtual World interface. This is an unprecedented coherency of virtual and real space.

The Earthmine demo is cool and the Flash API offers interesting ways to customize the street view with 2d & 3d annotations but the really killer thing is their dataset. As alluded to, they’re building an address space for the real world. So if you’re in San Francisco and you have an AR browser that uses the Earthmine API (rumors that Metaio are working on something here…) you can add an annotation to every STOP sign in The Mission so that a flashing text of “WAR” appears underneath. With the current GPS location strategy this would be impossible due to it’s relatively poor resolution (~3-5 meters at best). You could use markers but you’d need to stick one on every STOP sign. With Earthmine you can know almost exactly where in the real world you’re anchoring the annotation… and they can know whenever you click on one. Sound familiar?

Augmented reality suggests the most significant shift in computation since the internet. As we craft our computers into smaller and smaller mobile devices, exponentially more powerful and connected, we’re now on the verge of beginning the visual and locational integration of the digital world with the analog world. We’ve digitized much of human culture, pasted it onto screens and given ourselves mirror identities to navigate, communicate, and share in this virtual space. Now we’re breaking open the box and drawing the cloud across the phenomenal world, teaching our machines to see what we see and inviting the world to be listed in the digital Yellow Pages.

So, yeah, now your AR experience of the world is covered in billboards, sloganeering, propaganda, and dancing dinosaurs all competing for your click-through AdSense rating. A big consideration, and a topic that came up again & again at AR DevCamp, is the overwhelming amount of data and the need to filter it to some meaningful subset, particularly with respect to spam and advertising. A glance across the current crop of iPhone AR apps reveals many design interface challenges, with piles of annotations all occluding themselves and your view of the world. Now imagine a world covered in layers each with any number of annotations. UI becomes very important. Andrea Mangini & Julie Meridian led a session on design & usability considerations in AR that could easily be a conference of it’s own. How do you manage occlusion & sorting? Level of detail? What does simple & effective authoring of annotations on a mobile device look like? How do you design a small but visible environmental cue that an annotation exists? If the URL convention is an underlined text, what is the AR convention for gently indicating that the fire hydrant you’re looking at has available layers & annotations? Discoverability of the digital links within the augmented world will be at a tension with overwhelming the view of the world itself.

When we consider the seemingly-inevitable development of eyewear with digital heads-up display, occlusion can quickly move from helpful to annoying to dangerous. No matter how compelling the augmented world is you still need to see when that truck is coming down the street. Again, proper design for human usability is perhaps even more critical in the augmented interface than in a typical screen interface. Marketing and business plans aside, we have to assume that the emergence of truly compelling and valuable technologies are ultimately in line with the deep evolutionary needs of the human animal. We’re certainly augmenting for fun and art and engagement and communication but my sense is that, underneath all these we’re building this new augmented reality because the power & adaptive advantage mediated through the digital domain is so great that we need it to integrate seamlessly with our mobile, multi-tasking lives. It’s been noted by others – Kevin Kelly comes to mind – that we’re teaching machines to do many of things we do, but better. And in the process we’re making them smaller and more natural and bringing them closer and closer to our bodies. Ponderings of transhumanity and cyborgian futures aside, our lives are being increasingly augmented and mediated by many such smart machines.

DARPA wasn’t at AR Dev Camp. Or at least if they were, they didn’t say so. There was a guy from NASA showing a really cool air traffic control system that watched aircraft in the sky, tagged them with data annotations, and tracked their movements. We were shown the challenges to effectively register the virtual layer – the annotation – with the real object – a helicopter – when it’s moving rapidly. In other words, the virtual layer, mediated through a camera & a software layer, tended to lag behind the 80+ mph heli. But in lieu of DARPA’s actual attendance, it’s worth considering their Urban Leader Tactical Response, Awareness & Visualization (ULTRA-Vis) program to develop a multimodal mobile computational system for coordinating tactical movements of patrol units. This program sees the near-future soldier as outfitted with a specialized AR comm system with a CPU worn on a belt, a HUD lens over one eye, a voice recognition mic, and a system to capture gestures. Military patrols rely heavily on intel coming from command and on coordinating movements through back-channel talk and line-of-sight gestures. AR HUDs offer simple wayfinding and identification of team mates. Voice commands can execute distributed programs and open or close comm channels. Gestures will be captured to communicate to units both in an out of line-of-sight and to initiate or capture datastreams. Cameras and GPS will track patrol movements and offer remote viewing through other soldier’s cameras. But most importantly, this degree of interface will be simple, fluid, and effortless. It won’t get in your way. For better or for worse, maximizing pack hunting behaviors with technology will set the stage for the future of human-computer interaction.

After lunch provided by Qualcomm, Anselm Hook led an afternoon session at AR DevCamp titled simply “Hiking”. We convened in a dark and hot room, somewhat ironically called the “Sun Room” for it’s eastern exposure, to discuss nature and what, if any, role AR should play in our interface with the Great Outdoors. We quickly decided to move the meeting out into the parking lot where we shared our interests in both built and natural outdoor environments. A common theme that emerged in words and sentiment was the tension between experience & distraction. We all felt that the natural world is so rich and special in large part due to it’s increasing contrast to an urbanized and mechanized life. It’s remote and wild and utterly disconnected, inherently at peace in it’s unscripted and chaotic way. How is this value and uniqueness challenged by ubicomp and GPS and cellular networks? GPS & cellphone coverage can save lives but do we really need to Twitter from a mountain top? I make no judgement calls here and am plenty guilty myself but it’s worth acknowledging that augmented reality may challenge the direct experience of nature in unexpected ways and bring the capacity to overwrite even the remote corners of the world with human digital graffiti.

But remember that grove of trees I mentioned before, tagged with data annotations? Imagine the researchers viewing those trees through AR lenses able to see a glance-able color index for each one showing CO2, O2, heavy metals, turgidity, growth, and age. Sensors, mesh nets, and AR can give voice to ecosystems, cities, communities, vehicles, and objects. Imagine that grove is one of thousands in the Brazilian rainforest reporting on it’s status regularly, contributing data to policy debates and regulatory bodies. What types of augmented experiences can reinforce our connection to nature and our role as caretakers?

On the other hand, what happens when you and the people around you are each having very different experiences of “reality”? What happens to the commons when there are 500 different augmented versions? What happens to community and society when the common reference point for everything – the very environment in which we exist – is malleable and fluid and gated by permissions and access layers or overwrought with annotations competing for our attention? What social gaps could arise? What psychological ailments? Or perhaps more realistically, what happens when a small class of wealthy westerners begin to redraw the world around them? Don’t want to see other people? No problem! Just turn on the obfuscation layer. Ugly tenements ruining your morning commute? Turn on some happy music and set your iGlasses to the favela paintshop filter! Augmentation and enhancement with technology will inevitably proceed along economic lines. What is the proper balance between enjoying our technological luxuries and responsibly curating the world for those less fortunate? Technology often makes the symptoms look different but doesn’t usually eradicate the cause. In the rush to colonize the augmented reality, in the shadow of a wavering global economic system and deep revision of value and product, now is the best time and the most important time to put solutions ahead of products; to collaborate and cooperate on designing open, robust, and extensible systems; and, in the words of Tim O’Reilly, to “work on stuff that matters”.

At the end of the day, pizza’s arrived (Thanks MakerLab!), beers were opened (Thanks Layar & Lighting Labs), and the buzzing brains of AR DevCamp mingled and shared their thoughts. Hearts alit, I’ll be forgiven some sentimentality to suggest that the Hacker Dojo had a soft, warm glow emanating from all the fine folks in attendance. Maybe it was like this around the Acid Tests in the 60’s (with more paisley). Or the heady days of PARC Xerox in the 80’s (with more ties). That growing inertia and sense of destiny at being at the right place at the right time just at the start of something exceptional…

Special thanks to Andrea Mangini for deep and ranging discussions about all this stuff, among many other things.

“Imagine an environment where most physical objects know where they are, what they are, and can, (in principle) network with any other object. With this infrastructure, reality becomes its own database. Multiple consensual virtual environments are possible, each oriented to the needs of its constituency. If we also have open standards, then bottom-up social networks and even bottom up advertising become possible. Now imagine that in addition to sensors, many of these itsy-bitsy processors are equipped with effectors. Then the physical world becomes much more like a software construct. The possibilities are both scary and wondrous.” (Vernor Vinge – intro to ISMAR 2009)

Imagine walking around a town wearing your stylish Ray Ban augmented reality glasses (because hand-held mobile devices will become a significant limiting factor to experiencing the annotated world). You see small transparent white dots glowing on people and objects indicating that they contain accessible cloud content. Maybe you “select” (by whatever mechanism constitutes selection through a pair of eyeglasses) a bench on the sidewalk then view a flyout markup indicating that the bench was commissioned by the Bruce family in memorium of Aldis Bruce, manufactured by the Taiwanese Seating Concern. You click through the family link to see a brief bio of Aldis with a set of links to his life story, works, etc…

In the upper corner of your view a light begins to blink indicating a new feed is available in your subscription list. You select and expand, showing a menu item for Bob’s Neighborhood Chat. Initializing this feed draws a new green dot over the bench, indicating that Bob has published information tagged to it. You click and Bob’s markup flies out with text stating, “Tuesday, March 11, 2010: Don & Charise Ludemeyer celebrated their 25th wedding anniversary by returning to the place where they first kissed in 1985.” A link below this offers the couple’s personal website, a photo gallery & playlist of their wedding, and then a link to more public markups about the bench.

Clicking through the “more” link offers a list of other public comments. You choose the Sur 13 layer just to see what the local hoods are up to. Flyout: “Hernandez Bros. shot down by Westside Brownshirts, Sept. 23, 2009. RIP, locos.” Then, drawn over, a bit-crushed graffiti logo “WSB” animates across the view, hacked into the Sur 13 layer by Brownshirts. A click through would open the full Brownshirt regional layer but you already feel like a trespasser on suddenly dangerous turf.

Unsettled, you call up the local Police layer. A trailing list of crimes in a 5mi radius begins scrolling. You narrow the search to your current location with a 2 week time horizon. 3 yellow car break-ins glow indicators along the road, followed by a red assault marker 10 feet down the walk, and then 2 more blinking red markers at the bench. You hover over the bench markers and learn of two shootings here within the last 4 days.

You open up the iCabNow utility, send up your beacon, and wait nervously for Yellow Cab to find you. You thumb back to the Ludemeyer markup and click through to find the playlist from their wedding. As you hop into the cab a few moments later, the theme from Miami Vice swells up in your earbuds, sending you off from this time-twisted place. You call up the WordTweet micromarker app and make a traveler’s note: “This is a dangerous bench with an old heart.” Click “Publish” and a new feed indicator appears, offering your own layer update to subscribers.

Hirzberger Events – Digital Wallpaper from Gregor Hofbauer on Vimeo.

As an aside, I’ll be posting more videos given the highly visual nature of a lot of the cool emerging tech these days.

With all the hype flying around Augmented Reality lately, it’s easy to assume the nascent tech is just another flash-in-the-pan destined to burn out in a fury of marketing gimmickry & sensational posturing. Yet, it’s informative to consider the drivers pushing this trend and to tease out the truly adaptive value percolating beneath the hype. As we survey the last 40 years of computation we see vast rooms of tube & tape mainframes consolidating into single stacks & dense supercomputers. These, in turn, rode manufacturing advances into smaller components and faster processors bringing computing to the desktop. In the last 10 years we’ve seen computation un-encumber from the location-bound desktop to powerful, free-roaming mobile platforms. These devices have allowed us to carry the advantages of instant communication, collaboration, and computation with us wherever we go. The trends in computation continue towards power, portability, and access.

Specific implementations aside, augmented reality in it’s purest, most dilute form, is about drawing the experience of computation across the real world. It’s about point-and-click access to the data shadows of everything in our environment. It’s about realizing social networks, content markups, and digital remix culture as truly tangible layers of human behavior. Augmented reality represents another fundamentally adaptive technology to empower individuals & collectives with instant access to knowledge about the world in which we’re embedded. It breaks open both the digital & mental box and dumps the contents out on the floor.

There is a fascinating convergence at play here that, at a glance, seems almost paradoxical. While the contents of our minds are moving beyond the digital containers we’ve used to such creative & collaborative advantage, out into the phenomenal world of things & critters, the physical hardware through which this expression is constructed & mediated is miniaturizing and moving closer & closer towards our physical bodies. DARPA is funding research to push AR beyond current device limitations, envisioning transparent HUDs, eye-trackers, speech recognition, and gestural interfaces that release soldiers from the physical dependencies of our current devices. Today’s mobiles (and the limited AR tech built on them) compete directly with the other most adaptive human feature: our hands. Truly functional mobile comm/collab/comp must be hands-free… and this is the promise taking form in the emerging field of neurotechnology.

Nanomaterials, optogenetics, SPASERs, advanced robotics, neurocomputation, and artificial intelligence are merely a handful of the modalities shaping up to enable tighter integration between humans, machines, and the digital sphere. Advances in understanding the communication protocols and deep brain structures that mediate the human interface between our sensorium and the perceived world are presenting opportunities to capture & program our minds, to more accurately modulate the complexities of human emotion, creativity, trust, & cognition, and to build more expressive interfaces between mind and machine. Augmented reality is co-evolving with augmented physiology.

In it’s current and most-visualized form, augmented reality is clunky and awkward, merely suggesting a future of seamless integration between computation & cognition. Yet the visions being painted by the pioneers are deeply compelling and illustrate a near-future of a more malleable world richly overlaid with information & interface. As AR begins to render more ubiquitously across the landscape, as more & more phones & objects become smart and connected, the requirements for advancing human-computer interface will create exceptional challenges & astonishing results. Indeed, imagine the interface elements of a fully-augmented and interactive merging between analog & digital, between mind & machine… How do you use your mind to “click” on an object? How will the object communicate & interact with you? How do you filter data & interactions out from simple social transactions? How do you obfuscate the layers of data rising off your activities & thoughts? And what are the challenges of having many different opt-in or opt-out realities running in parallel?

Humans have just crossed the threshold into the Information Age. The sheer speed of the uptake is mind-bending as our world is morphing everyday into the science fictional future we spent the last century dreaming of. We may not really need the latest advances in creative advertising (similarly driven to get closer and closer to us) but it’s inarguable that both humans & the planetary ecology would benefit from a glance at a stream that instantly reveals a profile of the pollutants contained within, tagged by call-outs showing the top ten contributing upstream sources and the profiles of their CEOs – with email, Facebook, Twitter, and newsburst links at the ready. Examples and opportunities abound, perhaps best left to the authors and innovators of the future to sort out in a flurry of sensemods, augs, and biosims.

There are, of course, many challenges and unforeseen contingencies. The rapid re-wiring of the fundamental interface that such “capably murderous” creatures as us have with the natural world, and the attendant blurring of the lines between real & fabricated, should give pause to the most fevered anticipatory optimists. In a very near future, perhaps 10 or 15 years ahead, amidst an age of inconceivable change, we’ll have broken open the box, painted the walls with our minds, and wired the species and the planet to instantaneous collaboration and expression, with massively constructive and destructive tools at our fingertips. What dreams and nightmares may be realized when the apes attain such godhood? When technology evolves at a lightning pace, yet the human psyche remains at best adolescent, will we pull it off without going nuclear? Will the adaptive expressions of our age save us in time? I think they will, if we design them right and fairly acknowledge the deeply biological drivers working through the technologies we extrude.

[Acknowledgements: Tish Shute & Ugo Trade; Zack Lynch and his book The Neuro Revolution; conversations with fellow researchers at IFTF; and many others listed in the Signtific Lab tag for ProgrammableEverything.]

street tests from Najork on Vimeo.

Check ‘em out if you’re interested in the fascinating & accelerating field of BCI. Also feel free to add your own Signals you see in the world or are engaging in your professional research.

Cheers!

[Extensive photo album here.]

Japan crossed with Mexico. Hack, mash, and lash everything together. Very hot and thick, humid and prone to short heavy rains. Bangkok is larger than expected, with a higher skyline. Slum-like in many ways but comfortable. Dirty, aged, grafitti’d, tagged, polluted, smelly, hungry, buggy, feral. Friendly, smiley, reverent, strong, spiritualized, watery, creative, delicious, surviving with tenacity. Temples & tenements, luxury hotels and megamalls. Insane traffic and transport. Little regard for lanes or right of way. Swarms of motorbikes, vespas. Cheap and dangerous tuk tuk 3-wheelers. Families piled onto scooters, kids asleep, baggage strapped on. Traffic flow like a logjam, shifting metal slabs moving within inches of each other, victory goes to the bold in a cloud of exhaust. The mighty Chao Phraya cutting its way through Bangkok and out to the coast, it’s headlands in the foothills below Burma. These are river people, with traffic on the waterways as busy and chaotic as the streets. The river is deep, a 1/4 mile wide, running green & tan, dirty and littered with commercial & vegetal detritus. After the rains clumps of fallen jungle float on its surface, carried down from farms and foothill tributaries. Black & yellow birds land on leafy branches half submerged to dine on nuts and berries. Water taxis from hotel to Sky Train. Fantastic monorail, the SRT, its cement track a modern work of civil engineering adding to the Tokyo vibe of downtown Bangkok.

Tangled mess of black utility cable slashing horizontal lines across most everything, tied in to huge transformers, burnt metal grills pumping amperage for the teeming metropolis of 6 million. The twisted infrastructure grows organically like a banyan, stretching out axonal to connect and communicate. Most buildings are old haggard tenements, their facades stained with a dark grey wash like grease and ash drawn out of the thick air. Structures that seem abandoned, uninhabitable, are strung with drying laundry drawn perpendicular to the necessarily ubiquitous swamp coolers lining the sides of each floor. Broken concrete fields under freeway overpasses offer football grounds lined by graffiti mural walls under chainlink divisions.

Downtown, luxury malls with Louis Vuitton and Burberry fronted by large altars of golden Buddha’s and Ganesha’s, black marble elephants flecked with gold, yellow floral garlands and incense offered by shoppers to their immaterial gods. A sign at Wat Phrao Keo in broken Thaiglish sagely, if not inadvertently, warns visitors to “Beware of your valuable possessions”. Technology, commerce, wealth, and western aesthetics have moved in with the economic development afforded here as in every other large city by the realities of globalized communication and trade.

Down crowded alleyways lined with merchant stalls and open air ad hoc kitchens, thick with pedestrians, cars, tuk tuks, and manic motorcyclists weaving through the narrow channels, over rooftop patios caged against some unseen menace, rise countless golden and white and glittery temple spires. Buddhist Wats take residence everywhere, themselves seemingly hacked into the dense fabric of the city, rising like aspirational fruiting bodies of ancient mycelial webs. Wat Arum, Wat Pho, Wat Phra Keo & the Grand Palace, and innumerable others. Religion & myth is woven throughout the populace. Every building has it’s own adjacent spirit house offering residence to the disincarnate lest they move into your own home. City walls are tacked with incense holders between stores. Banyans breaking through the sidewalks are wrapped with rainbow sashes honoring their freakish holy treeness. Every taxi has a statue on the dash or mala hanging from the rearview or Buddhist stencil on the headboard or any combination of the aforementioned. A 3-day Buddhist holiday shut down all government and banking.

The current Thai king is the longest reigning monarch of the modern age, holding office since 1950. Thailand was the only East-Asian country to resist British colonialism, sparing its autonomy by ceding a few bits of territory along the Burmese & Malay borders. Indeed there are long running conflicts with the Burmese, and Buddhist Thailand is in the midst of an insurgency along the Malaysian border from an advancing Islamic populace. The cabinet of the prime minister and the military have provided ongoing political theater as each vie back and forth for the seat of power. Most transfers of power, even in the case of multiple coups, have been bloodless. The Thai people themselves seem to have little interest in these power games, preferring a life of pragmatic spirituality while maintaining a deep abiding love and respect for the king. The two possibly mortal social offenses in Thai society are speaking ill of the Buddha and speaking ill of the king.

Farming is honored. Rubber trees and palms cover most southern land, providing two of the country’s largest exports. The Thai peninsula includes all the most breathtaking exotic tropical beach locations you could imagine, including the stunning Railay Bay – famed for the movie The Beach. Beautiful light blue waters, ridiculously warm and salty, stretched for ages across the gulf. Koh Samui running on Full Moon inertia, tourist trinkets, and scattered luxury resorts sheltered from the hustle. Low inland jungles bring minimal shade to island shanties in seemingly impossible poverty. Yet they survive & persist and move through generations like the rest of us. Koh Phangan also still milking their internationally notorious Full Moon Rave scene, adding a Half Moon party to underwrite the Euro draw. Even away from the main strips the beach scenes has a fun accidental Burning Man vibe, a shoreline esplanade of shanty bars and sound systems. Expats all over the place. Seems easy to get lost for months, years, decades in some seaside shack eating fruit and fish in a poor man’s paradise. Impossible walls of insects whip up into sudden frenzy, a cacophonous wail of screamapillars, giant cicadas that still don’t seem anywhere near big enough to make such a pitch. Monkey troops swing across canopies carpeting tall rock slabs jutting from the water. A rock climber’s joy, sheer faces hung with dripping stalactites and pocked with rope tie-ins. These tall rocks are scattered by the hundreds – thousands? – across the Gulf of Thailand and Andaman Sea. A boater’s paradise. You could spend months exploring thin beaches stretched around the edges of countless small jungle rock islands.

In the South, each night was attended by thunderstorm, often over sea or above the island peaks. Big black charcoal canvas lumbering across, flicker flashed with lightning bursts every few moments, often too distant to hear the thunderclap, then a sudden ear-shattering rend of ozone right above. When the heavy rains hit they come quickly and with ferocity. Never seen rain like it. So thick that it occluded line of sight to 20 meters or so, hiding everything beyond in watery showers. From the steep island peaks water rushes down in sudden rivers cutting through beach sands, pushing tan clouds out into the bay, a shimmering clear layer of fresh water forcing the saline back out over the ocean’s surface. Giant raindrops agitate the bugs forcing them to take flight in peppery swarms. Small opportunistic swift-like birds take to the skies darting and arching, turning and diving to pluck the insects mid-air in some ancient deeply programmed ballet of the food chain. Life goes on. It must. When rains come often and fiercely you can’t just drop your business. This was especially so in Bangkok whose streets are lined with tirelessly deployed open markets bare to the sky save for a small canvas over each. In 20 or 30 minutes the rain will likely pass so there’s no point in worrying much about the interlude.

While the deep south is struggling with a mounting Islamic insurgency, and the peninsula is attending the construction of more new mosques, the buddhist majority continues to permeate life with the spirit of their patron, accompanied by a host of Garuda and Nagas and a menagerie of mythic beasties syncretized from India and China. If Thai Buddhist Bangkok is feral and lashed and relentlessly modded in ghetto slapdash, the Bangkok Chinatown is 10x more so compressed into tighter alleyways, with more people and motorcylces (Vespas apparently seek Chinatown to live out their golden years), hung with impossibly more spaghetti cables, and festooned with walls of neon Mandarin signage casting a little too much light onto freakish displays of animal carcass and presumably inedible seafood and giant transparent sacks of fried pork product and stall after stall of fashionable Versace & Loius Vuitton knock-offs. Imagine threading your way down a dark, narrow alley lined with flea market stalls and no-health-code/no-insurance open air cart kitchens, filled with people pressing in all directions through dense heat and smell and rot, then send a motorcycle down the alley every few moments to do battle with cross-traffic carts and tuk tuks. Now imagine the alley is a whole network labyrinth covering multiple blocks between several-story tenaments streaked with black soot and stain and hung with drying clothes and black cables. This is why we western pansies stay in the nice hotel with A/C and a pool.

The final capper to the trip was in Bangkok the night before our departure. After the evening rains subsided, my partner and I went down to the pool for a night swim, around 9pm. Refreshing and fun we frolicked and generally soaked up the remaining moments of our stay. Then, in the poolside darkness moving low between the lounge chairs, I saw a large reptilian form lumbering along. “Dude, there’s a fricken alligator coming towards the pool!” I exclaimed excitedly. As it marched into the light we realized it was actually a monitor lizard – Varanus salvator, to be precise – about 5-6ft long with a fattened belly like it just ate a dog or possibly a small European child. “If that thing gets in the water, we get out immediately” I said with some urgency. I knew it could swim and see underwater much better than we could. No reason to tangle with a 6ft thunderlizard in a foreign country with questionable health care. Sure enough the beast slipped into the pool and sidled along the swim-up bar. We hopped out, laughing nervously, and I approached the lizard from a careful distance. Grabbing the pool attendant I motioned towards the monster. “That’s bad”, he said in a way that suggested that, bad as it may be, it wasn’t unusual. And so he casually splashed the creature with water nudging it along until it climbed out of the pool slowly, begrudgingly, made it’s way back into the riverside brush. It was easily the biggest lizard I’d ever seen in the wild.

The final day we were denied pool access during a particularly solid rain. When it’s always 90+ degrees & 90+% humidity, swimming in the rain is quite nice. But no, we were not allowed. “Why?” I protested. “Lightning” retorted the attendant. Fair enough, I thought. Then, in a casual but cautionary aside, the attendant reflected, “We had an accident last year”. This is the Bangkok Riverside Marriott, a fancy if not dated family hotel. Apparently buried somewhere deep in the boilerplate legalese fine print of our hotel contract is the clause, “Marriott Properties takes no liability in the event of any hotel guest or visitor getting suddenly struck by lightning and then slowly eaten by ferocious monitor lizards”.

Themes: interoperability, open source, simulations, visualizations, breaking down the walls, and being stuck with Second Life. Little emphasis on chat and social networking, per se. Much more emphasis on architectures & component solutions.

Trends from Virtual Worlds Roadmap: simulation & training, health care, augmented tourism, mixed-reality museums, live sporting events in VW’s, virtual meetings.

Overview:
While the hype over virtual worlds has faded, many serious researchers continue to do fascinating work in the territory. Monetization of a good VW strategy is still needed but this goal seems to have receded into the future for many of the speakers, as well as many of the enterprise-scale companies investigating these spaces who seem less interested in making money (either through direct development and monetization or by riding the public hypetrain) and more interested in gaining efficiencies and trimming overhead (teleconferencing, remote collaboration).

Google (O3D), Intel (Cable Beach), Sun Microsystems (Project Wonderland), Samsung (Virtual Worlds Roadmap), and Nokia (supporting REalXtend) were all present, as well as many Stanford researchers, including the folks building Sirikata. Many are working to extend the OpenSim fork of the Second Life platform. None of them seem to be working towards direct productization (though Google wants O3D to be the in-browser standard for 3D content) but each were working to advance the platform and explore future possibilities.

With monetization off the table and money drying up, researchers are moving to embrace open source solutions (OpenSim, ScienceSim, Ogre3D) and pushing for open standards (OpenId, OAuth, XMPP) and flexible API’s. Almost everyone mentioned a desire to move away from the proprietary walled-garden approach towards an integrative one that looks to the success of social network strategies. While celebrating open source development of Second Life forks, almost everyone bemoaned being stuck on the platform, often underscoring the feeling with a groan that “there’s nothing else”.

Authoring was rarely addressed with content instead being re-purposed from upstream solutions, eg using 3DSMax & Maya content to build world content. Collada was uniformly mentioned as the exchange format. Most developers still want to shoehorn other modalities (eg PowerPoint, web browsing, document collab, etc) into the VW space. Some examples inadvertantly showed the clunkiness of current solutions. I asked why a technology like PowerPoint is any better in 3D than in 2D, eliciting a long pause from the presenter. There’s still a lot of ambition on the part of developers but not always a ton of common sense.

However, IBM’s manager of service design and service systems research, Susan Stucky, gave me the most reasonable answer I’ve heard yet about why it’s important to move 2D modalities into 3D. She said that for collaborative telepresence it was very helpful to have access to everything you would normally have access to in a meeting. Speaking with her at the break, she told me how IBM has found that the greatest use of their Second Life investment has come from the ability to bring employees and clients from around the world together into a collaborative space. They’ve held conferences, run meetings, and explored simulations of project management strategies. For her, the ROI was gained by telepresence & simulations.

And for me, I had a breakthrough speaking with Susan. One of the most compelling yet least-obvious values of collaboration in virtual worlds is the sense of embodiment conveyed by the presence of the avatar. Identity, social cohesion, team building, and friendship arise more naturally when those engaged are perceived as physically present. Self-awareness and the projection of self onto others is still quite bound to our physical bodies. Perhaps combining the embodiment of avatars with in-world access to knowledge & productivity tools represents a more effective modality for non-local collaboration. I’m not sure how this compares to video teleconferencing but I feel there’s a lot of depth to be explored in how virtual embodiment reinforces social cohesion & collaboration (attn: PhD candidates).

Other notables: Henry Lowood (Stanford Curator of History of Science, Media, & Genetics) speaking on The Ultimate Archive: building virtual museums of virtual world platforms inside virtual worlds (eg a virtual museum with a room that lets you play the first Doom level as it was originally). He noted both “perfect capture” (all the data can be archived) and “perfect loss” (experiences, emotions, and deleted content cannot be captured) in VW archiving. Sheldon Brown (Center for Research in Computing in the Arts, UCSD) showed his mind-bending work Scalable City and called for procedurally deriving world assets and behaviorally deriving world experiences.

Analysis:
Virtual Worlds have lost funding and are presently in the Valley of Hype. Effective monetization strategies have yet to reveal themselves. However, there is value to the enterprise in leveraging virtual worlds for telepresence and collaboration, simulation & training. The VW community is moving the R&D towards openness: open source components, open standards, interoperability, and engaging with the platforms and principles of social networks to enhance connectivity and move away from the Walled Garden. The most interesting work with virtual worlds continues to be in the deeper realms of behavior, psychology, telepresence, and simulations. Graphically, everyone is apparently stuck in Second Life. A smart, well-funded private investor would build a platform with the competitive graphics capabilities (surface mesh, brep, kinematics, HLSL, etc), a powerful and scalable object model that can push to XML/RDF/RSS, a powerful simulation engine with an expressive visualization/analytics front-end, a REST/JSON API capable of talking to agents, tools, and other VW’s (as well as Twitter, Facebook, LinkedIn, SMS, Playstation Network, XBox Network, etc), integrate ActiveX embedding of 2D tools (Office apps, browsers, etc), enable a content marketplace built around highly expressive and personalizable avatars and fetish objects, and cultivate a 3rd part service ecosystem supporting all of the above.

Is this so hard? ;)

I stayed in Las Vegas for a few nights this week to see Jane’s Addiction at The Pearl. A large part of me loathes much of what Vegas is (and by “Vegas” I’m mainly referring to The Strip and its satellites – no offense to the folks who live in the city) yet I can’t help but be mesmerized and amazed at the sheer scale of fantasy on sale there in the wasteland of the Nevada high desert. It is by all accounts an impossible mirage, timeless and ephemeral, drawing in the seekers, fleecing them, and sending them back home like it never existed. Inevitably, it seems it will fall back into the desert as Lake Meade dries up and the drought deepens, leaving behind skeletons of a once mighty empire. Caesar’s Palace may retain it’s name but Nero is the ruler of today’s Vegas.

Anyway, here are my tweets from the trip, in chronological order:

- Heading off to Sin City for glittering nights & saltine days before it all dries up & blows away. #NIN/JA2009 New Aeon Rat Pack 8:55 AM May 17th
- Have successfully played my role as cattle/combatant/customer in SanJoseAirport security theater. Now matriculated to cargo. 10:22 AM May 17th
- Tarmac running to the jetwash mirage of Las Vegas. 12:06 PM May 17th
- Vegas directs its formidable will at constantly maintaining the illusion of plenty. Super Size everything while the desert bides its time… 4:41 PM May 17th
- Vegas, in a nutshell: http://twitpic.com/5ew00 10:21 PM May 17th
- everything about this city is designed to separate me from my money. call me the mark. 12:29 AM May 18th
- Vegas commodifies dreams and the easy score, selling back crumbs at criminal markups, preying on mammon & ruin. 12:02 PM May 18th
- A sign of my age: hoping to trade my #NIN/JA floor tickets for seats. 1:13 PM May 18th
- Little fluffy clouds march relentlessly across the ancient Nevada desert as spacemen floating high above tweet us thermospheric thoughts. 1:45 PM May 18th
- As growth stalls, Vegas withdraws into the strip to focus on sustaining the mirage. The illusion thrives at the expense of the sprawl. 2:35 PM May 18th
- Recent NPR story spoke of tracts of abandoned LV tenaments haunted by erratic chirpings: the sound of fire detectors with dying batteries. 2:40 PM May 18th
- About 2M people inhabit Las Vegas. Nellis AFB brought federal stimulus; the mob & Howard Hughes built The Strip. 5:19 PM May 18th
- Deserts are like seas, vast & deep. In this The Strip is a glowing lure above the gaping maw of a dark desert angler. 5:25 PM May 18th
- You think you’re about to score a nice meal but really you are the prey about to feed something much larger. 5:26 PM May 18th
- got tix sorted. now heading to The Pearl for NIN/JA with @jingleyfish & friends. w00t! 7:28 PM May 18th
- Goddamn i love Jane’s Addiction 12:31 AM May 19th
- crawling the vegas strip with the good dr 1:17 AM May 19th
- new dreams waking with the sun on the fiery vegas strip, raging towards another night 5:49 AM May 19th
- ack. marinating in ambient cigarette smoke on the casino floor. 4:31 PM May 19th
- your trowelled-on cake facade masks the withering age of dessicated bones, too long standing on sore heels to hawk & bark a distant fantasy 4:37 PM May 19th
- Ruminating w/ @jingleyfish about the resource usage profile of the Vegas strip. How much does this desert fantasy consume? Is it a threat? 5:52 PM May 19th
- To paraphrase the bubbling hatery, is Vegas a “cankar needing to be excised”? Carbon tax would likely crush phallic wavings of Wynn et al. 5:56 PM May 19th
- sleepless pineal cascade, flush with endogenous indole, wondering if im really still stuck in this airport 7:27 PM May 19th
- on the ground rolling back to santa cruz. crowd-induced stabby mysanthropy subsiding. actual sleep nigh iminent. 10:24 PM May 19th