Me, in the Kinect point cloud (video still).

my home project: kinect hacking

Over the weekend I bought a Kinect and wired it to my Mac. I’m following the O’Reilly/Make book, Making Things See, by Greg Borenstein (who kindly tells me I’m an amateur machine vision researcher). With the book I’ve set up Processing, a JAVA-lite coding environment, and the open source Kinect libraries, SimpleOpenNI & NITE. I’ve spent a good chunk of the weekend reading a hundred pages into the book and working through the project tutorials and I’ve generated some interesting interactions and imagery. There’s also a ton of tutorial vids on You Tube, natch, to help cut through the weeds and whatnot.

Continue reading


weak signals in 3d print disruption

Peter Thiel’s foundation gives $350,000 to back Modern Meadow’s efforts to create bioprinted meats.

An engineer prints a working lower receiver for an AR-15 using a 15-year old Stratasys printer.

A Glasgow professor modifies a 3D printer to make pharmaceuticals.

More people manipulating matter with cheaper hardware. The rising tide of technology lifts all boats…



“insight, perspicacity, lyrical beauty and complete hogwash

I just have to share this wonderful blurb from Bruce Sterling about my article, Extended Senses & Invisible Fences:

*Y’know, it’s crazy-making to read stuff like this. It’s so full of insight, perspicacity, lyrical beauty and complete hogwash… and there’s no possible way to know which parts are which.

*It’s like a tsunami is coming and someone hands you a witch’s broom.

When I first read this I was struck by how he managed to catch a fairly subtle piece of my character that indeed deeply informs my writing: I aim for no truth, merely sharing my own experience, equal parts analytic and trickster, swept up with some good ol’ mysticism.

Praise from Caesar is praise indeed! Thanks, Bruce!

inviting machines into our bodies – big think

I have a new article over at Big Think looking at trends in wireless implant technology and the vulnerability profile presented by our emerging integration with connected biodevices. This article builds on my previous post here, Ubicomp Getting Under Your Skin? So Are Hackers.

From the intro:

In what amounts to a fairly shocking reminder of how quickly our technologies are advancing and how deeply our lives are being woven with networked computation, security researchers have recently reported successes in remotely compromising and controlling two different medical implant devices. Such implanted devices are becoming more and more common, implemented with wireless communications both across components and outward to monitors that allow doctors to non-invasively make changes to their settings. Until only recently, this technology was mostly confined to advanced labs but it is now moving steadily into our bodies. As these procedures become more common, researchers are now considering the security implications of wiring human anatomy directly into the web of ubiquitous computation and networked communications.


New Aesthetic Filmmaking – Clouds: DSLR + Kinect + RGBD Toolkit

The work of James George caught my attention when he began publishing still images generated by mixing inputs from a DSLR camera paired with a Kinect scanner. He & partner, Jonathan Mind, recently produced this thoroughly compelling future-now video from the same process, using their open framework software, RGBD Toolkit, to manage the mapping and in-scene navigation. The camera is fixed but since the Kinect produces a 3D scene you can navigate around the captured image. Where forms in the camera field cast shadows, i.e. where the Kinect cannot scan past e.g. an occluding arm & hand, you see stretching and warping of the 3D mesh and image map. The effect is uncannily similar to the scenes in the film version of Minority Report when Tom Cruise’s character watches holovids of his son & wife, their forms trailing along the light path of the holoprojector.George & Mind frame this video as an exploration of emerging techniques and technologies in filmmaking. Also, they talk about coding and geekery and other cool stuff.

Clouds: beta from DEEPSPEED media on Vimeo.

Clouds is a computational documentary featuring hackers and media artists in dialogue about code, culture and the future of visualization.

This is a preview of a feature length production to be released later this year.

By Jonathan Minard ( and James George (
Made with

The State of Augmented Reality – ARE2012

Last week I attended and spoke at the Wednesday session of ARE2012, the SF Bay Area’s largest conference on augmented reality. This is the 3rd year of the conference and both the maturity of the industry and the cooling of the hype were evident. Attendance was lower than previous years, content was more focused on advertising & marketing examples, and there was a notable absence of platinum sponsors and top-tier enterprise attendees. On the surface this could be read as a general decline of the field but this is not the case.

A few things are happening to ferry augmented reality across the Trough of Disillusionment. This year there were more headset manufacturer’s than ever before. The need for AR to go hand’s-free is becoming more & more evident [my biases]. I saw a handful of new manufacturers I’d never even heard of before. And there they were with fully-functional hardware rendering annotations on transparent surfaces. In order for AR to move from content to utility it has to drive hardware development into HUD’s. Google see’s this as does any other enterprise player in the mobile game. Many of the forward-looking discussions effectively assume a head’s-up experience.

At the algorithmic level, things are moving quickly especially in the domain of edge detection, face tracking, and registration. I saw some really exceptional mapping that overlaid masks on people’s faces in realtime responding to movement & expressions without flickers or registration errors (except for the occasional super-cool New Aesthetic glitch when the map blurred off the user’s face if they moved too quickly). Machine vision is advancing at a strong pace and there was an ongoing thread throughout the conference about the challenges the broader industry faces in moving facial recognition technology into the mobile stack. It’s already there and works but the ethical and civil liberty issues are forcing a welcomes pause in consideration.

Qualcomm was the sole platinum sponsor, promoting its Vuforia AR platform. Sony had a booth showing some AR games (Pong!?) on their Playstation Vita device. But pretty much everyone in the enterprise tier stayed home, back in the labs and product meetings and design reviews, slowly & steadily moving AR into their respective feature stacks. Nokia is doing this, Google of course, Apple has been opening up the camera stream and patenting eyewear, HP is looking at using AR with Autonomy, even Pioneer has a Cyber Navi AR GPS solution. The same players that were underwriting AR conferences in exchange for marketing opportunities and the chance to poach young developers are now integrating the core AR stack into their platforms. This is both good & bad for the industry: good because it will drive standardization and put a lot of money behind innovation; bad because it will rock the world of the Metaio’s & Layar’s who have been tilling this field for years. Typically, as a young technology starts to gain traction and establish value, there follows a great period of consolidation as the big fish eat the little ones. Some succeed, many fall, and a new culture of creators emerges to develop for the winners.

So here we are. Augmented reality is flowing in three streams: Content and marketing grab eyeballs and easy money while conditioning the market to expect these experiences; developers extend the software stack towards real-time pixel-perfect recognition & mapping, enabling the solutions to actually, um, solve problems; and hardware manufacturers labor to bring AR into the many transparent surfaces through which we interact with the world, freeing our hands and augmenting our views with ubiquitous networked data. Across these domains sit content shops, emerging start-ups, the leading innovators ala Layar & Metaio, and the big fish enterprise companies that have had a piece of the tech game for years & years and aren’t going to miss out if AR goes supernova. The market is a bit shaky and very much uncertain for the SMB’s but it’s certainly maturing with the technology.

My sense is that everybody gets that AR is here to stay and has deep intrinsic value to the future of mobility and interface. How this will impact the many passionate folks curating & cultivating the field from the bottom-up remains to be seen.

Extended Senses & Invisible Fences – ARE2012

I recently gave a talk at ARE2012 about emerging interactions in the networked city. It’s a broad overview of ubicomp and how it is modulating our experience of ourselves, each other, and our environment. I’ll be writing a follow-up article with more info.

The Church of Perpetual Reconfiguration

“What??” he yelled, in near shock. “You’ve got to be kidding me. Did you talk to the contractor?”

A short pause.

“Ok. Ok. Stop. What do you mean he’s not engaged? He’s a civ! He has to be!” This was going sideways fast.

“I’m calling my attorney.”

He de-coupled from the stemlink and rubbed his temples, struggling to push through the numbness now wrapping itself around his head. He closed his eyes, groping for the sense of it all. Then with a fast spin he flicked his stem and coupled to his attorney, Ersatz Bolzman.

“Satz. This is Bentley Statmaker. I’ve got a problem.”

Bentley proceeded to share with Bolzman the unsettling news that his frenetic & frazzled assistant had just conveyed: how the Biomoss got corrupted and was presently deviating wildly from the 19th century Craftsman remodel he had ordered and god-damned paid for and was now 2 wings and several buttresses into printing out a 13th century gothic cathedral. His home, a modest 3 bedroom on the South Side in dire need of a significant retrofit and well out of style to boot, was expanding at a nonlinear rate – so much so that the houses of his immediate neighbors were now food for its architectural extrusions. Possibly worse, the adjacent street was being overtaken by an aggressively-compiling abbey.

“What…the fuck, Satz?? What the hell am I supposed to do? The neighbors are on me, the city has sent a swarm of inspection flies – they’re mapping the whole deal, blaring code violations non-stop. I can only imagine the stacks at the planning department spinning wildly with dollar signs in their beady little machine eyes…” He trailed off, unsure of his course, whether he should strain to feed the hungry anger or just give in and veer downward into an impotent malaise. He knew there was nothing he could do but prepare for the imminent onslaught of legal battles, the deep apologies to the neighbors, the pay-offs to his HOA… God, that bastard Malamut would have him run out of the co-op for sure this time.

“Calm down, Bent. I’ll file for a writ of suspension, tie up the city works a bit. Pretty sure I can get those inspectors recalled to central too. And it’s not just happening to you, ya know. A whole condo development on 4th & Jobs just went Gaudi. Was supposed to be a nice post-decline piece, all glass and bioplastics, but the mossers crapped out and now it’s just a crazy riot of spires and ceramic tiles and weird biomorphic flourishes. Pretty much totally wiped out the high school across the street. Apparently, some agitprop collective of biomods hacked into the genome. Oh, and that new wing of the Hu Jintao Child Workers Memorial Colliseum..? Went from Classical Ming to Retro Koolhaas in the blink of an eye. The Imperial ruler is pissed,” he emphasized sourly. “You think you got legal troubles? Lemme tell ya…”

Talking with Ersatz calmed him down a bit. At least enough to cover the gap until the neurografts did their thing and started metabolizing the flood of endorphins & cortisol washing through his system. Then, with an almost audible sigh, the grafts began to release a steady drip of Relaxipam into his brain. His affect dulled, the tension of his musculature eased, and he was left with a vague longing for simpler times that probably never really existed but for the halcyon filter of nostalgia.

Midway into the 21st century the industrial labs toiling away for decades on nanotech started to merge into the university labs working on procedural construction, and that whole mess got mashed up with the Asian street gangs and neurocartels delving deeply into the forbidden wizardry of biosynthesis. The end result was a pretty wild chimera of hybridized designer nanomanufacturing: Biomoss. The stuff was the same thing as fire, metal, steam, electricity, hydrocarbons, and atomic physics. Or rather, it was the result of the same quest that drove humanity to harness each of those divine elements of nature and bend them to its will. Biomoss was both the penultimate subjugation and celebration of life itself. It was a means to not only design life but to program it to construct anything we could think of.

We made fuel first, of course. Algae mats that grew to cover most of Asia, expressing sweet crude like shiny black dew. Then we had to engineer vast carbon sinks and tar cleaners – and universal rice that could grow anywhere now that all the Eastern paddies were gone. That was a bit of a detour. Everyone started making all kinds of foods, natural and bizarre. The Southern Federalist Nation of God practically exploded with diabetes until a biomod made an insulin virus. Then they all started collapsing from hypoglycemia. Humans, of course, are pretty great at ideation, not so great at implementation.

The next big step moved beyond simple protein expression. This phase was about building and mass-producing living constructors that could take instruction sets and extrude things. All kinds of things. You could flash a constructor gel, give it a slab of substrate, and watch it build a cup, a toaster, a mobile, even a stemlink. Then they started making vehicles, buildings, bridges, factories. With increasing scale, food for the constructors became a limiting factor.

Initially, substrate was basically a slab of hydrocarbons bound to a crystalline lattice and studded with various functional groups. Esters, aromatics, polymers, and biomolecules. The ingredients of your standard organic stew. This posed problems. There was concern that the constructor gels might start eating up the local flora & fauna. More importantly to the benefactors of the massive public & private grants funding this work, constructing organics only gets you so far. They wanted to build product and devices, machines and monoliths. They needed inorganics. The constructors would be modified to chew through minerals, metals, lanthanides, and solid state compounds, then recompile and extrude them in forms. This also posed problems.

In 2047 a federal appeals court ruled against Pratt Lapman Assuary, Inc. in a class action lawsuit filed by the People’s Republic of Colorado. The plaintiff’s argued that due to gross neglect and moral corruption Pratt Lapman Assuary Inc. did knowingly and willingly administer a swarm of replicating bio-constructors outside the town of Gunniston, Colorado for the sole and express purpose of recompiling the region’s vast titanium deposits into a 3000 foot tall statue of the company’s founder, Chapman Assuary. As the constructors extracted and processed the resource, replicating in multitudes to match the scale of the work, it was only by a programmatic glitch in the genetics that the scalar vector botched and the replicators came to a halt. To this day there remains a 2000 foot headless titanium statue of Assuary, guarded by judicial writ, it’s shiny figure being slowly decomposed by the local, court-appointed bioscrubbers, and sold off bit by bit for the benefit of the Gunniston School District. Despite the loud & irate sputterings & protestations erupting from Pratt Lapman Assuary Inc. the World Court refused to review the case.

By 2050 international conventions were in place forbidding unrestrained nano-bio replication. This pretty much stopped conventional use of constructor gels for several years until epigenetic containment switches and network ID’s were engineered into the germ lines in 2057. The first successful line to emerge from the new regulatory environment became the prototype for Biomoss.

Biomoss is notable in that it is not really a moss, per se. In fact, by microscopy it appears more solid and structured like a mass of very tiny lady bugs covered in salt crystals. It gets its name from both the viridian hue of the individual’s carapace and the way it masses and accretes across surfaces. Furthermore, unlike constructor gels, it does not require a specific pre-configured substrate or uniform material base. Instead, Biomoss will digest anything that it’s targeted for and then recompose the digested mass into a product of similar composition. This adds additional containment to its capabilities by limiting its output to roughly the same as its input. Roughly. But the morphology of the thing is inconsequential. Or rather, the output is customizable as long as it stays within the bounds of the source.

Once Biomoss was productized, innumerable design shops spun up offering precision bespoke manufacturing, customized & stylized by in-house computects. A whole industry grew around the artistry of designed bio-extrusion, resurrecting the styles of historic starchitects, collaging & remixing the canon of industrial design, and making international figures of the leading composers tubing about the globe to paint their visions in transcendent and impossibly expressive mega-structures.

These facts were little comfort to Bentley Statmaker.

After a quick hop across the tube line and a relay along the ped tiles, Bentley Statmaker arrived at his home. Which is to say, Bentley Statmaker arrived at his shiny new gothic cathedral. It was enormous, dark & brooding, hung from the heavens themselves and, frankly, beautiful. But not in this neighborhood. And high holiness or not, historical re-creation be damned, Malamut and the board would not approve. No. They’d more likely have him drawn & quartered, strung up limp and emptied from the highest spire.

Pushing through the noisy crowd of angry neighbors, drooling gawkers, flashing newsbots, countless projectagrams, and a gathering collective of syncretic worshippers, Bentley heaved open the tall, wooden cathedral doors, stared blankly for a moment at the vast, arching, vaulted nave that used to be his tired living room, and then proceeded defiantly inward. He slammed the great doors behind him with a deep boom and thoughtlessly, aimlessly, walked down the aisle taking his seat along a bland and stern pew. Eschewing formality, he let himself slack, took out a cigarette, lit it after a few tries, and inhaled deeply. He immediately sputtered and coughed before defiantly drawing a second drag of the blue smoke, as if to authoritatively subdue the instinctual protestations of his corpus. In the sudden silence he could just apprehend the delicate crunching and printing of the Biomoss, like the sound of soft snow settling on an empty field.

Suddenly and without due respect to his sanctuary there was a great pounding on the door. Amidst the muted yelling of Malamut he made out the decree of Bentley’s imminent eviction. He imagined Malamut as a tired & troubled Martin Luther nailing up his Ninety-Five Theses on the door of the Castle Church. Here he sat, Pope Bentley the First, grinning slyly while billions of micro-miniaturized bio-machines, seized by some unknowable possession of Spirit, labored towards the monumental construction of this, his great abbey. He keyed his stemlink and coupled to Bolzman.

“Satz. Bentley. Listen… I think I’m gonna keep the church. Yeah, seriously. Hey, can you look up precedent for religious protection? And maybe seizure & eminent domain. I think there was something with the New People’s Temple of Jerseytown maybe ten years ago…”

He took another long drag, holding in the hot smoke as the evening light began to shine through the stained windows of the celestory. Bold indigos and feverish carnelians, glowing laurels and immaculate ambers, all caught fire with the Stations of the Cross surrounding the transept. If he let himself go for a moment, he swore he could hear a great and distant chorus rising up, washing away the pages of time with the enduring hope of the sangreal, and filling the nave with a resounding affirmation of his Grand Papacy.

“Oh, and Satz..? I think I’m gonna need a priest.”

What I’m Up To

I haven’t been blogging much lately and, in lieu of an actual article, here’s an overview of where my time & energy have been…

I’ve started a new job in an engineering group at a Fortune 500 company that’s delving into personal cloud deployment with multi-point device access. It’s pretty interesting tech moving into the same space as the Apple home media ecosystem but with a different angle on ownership. The idea is that you control your content & file management rather than trusting a 3rd party to hold it on some remote server. The implementation is pretty nice and the experience is good for such a young product line. While I don’t really have the job description I want, the organization is pretty loose and I’m following the “great employee guideline” of not being defined by my role. Of note, the commute adds 2 hours on top of an 8-hour day so it’s been a bit tricky to get used to the new schedule and the attendant physical overhead.

I’m also collaborating with the Hybrid Reality Institute run by Parag & Ayesha Khanna to contribute research and help grow client opportunities. This is a part-time volunteer gig at the moment but may dovetail with my consulting work at some point in the future. I’ve admired Parag’s efforts since reading his book, The Second World, and have been excitedly following his & Ayesha’s work growing HRI. I recently had the chance to chat with Parag for the first time and was very impressed with his friendliness and the incredibly dynamic life he & Ayesha lead. For my part, I’ll be looking at the broad & somewhat over-trod area of smart cities and urban development. It’s a cool challenge for me to map & articulate such complex systems; to integrate my interests in mobility, social structures, embedded systems, augmented reality, and CAD/BIM architecture; and to tease out hopefully novel approaches to understanding urban dynamics and bending development towards greater efficiency, equity, and sustainability.

On the side, I’ve got a new song I’ve just sent out for final mix & mastering. It’s a chunky hip hop beat with a big dubby flavor and some nice melodic elements. The whole track is built around this old recording of a “rasta elder” speaking on a radio station in, like, 1978. I’ve chopped it up and pulled out a couple of the most compelling bits. I love dub & reggae and have a fondness for rastafari culture in general so this song is really a realization of marinating in this stuff for some time. Particularly in the last 2 years I’ve been deep in the Dub Chamber trying to reverse engineer the dub reggae sound from a large archive of music produced by Studio One, Lee Perry, Trojan Records, King Tubby etc…. In the past few months it got mashed into my hip hop efforts with the result being this song, Man Crab. I’ll hopefully be publishing and promoting this track within the next month. More info as it proceeds but I’m working with a great engineer and can’t wait to hear the final result!

Finally, I’m talking with a German film maker who recently returned from Caracas, Venezuela. He approached me last year after finding my Sathorn Unique project and asked if I’d be interested in doing some soundtrack work for his documentary about La Torre de David, a 45-story abandoned skyscraper now home to literally thousands of squatters. He returned with a bunch of media to compile the documentary. I’ll be plundering the audio files for stuff to work into & inform the music. So, I may be carving out a very rarified niche as a producer who writes soundtracks for weird abandoned skyscrapers. :)

Anyway, I’ve got a lot of other stuff kicking around in the ol’ mind tank that will hopefully congeal into some coherent articles in the near future. In the mean time, thanks for reading!

GRASP, General Robotics, & Swarming Drones

This truly fantastic video went around the interwebs last week, featuring the work of U Penn robotics lab, GRASP. Notably, the video shows groups of small quadrotors flying in formation, following paths, and generally exhibiting both autonomy and collective behavior.

Flush with defense moneys, the GRASP team are doing some pretty amazing work. A survey of their current projects could be the basis for hundreds of scifi novels yet many of them are right on the edge of reality. Research includes:

Haptography – haptic photography
Reasoning in reduced information space
HUNT – heterogenous unmanned networked teams
Omni-directional vision
SUBTLE – situation understanding bot through language & environment
Modular robotics lab
SWARMS – scalable swarms of autonomous robots and mobile sensors

Peep their published lab papers for even deeper FutureNow goodness, eg “Multi-vehicle path planning in dynamically changing environments”.

Machine Aesthetics Video – Robot Readable World

BERG creative director, Timo Arnall, has published a video collecting “found machine-vision footage”. In his words:

How do robots see the world? How do they gather meaning from our streets, cities, media and from us? This is an experiment in found machine-vision footage, exploring the aesthetics of the robot eye.

I think it gets particularly poignant about 4 minutes in when the face tracking & recognition alphas make human TV hosts into odd, simplified charicatures, at once de-humanizing the hosts while betraying the limited sophistication of machines like children trying to capture the world in colorful crayons. Bonus points for the creeping irony of machines learning about humans through TV.

Robot readable world from Timo on Vimeo.

Paddy Ashdown: The Global Power Shift (TED)

A great talk on the shifting world by a distinguished and engaging speaker, Paddy Ashdown.

“I believe we are condemned, if you like, to live at just one of those moments in history when the gimbals upon which the established orders of power is beginning to change and the new look of the world, the new powers that exist in the world, are beginning to take form. These are nearly always highly turbulent times.”

Discontinuity & Opportunity in a Hyper-Connected World

[This paper was originally published for a government report on discontinuity & change management.]

We live in a time of large-scale, non-linear change driven by the twin engines of globalization and hyper-connectivity. Change is, of course, constant but we now have such extreme visibility into the farthest corners of the world that the amplitude of change appears much greater than ever before. Many of us are, for the first time, globally connected and wired to real-time data streams that carry information and emotion across the world instantaneously. When we look through this lens of hypermedia we are confronted by fast-moving, asymmetric complexity that seems to be slipping out of control. The landscape is moving more quickly than we are able to respond. This is deeply challenging to our sense of security.

As Americans, we face a highly multipolar world. We feel the decline of U.S. exceptionalism and the attendant existential crisis of this realization; the ongoing global financial malaise and the emerging debt crisis threatening to break apart the European Union; the rise of China as a dominant world power and the implicit criticism of democracy that comes from its economic success; and the evolution of Islam as an explicit criticism of western prosperity. We are realizing the massive power of finance & energy cartels while struggling with ultraviolent drug cartels. We feel the impacts of domestic unemployment amidst weekly reports of record corporate profits. Capital is moving away from mature western markets for the young labor pools of the developing world. Fund managers are betting more on decline than investing in growth. There is a growing sense that western governance is failing in its charter to effectively manage the prosperity & security of its citizenry, and that selfishness, partisanship, and corruption have undermined the political process.

In the United States there is arguably a crisis of confidence in governance. We face extreme partisanship among policy makers and their apparent inability to effectively govern on domestic issues. Congress has a 20% approval rating. 73% of Americans believe the country is moving in the wrong direction. On domestic issues, the popular narrative of U.S. governance is one of bickering, incompetence, and failure.

So if there is a crisis of confidence, is there an actual crisis in governance? Recently the debt Supercommittee failed to agree on a solution for the deficit. This past July, the largely-manufactured budgetary impasse shook confidence in U.S. governance contributing directly to the S&P downgrade of our hallowed AAA credit rating. To quote the S&P report, the downgrade “reflects our view that the effectiveness, stability, and predictability of American policymaking and political institutions have weakened at a time of ongoing fiscal and economic challenges”. Even closer to home, the American Society of Civil Engineers recently reviewed U.S. infrastructure with a grade of “D” stating that it would take $2.2 trillion over the next 5 years to bring our roads, bridges, railways, water and energy systems, and waste treatment capacity up to 1st world standards. These are the fundamental needs required to keep a country functional & efficient.

Looking at recent statistics, the U.S. Commerce Department charts wages & salaries at only 44% of GDP – the lowest since 1929. Corporate profits, on the other hand, now contribute 10% of GDP – the highest on record since that auspicious year, 1929. The U.S. Bureau of Labor Statistics estimates unemployment at 9% though real measures of unemployment that include the under-employed and those who have given up looking for work are estimated closer to 16%. Among young adults age 16-24, 50% are without work – the highest number on record since 1948. The majority of unemployed no longer receive state benefits. Tens of thousands of service members are returning to joblessness & homelessness. The 2010 U.S. Census Bureau estimates that 46 million people are living in poverty – 15% of the nation. This number has been increasing annually for the past 3 years. These trends are undermining the legitimacy of the US government both at home and abroad, and contributing to the social unrest sensationally illustrated by the rise of both the Tea Party and Occupy Wall Street movements.

Typically, when we observe these statistical trends in other countries we see a growing segment of the populace more exposed to gang indoctrination, co-option by religious fundamentalism, and coercion by home-brewed militias. This unfortunate reality is not lost on policy makers, as telegraphed most recently by Congressional attempts to reconfigure the legislative landscape of the Homeland as a domestic battleground.

While national statistics are indeed worrisome, the situation at the local & regional level is more varied and offers some hope. There is a shift towards state’s rights as illustrated by the more libertarian aspects of the Tea Party and the GOP narrative against so-called big government, but also in many state legislatures on both sides of the aisle. While often ideologically driven, this shift towards state governance is a response to the limitations of central management across such a large and complex territory as the United States. Perhaps more interestingly, we see a shift to municipal power as urban populations swell and major cities take ownership of their roles as economic engines. Mayors are gathering more influence over state and federal policy, and are making more lucrative partnerships with global allies.

Yet, there are huge budgetary challenges for both states and municipalities, with states often pushing their own budgetary problems down to the county & city level. There is even talk of an emerging municipal debt bubble as cities issue more bond debt to cover their existing debt costs. The U.S. just witnessed the largest municipal bankruptcy in history when Jefferson County, Alabama, failed to cover its sewage bonds. This is the downward cycle of U.S. infrastructure & budgetary mis-management laid bare.

The picture of local and regional governance is a patchwork of attempts (successes and failures) to address the many challenges confronting us locally and handed down from state and federal institutions. As higher-order governors lose legitimacy, states & regions will work to sidestep their authority and to innovate around budgetary shortfalls and non-local obstacles. Progressive states agitate for marijuana legalization and same-sex marriage, conservative states assail big government and immigration, southwestern border states are dealing with the spill-over from Mexico’s narcowar, and many regions across the country are absorbing diverse and extreme climate impacts potentially driving food production, water supplies, and population movements. So while large, productive cities are generally seeing more cohesion there is a significant risk of increased balkanization across regions and states.

U.S. governance is clearly challenged on many domestic fronts. In operational terms, we’re falling short. Governing institutions are too big and too slow to respond to such accelerated change. If we’re failing to manage the present, how can we prepare for the future? There is too much complexity to effectively predict change and yet there’s too much institutional friction to adequately invest in broad resilience. This combination poses tremendous risks to domestic security. The snapshot of social unrest in America arises from two primary drivers: the fear of U.S. decline and the sense that Democracy is no longer working (represented by the Tea Party and OWS movements, respectively). Both are rooted in a lack of jobs, diminishing access to prosperity, and growing insecurity in the face of poorly managed discontinuities. When government fails to meet it’s charter, it loses legitimacy. When conventional channels for change are closed, the gap widens between governors and the governed.

For better and for worse, a lot of innovation happens in the gaps. There is innovation in governance itself, as in the Gov 2.0 & OpenGov initiatives to standardize operational data across organizations, to publicize the data, and to invite the public to work with the data and develop 3rp party applications. Deputizing the crowd to help with governance can offer tremendous opportunities for innovation, as exemplified by tools such as Oakland Crimespotting and the Everyblock platform. The citizenry is becoming more digital and addressable with direct polling, crowdsourcing, and experiments in electronic voting. Transparency initiatives, such as the Sunlight Foundation, build web platforms to track and reveal the influence of money in politics. The growth in mobile/social/location platforms empowers tremendous opportunities in civic innovation, as does the emergence of embedded instrumentation in the built environment. Tech collectives and hacker spaces, experiments in local and digital currencies, slow food and Buy Local movements, increased community volunteerism and more public-private partnerships – all of these examples build local resilience and enable communities to take care of themselves.

Many of these efforts follow open source models that enable fast innovation and iteration across diverse non-local nodes, avoiding hierarchies and direct leadership in favor of feedback loops and emergent self-governance. These models gained popularity with the open source software movement but have since expanded to include innovation in open hardware and fabrication, science and robotics, economics (there is an estimated $10 trillion informal economy growing in the gaps globally), and political movements. Open source templates have enabled new models of power such as Occupy Wall Street and Anonymous, many aspects of the Iraqi insurgency, and the dangerous ecosystem of adaptation and innovation found in the IED marketplaces of Iraq and Afghanistan. The ability to maintain such open source models of organization has been radically empowered by mobile telephony, SMS, and social media. The ability to globally broadcast, communicate and collaborate has enabled a new breed of citizen reporting pushed out through platforms like You Tube and Twitter. Rapid SMS communication across mobile devices enables fast stigmergic coordination that can mobilize people en masses with a moment’s notice. The Green Revolution in Tehran, the Arab Spring, and the periodic support calls sent out by OWS groups are all examples of how borderless, frictionless hyper-connectivity empowers a patchwork of active tribes, locally and virtually.

Gaps in governance empower innovators and competitors alike. Actors exploit the gaps and seek to influence or undermine governance in order to open more gaps. Super-empowered individuals like Bill Gates and Eric Schmidt work to influence conventional channels of policy-making while restructuring the regulatory landscape to better enable their businesses. Activist billionaires like Warren Buffet, George Soros, and Sir Richard Branson use their weight and influence to change world affairs, as do libertarians like Peter Thiel and anarcho-capitalists like the Koch brothers. Some super-empowered actors are feral and may not appear to be powerful yet manage to inflict exceptional discontinuities on their targets. Arms dealer, Victor Bout, has been a significant driver of unrest in Africa. The head of the Sinaloan cartel, Joaquin Guzman, has helped deconstruct Mexican governance into a lawless war zone. Henry Okah, the leader of MEND in Nigeria, used a small group of lo-tech saboteurs to target critical pipeline infrastructure reducing crude output by 50% and costing western oil interests billions in production revenue. Cartels and criminal networks operate on international scales moving billions of dollars to influence authorities and outwit enforcers. Tech-enabled sociopolitical collectives like Anonymous and Wikileaks deputize themselves as moral enforcers, exposing secret agendas and arbitrating punishment. These actors walk the same stage as multinational corporations and NGO’s that have no built-in allegiance to the United States or, in some cases, to democracy itself. All of these actors exert their will on the world by building influence and exploiting the gaps. All of them are empowered by hyper-connectivity and cheap computation to coordinate, collaborate, and influence at all scales.

This is an age of hypermedia and hyper-politics. There are almost 3 billion internet users, globally. There are over 5 billion mobile subscribers – this is 77% percent of humanity. Last year, in 2010, over 6.9 trillion text messages were sent & received. Humanity has global, instantaneous communication; immediate amplification of emotion, ideology, witnessing, discovery, innovation, and iteration. We are sharing what works and what doesn’t in all domains and endeavors. Everyone is being lifted by this rising technological tide. Small-scale power is amplifying exponentially through ubiquitous computation and mobile communication. Power is re-distributing across the globalized, hyper-connected landscape in such a way that a small, minimally-funded group can generate exponential disruptions. In a mediated world, we see a new war of narratives competing for mindshare across hypermedia, cultivating borderless affinities and ideologies, and offering a global voice to disenfranchised and exploited groups. Top-down governance, unable to extend control so far over such large-scale discontinuities, is yielding space to flattened hierarchies and self-governance. All institutions are being forced to evolve and adapt to this new landscape, as all efforts to suppress it will inevitably fail and only drive more turbulence.

Complexity is an expression of information, and hypermedia is a complexity feedback loop of revealing, sharing, and iterating. Hypermedia, in all it’s varied forms, is injecting unprecedented amounts of information into our awareness. This widening perception of complexity drives behavioral uncertainty as people and institutions feel increasingly overwhelmed and lost in the noise. The world wide web has driven massive discontinuities into almost every business model, organization, and political objective. Mobile telephony coupled to social networks has given voice to the real-time status of the majority of people on the planet. In this maelstrom of asymmetrical disruption, chaos appears to be the new norm though this will likely reveal itself to be the turmoil attending a broad shift towards a new order of stability.

Complex systems across many scales have moved into a late conservation phase and are beginning to release their organizational capacity. Legacy institutions have grown far too optimized and narrow to absorb the turbulence unleashed by globalization, ubicomp, and mobile telephony. Systems have destabilized in order to make the phase change into whatever next basin of stability awaits. Governance is necessarily challenged and states will inevitably give some degree of power & influence as capital flows out of the West; as more empowered actors take the global stage; as non-local relationships shift affiliation and allegiance; as borders are antiquated by the internet and the cell phone; and as over-extended unions fracture and balkanize. Centralized control structures are not adequate to manage such large scales of nested and inter-dependent complex adaptive systems. But fortunately, the same drivers that have introduced so much discontinuity and have challenged governance as we know it are helping construct the new forms of distributed, participatory governance. Hyper-connectivity, hyper-visibility, and hyper-empowerment are driving a global peer review of legacy institutions in a patchwork attempt to define Civilization 2.0. The process is turbulent and the future is cloudy but we’ll likely land on solid ground eventually.

A Few More Notes on Machine Aesthetics

Olympus glitch, from Year of the Glitch

Scott Smith has a nice article about Our Complicated Love-Hate Relationship With Robots, exploring how robotics have been seeping into the public dialog of late. A couple of the links he cites were good reminders of previous work looking at the aesthetics of machine perception, notably Sensor-Vernacular from the fine folks at BERG and The New Aesthetic Tumblr by James Bridle.

If humanity is a reflection on the experience of perceiving and interacting with the world, what role does machine perception play in this experience? And if nature acts through our hands, to what ends are flocking drones and herds of autonomous machines? A taxonomy of machine perception seems necessary to understand the many ways in which the world can be experienced.

New Aesthetics of Machine Vision

I’ve grown fascinated by the technology of machine vision, but even more so with the haunting aesthetics captured through their eyes. There’s something deeply enthralling and existentially disruptive about the emergence of autonomous machines into our shared world, watching us, learning about us, and inevitably interacting with each other. It’s like a new inorganic branch of taxonomy is evolving into being. Anyway, two recent notes on this topic…

The first is this short series of images taken from a UAV drone and featured in the ACLU report, Protecting Privacy From Aerial Surveillance [PDF]. There’s a decent summary of the report at the New York Times.

Makes me think of Ian McDonald’s excellent novel, Brasyl, and the ad hoc indoctrination of Our Lady of Perpetual Surveillance into the extended canon of casual Orishas.

The second item of note is this haunting video of a 3D Scanner wandering the streets of Barcelona. It’s not any sort of smart machine – it’s just a dumb handheld scanner hitching a ride on a creative human – but it again evokes the aesthetic of a world seen through eyes very different from our own. The video really grabs me about a minute in:

alley posts from James George on Vimeo.

It seems to show a bizarre ghost world or a glimpse from another dimension into ours. The aesthetic (and the tech) is similar to LIDAR, which I had the luck to play around with a couple years ago – and which Radiohead employed to a very interesting end:

In some ways, I want to see these visions as analogous to the view through a wolf’s eyes in the 80′s flick, Wolfen (at 0:24 in this trailer):

Seeing through the eyes of machines isn’t especially new but it’s the awareness of the many adjacent, convergent technologies of pattern recognition, data analysis, biometrics, autonomous navigation, swarming algorithms, and AI that adds pressure to the long-held notion that machines might someday walk our world of their own accord. It seems much closer than ever before so it’s fascinating to watch the new aesthetics of machine vision move into the popular domain.