Note: This post has been updated with images of the ISIS system taken from the patent application posted here.

Note: story updated below.

Much has been made over the past week of the former Abraxas surveillance company Trapwire, yet we somehow still don’t know very much about the company’s technological powers.

There’s been plenty of speculation floating around alleging that its system is more powerful than face recognition, but there’s no evidence to prove this beyond comments made in private by intelligence firm Stratfor’s Fred Burton — who, as Wired points out, has engaged in some seriously sleazy backdoor dealing to hook Trapwire up with as many contracts as possible because his firm takes a cut.

We would do well not to trust the used car salesman about the quality of his product. So we frankly don’t know what the company is capable of.

Luckily for us, or unluckily, we don’t need to bother with speculation about whether or not Trapwire’s system can see inside my house from the camera across the street or cannot, or can connect my travel patterns to my face print to my credit card history or not. 

We already know, as my colleague Jay Stanley wrote yesterday, that “we are barreling full speed toward a surveillance society.” (I’d take that statement one step further by arguing, as I have elsewhere on this blog, that in the most important respects (i.e. technology and the rule of law) we are already there.)

The “on the road” or “already arrived” quibble notwithstanding, the point remains: Even if Trapwire isn’t all its promoters make it out to be, we have abundant reasons to be concerned about what similar, already existing technologies enable. And we don’t have to speculate about them, at all.

All the talk of Trapwire’s systems got me interested in looking into cutting-edge surveillance technologies related to networked video monitoring and analytics, to see where the research stands today. The following is just one example among many truly powerful and Orwellian-sounding schemes currently in the works. (Note: If you are interested in digging around in this stuff for yourself, visit the US Patent and Trademark Office website. The patent side is flush with creepy spook technology.)

“Imaging Systems for Immersive Surveillance”

The closest thing I found in the patent documents to the total surveillance system people (possibly wrongly) attribute to Trapwire is called “IMAGING SYSTEMS AND METHODS FOR IMMERSIVE SURVEILLANCE” (ISIS). 

Designed by a engineers at the MIT Lincoln Labs in Massachusetts for the Air Force, the technology aims to create a 360 degree surveillance field of vision, enabling camera operators to zoom in on someone’s face to ID them using face recognition tools without losing sight of the bigger picture around them. The system is already in place at Logan airport in Boston, MA, part of a DHS pilot program called “Wide Area Surveillance.”

Typical surveillance systems require agencies staff multiple people to monitor multiple camera feeds to allow them to see the full picture, but the “ISIS” system has a fix for that: 

…ISIS combines imagery from many cameras into a single, continuous image that appears to the end user as if it were a from a single lens-imager combination. As a result, ISIS enables a single user to monitor an entire wide-area scene–the number of users does not scale with the number of cameras (or the camera resolution).

The patent application says that the tool is able to detect someone’s face for identification at up to 200 meters.

But the system the scientists have designed isn’t just a bunch of cameras enabling a viewer to see in all directions and up close and far away, at the same time and in one image. It’s also a monitoring suite that gives the operator the ability to automatically track targets, ID people using face recognition, correct for color and low light, automatically store recordings of sought-after movements or other patterns (possibly including faces) in databases, and execute a wide range of other video analytics processes.

The goal appears to have been to cram all of these technical feats into one tool, to make it relatively cheap and available off the shelf for not just military clients but a range of government and non-government actors. The inventors see their product as a total surveillance suite:

An exemplary ISIS system may also provide a memory, or data storage, solution that is capable of handling these very high data rates (e.g., 240 million pixels at eight frames per second). ISIS also provides a tiled, multi-resolution compression scheme, accelerated through hardware-based compression in combination with an image client-server architecture, that allows for efficient bandwidth usage and fast playback speeds. 

[0047] Finally, embodiments of ISIS provide a combination of image processing, viewer software design, and video analytics that enable the user to interact with the very large data in an efficient way. An illustrative ISIS viewer gives one or more users the ability to maintain wide-area situational awareness with a contextual view while simultaneously viewing different parts of the scene at high resolution through the means of the virtual pan/tilt/zoom view of the scene. Automated detection and tracking of moving objects in the scene can cue users to activity of interest instead of requiring an active search of the scene for activity.  

Sounds scarier than what we’ve heard confirmed about Trapwire, no? And as with every surveillance tool developed by the military for foreign war fighting, the technology will someday soon come home to roost. The patent applicants wrote:

Exemplary ISIS systems can also be used for sports, entertainment, and news broadcasting. For instance, a portable ISIS system can be used instead of or as a supplement to a conventional camera crew to provide coverage of sporting events, parades, and breaking news. An ISIS system mounted in baseball or football stadium or hung from the ceiling of a basketball arena can provide real-time, 360-degree coverage of the entire field or court. ISIS can provide such coverage to multiple viewers simultaneously. The “virtual camera” features of ISIS mimic the pan/tilt/zoom (PTZ) functionality of conventional video cameras and allow television producers–and even individual viewers–to zoom in, zoom out, or follow particular sections of the shot. Exemplary ISIS systems give fans instant replay on demand, enable referees to review close calls, and allow investigators to track the movements of suspects through crowded areas. 

It seems like the engineers and the Air Force intend for their product to be used widely. Towards that end, they designed it with simplicity of install, ease of use and cost in mind.

…[T]he unification of an illustrative ISIS systems into a complete, end-to-end, transportable platform means that the solution is cost effective, quickly deployable and bypasses the need for a system integration effort in which similar components are purchased separately. 

The patent was granted on July 5, 2012. When can we expect to see this tool on the commercial law enforcement market?

Meanwhile, when can we expect our lawmakers to impose some commonsense limits on the use of extremely powerful surveillance tools like the ISIS system? 

After all, the Fourth Amendment was written to protect us from improper government intrusion into our private lives. The people who wrote it could never have imagined the ways in which the government now violates the spirit of that protection using advanced technologies. Far too often the courts and Congress can’t or don’t want to push for a recognition of the fact that government — aided by advanced technology — has grown too powerful.

It’s past time for that to change.

UPDATE 7/15/13: The Boston Police Department apparently used this system to monitor visitors and Boston residents at the July 4, 2013 celebrations. 

BPD Deputy Daley explaining Imaging System for Immersive Surveillance (ISIS), utilized during 4th of July. pic.twitter.com/Vq2bY1elhs

— Boston Police Dept. (@Boston_Police) July 4, 2013