The National Security Agency
is harvesting huge numbers of images of people from communications that
it intercepts through its global surveillance operations for use in
sophisticated facial recognition programs, according to top-secret
documents.
The
spy agency’s reliance on facial recognition technology has grown
significantly over the last four years as the agency has turned to new
software to exploit the flood of images included in emails, text
messages, social media, videoconferences and other communications, the
N.S.A. documents reveal. Agency officials believe that technological
advances could revolutionize the way that the N.S.A. finds intelligence
targets around the world, the documents show. The agency’s ambitions for
this highly sensitive ability and the scale of its effort have not
previously been disclosed.
The
agency intercepts “millions of images per day” — including about 55,000
“facial recognition quality images” — which translate into “tremendous
untapped potential,” according to 2011 documents obtained from the
former agency contractor Edward J. Snowden. While once focused on
written and oral communications, the N.S.A. now considers facial images,
fingerprints and other identifiers just as important to its mission of
tracking suspected terrorists and other intelligence targets, the
documents show.
“It’s
not just the traditional communications we’re after: It’s taking a
full-arsenal approach that digitally exploits the clues a target leaves
behind in their regular activities on the net to compile biographic and
biometric information” that can help “implement precision targeting,”
noted a 2010 document.
One
N.S.A. PowerPoint presentation from 2011, for example, displays several
photographs of an unidentified man — sometimes bearded, other times
clean-shaven — in different settings, along with more than two dozen
data points about him. These include whether he was on the
Transportation Security Administration no-fly list, his passport and
visa status, known associates or suspected terrorist ties, and comments
made about him by informants to American intelligence agencies.
It
is not clear how many people around the world, and how many Americans,
might have been caught up in the effort. Neither federal privacy laws
nor the nation’s surveillance laws provide specific protections for
facial images. Given the N.S.A.’s foreign intelligence mission, much of
the imagery would involve people overseas whose data was scooped up
through cable taps, Internet hubs and satellite transmissions.
Because
the agency considers images a form of communications content, the
N.S.A. would be required to get court approval for imagery of Americans
collected through its surveillance programs, just as it must to read
their emails or eavesdrop on their phone conversations, according to an
N.S.A. spokeswoman. Cross-border communications in which an American
might be emailing or texting an image to someone targeted by the agency
overseas could be excepted.
Civil-liberties
advocates and other critics are concerned that the power of the
improving technology, used by government and industry, could erode
privacy. “Facial recognition can be very invasive,” said Alessandro
Acquisti, a researcher on facial recognition technology at Carnegie
Mellon University. “There are still technical limitations on it, but the
computational power keeps growing, and the databases keep growing, and
the algorithms keep improving.”
State
and local law enforcement agencies are relying on a wide range of
databases of facial imagery, including driver’s licenses and Facebook,
to identify suspects. The F.B.I. is developing what it calls its “next
generation identification” project to combine its automated fingerprint
identification system with facial imagery and other biometric data.
The
State Department has what several outside experts say could be the
largest facial imagery database in the federal government, storing
hundreds of millions of photographs of American passport holders and
foreign visa applicants. And the Department of Homeland Security is
funding pilot projects at police departments around the country to match
suspects against faces in a crowd.
The N.S.A., though, is unique in its ability to match images with huge troves of private communications.
“We
would not be doing our job if we didn’t seek ways to continuously
improve the precision of signals intelligence activities — aiming to
counteract the efforts of valid foreign intelligence targets to disguise
themselves or conceal plans to harm the United States and its allies,”
said Vanee M. Vines, the agency spokeswoman.
She
added that the N.S.A. did not have access to photographs in state
databases of driver’s licenses or to passport photos of Americans, while
declining to say whether the agency had access to the State Department
database of photos of foreign visa applicants. She also declined to say
whether the N.S.A. collected facial imagery of Americans from Facebook
and other social media through means other than communications
intercepts.
“The
government and the private sector are both investing billions of
dollars into face recognition” research and development, said Jennifer
Lynch, a lawyer and expert on facial recognition and privacy at the
Electronic Frontier Foundation in San Francisco. “The government leads
the way in developing huge face recognition databases, while the private
sector leads in accurately identifying people under challenging
conditions.”
Ms.
Lynch said a handful of recent court decisions could lead to new
constitutional protections for the privacy of sensitive face recognition
data. But she added that the law was still unclear and that Washington
was operating largely in a legal vacuum.
Laura
Donohue, the director of the Center on National Security and the Law at
Georgetown Law School, agreed. “There are very few limits on this,” she
said.
Document
Identity Intelligence: Image Is Everything
An excerpt of a document obtained by Edward J. Snowden, a
former contractor with the National Security Agency, referring to the
agency’s use of images in intelligence gathering.
Congress
has largely ignored the issue. “Unfortunately, our privacy laws provide
no express protections for facial recognition data,” said Senator Al
Franken, Democrat of Minnesota, in a letter in December to the head of
the National Telecommunications and Information Administration, which is
now studying possible standards for commercial, but not governmental,
use.
Facial
recognition technology can still be a clumsy tool. It has difficulty
matching low-resolution images, and photographs of people’s faces taken
from the side or angles can be impossible to match against mug shots or
other head-on photographs.
Dalila
B. Megherbi, an expert on facial recognition technology at the
University of Massachusetts at Lowell, explained that “when pictures
come in different angles, different resolutions, that all affects the
facial recognition algorithms in the software.”
That
can lead to errors, the documents show. A 2011 PowerPoint showed one
example when Tundra Freeze, the N.S.A.’s main in-house facial
recognition program, was asked to identify photos matching the image of a
bearded young man with dark hair. The document says the program
returned 42 results, and displays several that were obviously false
hits, including one of a middle-age man.
Similarly,
another 2011 N.S.A. document reported that a facial recognition system
was queried with a photograph of Osama bin Laden. Among the search
results were photos of four other bearded men with only slight
resemblances to Bin Laden.
But
the technology is powerful. One 2011 PowerPoint showed how the software
matched a bald young man, shown posing with another man in front of a
water park, with another photo where he has a full head of hair, wears
different clothes and is at a different location.
It
is not clear how many images the agency has acquired. The N.S.A. does
not collect facial imagery through its bulk metadata collection
programs, including that involving Americans’ domestic phone records,
authorized under Section 215 of the Patriot Act, according to Ms. Vines.
The
N.S.A. has accelerated its use of facial recognition technology under
the Obama administration, the documents show, intensifying its efforts
after two intended attacks on Americans that jarred the White House. The
first was the case of the so-called underwear bomber, in which Umar
Farouk Abdulmutallab, a Nigerian, tried to trigger a bomb hidden in his
underwear while flying to Detroit on Christmas in 2009. Just a few
months later, in May 2010, Faisal Shahzad, a Pakistani-American,
attempted a car bombing in Times Square.
The
agency’s use of facial recognition technology goes far beyond one
program previously reported by The Guardian, which disclosed that the
N.S.A. and its British counterpart, General Communications Headquarters,
have jointly intercepted webcam images, including sexually explicit
material, from Yahoo users.
The
N.S.A. achieved a technical breakthrough in 2010 when analysts first
matched images collected separately in two databases — one in a huge
N.S.A. database code-named Pinwale, and another in the government’s main
terrorist watch list database, known as Tide — according to N.S.A.
documents. That ability to cross-reference images has led to an
explosion of analytical uses inside the agency. The agency has created
teams of “identity intelligence” analysts who work to combine the facial
images with other records about individuals to develop comprehensive
portraits of intelligence targets.
The
agency has developed sophisticated ways to integrate facial recognition
programs with a wide range of other databases. It intercepts video
teleconferences to obtain facial imagery, gathers airline passenger data
and collects photographs from national identity card databases created
by foreign countries, the documents show. They also note that the N.S.A.
was attempting to gain access to such databases in Pakistan, Saudi
Arabia and Iran.
The
documents suggest that the agency has considered getting access to iris
scans through its phone and email surveillance programs. But asked
whether the agency is now doing so, officials declined to comment. The documents also indicate that the N.S.A. collects iris scans of foreigners through other means.
In
addition, the agency was working with the C.I.A. and the State
Department on a program called Pisces, collecting biometric data on
border crossings from a wide range of countries.
One
of the N.S.A.’s broadest efforts to obtain facial images is a program
called Wellspring, which strips out images from emails and other
communications, and displays those that might contain passport images.
In addition to in-house programs, the N.S.A. relies in part on
commercially available facial recognition technology, including from
PittPatt, a small company owned by Google, the documents show.
The
N.S.A. can now compare spy satellite photographs with intercepted
personal photographs taken outdoors to determine the location. One
document shows what appear to be vacation photographs of several men
standing near a small waterfront dock in 2011. It matches their
surroundings to a spy satellite image of the same dock taken about the
same time, located at what the document describes as a militant training
facility in Pakistan.
