"Successful Baggage Screening Relies on Human Factors"

Wednesday, March 27, 2002

Successful Baggage Screening Relies on Human Factors
Airport Security Report


Human operators could well be the determining factor as to whether
government-certified computer tomography X-ray (CTX) machines improve
aviation security, according to industry sources. The CTX machines have
high false-alarms rates which require significant human intervention in
the process, these sources told ASR. 

The concerns about explosives detection devices being used to screen
passengers and luggage at the nation's airports are being expressed as
the aviation industry moves to tighten security. Critics are again
bringing up questions that have been posed for nearly a decade about the
detection capabilities, alarm rates and throughput of explosives
detection system (EDS) machines, such as CTX units. In a recent nine-day
span, CTX machines provided false positive alarms that forced the
evacuations of two West Coast airports on three separate occasions. The
Department of Transportation "zero tolerance" policy has forced a total
of 156 airport terminal/concourse evacuations from Oct. 31 to March 16. 

In the final analysis, the ability to detect dangerous items, not the
throughput rate, will be most important criteria for the success or
failure of the EDS machines, argued Douglas Laird, vice president of BGI
International Consulting Services. 

"The premise has to be to keep bombs off the aircraft," he declared. "Do
we want to use the UK [baggage screening] system where you can't find
the bombs, compared to a system where we can? I think the answer is
pretty clear - find the bombs." 

An EDS machine examines the density of objects inside luggage and
compares them to a database of known explosives. Industry experts say
the machines perform well in finding large explosives, but they are less
reliable in finding explosives made with the latest technology, such as
sheet explosives. 

The final part of the EDS examination relies on a trained operator to
check an image of a bag's contents against a database of images. The
operator must clear the bag for loading or signal the bag for further
inspection. 

"Throughput of baggage through these machines is not the right figure of
merit to measure aviation security," asserted Thomas Hartwick, chairman
of a National Academy of Sciences panel on aviation security. "We don't
want to wear out our operators. We want them to be fresh to work
together with CAPPS [computer assisted passenger prescreening system]
and EDS in order to find bombs and find terrorists." 

Hartwick told ASR that the panel's second report on aviation security, a
study on human factors and deterrence, will be released this week. The
first report was published in November 1999 

San Francisco International Airport (SFO) has an integrated baggage
screening system that uses PerkinElmer Z-Scan-7 X-ray machines at the
first level of screening. Most European airports also use a traditional
or enhanced X- ray machine at the first level of screening. Trained
operators, viewing images from a remote location, single out bags for
further examination. A bag will go to an InVision Technologies CTX9000
machine if the operator selects it for further screening or if the
operator can't clear it in the allotted time. 

Laird said that a trained operator can clear nearly all bags if given
sufficient time to look at them. "You've given that human operator a CTX
machine that is unsurpassed. It can't be matched by an X-ray machine,"
he said. "A good operator can clear 99.8 percent of so-called alarms ...
If you ran those same items through an X-ray, it wouldn't even alarm." 

Throughput remains the biggest problem with EDS machines, according to a
former federal security manager who requested anonymity. Manufacturers
not only bump up the throughput numbers they release publicly, but the
Federal Aviation Administration (FAA) certification process does not
take into account bag reconciliation by human operators. During
certification in 1994, the FAA measured the throughput of the CTX5000SP
at 254 bags per hour. In order to meet the FAA's throughput standard,
InVision operated two 5000SPs in parallel for the evaluation, termed
them "one system," and passed the 450 bags per hour standard. 

EDS machines alarm on 28 percent of screened bags, on average.
Post-Sept. 11, a single airport ticket counter station processes about
30 passengers and 45 checked bags per hour. A ticket counter operating
10 stations at full capacity with one takeaway belt would then handle
450 bags per hour. Multiple EDS machines would be needed in the baggage
system just to handle the bag load. 

The quickest method is to have a human operator reconciling bag images.
This would take on average 30 seconds per bag. A multi-EDS baggage
system screening 450 bags per hour would alarm on 126 bags. 

Since Sept. 11, the FAA has changed its requirements for baggage
reconciliation to allow the use of explosive trace detection (ETD) or
physical inspection of bags, the source said. Only systems that have
properly trained EDS operators were allowed exemptions. 

San Francisco does not use EDS machine at the first level of screening.
But selected bags are labeled with a radio frequency identification
(RFID) tag so that they default to the EDS system at the third level of
screening, the source said. 

GAO, DOT Skeptical of EDS Machines 

The two main provisions of the Aviation and Transportation Security Act
of 2001 is the federalization of security checkpoint screeners and the
mandate to install EDS machines at all commercial U.S. airports by Dec.
31, 2002. But only about 160 EDS machines have been deployed at 50 U.S.
airports so far, and the FAA estimates more than 2,000 machines are
needed to meet the law's deadline. 

Concerns about the EDS machines have been expressed in testimony by the
General Accounting Office (GAO) and the DOT Office of Inspector General
(DOT/OIG) dating back to the 1980s about screener performance and the
evolution of EDS machines. Testimony by DOT/OIG in 1998 shortly after
test EDS machines were installed in U.S. airports identified concerns
about false alarm rates, detection rates and throughput rates. 

The first EDS machine deployed at airports, InVision's CTX5000SP,
performed differently in an airport environment than it did during
certification testing, said Alexis Stefani, deputy assistant inspector
general, in May 1998 congressional testimony. False alarm rates were as
high as 169 percent above the standard set during certification, she
said. Test bags used during certification testing didn't properly
represent how passenger baggage would be affected by various weather
conditions. The machines were screening only 200 bags per day when they
could screen as many as 225 bags per hour, Stefani testified in 1998. 

In November last year, DOT/IG Kenneth Mead said that EDS machines
continued to be "underutilized and understaffed." He said that over one
weekend his inspectors at an airport observed that only seven bags out
of 62 passengers who checked luggage where screened through an EDS
machine during a 30-minute period. Only one carrier at that airport
required all checked luggage to be screened, Mead said. 

Earlier this year, Mead told a congressional committee that during a
one- hour airport observation on Jan. 18, only 15 of 110 bags checked by
passengers were screened through an EDS machine. 

Airlines complain that the EDS machines are slow and produce false alarm
rates about 30 percent of time - far above the certification limit. But
linking EDS scanning to CAPPS was the main reason why the airlines never
increased usage of CTX machines. If an airline was able to keep its
number of selected passengers low, then EDS machines would be used less.


Government Orders 100 EDS Units 

On March 6, the Transportation Security Administration (TSA) ordered 100
EDS units from InVision Technologies of Newark, Calif. at a cost of
$169.8 million. That order consists of smaller CTX 2500 versions and
mid-size CTX 5500DS models to be delivered this summer. Additionally,
the TSA ordered part kits for another 300 units of the same models. 

InVision signed a letter of intent with CoorsTek on March 14 for the
company to become a manufacturing subcontractor supporting InVision's
production expansion. Golden, Colo.-based CoorsTek will manage
InVision's supply chain for all CTX 2500 and CTX 5500DS systems. 

InVision was the first company to have a machine certified for EDS
screening by the FAA's Hughes Technical Center in Atlantic City, N.J.,
back in 1994. It now owns more than 90 percent market share for
certified EDS systems. 

New York-based L-3 Communications [LLL], the second company certified
for EDS machine production, has no current orders from the government,
but is expecting some soon, said Evan Goetz, a company official. 

Before Sept. 11, EDS machines worked in combination with CAPPS to screen
passengers' checked luggage. CAPPS was not intended to screen passengers
or their carry-on luggage, nor were EDS machines intended to screen
large volumes of luggage. 

'Systems' Approach Taken For Security 

To meet the Dec. 31 deadline to have EDS equipment in place, the TSA has
authorized using several methods in a multi-layered security system. All
the procedures are currently being used, but not at all airports: 

   * EDS machines for baggage screening 

   * K-9 units trained to sniff explosives

   * ETD devices for searching passengers and both carry-on and checked
luggage

   * CAPPS for identifying high-risk passengers

   * Manual bag searches

   * Matching checked luggage to an individual passenger on originating
flights only

   * Secondary random passenger gate screening by airlines

TSA Under Secretary John Magaw last month said the agency would embark
on a trial program of matching bags to passengers on connecting flights.
A TSA official refused to provide details of the trial program, but said
it would start soon. 

Also, Magaw told a Senate appropriations committee March 21 there is no
one existing item or technology that covers all explosives detection
threats. The TSA will solve problems on an airport-by-airport basis by
using "two or three" technologies together, he said. Magaw also said
that the TSA is budgeting up to $175,000 for installation costs of every
EDS machine. 

Comparison of Bulk Detection Technologies 

Technology                             Specificity1  Film   
                                                     Safe

Fluoroscopic Imaging (FI)              Poor          No     
Scanning Black and White X-ray Imager  Poor          Yes    
Dual Energy X-ray (Vivid)              Good          Yes    
Backscatter X-ray                      Moderate      Yes    
Low-Dose Backscatter X-ray             Moderate      Yes    
Computer Tomography (CT) X-ray         Good          No     
Quadrople Resonance (QR)               Excellent     Yes    
Thermal Neutron Activation (TNA)       NA            Yes    
Pulsed Fast Neutron Analysis (PFNA)    Good          Yes    
Gamma Backscatter                      Moderate      No     
Millimeter Wave Imaging                Moderate      Yes    
Microwave Imaging                      Moderate      Yes    

Technology                             Cost       Portable   
                                                  Versions
                                                  Available
Fluoroscopic Imaging (FI)              Low        Yes        
Scanning Black and White X-ray Imager  Moderate   No         
Dual Energy X-ray (Vivid)              High       No         
Backscatter X-ray                      High       No         
Low-Dose Backscatter X-ray             High       No         
Computer Tomography (CT) X-ray         Very high  No         
Quadrople Resonance (QR)               Moderate   Yes        
Thermal Neutron Activation (TNA)       High       No         
Pulsed Fast Neutron Analysis (PFNA)    Very high  No         
Gamma Backscatter                      Low        Yes        
Millimeter Wave Imaging                Unknown    NA         
Microwave Imaging                      Unknown    NA         

1 Poor - no discrimination between low-Z and high-Z materials. Moderate
- can distinguish between low-Z and high-Z materials, but no
discrimination between vaous low-Z materials. Good - can distinguish
between various low-Z materials. Eellent - can identify specific drugs.
Source: "Guide for the Selection of Drug Detectors for Law Enforcement
Applications," National Institute of Justice, August 2000 

Recent Airport Evacuations Involving Certified EDS Machines 

   * March 7, Los Angeles: InVision CTX 5500 alarmed on a food processor
in passenger luggage, forcing an evacuation.

   * March 4, Los Angeles: InVision CTX 5500 alarmed on an exact replica
of a hand grenade, forcing an evacuation. 

   * Feb. 26, Sacramento: InVision CTX 2500 alarmed on a Disney Mickey
Mouse snow globe, forcing an evacuation. 

Was Salt Lake's Performance Golden? 

On Jan. 18, the Salt Lake City International Airport (SLC) security
program for the Winter Olympic Games kicked off. The federal government
kicked in $12.7 million to beef up security so that 100 percent of
passengers' checked luggage was screened using various methods. After
the experience hosting the Olympics, the airport's system can be seen as
a model for systems at other airports. The security program consisted
of: 

   * Nine computer tomography X-ray (CTX) machines: seven new, two
existing. The airport uses three types of CTX machines: InVision 2500,
InVision 5500 and Vivid. 

   * 84 explosive trace detection (ETD) units for screening passengers
on 11 airlines. The wand-like devices were used to scan passenger hands
and luggage for traces of explosive material at the ticket counters and
skycap check-in areas. 

   * Five federal K-9 explosive detection dogs were added to Salt Lake's
five K-9 units. Each dog worked with a SLC police officer.

   * Around 400 additional personnel were added during the Olympics. 

No security incidents or arrests occurred during the two weeks of
events, but that didn't silence critics. Nearly 20 Federal Aviation
Administration (FAA) special agents wrote a letter to TSA Under
Secretary John Magaw charging that the FAA disregarded or compromised
security at the airport. 

The agents said FAA officials failed to evacuate the airport or rescreen
passengers after a metal detector was found unplugged; checkpoints that
failed to search security personnel who were allowed to bypass metal
detectors and X- ray machines; and airlines incorrectly screened and
searched passengers at gates. The TSA is investigating the allegations. 

One industry source said the government "threw the kitchen sink of
security" into the airport during the Olympics. But not all the
resources were used efficiently. The source said ETDs were used as a
level one screening method, which isn't a productive use of those units.
ETDs were used on the outside of passenger baggage first. Only after
that was the inside of the baggage checked. 

"I don't think the execution was well done because they continuously
placed ski jackets, carry-on bags, everything, on stainless steel tables
that could contaminate anything that touches it," the source said. 

Salt Lake officials could not be reached for comment.


 Do you have an opinion about this story?
Share it with other readers in our CAA Discussion Forums

http://www.californiaaviation.org/cgi-bin/dcforum/dcboard.cgi?conf=DCConfID8

*****************************************