WASHINGTON -- A thorough cost-benefit analysis that includes an assessment of meaningful alternatives is needed to reveal the potential security advantages of deploying new detector systems to screen cargo for nuclear and radiological materials at U.S. ports and border crossings. It is likely that the costs will exceed the savings gained from improved efficiency of the screening systems, says a new report from the National Research Council. There are shortcomings in the U.S. Department of Homeland Security's testing of these new detector systems, therefore the DHS secretary cannot conclude whether they will consistently outperform the current ones. DHS should not proceed with further procurement of these new detector systems until the issues are addressed and the systems have been shown to be a favored option in the cost-benefit analysis.
The 2006 SAFE Port Act requires that all containers coming into the United States through major entries be scanned for radiation, and "to the extent practicable, the Secretary shall deploy next generation radiation detection technology" to enable such scanning. In response, the department's Domestic Nuclear Detection Office (DNDO) requested proposals for advanced spectroscopic portals (ASPs), the next generation of radiation detectors for cargo screening, to replace the current system of radiation portal monitors and handheld radioisotope identifiers, which have known deficiencies. Before DHS can proceed with full-scale procurement of ASPs, Congress required the secretary to certify that they will provide a "significant increase in operational effectiveness" over continued use of the existing screening devices. If ASPs are certified, DHS may spend more than $1 billion to purchase these detector systems, with a possible net lifecycle cost of more than twice that figure. Currently, DHS is testing and evaluating the ASPs to inform the secretary's decision.
Congress asked the Research Council to advise DHS about testing, analysis, costs, and benefits of ASPs before this certification decision is made. The ASP testing and evaluation program encountered some delays in 2008, which created the opportunity for this interim report to provide advice on how testing, evaluation, and the cost-benefit analysis should be completed.
Testing of ASPs before 2008 had serious flaws, a number of which DNDO has acknowledged and addressed, said the committee that wrote the new interim report. In 2008, DNDO carried out physical tests to evaluate some of the limits of the ASP systems, but inadequacies remain. These include a lack of modeling to complement the physical tests and small test sample sizes that limit the confidence of comparison testing between the old and new detector systems.
The committee recommended an iterative testing approach, using computer models to simulate performance of the detector systems and physical experiments to test the models and identify needs for refinements. Then the needed model refinements could be undertaken. This iterative modeling and testing approach will allow DHS to gain a better understanding of the detector systems' performance, the committee said.
Rather than focusing on a single procurement to replace current screening technology, testing of ASPs should be viewed as a first step in improving and adapting the detector systems, the committee recommended. DHS should develop a process for continuous improvement that could address and exploit changes in technology and the nature of commerce so the system is not outdated or obsolete by the time it is fully deployed. DHS should begin this process by deploying the unused ASPs they already own in real ports of entry.
To determine whether the costs for these systems are reasonable and justified, a careful assessment will be needed to reveal the advantages of ASPs among alternatives, the committee said. The cost-benefit analysis should include a clear statement of the objectives of the program; an assessment of meaningful alternatives; and a comprehensive, credible, and transparent analysis of in-scope benefits and costs. The benefit assessment should show how the procurement contributes, relative to other possible DHS efforts and expenditures, to improving security with respect to prevention of the detonation of a nuclear or radiological device, which is the primary objective of the ASP program. A cost-benefit analysis that is silent on this subject would be incomplete, the committee noted.
The study was sponsored by U.S. Department of Homeland Security. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies. They are independent, nonprofit institutions that provide science, technology, and health policy advice under an 1863 congressional charter. Committee members, who serve pro bono as volunteers, are chosen by the Academies for each study based on their expertise and experience and must satisfy the Academies' conflict-of-interest standards. The resulting consensus reports undergo external peer review before completion. For more information, visit http://national-academies.org/studycommitteprocess.pdf. A committee roster follows.
Copies of EVALUATING TESTING, COSTS, AND BENEFITS OF ADVANCED SPECTROSCOPIC PORTALS FOR SCREENING CARGO AT PORTS OF ENTRY: INTERIM REPORT are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at HTTP://WWW.NAP.EDU. Reporters may obtain a copy from the Office of News and Public Information (contacts listed above).
[ This news release and report are available at HTTP://NATIONAL-ACADEMIES.ORG ]
NATIONAL RESEARCH COUNCIL
Division on Earth and Life Studies
Nuclear and Radiation Studies Board
COMMITTEE ON ADVANCED SPECTROSCOPIC PORTALS
ROBERT C. DYNES 1 (CHAIR)
Professor
Department of Physics
University of California, San Diego; and
Professor of Physics
University of California
Berkeley
RICHARD E. BLAHUT 2
Henry Magnuski Professor
Department of Electrical and Computer Engineering
University of Illinois
Urbana-Champaign
ROBERT R. BORCHERS
Chief Technology Officer
Maui High Performance Computing Center
University of Hawaii
Maui
PHILIP E. COYLE III
Private Consultant and Senior Adviser
World Security Institute
Sacramento, Calif.
ROGER L. HAGENGRUBER
Research Professor;
Director, Office for Policy, Security, and Technology; and
Director, Institute for Public Policy
University of New Mexico
Albuquerque
CARL N. HENRY
Private Consultant
Albuquerque, N.M.
JOHN M. HOLMES
Deputy Executive Director of Operations
Port of Los Angeles
Los Angeles
KAREN KAFADAR
Professor
Department of Statistics
Indiana University
Bloomington
C. MICHAEL LEDERER
Research Chemist and Deputy Director Emeritus
University of California Energy Institute
University of California
Berkeley
KEITH W. MARLOW
Consultant
Orion International Technologies Inc.
Albuquerque, N.M.
JOHN W. POSTON SR.
Professor
Department of Nuclear Engineering
Texas A&M University
College Station
HENRY H. WILLIS
Professor of Policy Analysis
Pardee RAND Graduate School; and
Policy Researcher
RAND Corp.
Pittsburgh
RESEARCH COUNCIL STAFF
MICAH LOWENTHAL
Study Director
1 Member, National Academy of Sciences
2 Member, National Academy of Engineering