Army, Marine Corps Look for Better Data on Simulator Effectiveness.

Army and Marine Corps officials have praised simulators because they cost less than live combat training. With the flip of a button, instructors can change settings from desert to jungle, run different scenarios and give instant feedback to troops instead of having to set up elaborate, expensive and time-consuming war games.

However, neither service is able to precisely measure just how much money is being saved, or to what degree simulators are able to improve troop performance, said an August report by the Government Accountability Office.

Part of the problem is that while the Army and Marine Corps have subjective ways of documenting that virtual training improves proficiency, they don't have an objective method to measure how much it helps prepare troops for combat or live exercises, said Sharon Pickup, GAO director of defense capabilities and management. "The Army needs an institutional approach," she said.

Both services need to establish metrics to calculate just how effective their simulators are, said the report. Furthermore, they need a more comprehensive method to compare the costs of live and virtual training, it said.

The Army's program executive office for simulation, training and instrumentation has recognized the need for more information on the value of what it does, said Army Lt. Col. Johnny Powers, PEO STRI's product manager for simulation to mission command interoperability.

It currently is studying how to best evaluate the usefulness of virtual training--both by using data and subjective measures, he said.

"This study will provide methodologies to help determine the return on investment for the use of simulation tools to provide command and staff training to units," he told National Defense in an email.

There are many methods the Army uses to measure the value of simulation for individual and crew-level training, but there is no overall methodology to determine if they actually "increase an organization's collective task proficiency" Powers said. Currently, "commanders and other leaders subjectively assess unit task proficiency based on published standards and the assessor's experience level."

The Army also routinely collects data on how effectively an individual is using a simulation, and a soldier must meet doctrinal standards before moving on to the next phase of virtual or live training, he said.

Marine Corps training is based on its Systems Approach to Training (SAT), Laura Junor, deputy assistant secretary of...