Data Cleansing Improves Federal Government Outcomes.

• Author: Sweeney, Patrick

The 2024 fiscal year budget released by the Defense Department has $145 billion earmarked for research, development, test and evaluation funding, including $1.8 billion for artificial intelligence and $1.4 billion for systems like joint all-domain command and control, also known as JADC2.

The budget clearly prioritizes the continued investment in modernizing and innovating the department's operations.

Investments of this scale in AI, machine learning, data analytics and other emerging technologies can be undermined by poor data hygiene.

Even when info-tech systems are capable of processing enormous amounts of data, the adage of "garbage in, garbage out" still applies. Without accurate, reliable and sortable data, systems like the Federal Procurement Data System will only provide limited insight and diminished return on investment. Data quality cannot be achieved without strategic data cleansing.

Data cleansing--the process of identifying and correcting or removing inaccurate, incomplete or irrelevant data from a dataset--is not a one-size-fits-all approach. Depending on the different types of data being collected across various channels, agencies will need to determine what kind of clean-up is required to fully take advantage of advanced analytics.

With messy data, agencies risk relying on incorrect analyses that can lead to poor decision-making. By cleansing data, agencies can ensure that they are working with accurate, complete and relevant information that will actually improve operational efficiency.

With a cleansed data set, agencies can supercharge their IT system's capabilities in several ways, including increased accuracy and efficiency. Data cleansing can help agencies save time and resources by automating the process of identifying and correcting errors, reducing the need for manual data entry and validation.

It can also facilitate better data integration. Data cleansing can help agencies integrate data from multiple sources more effectively, ensuring that they are working with a complete and accurate dataset.

It can also improve analysis quality. By investing in data cleansing, agencies can ensure that they are working with high-quality data that is reliable and trustworthy, which can help them build better models and make better decisions.

The ramifications of poor data hygiene for agencies can manifest in several areas, including audits. Consider the Defense Department's financial audit process, for example.

The department has...