A novel element of comprehensive program evaluation: Using a domain matrix design

• Published date: 01 October 2022
• Author: Catherine Walker O'Neal,Mallory Lucier‐Greer
• Date: 01 October 2022
• DOI: http://doi.org/10.1111/fare.12673

RESEARCH

A novel element of comprehensive program

evaluation: Using a domain matrix design

Catherine Walker O’Neal

1

|Mallory Lucier-Greer

2

1

Department of Human Development and Family Science, The University of Georgia, Athens, Georgia, United States

2

Department of Human Development and Family Science, Auburn University, Auburn, Alabama, United States

Correspondence

Catherine Walker O’Neal, Department of

Human Development and Family Science,

The University of Georgia, 107 Family

Science Center II (House D), Athens,

GA 30602-2622, USA.

Email: cwalker1@uga.edu

Abstract

Objective: This brief presents a novel element of program

evaluation, termed a domain matrix design, and demon-

strates the application of the design element with an exam-

ple from a collaborative evaluation project focused on the

Department of the Air Force’s Personal Financial Readi-

ness program.

Background: Utilizing sound scientific principles and

employing a replicable approach to program evaluation,

the domain matrix design is an adaptable approach to

quantitative data collection for program evaluation that

minimizes participant survey burden while maximizing

available data.

Method: This goal is primarily achieved through the iden-

tification of domains of measurable training outcomes that

are pertinent to the evaluation and then the strategic

assignment of program participants to survey assessment

groups that provide sufficient, but limited, overlap of these

domains.

Results: The proposed approach provides sufficient data on

all outcome domains, and analyses can be utilized to exam-

ine how each domain is linked to other domains as outlined

in the program’s logic model, while also minimizing the

participant burden and streamlining data collection.

Conclusion: This brief serves as an example of bridging

evaluation to meet internal program stakeholders’needs

with opportunities to contribute to the broader research

community, specifically the fields of family science and

evaluation. That is, the example project demonstrates how

evaluation can meet both the internal needs of program

stakeholders (e.g., does this program work for us? what

modifications are needed to enhance program effective-

ness?) and the external needs of the research community

Received: 18 September 2020Revised: 3 February 2021Accepted: 29 May 2021

DOI: 10.1111/fare.12673

© 2022 National Council on Family Relations.

1444 Family Relations. 2022;71:1444–1451.wileyonlinelibrary.com/journal/fare