Lessons from the Field. Continuous Quality Improvement: A multiyear HMRF case study of best practices in outreach program excellence

• Published date: 01 December 2023
• Author: Tyler S. Nesbit,Victor W. Harris,Brian Visconti,Charles Bradford Sewell,Kate Fogarty,Jeneé C. Duncan,Liva LaMontagne,Nelly Nelson
• Date: 01 December 2023
• DOI: http://doi.org/10.1111/fare.12826

BRIEF REPORT

Lessons from the Field. Continuous Quality

Improvement: A multiyear HMRF case study of best

practices in outreach program excellence

Tyler S. Nesbit

1

|Victor W. Harris

1

|Brian Visconti

1

|

Charles Bradford Sewell

2

|Kate Fogarty

1

|Jeneé C. Duncan

1

|

Liva LaMontagne

1

|Nelly Nelson

3

1

Department of Family, Youth and

Community Sciences, University of Florida,

Gainesville, FL

2

Purdue University Extension, West

Lafayette, IN

3

University of Florida Institute of Food and

Agricultural Sciences, Gainesville, FL

Correspondence

Tyler S. Nesbit, P.O. Box 110310, Gainesville,

FL 32611-0310, USA.

Email: tnesbit@ufl.edu

Funding information

Funding for the SMART Couples Florida

Project was provided by the United States

Department of Health and Human Services,

Administration of Children and Families,

Grant 90FM0079. Services are available to all

eligible persons regardless of race, gender, age,

disability, or religion.

Abstract

Objective: We describe the process and outcomes of develop-

ing continuous quality improvement (CQI) procedures for a

multiyear, multimillion-dollar healthy marriage and responsi-

ble fatherhood (HMRF) relationship enhancement education

program. We present lessons learned, including adaptations

used to move all programming online due to the COVID-19

pandemic.

Background: Continuous quality improvement (CQI) is a

set of best practices that are often neglected in outreach

programming due to challenges associated with funding,

available expertise, and fear of underwhelming results.

However, this practice provides valuable insight and bene-

fits to programs and participants and can be implemented

without interrupting program delivery.

Method: We developed a “living”CQI plan over the course

of 5years using three sources of data to track, evaluate,

and inform CQI high-performance decision-making: pro-

gram data, fidelity data, and outcome data.

Results: A sample of the preliminary descriptive quantita-

tive results is presented including program registrations

and show rates, facilitator effectiveness scores, interven-

tion outcomes, and participant responses to online delivery

to illustrate how the three types of data collected are used

in the SMART Couples Project to support CQI efforts.

Conclusions and Implications: Our study demonstrates the

benefits of using CQI as a powerful tool for program

improvement, with staff and participants alike. It is the

nature of the CQI process to be amenable to changes,

including unforeseen disruptions in program delivery.

Implementing an intentional formative and summative

Received: 8 July 2022Accepted: 11 November 2022

DOI: 10.1111/fare.12826

© 2023 National Council on Family Relations.

Family Relations. 2023;72:2627–2646. wileyonlinelibrary.com/journal/fare 2627

CQI strategy provides benefits to social outreach and family

life education programs across delivery formats and contexts.

KEYWORDS

continuous quality improvement, evaluation research, family life

education, outreach program best practices, relationship education

Family fragmentation costs state and local taxpayers an estimated 112 billion dollars nationally

per year in foregone tax revenues, justice system expenses, Temporary Assistance for Needy

Families (TANF), Medicaid, State Children’s Health Insurance Program (SCHIP), and child

welfare program expenditures (Scafidi, 2008). Estimates of social and emotional costs in terms

of lost productivity and decreased quality of life are generally much higher (Raley &

Sweeney, 2020; Schramm, 2009). Research-based relationship and marriage education (RME)

and family life education (FLE) programs provide theoretically grounded and data-based

approaches to address the economic and human costs of family fragmentation and child welfare

(Futris & Adler-Baeder, 2013). Continuous quality improvement (CQI) of outreach programs

through tracking and acting on program, fidelity, and outcome data represent critical best prac-

tices for the future and relevance of family science as a discipline.

Program evaluation

Program evaluation is critical to assessing the validity of family and social science outreach

programs (Dahlberg & McCaig, 2010). Common reasons for evaluation include assessing pro-

gram impact (e.g., by measuring client knowledge and skill attainment, program reach, and

client satisfaction), providing justification and accountability to program stakeholders

(e.g., to influence relevant policy), and improving program implementation (e.g., identifying

what is and is not working, making changes to delivery, services, and/or resource allocations

as needed; Darling et al., 2022). Two broad categories of evaluation are formative and sum-

mative evaluation. Formative evaluation occurs in the early and middle stages of program

implementation to assess the degree to which program components contribute to program

goals and what changes are needed to advance program objectives (Boothroyd, 2018). For

example, process evaluation is a type of formative evaluation that assesses how well a pro-

gram is being implemented as intended (i.e., program fidelity; Centers for Disease Control

and Prevention, n.d.). Summative evaluation occurs near or at the end of implementation to

assess the overall outcomes achieved and to determine the value of continuing or adapting a

program plan of action (Boothroyd, 2018).

Ideally, the evaluation process commences well before implementing programming in

ordertosetgoalsandmonitorprogress(Darlingetal.,2022; Newcomer et al., 2015). Despite

the benefits of program evaluation to the delivery and effectiveness of RME and FLE and

other social science outreach programs, many programs neglect this process due to challenges

associated with funding, available expertise, and fear of underwhelming results (Hawkins

et al., 2020). Certainly, program evaluation can be time-consuming and resource intensive,

often involving hiring an external evaluator. When an evaluation is undertaken with an aim

to appease grant funders or to satisfy other external stakeholders, it may be viewed with dis-

dain by program management and staff as an onerous task disconnected from the real-world

impacts of program participants (Newcomer et al., 2015). However, many approaches to pro-

gram evaluation offer multiple benefits for internal programmatic purposes, and in some

cases, can be implemented using simple approaches in real time without impacting program

activities (Darling et al., 2022, p. 173).

2628 FAMILY RELATIONS