Is It Still Possible to Collect Nationally Representative Marriage Data in the United States? A Case Study From the CREATE Project

• Published date: 01 October 2022
• Author: Spencer L. James,Jeremy B. Yorgason,Erin K. Holmes,David R. Johnson,Dean M. Busby
• Date: 01 October 2022
• DOI: http://doi.org/10.1111/fare.12577

S L. J, J B. Y,  E K. HBrigham Young

University

D R. JThe Pennsylvania State University

D M. BBrigham Young University

Is It Still Possible to Collect Nationally

Representative Marriage Data in the United States?

A Case Study From the CREATE Project

Objective: Tounderstand challenges in the data

collection environment for collecting nation-

ally representative data and discuss one study’s

response to these challenges.

Background: The United States is undergoing

impressive and transformational social change

related to marriage. Social scientists’ ability

to study such changes are contingent on being

able to minimize sampling error (the difference

between the sample and the population), accom-

plished most reliably by collecting representa-

tive survey data and making concomitant gen-

eralizations from them. Given the expected low

response rates in contemporary surveyresearch,

it is natural to ask whether it is still possible

to collect high-quality, nationally representative

survey data on marriage and family.

Method and Results: This article presents an

argument about the importance of continuing to

collect nationally representative data on mar-

riages and families, a discussion about the chal-

lenges associated with the task, an example of

how one project managed and dealt with that

School of Family Life, Brigham Young University, 2095

JFSB, Provo, UT,84606 (spencer_james@byu.edu).

Key Words: marriage, nationally representative, survey

research.

environment, and data comparisons between

multiple sampling methods.

Conclusion: Despite the challenges, scholars

must continue to pursue nationally representa-

tive data to inform knowledge of national trends

and relationships.

Implications: A greater focus on sampling

methods and a study’s concomitant general-

izability and external validity will improve

knowledge of trends in marriage and family

relationships, leading to improved therapeutic

practice; more informed policymaking; and

better theoretical, conceptual, and methodolog-

ical understanding of key family processes and

outcomes.

The United States is undergoing impressive and

transformational social change. These changes

hold important implications for the well-being

of current and future generations. One partic-

ularly turbulent aspect for many Americans is

the ever-shifting role of family life, in large

part due to changes in the economy and related

increases in income and wealth inequality

(Blau, 1998; Ellwood & Jencks, 2004; Piketty

et al., 2016). An increasing proportion of the

American population is experiencing family and

social change, including delaying and forgoing

marriage; increasing poverty and nonmarital

1428Family Relations 71 (October 2022): 1428–1443

DOI:10.1111/fare.12577

Nationally Representative Marriage Data1429

childbearing, relational instability, and high

divorce rates; and rising levels of cohabitation,

all of which are thought to be tied to global

trends encompassed in the second demographic

transition (Lesthaeghe, 2010).

Tracking and accurately capturing these

changes requires a large undertaking, and there

have been discussions on the need for new

nationally representative data (House, 2015;

Manning et al., 2015) to address emerging ques-

tions about American family life (Moft et al.,

2015; Raley, 2015; Seltzer, 2015). Given the

increasing complexity of contemporary family

relationships and demographics (Cherlin, 2010),

social scientists’ ability to respond to such calls

to action and the individual and societal benets

of social science research are contingent upon

our ability to (a) collect data that mirror the

population, (b) draw appropriate conclusions

based on a sample’s generalizability,(c) analyze

the data rigorously in accordance with prevail-

ing scientic and statistical standards, and (d)

do all of these things in a timely and orderly

fashion, without imposing unnecessary burden

on respondents (Manning et al., 2015). This is

a tall task, as survey responses have declined

substantially, in part due to inundation from

surveys of varying quality and motives that

levy signicant costs on respondents (Dutwin

et al., 2014).

T C  C M

S D

Given that response rates to current surveys

are often in the high single-digits to low teens

(Kohut et al., 2012), it is natural to ask whether

it is even possible to still collect high-quality,

nationally representative survey data on mar-

riage and other family-related topics. In this

article, we address these issues by discussing

the importance of collecting nationally repre-

sentative data, especially regarding marriage, in

a challenging data collection environment. We

articulate the particular challenge of falling sur-

vey response rates and note several approaches

researchers have pursued to reduce survey non-

response bias, as well as other ways to col-

lect data either via random probability samples

or by using nonprobability samples more judi-

ciously. We then introduce readers to the Couple

Relationships and Transition Experiences study

(CREATE), a nationally representative study of

newly married couples aged 18 to 36 years, 90%

of whom married during 2014. Although this

study does not respond wholly to the needs dis-

cussed (Manning et al., 2015), we have success-

fully collected a nationally representative dataset

of almost 2,200 newlywed couples in a challeng-

ing data environment.

W C N R

D

Addressing emerging questions about contem-

porary American families will require new

nationally representative data (House, 2015;

Manning et al., 2015). Importantly, one can rea-

sonably ask why nationally representative data

are essential. After all, modern data collection

techniques make collecting data from across

wide geographic areas like the United States

relatively quick and affordable. For instance,

many marketing research rms employ panels of

individuals who are paid to take surveys. Such

individuals are often attentive survey-takers

who complete the study in a timely fashion.

Meanwhile, the companies often ensure that

their panels match national characteristics of

the United States or other relevant populations.

Why not use our limited resources in this way?

After all, couldn’t we get more data for less

money in a shorter amount of time?

Unfortunately,this data utopia is but a mirage.

The reason why we require nationally represen-

tative data to assess shifting patterns in con-

temporary American family life boils down to

two words: sampling error.1Sampling error can

be conceptualized as the difference between the

“true” population parameter and the estimate

obtained from a given sample. How large the

sampling error is reects the amount of varia-

tion we would expect to nd over all possible

samples collected with identical sampling and

data collection/analysis methods. In part, this is

because people who respond to surveys tend to

be systematically different from people who do

not. People who respond to surveys will be over-

represented on commercial panels. And these

differences will result in sampling error.

As we discuss subsequently, sampling

error is far more likely to inuence estimates

1Weacknowledge other sources of bias, such as nonsam-

pling error (e.g., survey design and interpretability) butfocus

on sampling error here because minimizing sampling error

is what differentiates representative samples from nonrepre-

sentative ones.