The purpose of the current study was to test the factor structure and scale quality of data provided by caregivers about the home environment and child behavior at home using the Elementary School Success Profile (ESSP) for Families. The ESSP for Families is one component of the ESSP, an online social-environmental assessment that also collects information from students and teachers. Confirmatory factor analyses with Mplus and weighted least squares means and variances adjusted estimation took into account the hierarchical nature and ordinal level of the data. The sample comprised caregivers of 692 third- through fifth-grade students from 13 elementary schools in four districts. A primary model and an alternative model were tested. Models were tested on a random calibration sample and validated with another sample. A nine-factor first-order solution demonstrated superior fit to the data. Scores from the nine scales also demonstrated acceptable internal consistency reliability. Implications for practice and further research are presented.
KEY WORDS: confirmatory factor analysis; elementary school; reliability; social environment; validation
The theoretical and empirical knowledge bases of developmental psychopathology (Costello & Angold, 1996; Sroufe, 1997), risk and resilience (Rutter, 2001; Sameroff, 2000), ecological theory (Bronfenbrenner, 1992), and developmental contextualism (Lerner, 1986) suggest that students' social and academic behaviors in schools are influenced by their experiences in the social environment. Research supporting these theories demonstrates the impact that the social environment has on concurrent and future child outcomes (for example, Barnard, 2004; Chase-Lansdale & Gordon, 1996). In spite of the pivotal role of social-environmental experiences in child development, few well-validated social-environmental assessments exist for school-based practitioners. An exception is the School Success Profile (SSP), a self-report social-environmental assessment for middle and high school students, which was developed in the early 1990s (G. L.Bowen, Richman, & Bowen, 2002; G. L. Bowen, Rose, & Bowen, 2005). Based on an "eco-interactional-developmental" perspective (G. L. Bowen et al., 2002), the SSP assesses dimensions of the neighborhood, school, peer system, and family as experienced by youths. Youths' perceptions of dimensions of their own psychological and physical well-being and school performance are also assessed. Individual- and group-level data are provided to schools to guide change efforts.
In response to practitioner requests for an elementary version of the SSP, the Elementary School Success Profile (ESSP) (N. K. Bowen, 2006; N. K. Bowen & Powers, 2005; Woolley, Bowen, & Bowen, 2004) was developed through an iterative process incorporating expert, practitioner, and respondent feedback. The ESSP makes it possible to assess the social environment of children in grades 3 through 5. The ESSP assesses the same major domains as the SSP but with differences in specific dimensions that reflect the developmental stage of middle childhood. Because the ESSP assesses younger children who might not be developmentally capable of providing valid self-report data about all aspects of their social environments, ESSP data are collected from caregivers and teachers in addition to children.
Together, the three components of the ESSP collect data about the following social domains: neighborhood (child and caregiver report), school (child and caregiver report), friends (child report), caregiver education involvement (caregiver and teacher report), and family (child and caregiver report). Data are also collected on social behavior at home and school (caregiver and teacher report) and school performance (teacher report). Sources of data within domains overlap when appropriate, providing multiple perspectives on the same social environmental factors.
Analysis of data collected from caregivers supported the reliability of scales, with alphas ranging from .69 to .94 (N. K. Bowen, 2008). Results of exploratory factor analyses indicated adequate convergent and discriminant validity of dimensions. Construct validity has been supported through significant correlations between scales on the ESSP for Families (previously called the "ESSP for Parents") and similar scales on the ESSP for Teachers and the ESSP for Children. More information on the three-part ESSP, including its theoretical approach, is available in previously published work (N. K. Bowen, Bowen, & Woolley, 2004; see also http:// schoolsuccessonline.com).
Like the SSP, the ESSP provides individual- and group-level reports that are used by school staff to guide intervention efforts and promote school success. The theory of change derived from the eco-interactional-developmental perspective (G. L. Bowen et al., 2002) posits that social-environmental factors influence well-being, which in turn influences behavior and performance. According to this theory of change, intervening in malleable environmental dimensions is expected to promote school success. The purpose of the ESSP is consistent with recent developments in the school intervention literature, such as calls for evidence-based practices, school-level prevention approaches and data-driven decision making and a focus on social and emotional learning issues in addition to academic issues (Graczyk, Domitrovich, Small, & Zins, 2006).
The current study adds to the validation evidence for the caregiver component of the ESSP for Families. In addition to providing a solution to the potential problem of unreliable data from child informants, collection of information about student performance from multiple sources and across different settings is considered a hallmark of high-quality assessment (Merrell, 2003; Pepler & Craig, 1998; Silverman & Saavedra, 2004). Documented differences in perspective among sources regarding the same phenomena support the importance of obtaining multiple views (Achenbach, McConaughy, & Howell, 1987; Offord et al., 1996; Pepler & Craig, 1998). Pepler and Craig, for example, outlined the strengths and limitations of different sources, concluding that a combination of information from multiple sources provided the most complete understanding of phenomena. Data collected with the ESSP for Families complement the data collected from children and teachers. Because caregivers have the most knowledge of their own children over time and across settings (Pepler & Craig, 1998), they may be the most important source of information on children (Achenbach et al., 1987). Establishing the quality of data collected with the ESSP for Families, therefore, is a critical step in validating the ESSP as a whole.
The purpose of the current study was to test the factor structure and scale quality of family-report items assessing the home environment and child behavior at home. Findings of the study permitted progress toward an additional goal: reducing the length of the ESSP for Families. The caregiver report component of the ESSP is the longest of the three survey components, with 185 items and an average response time of 30 minutes. School staff members have commonly reported that length has prevented some respondents from completing the ESSP for Families. Excessive length might also be expected to reduce the reliability of data collected from caregivers. The present analysis included only items belonging to scales assessing child social behavior at home and the home environment, focusing on 50 items from the original 185-item instrument.
Caregivers of 692 third- through fifth-grade students attending 13 low-performing schools in four school districts in a southeastern U.S. state constituted the sample. Each school was participating in one of four ESSP projects during the 2008-09 school year. The schools by district, number, percentage of responses per school, and percentage of urbanicity/rurality of the sample are displayed in Table 1. Five elementary schools in two rural, economically depressed districts (districts C and D) (see Table 1) constituted 64% of the sample. In a project in more urban district (district B) (see Table 1), students in five elementary schools were randomly selected from among low-achieving third-, fourth-, and fifth-graders. Data were collected from about 63.7% of the targeted sample in these schools, primarily because one of the five schools failed to collect data. In another district (district A) (see Table 1), all fourth-graders in four schools were targeted. Students in three of the schools were in the second year of a three-year project that started when the students were third-graders. School number four (see Table 1) was a new school that was added to the project because many of its students had been reassigned from the three schools originally targeted. Data were collected on about 80% of the targeted population in this project.
The ESSP is designed to be useful to schools wanting to assess only those students who are struggling with social or academic problems in school or schools wanting to understand the positive and negative influences and experiences of all their students. By combining samples...