Welcome to The Community Guide! Let us know what you think of the website by completing this quick survey.

Health Equity: Out-of-School-Time Academic Programs – General

Tabs

What the CPSTF Found

About The Systematic Review

The CPSTF findings for out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content) are based on evidence from a meta-analysis published in 2006 (Lauer et al., 35 studies, search period 1985–2003) combined with more recent evidence (25 studies, search period 2003-2011). Of the 35 studies included in the Lauer et al. meta-analysis, this review excluded three studies which reported only school grades, a relatively subjective measure of reading and math achievement. The CPSTF used evidence from an independent systematic review of summer school programs to confirm Community Guide review findings (Cooper et al., 2000).

The effectiveness evidence is based on a systematic review of all available studies, conducted on behalf of the CPSTF by a team of specialists in systematic review methods, and in research, practice, and policy related to the use of educational interventions for the promotion of health equity.

Context

Overall, children from low-income and racial and ethnic minority populations in the United States have lower academic achievement levels than children from higher-income and majority populations. As a long-term consequence, these children often grow to be adults with lower income levels and poorer health, perpetuating a "cycle of poverty" (Duncan et al., 1998). Out-of-school-time academic programs aim to interrupt this cycle by helping children who are at risk for low academic achievement.

Summary of Results

Detailed results from the systematic review are available in the CPSTF Finding and Rationale Statement pdf icon [PDF - 234 kB].

The systematic review included 57 studies of out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content).

  • Out-of-school-time academic programs led to modest improvements in academic achievement, as measured by standardized achievement tests. The degree of effectiveness was largely dependent on program focus.
    • General academic programs
      • Reading achievement: 0.09 standard deviations (SD; 21 studies)
      • Math achievement: 0.06 SD (20 studies)
  • Summer programs were more effective than after-school programs, especially for general academic programs.

Summary of Economic Evidence

Detailed results from the systematic review are available in the CPSTF Finding and Rationale Statement pdf icon [PDF - 234 kB].

The economic review included 14 studies for out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content). Only program costs were reported. Monetary values are reported in 2012 U.S. dollars.

  • Annual program costs ranged from $623 to $8705 per student and varied greatly by hours of operation (14 studies).
  • Hourly costs per student ranged from $3.06 to $13.17 (11 studies).
  • Major cost drivers included salaries for teachers and staff, costs for facilities and utilities, and transportation costs, with salaries being the largest expense.

Applicability

Based on the settings and populations from included studies used in the systematic reviews of out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content), results are applicable to

  • Urban K-12 schools in the United States
  • Low-income populations
  • Summer and after-school programs

Evidence Gaps

Additional research and evaluation are needed to answer the following questions and fill existing gaps in the evidence base for out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content).

  • Which features of out-of-school-time academic programs contribute to their effectiveness?
  • How does intervention effectiveness vary by the following?
    • Student characteristics (e.g., race or ethnicity, socioeconomic status)
    • Program focus
    • Degree of student participation in combination with program duration (short programs with full participation or long programs with minimal participation are likely to be less effective)
  • What activities are used in the intervention and control groups?
  • What are the long-term effects of out-of-school-time academic programs?
  • How can implementation of, and participation in, out-of-school-time academic programs be improved?
  • Transportation may be a challenge for children in low-income families. What are the best ways to address this barrier?
  • With its increased academic focus, how effective are the 21st Century Community Learning Centers programs (James-Burdumy et al., 2007)?
  • What are the actual costs of out-of-school-time academic programs?
  • What are the cost variations of different program types used in different locations?
  • What is the cost-benefit of out-of-school-time academic programs?

Study Characteristics

The following characteristics describe studies included in systematic reviews of out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content).

  • All included studies were conducted in the United States.
  • Of the evaluated programs, 37 (65%) were implemented in urban areas.
  • Programs were implemented for elementary school students (28 programs; 49%), both elementary and middle school students (8 programs; 14%), middle school students (7 programs; 12%), middle and high school students (3; 5%), high school students (7 programs; 12%), or all students (4 programs; 7%).
  • The studies evaluated similar numbers of summer (29) and after-school (28) programs.
  • Study populations were predominantly racial/ethnic minorities (i.e., Black and Hispanic). Specifically, studies included majority Black students (25 studies; 43%), majority Hispanic (4 studies; 7%), majority non-White (unspecified, 2 studies; (4%), majority White (7 studies; 12%), mixed (4 studies; 7%), or did not report race/ethnicity (15 studies; 26%).
  • Most studies evaluated majority low SES populations (42 studies; 74%).
  • Studies included similar numbers of male and female students.
  • Programs employed tutoring or individualized instruction as the instructional method (17 programs; 30%), group instruction (24 programs; 42%), both tutoring and group instruction (10 programs; 18%), or did not report didactic method (6 programs; 11%).
  • Programs were reading-focused (23 programs; 40%), math-focused (7 programs; 12%), general (23 programs; 40%), or had minimal academic content (4 programs; 7%).

Publications