Social Determinants of Health: Out-of-School-Time Academic Programs – Minimal Academic Content

Related Pages

Findings and Recommendations

The Community Preventive Services Task Force (CPSTF) finds insufficient evidence to determine whether out-of-school-time academic programs with minimal academic content improve academic achievement among academically at-risk students.

The CPSTF also has related findings for the following out-of-school-time academic programs:

The achievement gains from out-of-school-time academic programs do not, by themselves, guarantee academic achievement in later years. Ongoing school and social environments that support learning and development are essential.

Because academic achievement is linked with long-term health, and because out-of-school-time academic programs are commonly implemented in racial and ethnic minority or low-income communities, these programs are likely to improve health equity. Equity in health is widespread, achievable equality in health and in the major social determinants of health in all the principal social divisions of a population.

The full CPSTF Finding and Rationale Statement and supporting documents for Social Determinants of Health: Out-of-School-Time Academic Programs are available in The Community Guide Collection on CDC Stacks.

Intervention

Out-of-school-time academic programs are programs provided outside of regular school hours to students in grades K-12 who are either low-achieving or at risk of low achievement. Out-of-school-time academic programs are offered during the school year—usually after school hours or during summer recess.

Programs must include minimal academic content, such as supervised time for students to complete their homework or receive homework assistance.

To address other goals, programs also may include sports and recreation, snacks, or counseling. Attendance is most often voluntary, though students may be required to participate under certain circumstances (e.g., to avoid retention in grade).

About The Systematic Review

The CPSTF findings for out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content) are based on evidence from a meta-analysis published in 2006 (Lauer et al., 35 studies, search period 1985–2003) combined with more recent evidence (25 studies, search period 2003-2011). Of the 35 studies included in the Lauer et al. meta-analysis, this review excluded three studies which reported only school grades, a relatively subjective measure of reading and math achievement. The CPSTF used evidence from an independent systematic review of summer school programs to confirm Community Guide review findings (Cooper et al., 2000).

Study Characteristics

  • All included studies were conducted in the United States.
  • Of the evaluated programs, 37 (65%) were implemented in urban areas.
  • Programs were implemented for elementary school students (28 programs; 49%), both elementary and middle school students (8 programs; 14%), middle school students (7 programs; 12%), middle and high school students (3; 5%), high school students (7 programs; 12%), or all students (4 programs; 7%).
  • The studies evaluated similar numbers of summer (29) and after-school (28) programs.
  • Study populations were predominantly racial/ethnic minorities (i.e., Black and Hispanic). Specifically, studies included majority Black students (25 studies; 43%), majority Hispanic (4 studies; 7%), majority non-White (unspecified, 2 studies; (4%), majority White (7 studies; 12%), mixed (4 studies; 7%), or did not report race/ethnicity (15 studies; 26%).
  • Most studies evaluated majority low SES populations (42 studies; 74%).
  • Studies included similar numbers of male and female students.
  • Programs employed tutoring or individualized instruction as the instructional method (17 programs; 30%), group instruction (24 programs; 42%), both tutoring and group instruction (10 programs; 18%), or did not report didactic method (6 programs; 11%).
  • Programs were reading-focused (23 programs; 40%), math-focused (7 programs; 12%), general (23 programs; 40%), or had minimal academic content (4 programs; 7%).

Summary of Results

  • Out-of-school-time academic programs led to modest improvements in academic achievement, as measured by standardized achievement tests. The degree of effectiveness was largely dependent on program focus
    • Minimal academic programs: Reading achievement: 0.07 standard deviations (SD) (1 study); Math achievement: 0.04 SD (1 study)

Summary of Economic Evidence

An economic review of this intervention was not conducted because the CPSTF did not have enough information to determine if the intervention works.

Applicability

Applicability of this intervention across different settings and populations was not assessed because the CPSTF did not have enough information to determine if the intervention works.

Evidence Gaps

  • Which features of out-of-school-time academic programs contribute to their effectiveness?
  • How does intervention effectiveness vary by the following?
    • Student characteristics (e.g., race or ethnicity, socioeconomic status)
    • Program focus
    • Degree of student participation in combination with program duration (short programs with full participation or long programs with minimal participation are likely to be less effective)
  • What activities are used in the intervention and control groups?
  • What are the long-term effects of out-of-school-time academic programs?
  • How can implementation of, and participation in, out-of-school-time academic programs be improved?
  • Transportation may be a challenge for children in low-income families. What are the best ways to address this barrier?
  • With its increased academic focus, how effective are the 21st Century Community Learning Centers programs (James-Burdumy et al., 2007)?
  • What are the actual costs of out-of-school-time academic programs?
  • What are the cost variations of different program types used in different locations?
  • What is the cost-benefit of out-of-school-time academic programs?

Implementation Considerations and Resources

Despite the finding of insufficient evidence, the following are considerations for implementation drawn from studies included in the evidence reviews of out-of-school-time academic programs (reading-focused, math-focused, general, minimal academic content), the broader literature, and expert opinion.

  • For many federal programs, oversight is the responsibility of the state, and compliance with program requirements and enforcement are commonly incomplete
  • Some school districts fail to notify parents about the availability of free programs, such as Supplemental Educational Services, and the programs are underused
  • Most out-of-school-time academic programs are voluntary, and attendance may be especially low for students most in need
  • Transportation to and from programs may be an issue for potential out-of-school-time academic programs participants
  • Inadequate staff training and staff turnover can make programs inefficient