Help us improve the Community Guide website Join a Usability Testing Session
The Guide to Community Preventive Services (The Community Guide) Go to site home page About the Task Force

S M L XL

Submit your email address to get updates on The Community Guide topics of interest.

Promoting Health Equity Through Education Programs and Policies: Out-of-School-Time Academic Programs

Task Force Findings

The Community Preventive Services Task Force issued separate findings for four types of out-of-school-time academic programs.

  • Reading-focused out-of-school-time academic programs are recommended on the basis of strong evidence of effectiveness in improving the reading achievement of academically at-risk students in grade levels K-3.
  • Math-focused out-of-school-time academic programs are recommended on the basis of sufficient evidence of effectiveness in improving the math achievement of academically at-risk students. Effects appear greater for older students (grade levels 7-12), compared with younger students (grade levels 2-5), although the number of studies is too small to draw a conclusion.
  • General out-of-school-time academic programs that do not focus on a specific subject are recommended on the basis of sufficient evidence of effectiveness in improving the reading and math achievement of academically at-risk students, though the magnitude of each effect is smaller than those from reading- and math-focused programs.
  • Out-of-school-time academic programs with minimal academic content have insufficient evidence to determine the effectiveness of these programs, which typically only provide homework assistance or time to complete homework.

The achievement gains apparent after out-of-school-time academic programs do not, by themselves, guarantee academic achievement in later years. Ongoing school and social environments that support learning and development are essential.

Because academic achievement is linked with long-term health, and because out-of-school-time academic programs are commonly implemented in racial and ethnic minority or low-income communities, these programs are likely to improve health equity. Equity in health is the widespread, achievable equality in health and in the major social determinants of health in all the principal social divisions of a population.

Read the full Task Force Finding and Rationale Statement for details including implementation issues, possible added benefits, potential harms, and evidence gaps.

Intervention Definition

Out-of-school-time academic programs are programs provided outside of regular school hours to students in grades K-12 who are either low-achieving or at risk of low achievement. Out-of-school-time academic programs are offered during the school year—usually after school hours—or during summer recess.

These programs must include an academic component, which can range from minimal academic content, such as supervised time for students to complete their homework or receive homework assistance to more intensive tutoring or remedial classes focused on specific subjects, such as reading or math. To address other goals, programs also may include sports and recreation, snacks, or counseling. Attendance is most often voluntary, though students may be required to participate under certain circumstances (e.g., to avoid retention in grade).

Overall, children from low-income and racial and ethnic minority populations in the United States have lower academic achievement levels than children from the higher-income and majority populations. As a long-term consequence, these children often grow to be adults with lower income levels and poorer health, perpetuating a "cycle of poverty" (Duncan et al., 1998). Out-of-school-time academic programs aim to interrupt this cycle by assisting children who are at risk for low academic achievement.

About the Systematic Review

This Task Force finding is based on evidence from a meta-analysis published in 2006 (Lauer et al., 35 studies, search period 1985–2003) combined with more recent evidence (25 studies, search period 2003-2011). Of the 35 studies included in the Lauer et al. meta-analysis, this review excluded three studies which reported only school grades, a relatively subjective measure of reading and math achievement. The Task Force used evidence from an independent systematic review of summer school programs to confirm Community Guide review findings (Cooper et al., 2000).

The effectiveness evidence is based on a systematic review of all available studies, conducted on behalf of the Task Force by a team of specialists in systematic review methods, and in research, practice, and policy related to the use of educational interventions for the promotion of health equity.

Results

  • Out-of-school-time academic programs led to modest improvements in academic achievement, as measured by standardized achievement tests. The degree of effectiveness was largely dependent on program focus.
    • Reading-focused programs: Reading achievement improved by a median of 0.31 standard deviations (SD) (interquartile interval [IQI]: 0.02 to 0.58; 23 studies).
      • 0.43 SD (IQI: 0.11 to 1.05) for students in grades K-3 (17 studies)
      • ‑0.02 SD (IQI: ‑0.06 to 0.06) for students in grades 4-12 (6 studies)
    • Math-focused programs: Math achievement improved by a median of 0.12 SD (range: ‑0.32 to 1.33; 6 studies)
      • The range of effects was ‑0.32 to 0.07 for grades 2-5 (3 studies)
      • The range of effects was 0.18 to 1.33 for grades 7-12 (3 studies)
    • General academic programs:
      • Reading achievement: 0.09 SD (IQI: 0.00 to 0.26; 21 studies)
      • Math achievement: 0.06 SD (IQI: ‑0.01 to 0.24; 20 studies)
    • Minimal academic programs:
      • Reading achievement: 0.07 SD (1 study)
      • Math achievement: 0.04 SD (1 study)
  • Summer programs were more effective than after-school programs, especially for general academic programs.

Study Characteristics

  • All included studies were conducted in the U.S.
  • 37 (65%) of evaluated programs were implemented in urban areas
  • 28 (49%) of programs were for elementary school students (K-12), 8 (14%) for both elementary and middle school students, 7 (12%) for middle school students, 3 (5%) for middle and high school students, 7 (12%) for high school students and 4 (7%) for all students.
  • The studies evaluated similar numbers of summer and after-school programs (29 and 28, respectively).
  • Study populations were predominantly racial/ethnic minorities (i.e., Black and Hispanic). Specifically, 25 (43%) of studies evaluated majority Black students, 4 (7%) majority Hispanic, 2 (4%) majority non-White (unspecified), 7 (12%) majority White, 4 (7%) mixed, and 15 (26%) did not report race/ethnicity.
  • 42 (74%) of studies evaluated majority low SES populations.
  • Studies included similar numbers of male and female students.
  • 17 (30%) of programs employed tutoring or individualized instruction as the instructional method, 24 (42%) employed group instruction, 10 (18%) used both tutoring and group instruction, and 6 (11%) did not report didactic method.
  • 23 (40%) of programs were reading-focused, 7 (12%) were math-focused, 23 (40%) were general academic and 4 (7%) were minimal academic

Applicability

  • Based on the settings and populations from included studies, results are applicable to:
    • Urban K-12 schools in the United States
    • Low-income populations
    • Summer and after-school programs

Economic Evidence

Fourteen studies were included in the economic review and reported only program cost.

  • Program costs varied largely according to program operation hours.
  • Hourly costs per student ranged from $3 to $12.90 (11 studies).

The economic evidence is based on a systematic review of all available studies, conducted on behalf of the Task Force by a team of specialists in systematic review and economic methods, and in research, practice, and policy related to the use of educational interventions for the promotion of health equity.

Considerations for Implementation

The following considerations are drawn from studies included in the evidence review, the broader literature, and expert opinion.

  • For many federal programs, oversight is the responsibility of the state, and compliance with program requirements and enforcement are commonly incomplete.
  • Some school districts fail to notify parents about the availability of free programs, such as Supplemental Educational Services, and the programs are underused.
  • Most OSTA programs are voluntary and attendance may be especially low for students most in need.
  • Transportation to and from programs may be an issue for potential OSTA participants.
  • Inadequate staff training and staff turnover can make programs inefficient.

Supporting Materials

Publications

Knopf JA, Hahn RA, Proia KK, Truman BI, Johnson RL, Muntaner C, Fielding JE, Jones CP, Fullilove MT, Hunt PC, Qu S, Chattopadhyay SK, Milstein B, and the Community Preventive Services Task Force. Out-of-school-time academic programs to improve school achievement: a Community Guide health equity review. Journal of Public Health Management & Practice 2015;21(6):594–608.

Community Preventive Services Task Force. Out-of-school-time academic programs are recommended to improve academic achievement and health equity. Journal of Public Health Management & Practice 2015;21(6):609–12.

Read other Community Guide publications about Promoting Health Equity Through Education Programs and Policies in our library.

Promotional Materials

Community Guide News

More promotional materials for Community Guide reviews about Promoting Health Equity Through Education Programs and Policies.

References

Cooper H, Charlton K, Valentine JC, Muhlenbruck L. Making the most of summer school: a meta-analytic and narrative review. Monographs of the Society for Research in Child Development 2000;65(1, Serial No. 260).

Duncan GJ, Yeung WJ, Brooks-Gunn J, Smith JR. How much does childhood poverty affect the life changes of children? American Sociological Review 1998;63:404-23.

Lauer P, Motoko A, Wilkerson S. Out-of-school-time programs: a meta-analysis of effects for at-risk students. Review of Educational Research 2006;76:275-313.




Disclaimer

The findings and conclusions on this page are those of the Community Preventive Services Task Force and do not necessarily represent those of CDC. Task Force evidence-based recommendations are not mandates for compliance or spending. Instead, they provide information and options for decision makers and stakeholders to consider when determining which programs, services, and policies best meet the needs, preferences, available resources, and constraints of their constituents.

Sample Citation

The content of publications of the Guide to Community Preventive Services is in the public domain. Citation as to source, however, is appreciated. Sample citation: Guide to Community Preventive Services. Promoting health equity through education programs and policies: out-of-school-time academic programs. www.thecommunityguide.org/healthequity/education/outofschooltime.html. Last updated: MM/DD/YYYY.

Review completed: December 2013