AT-A-GLANCE
Evaluation and Infant/Toddler Child Care
Every two years States and Territories engage in the process of developing plans for the use of the Child Care and Development Fund (CCDF) to provide assistance to families to pay for child care and to improve the quality of care. While the majority of funds are used for child care subsidy, a great deal of attention is given to the potential uses of the quality set-asides and earmarks. CCDF administrators and other early care and education leaders want to find the most effective uses of funds to improve services for children. Because the years from birth to three are critical to children's development, school readiness, and future success, the effective use of the infant/toddler earmark and other funds for infant/toddler services is of particular concern. This document provides information about ways States and Territories can use system and program evaluation to determine the effective use of funds for children under three.
There are two key ways States and Territories can think about the use of evaluation in infant/toddler child care. The first approach is to assess or evaluate the early childhood system for its ability to provide quality early care and education services for children under three. This can be done as part of a comprehensive early childhood systems planning process, such as the Maternal and Child Health Bureau's, or in a more targeted approach such as the method used by the National Infant & Toddler Child Care Initiative. System evaluation often combines objective (quantifiable) data measured against quality system indicators combined with subjective (qualitative) information gathered from early childhood system professionals and other stakeholders. System evaluation can provide the "baseline" to measure quality improvement efforts over time.
A second way of thinking about evaluation of infant and toddler child care is at the program level. States are increasingly interested in evaluating the impact of their investments of funds on infant and toddler program quality and caregiver practice. In the FY 2004-2005 CCDF plans, 48 States (including the District of Columbia ) and 3 Territories reported they used infant/toddler earmark funds for professional development of caregivers. Reported professional development activities included training (sometimes linked to a credential), scholarships/stipends, and the support and training of infant/toddler specialists to work with programs and caregivers. Obviously, understanding how professional development initiatives impact programs and caregiver practice is of primary concern when a large percentage of infant/toddler earmark funds are invested.
Developing Infant and Toddler Child Care Evaluation Efforts
Early care and education leaders can consider how evaluation efforts for birth to 3 services can be built into new initiatives or ongoing evaluation efforts. Possible approaches include:
- Incorporating evaluation requirements in contracts, particularly for new or large initiatives,
- Working with the higher education system and researchers to develop capacity, funding and interest in evaluating initiatives to improve the quality of infant/toddler child care and caregiver practice,
- Focusing existing data and information collection efforts on infants, toddlers, and their families, and
- Insuring that evaluation tools are measuring structural and process indicators that reflect the developmental needs of infants and toddlers.
Incorporating Evaluation Requirements in New and Existing Initiatives
Ohio , Kansas , and Washington State have incorporated evaluation efforts in their development of infant/toddler and health specialist networks and services.
Ohio's Infant Toddler Initiative Evaluation
First Steps is the Ohio Bureau of Child Care and Development initiative to create an infrastructure of infant/toddler specialists and services that could support the emerging needs of Ohio 's youngest and most vulnerable learners. First Steps is funded out of the infant toddler quality earmark and managed by the Ohio Child Care Resource and Referral Agency (OCCRRA). Twelve infant/toddler specialists were hired and placed in the resource and referral agencies across the State. In the pilot year a decision was made to determine the quality of infant toddler child care across all care settings and to "test out" various levels of intervention with providers to find out which model yields the greatest degree of change as evidenced by pre and post rating scale scores. More than 900 providers were recruited to make up a stratified sample and they were randomly assigned to one of three intervention groups: low, medium, and high. All participants have their settings assessed using either the ITERS or FDCERS. The low intervention group gets the profile mailed to them (45 percent). The medium intervention group gets a summary report and must attend community based training based on PITC (45 percent). The high intervention group gets the summary report reviewed with them on site, community training, on-site coaching and technical assistance, visits from Healthy Child Care Ohio nurses, curriculum resources, and training. To date more than 600 settings have been assessed, 275 technical assistance visits have been provided, and more than 1,100 infant/toddler caregivers have attended training. The intent is to use the data to analyze the findings and use it for strategic planning for improvement of infant/toddler child care and effective use of CCDF earmark funding.
Evaluation funded by: Ohio 's CCDF Infant/Toddler Earmark
Evaluation conducted by: Ohio State University
Contact Information:
Terrie Hare, Bureau Chief
Bureau of Child Care and Development
Office for Children and Families
Ohio Department of Job and Family Services
255 East Main Street , 3rd Floor, Columbus , OH 43215
E-mail: haret@odjfs.state.oh.us
Jamie Gottesman, Assistant Bureau Chief
Bureau of Child Care and Development
Office for Children and Families
Ohio Department of Job and Family Services
255 East Main Street , 3rd Floor, Columbus , OH 43215
Phone: (614) 466-1043
E-mail:gottej@odjfs.state.oh.us
Kansas' Infant/Toddler Project Evaluation
From the beginning of the Kansas Infant/Toddler project there was a desire to understand how the project could impact the quality of child care for its youngest children. The evaluation efforts began with a design of how to track the project services followed by a survey of provider awareness and use of the services. The survey also asked providers who had used the services whether they had made any changes in their practice. In 2000 the Kansas Association of Resource and Referral Agencies Infant/Toddler Project began another phase of its evaluation efforts by conducting a statewide census of infant/toddler child care capacity. This was followed by enrollment surveys in 2001and 2002. These efforts provided estimates of the number of infants and toddlers in care by setting. The next evaluation effort, The Child Care Quality Study, focused on establishing a baseline of infant/toddler child care quality, documenting the services received by the participants and evaluating the contributions of specialized training, technical assistance and resource materials to the quality of infant/toddler care. The study was completed with a stratified sample of 196 providers. Provider interviews and direct observations of program quality were conducted to provide a baseline description of provider quality characteristics, educational experiences, and child care quality during the first years of the Infant/Toddler Project. A second wave of interviews and observations were conducted: (a) to describe characteristics and quality one year after baseline, (b) to track participation over that year, and (c) to determine the relationship of participation in the Infant/Toddler Project to quality outcomes and change over a 1-year period. The results of the study indicate those providers who receive multiple services of the Infant/Toddler Project including professional development and training, technical assistance, and resource materials, made significant increases in the quality of care offered to infants and toddlers in Kansas .
Evaluation funded by: United Methodist Health Ministry Fund
Evaluation conducted by: David Norlin, Bethany College & Jane Atwater, University of Kansas
Evaluation reports at: http://www.kaccrra.org
Contact Information:
Lana Messner, Project Director
P.O. Box 2294, Salina, KS 67402-2294
Phone: (785) 823-3343
E-mail: lana@kaccrra.org
Washington State's Healthy Child Care Infant and Toddler Initiative
The Washington State Department of Health was interested in determining the effectiveness of their initiative to improve the quality of child care for infants and toddlers. The Initiative weaves together multiple strategies to enhance quality including: the development of local and State systems to link health, safety and early childhood programs, training and technical assistance to child care health consultants, consultation and training to providers and parents, and information and referral to parents to assist in the placement of infants and toddlers in child care settings. Healthy Child Care Washington (HCCW) worked with Organization Research Services to design and implement the evaluation. The HCCW evaluation tracks actual changes in consultative practices in systems, child care providers' awareness and behaviors, parent and child care providers' communication, and child care policies and environments. The initial evaluation implementation results report a "significant impact on the health and safety of children in child care."
Evaluation funded by: Washington State Department of Health
Evaluation conducted by: Organizational Research Services
Evaluation report at: http://www.healthychildcare-wa.org/EvaluationResults.htm
Contact Information:
Sonja Dorda, M.A., Washington State Department of Health
Maternal and Child Health
Healthy Child Care Washington Initiative
New Market Industrial Campus
7171 Cleanwater Lane , Bldg. 7, Olympia , WA 98504-7880
Phone: (360) 236-3575
E-mail: sonja.dordal@doh.wa.gov
Developing Capacity and Interest in Evaluating Infant/Toddler Child Care and Caregiver Practice
The Federal government and States are working with higher education and researchers to develop effective evaluation efforts of initiatives to improve the practice of caregivers and early childhood professionals caring for infants and toddlers. Here are some examples of evaluations that are in process that use CCDF funding and partner with higher education and researchers.
Did you know that the Child Care Bureau has played a significant role to help develop and fund evaluation capacity. Since FY 2000, the Child Care Bureau has awarded approximately $10 million per year to States, higher education institutions and research organizations to support child care research and evaluation, including infant and toddler child care studies.
Impact of the Creative Curriculum® on Caregiver Practice and Classroom Quality, Oklahoma
The State of Oklahoma Department of Education has sponsored a two-part study. The first is an implementation study including pre-and post-test scores on classroom quality and implementation and focus groups of participants. The second part is a randomized experimental study of preschool programs in Tulsa , Oklahoma . The study includes center-based infant-toddler programs and preschool programs using The Creative Curriculum® and programs not using The Creative Curriculum®. The State of Oklahoma has contracted the evaluation to David Connell. The earliest results of the implementation study are expected in late 2005. Further findings on child outcomes are expected in 2006.
Evaluation funded by: Oklahoma Department of Education
Evaluation conducted by: David Connell
Contact information:
Jill Soto, Center for Early Childhood Professional Development, College of Continuing Education
University of Oklahoma
1801 N. Moore Avenue , Moore , OK 73160
Phone: (888)-446-7608 or (405)-793-3409
E-mail: jsoto@ou.edu
Web Site: http://sde.state.ok.us/home/
Mississippi , Comparison of Three Methods of Right From Birth Training for Infant/Toddler Child Care Providers
This evaluation compares the effectiveness of three training methods incorporating the Right From Birth Training Series on caregiver practice and child care quality. The three training methods were compared to answer whether gains in caregiver knowledge led to actual changes in the quality of care they provided to young children; could the benefits of RFB training be demonstrated with less than six workshops; and would the benefits be greater if intensive on-site coaching is provided.
Evaluation funded by: Child Care Bureau & the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services
Evaluation conducted by: Sharon & Craig Ramey and colleagues
PowerPoint at: http://che.georgetown.edu/presentations/Baltimore%20CCB%20Presentation.ppt
Contact Information:
Sharon & Craig Ramey, Georgetown Center on Health and Education
Georgetown University , Box 571107
3700 Reservoir Road, N.W. , Washington , DC 20067-1107
Phone: (202) 0687-8818
E-mail: ctr5@georgetown.edu
Focusing Existing Data and Information Collection Efforts on Infants, Toddlers, and Their Families
States with resource and referral systems, State data capacity initiatives, and research grants have the potential to focus data and information collection efforts on babies/toddlers and their families and the early childhood system that serves them. All States and Territories can begin by using the information collected by the National Infant & Toddler Child Care Initiative (http://nccic.org/itcc) available in State profiles and an infant/toddler initiatives database that will help build understanding of the capacity, quality, and needs of infant and toddler child care programs and caregivers. Additional information is available through the National Child Care Information Center (http://nccic.org).
Insuring That Evaluation Tools Measure Indicators That Reflect The Developmental Needs of Infants And Toddlers
Choosing the right evaluation tool to determine the effectiveness of interventions to improve infant/toddler child care quality needs to be carefully considered. Common evaluation designs use assessment tools to measure quality indicators before and after services are delivered to determine the impact of the intervention. Because the relationship between caregivers and babies is key to the baby's development, it is particularly important to select tools that measure their interaction. The most common tool in use today is the Infant Toddler Environmental Rating Scale Revised Edition (ITERS-R) (2003), by Thelma Harms, Debby Cryer, and Richard M. Clifford. ITERS is designed to assess group programs for children from birth to 2.5 years of age and measures process quality. Process quality refers to environmental influences on quality, such as adult-child interaction, learning activities and other characteristics related to caregivers' styles of relating to children. (NCCIC). States are also using The Family Day Care Rating Scale (FDCRS) also by Thelma Harms and Richard M. Clifford, to assess family child care programs conducted in a provider's home. Another commonly used tool for infant/toddler child care settings is The Caregiver Interaction Scale (1989). It was developed by Jeffrey Arnett and is designed to assess the quality of caregivers' interaction with children. Many evaluations use multiple tools. The Evaluation of Mentoring Training Program for Infant/Toddler Center-Based Child Care Providers , Harrisburg , PA , is an example. The Capital Area Early Childhood Training Institute at Penn State in Harrisburg , PA. designed an onsite mentoring program that paired seasoned professionals with center-based infant/toddler child care providers. The Institute was interested in determining whether the program was effective in producing measurable change in provider behavior and program quality. A randomized trial evaluation was designed that employed four measures, including ITERS, the Knowledge of Infant Development Inventory (KIDI), measuring the caregiver's knowledge of child development, the Bloom Work Environment Survey, measuring the organizational climate of the center, and the Arnett Caregiver Interaction Scale. The findings supported the effectiveness of the mentoring program in improving quality of care.
Evaluation funded by: Pennsylvania Department of Public Welfare, Wells Foundation, Whitaker Foundation, and the Greater Harrisburg Foundation
Evaluation conducted by: Capital Area Early Childhood Training Institute
Paper at: http://caecti.org/docs/publication/mentoring_article.doc
Contact Information:
Richard Fiene, Ph.D., Director, Penn State ECTI
2001 North Front Street, Suite 314 , Harrisburg , PA 17102
Phone: (717) 233-5276
E-mail: rjf8@psu.edu
Resources
The National Child Care Information Center has produced a three-part series of documents that discuss child assessment, program evaluation, and accountability systems. Assessment and Evaluation: Becoming an Educated Consumer Part II: Program Evaluation (June 2005) provides information, definitions, examples and resources about program evaluation in early care and education services. Resources listed include:
Early Childhood Curriculum, Assessment, and Program Evaluation: Building an Effective, Accountable System in Programs for Children Birth through Age 8 (November 2003), a revised Joint Position Statement of the National Association for the Education of Young Children (NAEYC) and the National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE). This resource is available on the Web at http://www.naeyc.org/about/positions/pdf/CAPEexpand.pdf
Evaluating Early Childhood Programs: Improving Quality and Informing Policy (July 2003), Vol. 23, No.6, by Walter Gilliam and Valerie Leiter, produced by ZERO TO THREE. This resource is available on the Web at http://zerotothree.org/ztt_journal.html
Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action (January 2004), produced by the W.K. Kellogg Foundation. This resource is available on the Web at http://www.wkkf.org/Programming/ResourceOverview.aspx?CID=281&ID=3669
A Toolkit for Evaluating Initiatives to Improve Child Care Quality (2003), by Toni Porter, et al., produced by Bank Street College of Education, Institute for a Child Care Continuum. This resource is available on the Web at http://www.bankstreet.edu/iccc/toolkit.html
Using Results to Improve the Lives of Children and Families: A Guide for Public-Private Child Care Partnerships (n.d.), by the Child Care Partnership Project. This resource is available on the Web at http://nccic.org/ccpartnerships/results.pdf
When and How to Use External Evaluators (November 2002), by Tracy Rutnik and Marty Campbell , produced by the Annie E. Casey Foundation through the Funders Evaluation Initiative. This resource is available on the Web at http://www.aecf.org/publications/data/using_external_evaluators.pdf
Other publications and resources that are available to help evaluate CCDF programs and initiatives include:
The Program Manager's Guide to Evaluation This resource is available on the Web at http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eval/index.html
Measuring Program Outcomes: A practical approach . United Way. This resource is available on the Web at http://national.unitedway.org/outcomes/resources/mpo/
References
Fiene, Richard, 2005. How On-Site Mentoring Improves the Quality of Infant and Toddler Child Care Providers, A Summary of the Program and Evaluation . Paper presented at Society for Research in Child Development Biennial Meeting, Atlanta , Georgia . Accessed on-line May 29, 2005, Child Care and Early Education Research Connections. http://childcareresearch.org/location/ccrca6026
The National Child Care Information Center . Assessment and Evaluation: Becoming an Educated Consumer Part II: Program Evaluation (June 2005)
Ramey, Sharon & Craig, 2005. Right From Birth (RFB), Findings from a Randomized Controlled Trial Comparing 3 Methods of RFB Training . Power Point presentation accessed on-line May 29, 2005 at http://che.georgetown.edu/presentations/index.html.
|
