Lessons Learned

top-blog-header-4

TOP staff and evaluators had several tips for programs embarking on a similar study:

Take the time to develop relationships. A recurring theme in the story of this project is the strength of TOP’s partnerships. These relationships have been fundamental to carrying out such a large-scale study.

Be transparent. Let both partners and participants know exactly what you are studying and why. It will be vital to both recruitment and participation, and will ultimately improve data quality.

Don’t be afraid of change. Too often, we feel stuck within the confines of our previous decisions. A key strength of the TOP study has been the willingness to acknowledge what isn’t working and fix it. Be pragmatic in your choices and flexible in response to new information.

Allow for your study to expand. Programs and populations change over time; so, too, do the goals of research. Periodically review your research design to determine whether you’re really getting what you need to make decisions and inform others.

Be persistent about tracking procedures. The TOP Longitudinal Study has achieved such high participation rates across levels by being dogged about asking and following up.

“However long you think it will take, double it.” Data collection and analysis are rarely efficient and linear processes. Do your best to plan efficient methods of collection and synthesis, but know that no one gets everything right the first (or second, or third…) time.

Go big! Small additional investments in evaluation can yield large returns in terms of impact. The results of this study are exceptionally useful and compelling because of the scale of the project.

Keep your eyes on the prize. TOP staff, evaluators, and school faculty and administration are all committed to the value of early education, and the power of demonstrating its impact. Your research should be motivated by the same goals that guide your work: improving the lives of at-risk kids.

top-blog-footer4

TOP Key Features

top-blog-header-3

 

After interviewing the key players who have developed and conducted this ambitious study, and reviewing all reports and instruments, we have identified four key features that make this study distinctive.

Excellent Participation

TOP staff noted that communication is key to this relationship. top-blog-3-bulletsTOP built relationships with school districts, schools, teachers, and administrators, educating them about the goals of the study and why it was important to their overall mission.

Parents were provided with a longitudinal study brochure explaining the premise of the study. TOP program staff noted that generally the only families who do not give consent were guardians of foster children. In order to keep current TOP graduate and parent contact information, TOP staff sends out reminder postcards annually, and maintain a Facebook page (Top South, Top North, Top Northwest) to reach out to parents. Family consent was only half the battle. In order to collect data for the study, school districts’ and teachers’ willingness to cooperate was crucial.

In addition to education and incentivizing participation, WSU researchers have worked to make the teacher survey less cumbersome, so it takes less time for them to complete. Teacher participation has also been excellent. One TOP staff stated that teachers want more TOP children in their classrooms because of their readiness for school.

Targeted Use of Professional Researchers

A second key feature of the TOP study is the targeted use of professional researchers. Rather than simply handing over the reins of the project to WSU researchers, TOP has collaborated with them on the design, data collection, and communication of the results. There are multiple benefits to this approach of using professional researchers in targeted ways. It seems to create a nice collaborative relationship in which the researcher is closely working with staff to ensure they are producing something useful to them. Families and school districts may be more inclined to participate in a study when approached by an excellent local preschool, rather than a previously unknown university researcher.

Strategic Approach to Data

A theme throughout our interviews with TOP staff and WSU is focusing on the key information needed to understand TOP’s impact.

  1. Data or analyses are not strictly necessary to convey the point of the project and may be a waste of effort.
  2. Being selective about what data are presented is key to communicating to a broad audience.

It simplifies data collection greatly, and likely makes it easier to get district and teacher cooperation. Furthermore, it may be true that including individual-level variables is just not that important to persuading others.

There are multiple benefits to this strategic approach to data. It offers necessary focus: on the goals of the project; on respecting time and effort of partners and staff; and finally, on the intended audience. This is not a study designed to sit on a shelf; it’s a study designed to persuade laypeople about the value of high-quality preschool. Second, a strategic approach to data has offered staff and evaluators the flexibility to change what’s not working, or could be working better.

Commitment to Demonstrating Outcomes

Demonstrating outcomes is a central value of this organization. Being able to track and demonstrate effectiveness was always a secondary goal of TOP. The value of doing so seems self-evident to every person we spoke to: all perceived that continued support of TOP is dependent on being able to show long-term results on issues the public cares about.

The study aims to produce convincing data that quality early childhood programs improve children’s well-being – and doing so is considered to be essential to TOP’s ability to continue, grow, and thrive as a program.

top-blog-3-readinessThis level of commitment to demonstrating results can be rare in the world of early childhood services. Many early childhood program staff worry that evaluation takes time and money away from the real work of the organization – the work that will directly benefit children. In a context in which everything is tight – time, money, expertise – and the need is vast, growing, and desperate, it can be difficult to justify directing resources towards evaluation.

In a world in which demand is great and resources are limited, it is crucial that programs be as effective as they can be in helping their target populations. This requires tracking outcomes continuously, systematically weighing the evidence, and continuously looking for ways to improve. Evaluation is part of how you make sure you are helping kids to the best of your ability.

top-blog-footer3

TOP Methods and Results

top-blog-2-header

The TOP Longitudinal Study was designed to provide policy makers, business leaders, and other stakeholder evidence of TOP’s effectiveness over the long term.

Wichita State University researchers issued the first annual report on the longitudinal study in 2009. Over the course of the next 6 years, these reports have demonstrated clear quantitative results. Currently, the study comprises 810 TOP graduates and a control group of 2,596 comparable children.

Eligibility and Consent: The primary criterion to participate in the longitudinal study is eight months of continuous enrollment in the TOP program during the year preceding kindergarten entry. A consent form to participate in the longitudinal study is administered to parents or guardians. Consent covers a TOP graduate through elementary school. A second consent is obtained for middle and high school participation.

Teacher Survey: The social questionnaire asks teachers to compare the TOP graduates to the remainder of the children in their class in order to determine whether the TOP graduates used appropriate social skills more often than their peers.

School Performance Data: WSU researchers compare TOP graduates’ school performance data to a demographically-matched control group using descriptive statistics, including: averages and percentages.

Data Synthesis and Analysis: Since a primary goal of the study is to communicate with non-technical audiences such as members of the business community and policymakers, the reports largely rely on descriptive statistics.

top-blog-2-graphicResults

For all grades, teachers rated TOP graduates as having greater emotional maturity, greater ability to behave appropriately, and greater social competence than their classmates. Teacher estimations of TOP graduates’ social-emotional skills were statistically significantly higher than average.

For all grades, TOP graduates have noticeably fewer absences on average than their control group peers each year, from kindergarten through the 6th grade.

When the Wichita and Derby school districts are combined, TOP graduates have a lower percentage of special education placements than their control group peers.

Discipline referrals were taken into account to determine behavioral problems. Only repeat office visits were a measure of discipline referral. When looking at the total numbers of kindergarten through 6th grade students, TOP graduates were much less likely to have discipline referrals.

TOP children in 4th, 5th and 6th grades have a considerably lower percentage of students who do not meet state standards in both math and reading compared to the control group.

top-blog-2-footer

About TOP

top-blog-1-header

TOP: An Exemplary Program for Demonstrating Effectiveness

A critical component of collaborating to improve early childhood outcomes is tracking results to demonstrate effectiveness. The Kansas Children’s Cabinet and Trust Fund is a strong proponent of rigorous evaluation practices and continuous quality improvement. The Cabinet’s Blueprint for Early Childhood offers a vision of what success looks like, in broad terms and measurable goals while offering suggestions of specific measurement tools.

Over the past seven years, a Wichita preschool has engaged in a project unprecedented in Kansas early childhood education: tracking their graduates’ outcomes throughout grade school and continuing to follow them through high school graduation. The story of how this ambitious study came to be is largely the story of the school itself: Wichita’s The Opportunity Project, commonly referred to as TOP.

top-blog-1-mapThe first TOP Early Learning Center opened in 2003. Currently, three TOP Centers serve nearly 700 young children, preparing them for kindergarten and lifelong success. Since 2007, researchers at Wichita State University have been tracking TOP graduates as they progress through elementary and middle school to demonstrate TOP’s effectiveness to a wide audience of stakeholders, including community members, business leaders, and policymakers. This rigorous evaluation provides an opportunity to analyze the return on investment associated with this innovative program, which The Cabinet has presented in The TOP Early Learning Centers Longitudinal Study.

The TOP Longitudinal Study discusses the TOP model and describes the study and its results. The four key features that make the TOP study distinctive:

Excellent participation: The TOP longitudinal study can boast an impressively high level of cooperation among families, teachers, schools, school administration and districts. 95% of all TOP graduates have consented to participate.

Targeted use of professional researchers: TOP collaborates with WSU researchers on all aspects of the study, including design, data collection, and communication of the results.

Strategic approach to data: The study employs a thoughtful approach to collecting and reporting on data, which focuses on the key information needed to understand TOP’s impact.

Commitment to demonstrating outcomes: The study aims to produce convincing data that quality early childhood programs improve children’s well-being – and doing so is considered to be essential to TOP’s ability to continue, grow, and thrive as a program.

top-blog-1-footer

Today’s Block Tower is Tomorrow’s Downtown High-Rise: 21st Century Skills Start Early

As our younger generations grow more adept at technological advancements, educators and employers are searching for the tools to both encourage and enhance these skills. Incorporating technology alone may miss key components of preparing a future workforce, bringing about a more recent focus on what are called 21st Century Skills, or abilities such as collaboration, creativity, problem solving, and communication.

At the elementary, middle, and high school level, programs that address the need for 21st Century Skills are emerging both nationwide and in Kansas, including the EPIC Program (Education Practice and Immersion for Credit) recently piloted by the Kansas Enrichment Network  http://www.kansasenrichment.net. In cooperation with after-school programs such as the Boys & Girls Club and The City, as well as municipal authorities, the EPIC pilot program gave at-risk students in Salina and Hutchinson experiential learning opportunities with local industry and business leaders, and focused on conscientiousness, positive self-evaluation, social skills, and cognitive processes. To adequately prepare for the future, young people need to be capable of integrating their technological and media savvy with equally important talents in teamwork and relationships.

When does this start? Ask anyone in the field of Early Childhood, and he or she will tell you it begins well before Kindergarten. The development of 21st Century Skills is the foundation of quality toddler and preschool classrooms, like those supported by the Cabinet. Teachers arrange environments with changing materials, unexpected methods, and room for flexibility and experimentation, all the while encouraging questions, wonder, and creativity. In every interaction that looks like basic play – taking turns, arguing over toys, negotiating rules of games – young children are practicing how to be a good co-worker, contribute to a team, listen to others’ ideas, and accomplish tasks together. Today’s block tower is tomorrow’s downtown high-rise! Kansas is looking toward a bright future, and the skills to get there are being cultivated far earlier than you might think.

Kansas Businesses Recognized for Innovative Child Care Solutions

Experiences and environment during the earliest years of a child’s life are responsible for shaping the brain and setting a foundation for everything ahead. For all Kansans, the path to a brighter future depends upon safe, educational, attentive care for ages birth to 5, yet for those living in poverty, access to affordable, high-quality child care may be the single most important factor in making positive change for their families. Whether provided by family, in a center, or in a child care home, improving the quality of child care options in our communities is a key to bettering our state overall.

On February 2, 2016, over 200 child advocates, child care providers, community and business leaders, and policymakers convened for the second annual Symposium for Early Success. Hosted by the Partnership for Early Success, this event was created to elevate the conversation around issues impacting the state’s youngest children, with a focus this year on the importance of high-quality child care.

Four Kansas employers were recognized for their efforts to provide or enhance child care options, access, or affordability for their employees or within their communities. The 2016 Business Awards for Innovative Child Care Solutions honored:

  • The Hispanic Chamber of Commerce of Greater Kansas City, for promoting the development and adoption of Spanish-language training modules for child care providers and promoting in-home child care providers as small businesses in their communities.
  • Topeka Housing Authority, for including on-site child care services for working parents or parents looking for jobs.
  • Salina Regional Health Center, for conducting a community-wide, $6 million capital campaign called Every Child Matters to build, furnish, and equip the Donna L. Vanier’s Children’s Center.
  • Hill’s Pet Nutrition, for implementing a series of family-friendly policies, making it much easier for working parents to care for their children.

These employers are examples of the impact that is possible when the unique needs of a region are assessed and ideas turn into collaborative action. Each has recognized that building prosperous communities requires an effective workforce – which is best comprised of supported and strengthened families.

 

The Family Conservancy: Talk, Read, Play with a Child Every Day Campaign

Mother helping her little child with homework

 

Recent studies of early brain development prompted the American Academy of Pediatrics to recommend that pediatricians advise all parents to read aloud to children — beginning as infants. Even before a child can talk, hearing words from the adults in their lives helps their brains grow. In fact, 80% of a child’s brain is developed by age three and differences in brain development, based on language, can be detected as early as 18 months.

Studies show that children born into low-income families hear roughly 30 million fewer words by the age of three than their more affluent peers. This word gap leads to an alarming achievement gap – in school and life – between poor children and those born into more well-to-do families. Fortunately, there is a simple and effective way to eliminate this gap: Talking, Reading, and Playing with a child every day is proven to increase that child’s vocabulary and school success.

“Every parent wants their child to do well in school and in life. And the most important things you can do to ensure school success are to start early by talking, reading and playing with your child every day – beginning at birth,” said Dean Olson, Vice President of Programs at The Family Conservancy, a founding partner of the Talk, Read, Play educational campaign established by a coalition of agencies in 2009. The Talk, Read, Play campaign simplifies studies about brain development in young children into a few simple steps every parent and caregiver can take to help support a child’s early development: Talk,

Read, Play with your child every day. These simple activities help a child learn words, enjoy learning and get along with others … important skills for school success. Children’s success in school leads to positive outcomes within our communities: greater high school graduation rates, more skilled and contributing citizens, and a reduced need for welfare.

You can find ideas to make Talking, Reading and Playing with your child a part of your everyday routines by visiting www.TheFamilyConservancy.org or www.bornlearningkc.org.

#TalkReadPlayKC #CloseTheWordGap

The Cultural Awareness, Respect, and Engagement (C.A.R.E.) Toolkit Helps Home Visitors work with Refugee Families

Family.

Talking to families about the details of their lives is always an exercise in both empathy and tact. Now imagine that the family whose home you entered moments before not only speaks a different language but hails from a distant part of the world — a place they may have escaped under traumatic circumstances — and follows cultural practices you’ve never heard of, much less encountered first-hand.

That was the challenge facing home visitors working with refugee families in Kansas. Social workers are trained to deal with a variety of circumstances and experiences, but it’s hard for them to employ those skills when even the most basic interaction is fraught with potential misunderstandings. From etiquette to eye contact to working with interpreters, home visitors found themselves in uncharted territory — trying to teach people from other cultures to navigate new lives in Kansas without access to any of the familiar landmarks.

The Cultural Awareness, Respect, and Engagement (C.A.R.E.) Toolkit, introduced in the fall of 2015, was designed as a resource for home visitors in these situations. Created through a partnership between the Kansas Department of Health and Environment’s MIECHV (Maternal, Infant and Early Childhood Home Visiting) and Project LAUNCH programs, and KU’s Center for Public Partnerships and Research (CPPR), the C.A.R.E. toolkit offers introductory webinars on issues that come into play when communicating across cultures, as well as background information on the specific groups home visitors are likely to encounter in Kansas.

“Home visitors were running into a lot of new experiences,” said Adam Brazil, a CPPR researcher who oversaw the development of the toolkit. “Their jobs are very difficult, and any time you can weave in development or support … that’s going to help them, whether directly or indirectly.”

Topics covered by the C.A.R.E. toolkit range from explaining the concept of ‘culture’ to illuminating the diversity among cultures in areas such as world view (for example, how time is measured and understood, and whether more emphasis is placed on individual or collective needs) and behavioral factors (e.g. hand gestures and facial expressions, which can have radically different meanings from one culture to the next).

According to Brazil, the toolkit is about “equipping home visitors with some strategies to think critically about what they’re seeing and to reflect on themselves.”

Learning to recognize their own cultural paradigms helps home visitors understand that not all values are universal. People from other countries may have entirely different expectations regarding education, health care, politics, money, religion and family. Even among refugees from Burma — one of the primary refugee populations in Kansas — there are multiple ethnic subgroups, each with distinct traditions and beliefs. Working with an interpreter can bridge the language gap, but also presents its own set of challenges for a home visitor unaccustomed to working through an intermediary.

Both the toolkit and related training sessions have begun to address these and similar issues likely to confront those assisting refugee families with the assimilation process. Working in cooperation with Catholic Charities of Northeast Kansas and the Bethel Neighborhood Center in Kansas City, Kansas, the C.A.R.E. initiative offers support and advice for those who spend their days helping others.

To learn more about the C.A.R.E. toolkit, or test your own cultural competence, visit http://kshomevisiting.org/care-toolkit/

SIDS Awareness

5547912701_3bb86b4346_b

October is national SIDS Awareness Month, and the Kansas Infant Death & SIDS Network, Inc. (KIDS) has a number of events planned to mark the occasion (see below). Raising awareness about safe sleep practices is an ongoing challenge, however. KIDS Network Executive Director Christy Schunn notes that while many people have gotten the message that babies should be put to sleep on their backs, other factors that can endanger a sleeping infant are less widely recognized.

“We need to make sure that every infant is sleeping alone, on the back, in a crib,” said Schunn, explaining that ‘alone’ also means no bumpers, stuffed animals, loose blankets or other potential impediments to breathing. Given both their lack of muscular control and sensitive respiratory systems, “it doesn’t take a lot for a baby to stop breathing.”

A clutter-free crib is only part of the ideal sleeping environment. “We focus on safe sleep, but we know that breastfeeding and non-smoking environments reduce the risk of SIDS,” Schunn said. According to Dr. Michael Lu, associate administrator for Maternal and Child Health with the federal Health Services and Resources Administration (HRSA), up to a third of SIDS cases nationally involved smoking.

As understanding of the contributing factors linked to SIDS or Sudden Unexpected Infant Death (SUID) grows, KIDS and its many local and national partners are working to change the behaviors that put sleeping babies at risk. In addition to collaborating on smoking cessation programs, KIDS Network gives away hundreds of portable cribs at community baby showers, provides safe-sleep kits to hospitals and physicians, and last year trained home visitors in 100 of 105 Kansas counties to deliver safe sleep education. They also offer support to families who have endured such a devastating loss.

“Our mission started with bereaved families, and changing the outcome for other people,” Shunn explained. “It’s that energy, that passion, that grief that I think moves the needle.”

From improving child care legislation to putting cribs in the hands of expectant mothers, KIDS and its partner organizations are promoting not only the message about safe sleep but the means to make it happen — regardless of income or education, age or geography.

“Infant mortality is a depiction of how healthy our community is, but when we’re talking about ‘these babies,’ they’re our babies,” said Schunn.

Upcoming Events

Oct. 3: Community Baby Shower for Safe Sleep, Wichita

Oct. 10: Wade’s Ride fundraiser, supporting safe sleep programs at Newman Regional Health, Emporia

Oct. 17: Community Baby Shower for Safe Sleep, Topeka

Nov. 7: Luke’s Community Baby Shower for Safe Sleep, Geary County

Bullying: How to Recognize and Prevent It

Bullying has garnered national attention in recent years from parents, policymakers, educators, and practitioners alike. Bullying is defined as a repeated act of aggression toward a child who is weaker in some way, such as physical size or social status (Merrell et al., 2008; Olweus, 1991). Thus, the two primary features of bullying behavior that differentiate it from other kinds of aggressive behavior and normal peer conflicts are: 1) its repeated and chronic nature, and 2) an actual or perceived power imbalance between the child engaging in bullying and the child targeted.

Recent tragedies (e.g., the suicides of Tyler Clementi, http://www.tylerclementi.org/tylers-story; Megan Meier, http://www.meganmeierfoundation.org/statistics.html) bring to bear the most severe consequences of repeated exposure to bullying and peer victimization. Although a rare occurrence, these deaths make clear that early prevention and intervention is necessary to offset the likelihood of experiencing challenges later in life. Most commonly, those targeted by peers are at a heightened risk for experiencing a wide range of social, emotional, and academic difficulties, including loneliness, social anxiety, depressive symptoms, and challenges focusing in school (Card et al., 2008).

As the school year begins, we must be mindful that, while most children do not experience peer victimization or engage in bullying behavior, approximately 25% – 30% of children in elementary school do (Nansel et al., 2001). Although rates of bullying decline as children age, cyber bullying (e.g., occurring through electronic media, such as text messaging, Facebook, or online games) is thought to peak in middle school and persist as the most common form in high school.

The most effective strategies to prevent and intervene with bullying involve broad and collaborative responses, bringing families, schools, and community professionals together. As a starting place, on-going communication between children and the adults in their lives is needed to identify children at risk for being involved in bullying. Many times, those targeted do not actively report their experiences to anyone – thus the more we ask children about their experiences at school and with peers, the more we can learn about their risk for being involved in bullying.

Many resources are available online that discuss a wide range of intervention options and supports for students involved in bullying. For up-to-date, comprehensive information and research, visit http://www.stopbullying.gov/. For a cyberbullying specific resource, see http://cyberbullying.us/.

Anne Williford, PhD
Associate Professor
School of Social Welfare
University of Kansas