by James C. Sherlock
This is a follow-up to my Monday report on VPI+, a federally funded four-year pilot program to assess the value of the Virginia Preschool Initiative.
Today we will discuss what was not reported to the public. We will also assess the dreadful results of the pilot participants after those kids graduated and went on the kindergarten and first grade.
Clearly, SRI International (main report) and RAND (cost-benefit report) were directed not to disaggregate the results of the data they collected by division and school. Those, of course, are the levels that give parents enough information to evaluate the program.
What was revealed, at the very end of the main report, was that disadvantaged kids participating had made learning gains compared to their disadvantaged peers who did not attend, but
“like other state public preschool programs, by spring of first grade the differences were no longer statistically different.”
That heart-breaking outcome was left un-assessed.
The mandarins at VDOE (and perhaps the federal DOE) appear to believe that pre-school is too important for parents to get involved.
If given full information, some might challenge the program or decide it is not appropriate for their own children in their local school district.
Like the domestic terrorists some of them are considered in certain circles to be.
VDOE has an outstanding school quality profiles program, yet most disappointingly the VPI+ reports did not provide school quality data for the districts or individual public schools that participated in the pilot.
Federal education funding comes with the requirement to collect those data, but were not shared at any level of disaggregation.
The data disaggregated and published in the quality profiles program for each division and each school include:
- test scores;
- enrollment by racial and ethnic, disability, economically disadvantaged, and English learner subgroups;
- learning climate to include chronic absenteeism and disciplinary data;
- teacher quality to include teacher-student ratios and teacher training and experience; and
- school readiness of the children, those meeting literacy benchmarks the key measure in this pilot.
Eleven school divisions participated in the pilot to sufficient extent that they were considered in the evaluation. There is no school quality reporting, even by division, in the report.
Crucially, there is not enough data made public for parents to decide to send their children to their local public pre-school at four years old, or, soon, at three years old, as opposed to raising them at home.
Achievement disaggregation by demographic subgroups
The pilot was carefully and well crafted to discover whether the pilot helped the kids it targeted.
During the 4-year grant, the VPI+ program served more than 5,500 children in 13 school divisions. As planned, the number of 4-year-olds participating in VPI+ high-quality preschool slots in high-need communities across Virginia increased each year. In Year 3, VDOE added two more school divisions to the initiative.
Children were only eligible for VPI+ if their families’ incomes were at or below 200 percent of the Federal Poverty Level (FPL) and they met the age criteria – 4 years old by September 30th of their preschool year. More than half of enrolled children were from households living in poverty (at or below 100% of the FPL), ranging from 53–66% across the 4 years of the initiative. Three- fourths of mothers of VPI+ children reported having completed high school (ranging from 75– 77% across years).
According to family enrollment forms, about half of the children were Black or African American (ranging from 46–55% across years), around a quarter were Hispanic (ranging from 24–29%), and less than one fifth were White (ranging from 14–18%).
Most children spoke English at home; however, about one fourth to one third of children (ranging from 26–32% in any given year) were dual language learners who spoke a language other than English at home (typically Spanish).
Approximately 7–9% of VPI+ children were identified as having a disability or delay and had an Individualized Education Program (IEP) at some point during the preschool year.
Yet the findings in the public reports are not disclosed by division and school using the criteria in the school quality profiles.
Parents and taxpayers must ask why not?
Very high lapse rate of the pre-kindergarten advantage
The second major gap is that there is no discussion of why the children from the pilot reportedly showed up in kindergarten so measurably ready to learn and drifted back to level with the control group of kids of the same ages and demographics including socio-economic disadvantages by the spring of first grade.
The longitudinal (post VPI+ student performance) analysis that looked forward through first grade created a control group.
Longitudinal study sample and analysis approach
First, we limited our analyses to only those children who would likely have been eligible for VPI+ had they chosen to enroll. This means that they were (a) enrolled in one of the 11 school divisions for kindergarten in the 2017–2018 school year, (b) between the ages of 60 months and 71 months on September 30, 2017, and (c) were flagged as economically disadvantaged (defined as eligible for free and reduced-price lunch, receives Temporary Assistance for Needy Families, or eligible for Medicaid). Second, we used coarsened exact matching … to identify comparable groups of VPI+ and non VPI+ children for the analyses.
The findings were profoundly disturbing.
Longitudinal study findings
Children who attended VPI+ performed better on literacy assessments in the fall and spring of kindergarten than did children who did not attend VPI+, but by spring of first grade the differences were no longer statistically different.
The results in the second semester of the first grade were not statistically different from the control group of equally disadvantaged kids who did not attend VPI+ schools.
What the heck happened? Are we to call that success?
The Governor’s half truth. Enter Governor Northam. The Governor’s Executive Directive Four (2019) claimed success by masking the biggest problem:
The state’s most recent expansion of quality preschool through Virginia Preschool Initiative Plus, provided by a federal grant, resulted in the elimination of the school readiness gap between economically disadvantaged children and their non-disadvantaged peers, according to analysis by SRI International.
That same Directive failed to mention that VPI+ graduates failed to maintain their advantage by the Spring of their first grade, not over the Governor’s “non-disadvantaged peers,” but rather the equally disadvantaged control group.
The Governor’s assessment was a half-truth in the service of his political objectives.
“A lie that is half-truth is the darkest of all lies.” —Alfred, Lord Tennyson
What do parents need to know? It is not just reasonable, but necessary for the public, especially parents, to know the answers to several major questions. Parents don’t send their kids to kindergarten in “Virginia Public Schools,” but to a very specific neighborhood school in their district.
What were the outcomes in different districts and schools for different demographic groups within the pilot?
- Is a mother of an English learner supposed to assume that pre-K in her local public school will work for her children? How did Black kids in pre-K in Henrico Public Schools fare in the VPI+ pilot? In kindergarten and first grade thereafter?
- Why were 76% of the kids evaluated in year two of the pilot from only five of Virginia’s 132 school districts: Chesterfield (160), Henrico (358), Norfolk (188), Prince William County (199) and Richmond City (160).
- Does that concentration yield representative results? If so, why?
- What districts were asked to participate?
Discipline and attendance. Reading between the lines of the report, some of the classrooms were chaotic. How did that affect learning compared to classrooms in which discipline was not a problem?
How did kids with good attendance learn compared to the chronically absent?
What caused the incredible regression of the VPI+ kids to the mean of the disadvantaged, non VPI+ kids in Spring of the first grade?
- Are parents to assume that, having benefited from VPI+ and ready for kindergarten, that kindergarten was not ready for them?
- Were many of their neighborhood schools so bad that attending kindergarten and first grade added no value to what they learned in pre-K?
- If so, why can’t they get charter schools?
I’m still not going to discuss transportation — buses, bus drivers and special care — for four-year olds.
VDOE. Yet having failed to provide those answers, VDOE is going forward with a statewide implementation of a VPI program.
Because it is what they want to do, not because VPI+ was a success.
The pilot methods were designed to provide the answers. The Methods section of the SRI final resort illuminates what was done.
In January 2015, the Virginia Department of Education (VDOE) in the Commonwealth of Virginia was awarded a 4-year federal Preschool Development Grant (PDG) to expand high-quality preschool programs for at-risk 4-year-olds in 11 of Virginia’s 132 school divisions2 that ranked highest in need on key indicators.
SRI used a variety of data sources to learn about VPI+ implementation and impacts. These included: (1) extant data on student demographic and enrollment characteristics, teacher and program characteristics, and ratings from classroom and teacher quality observations; (2) logs on local coaching delivered and interviews with coaches; (3) summaries of technical assistance and training sessions provided by state partners to VPI+ coordinators, coaches, and family engagement coordinators; (4) interviews and surveys with VPI+ coordinators; (5) surveys of VPI+ teachers; and (6) direct assessments of children and teacher-completed checklists to measure outcomes in the areas of language and literacy, mathematics, approaches to learning, social and emotional development, and physical well-being and motor development.
The last two questions posed by SRI in its final report were
“(3) Did VPI+ improve school readiness skills and outcomes? What percentage of VPI+ children were ready for kindergarten? Did VPI+ impact participating children’s school readiness skills and outcomes? What are the longer-term outcomes for VPI+ participants?
(4) What factors supported and challenged successful implementation? What lessons were learned that can inform similar efforts?”
They did not share the answers with the public. Those questions are not answered in the public report.
- No data or analysis of outcomes by race, English language skills or disability was disaggregated and shared at the division or school level.
- What were the effects of chronic absenteeism? Of undisciplined classrooms? Of the high concentration of evaluated pilot participants from only five districts?
- No explanation of the collapse of the VPI+ advantage other than this:
“A longitudinal follow-up study found that children who participated in VPI+ performed better on literacy assessments in the fall and spring of kindergarten than did children who did not attend VPI+, but like other state public preschool programs, by spring of first grade the differences were no longer statistically different.”
In plain language,
“It (learning stagnation of kids who went to pre-K) always happens like that.”
No inquiry as to why.
Bottom line. I am a vocal public advocate for the kids who this program is supposed to benefit Yet vital information derived from the pilot was not shared with the public.
From me and their mothers to VDOE and Governor Northam:
Thanks for nothing.
