Making Educators Accountable for Student Outcomes

This is fifth in a series of articles about Virginia’s Standards of Learning assessments.

By Matt Hurt

In 2011 the Virginia Board of Education added a new criteria, Standard 7 Student Academic Progress, for evaluating teachers and administrators. Previously, 100% of the criteria used to evaluate educators had consisted of inputs — lesson delivery, lesson planning, school improvement planning, etc. — with no consideration of student outcomes. A teacher who arrived on time, delivered a captivating lesson during the principal’s classroom observation, and submitted impeccable lesson plans each week could receive an exemplary evaluation — even if his or her students failed to pass a minimum-competency test of grade-level standards, the Standards of Learning (SOLs), at the end of the year. Standard 7 changed the game by giving 40% weight to student outcomes.

Even with the new measure of student outcomes, evaluations did not always correlate to student outcomes. Many divisions implemented a pre-test/post-test process, which was intended to measure student progress over the course of the year. The pre-test, consisting of content that would be covered during the year, was administered at the beginning of the year and the post-test at the end. As one would expect, students always performed better after being exposed to the material than after. Most divisions considered this improvement a sign of student growth.

The problem with the pre-test/post-test scheme is obvious — students could “show progress” but still fail to meet the standards required to advance to the next grade. An analysis of the relationship between teacher ratings and SOL outcomes found in one division, for instance, that 45% of the teachers who had exemplary evaluation ratings and 40% with proficient ratings had fewer students passing the SOL test than is required for the school to be accredited (75% in English and 70% in other core areas).

By contrast, the analysis showed, divisions that better aligned evaluations with student outcomes realized significant improvements in SOL performance and saw a shrinking gap between racial/ethnic subgroups.

Wise County implemented a rigorous version of aligning evaluations with outcomes in 2012. SOL pass rates improved from 26th in the state in 2011 to 16th in 2012 (out of 132 public school divisions). Wythe County implemented a number of instructional improvements, including revamping the evaluation process in 2016 (as well as implementing the Comprehensive Instructional Program). SOL outcomes improved from 58th in 2015 to 13th in 2016. Both divisions have continued to improve their state standings since then.

Unfortunately, the Virginia Board of Education has moved in the opposite direction over the past few years. Focused on equity issues, including the persistent achievement gap between racial/ethnic subgroups, VBOE approved updates to guidelines that reduced accountability for student outcomes. Currently, according to the guidelines, student outcomes are “not the least weighted of the performance standards or less than 1 (10 percent); however, [they] may be weighted equally as one of the multiple lowest weighted standards.”

Schools and school divisions are entities, not individuals. Schools and divisions do not work with students to learn academic skills necessary to be successful in life. Teachers do that. Principals lead and support their teachers to make sure that happens. Superintendents make sure principals have the support and teachers have the resources they need.

Schools and school divisions cannot be held accountable for much of anything. Virginia’s school accreditation process attempts to hold schools accountable for student outcomes, but we have many examples of schools whose students perform below acceptable standards year in and year out. Unless individuals are held accountable, there is no real accountability. If a school needs 75% of the students to pass English SOL tests and 70% in other content areas to be accredited, and the principal and teachers don’t pull their load, accreditation won’t happen.

There are some pitfalls to holding individuals accountable. For example, students take their first SOL tests in third grade. If the Kindergarten, 1st-grade, and 2nd-grade teachers don’t give students rigorous instruction and support, a 3rd-grade teacher doesn’t have much of a chance getting their wards to pass their SOL tests. Similarly, if a school board overturns a principal’s decision to discipline a teacher for a serious infraction, then other teachers learn that they can do as they please. In such instances, it would be unconscionable to hold the 3rd-grade teacher or the principal accountable for students’ outcomes.

From the individual educator’s perspective, the experience in Wise and Wythe was positive. One problem that gets under the skin of hard-working educators is knowing that colleagues are not pulling their share of the load. Knowing that everyone is actually being held to the same performance standard is comforting. Teachers also buy into the system when students arrive in class each fall more proficient in their prerequisite skills than in the past.

When Wise and Wythe counties updated their accountability systems to tie educator evaluations to student outcomes, they did not experience a massive overturn in personnel either due to firings, disciplinary actions, or educators choosing to go elsewhere. The administrators in those divisions worked diligently to explain why the evaluation goals were reasonable and attainable. Teachers and staff internalized the new expectations, and they produced better results.

If we wish to eliminate the achievement gap, we must hold educators more accountable, not less, for student outcomes. As was demonstrated in “Poverty Not Destiny for Educational Performance,” and “How the Best Teachers Help At-Risk Students Succeed,” helping at-risk students may not be easy but it is achievable. If we downplay the role of individual student outcomes in educator evaluations, we cannot expect better outcomes for the educational system as a whole.

To read more about Virginia’s evaluation systems for teachers, principals, and superintendents, please visit VDOE’s Performance and Evaluation page.

Matt Hurt is director of the Comprehensive Instructional Program based in Wise.


Share this article



ADVERTISEMENT

(comments below)



ADVERTISEMENT

(comments below)


Comments

29 responses to “Making Educators Accountable for Student Outcomes”

  1. LarrytheG Avatar

    I got a little confused here:

    ” Unfortunately, the Virginia Board of Education has moved in the opposite direction over the past few years. Focused on equity issues, including the persistent achievement gap between racial/ethnic subgroups, VBOE approved updates to guidelines that reduced accountability for student outcomes. Currently, according to the guidelines, student outcomes are “not the least weighted of the performance standards or less than 1 (10 percent); however, [they] may be weighted equally as one of the multiple lowest weighted standards.”

    how did they reduce accountability and what can be measured beyond or different than academic progress? Is there some way to “measure” “equity”?

    And yes, the 3rd grade is the crucible. If you are a teacher in the 3rd grade and your kids come woefully unprepared, it won’t be the K, 1st and 2nd grade teachers that are held “accountable”.

    And I have to ask – how many parents would really understand how this accountability process really works if every grade is not measured and some kids who are learning disadvantaged are sent out for some period of time to reading and math specialists?

    This would seem to be a fairly heavy duty bureaucratic process… for the non-SOL grades.. where the teachers grading A, B, C may not track with the SOLs at all.

    I thank you once again for your practical and rational , non-partisan approach to this issue and it makes me wonder just how Charter schools would be required to do something like this – or not – and we treat them as automatically assuming they will do a better job.

    1. Matt Hurt Avatar

      The point I was trying to make with this article was that while the Board of Education states that they are trying to focus on equity, their strategy of making educators less accountable for student outcomes is very much antithetical to that goal.

      For the non-SOL grades, there are other means by which to measure student progress. Simply put, common assessments that everyone administers in a school or division are frequently used for that purpose.

      As far as pulling students from class, our most successful schools don’t pull them from math or reading class for help. They pull them during other times of the day for EXTRA help. It’s not good that they miss that other instruction (usually history or science), but there’s only so many hours in the day, and you do have to make priorities.

      1. LarrytheG Avatar

        I agree on the equity thing. I see where California is considering reducing math standards to focus more on equity – I could not disagree with that more vociferously, nor any version of it in Virginia.

        I would like to hear more about how to measure academic performance outside the SOL grades. It’s clearly needed IMHO.

        Finally, for kids that have serious learning disabilities, for example, dyslexia, or even being 2 grades levels back – their ability to “learn” is seriously impaired until they get specific help for that. They can be “in class” but inevitably they’re going to fall behind or the teacher is going to be put in the position of having to choose between the majority of the class advancing and staying on grade level.

        That makes that teacher’s job so much harder trying to balance the disparate needs.

        I knew a principal who would try to look at the academic status of each kid in an entire grade level prior to specific teacher/classroom assignment. He would then configure individual student membership in each classroom based on their academic status – trying to give each teacher a fair share of the harder to teach ones – as opposed to doing it willy nilly and the newest teacher getting way more of the harder ones.

        He also would hold weekly “reviews” that focused on the progress on the at-risk kids and decisions made when they were falling back.

        This school principal was not the norm in that school system, but he achieved school success over several years and was “kicked upstairs” to try to promote that approach school-wide – which is a much, much harder job.

        Which I also imagine is a similar challenge in the Region 7 approach – i.e. trying to get all school principals and teachers on board with the “model” approach.

        In all of this , we are talking about creating “standard models” as opposed to letting each principal and each teacher decide on their own how to accomplish the job.

        And really, not that much unlike the approach to SOLs and SOL curricula mapping. It just takes it to a different level in my mind.

        1. Matt Hurt Avatar

          Larry, your assumption that students with disabilities, or students who are behind will inevitably fall behind is not true. I’ll give you the case of JW Adams Combined School, a PRrK-8 school in Wise County. The achievement of their at-risk populations as so close to the non at-risk population that they won a national blue ribbon award for closing the gaps. Ricky Bolling is their principal, and if you asked him what they do different for their struggling students, such as SPED students, he will tell you that they do nothing different. They provide the instruction and supports that EACH STUDENT needs to be successful.

          Teachers in that building maintain the same rigorous instructional pace during core instruction for each student, and students who need additional help are pulled during their exploratory classes by their classroom teachers- those individuals who know the student needs the best, and who are best situated to help them. Therefore, those kids get EXTRA instruction, not help in lieu of regular core instruction.

          That principal you mentioned was smart. When you create “good” groups of kids and “hard to teach” groups of kids, that really screws with teachers’ expectations, and screws over the “hard to teach” kids.

          https://schoolquality.virginia.gov/schools/j-w-adams-combined#fndtn-desktopTabs-assessments

    1. Matt Hurt Avatar

      It’s political drama, not unlike all of the Hunter Biden mess. Serious folks would do well to disregard both.

      1. Nancy Naive Avatar
        Nancy Naive

        But not K12 CRT, right? That was real.

        1. LarrytheG Avatar

          I’d be curious to hear Mr. Hurt’s opinion on CRT also.

        2. Matt Hurt Avatar

          There are/were some players who want to take it to atrocious and harmful ends, but what’s actually being officially recommended is not negative in my opinion. However, it is problematic in how it’s being sold. Instead of engaging with folks who are resistant, they have been in their own policy bubble with like minded folks, and can’t understand what about this that folks oppose. More real conversations would make this a non-issue. However, nobody on either side are interested in the conversation. Each is comforted by the fact that they’re doing God’s work in opposition to their enemies who are working to keep them from it.

          Also, what they’re doing with this won’t move the needle with regards our subgroup gaps. Most of our teachers and administrators are sympathetic for our underperforming subgroup populations, so much so that they hold those kids less accountable. Some call this the soft bigotry of low expectations. What we need is training on how these kids can be successful- educators have to do their part, here are some folks who did it, and here’s what they did.

          1. LarrytheG Avatar

            Well said Matt. Thanks!

  2. LarrytheG Avatar

    One of my teacher friends informs me that if kids were identical widgets that he would be so much easier to “measure” progress!

    And earlier Mr. Hurtt had opined about “low expectations” which I interpreted to mean teachers awarding grades for work that may not be aligned with SOL measures so that a kid might get an A in reading but actually fail the SOLs for reading!

    Seems like unless there are standardized measures used in each grade , it will be hard to hold each teacher truly accountable.

    I foolishly thought that’s what was done early one but soon was “educated” as to the facts which are that SOLs are not tested in every grade and the academic status of a kid in a grade not tested for SOL may or may not be known with the same precision.

    That makes teachers in the SOL grades – on the bubble.

    And it don’t take much imagination in a neighborhood school that serves a low income demographic that it’s a tough gig for all teachers but especially so for the SOL teachers. So what teacher who has a choice of where to work in the less rural, more urban areas of Virginia are going to choose to be employed in a troubled school.

    Easy to point fingers and assign blame, but how do we fix this?

    And despite all the promises and hopes for “Charter” schools, if they take low income kids with poorly educated parents – they ain’t going to fix this very easy either also despite the Hocus Pocus Conservative advocates of Charters who we virtually never hear them also advocating for measuring student achievement and accountability in the Charters.

    1. Matt Hurt Avatar

      You’re exactly right. Many of our urban divisions serve as the training ground for their less urban neighbors. They recruit teachers from all over, spend a lot of time and resources bringing them up to snuff, only to have them leave for greener pastures as soon as a better offer is made. It’s a terrible thing, as it really hurts those divisions. However, I can’t blame the teachers for voting with their feet for the division with the better teaching conditions.

      I believe that this issue is one of the biggest visual, symptomatic issues affecting these divisions. If conditions in those places (whether it be school culture, discipline, cumbersome state mandated school improvement processes, etc.) could be improved, it would likely go a long way to stabilize the teacher and administrator positions in those places. This would in turn improve the capacity for improvement.

      1. James C. Sherlock Avatar
        James C. Sherlock

        As for teachers in tough schools leaving for greener pastures, that is not always the case. It is not true at the profoundly and consistently low performing MLK Jr. Middle School in Richmond which I have studied in detail. From a response to a FOIA request l was able to determine that out of a very large staff, all but about three (I am traveling and the spreadsheet is not in front of me) from the 2018-19 school year are still on staff today, including the principal and all three APs. And that school may be the worst performing middle school in Virginia. They are likely good people, and I congratulate them for trying, but they have shown no capability as a group to educate the kids in their care. The RPS is run for the adults, not the children.

        1. LarrytheG Avatar

          What do you think a Charter School (or other non-public school) would do different than RPS and would their teaching staff be different than RPS teaching staff?

          How would you hold Charter or non-public schools accountable for their performance?

      2. VaNavVet Avatar

        You mention discipline which usually means taking the student out of the classroom for some type of suspension. Teachers are expected to maintain a positive learning environment in the classroom via discipline of disruptive students. Unfortunately, this often impacts special needs students the most. Additionally, athletes do often miss many classroom hours which the struggling ones have a hard time compensating for. The attitude is that there is more to school than academics particularly for minority students.

        1. Matt Hurt Avatar

          When I worked in a PreK-8 school as an administrator, I only removed a student from class for disciplinary reasons for just long enough to get the student to stop the disruptive behavior that he/she was exhibiting in class. Then there was additional in-school suspension time assigned during non-core class instruction (lunch and/or exploratory classes). Some students misbehaved just to get out of class. When they realized that they would loose their “social” time with their friends instead of class time, they were much less likely to be involved in the misbehavior in the future. We also implemented progressive discipline, so more ISS was added during their lunch and/or exploratory classes for each infraction. We also called parents/guardians immediately upon each infraction. These strategies helped to decrease our discipline referrals each of the four years I was there.

  3. Dick Hall-Sizemore Avatar
    Dick Hall-Sizemore

    Outstanding article. Teachers should be held accountable, both in positive and negative ways.

    I always thought teachers, especially those with educational disadvantaged kids, should not be judged on SOL scores, but on the progress that the kids made while in his or her class. But, the study you cite disabuses me of that notion. It is best to hold kids to high standards.

    Matt, you identified the major flaw in the idea of tying evaluation to SOL results. If a teacher in the SOL grade gets a kid who has had inadequate preparation in previous ways, that teacher has a problem. However, you did not discuss how to get around that problem. Ideally, of course, all the K-3 teachers in a school view themselves as a team and view the SOL scores as reflecting on them all, not just the third grade teacher. I don’t know how often that happens. Is there a practical way to implement that concept or is that just what good principals do?

    1. Matt Hurt Avatar

      Very infrequently do the K-3 teachers view themselves as a team and take collective ownership of those 3rd grade scores, though they all should.

      There’s a couple of strategies to mitigate this issue. First, you have to hold principals accountable. Those elementary principals are responsible for making sure that the K-2 program in their school produces newly minted third graders each year who arrive to school in the fall with the vast majority of prerequisites that they need to be successful on the third grade skills. Second, I’m not convinced that there should be the same expectation for student performance set for every teacher, or even the same teacher from year to year. I believe that you have to evaluate the conditions at hand and then set reasonable, achievable, and rigorous goals for performance.

      1. LarrytheG Avatar

        This is what I’ve been told also by teachers I know – that prior to the SOL grades, efforts are made to ascertain the academic status of each child and it can vary a lot depending on different factors including the experience and competence of the teacher but also very much the child and children of low-education, low-income parents are often problematical in that regard. They do not get the same level of reading help from their parents, sometimes the parents work odd hours and the kids might be staying with relatives or grandparents and the parents life may not be stable as they move around and change jobs. Children in these life situations are harder to keep on grade level and more prone to fall behind especially over summer breaks.

        A teacher, even a very good one, is at a disadvantage when trying to teach such kids.

        We hear a lot about Charters and these disadvantaged kids but most private schools, charters, and the like are dealing with kids of much better life circumstances, kids with higher educated parents, kids and parents who are motivated and who live in more stable family situations.

        The public schools don’t do as well with these kids. It’s well known in places like Richmond and other but what’s less recognized is that even very good school systems like Henrico and Fairfax ALSO have numbers of such kids with lower SOL scores and persistent academic gaps. The common thread is the socio-economic status of the kids – who, through no fault of their own – NOR their teachers do not have the same lifestyle advantages as kids whose parents are better educated and work at higher paying jobs and have economic security , and a more stable home life.

        Mr. Hurtt speaks of the success of the SW CIP program – and clearly has had success with the economically disadvantaged kids by adopting “models” that “work” for teaching these kids.

        These “models” sound like collaborations of principals and teachers to work together to use techniques that are proven to work, but other systems outside of SW Va apparently don’t do things that way.

        I think there may also be a difference between the stereotypical “poor” rural family and children and urban poor families – one being the rural may live on family-owned land and urban in subsidized rental units that even in not in “projects” are in lower income neighborhoods. The schools that serve those neighborhoods have much higher numbers of economically disadvantaged kids AND the teaching positions are much more difficult and problematic and do not attract highly skilled veteran teachers but the opposite. Newbies and others who end up with much more limited options for employment and take these positions even though they may be woefully under-qualified for teaching large numbers of economically disadvantaged kids.

        The blame game intimates that teachers be evaluated and judged and fired if they do not perform, but in these low-income neighborhood schools – high quality replacement teachers are not standing in line to take over – it’s the opposite, teachers with options will avoid these schools like the plague because the jobs are so much more difficult and if they get a bad class and turn out a bad class, their career is at risk.

        It’s no secret, good teachers avoid these schools for employment because of the fear of being presented with more than they can handle and damage to their career for “failure”.

        They’ve successfully dealt with a version of it in Region 7 and I do believe how Region 7 has approached the issue is effective with smaller numbers of economically disadvantaged kids but I have doubts it could work in a place like Richmond or even in low-income neighborhood schools in Henrico, Chesterfield, Fairfax, etc.

        If Youngkins plan is Charters and/or to essentially “bus” ED kids to better public schools of their “choice” – I’m still a skeptic.

        I’m more than willing to agree to do it – with the proviso that ALL schools, no matter public or voucher or charter be fully accountable for SOL scores and if Charters and Voucher schools perform better – for whatever reason, I’m all for using more public money for more of them. Conversely, we also hold them accountable by requiring the same SOLs and if they fail then recognize that it takes more than calling any given school a Charter or voucher or “choice” school to actually change outcomes.

        Region 7 has significant success. We should build on it but I think it may not work as-is in the more urban poor schools and may need to be altered a little, but the idea that you may have newbie teachers dealing with large numbers of ED kids, it makes perfect sense to establish “models” like Region 7 has – but you also have to have a way to measure performance in the grades that are not SOL.

        .

        1. Matt Hurt Avatar

          The big thing that Region VII has going for it is that teachers are not taking every opportunity to get out as they are in some of the urban divisions. Think of it like this. What Region VII has done with the CIP is to erect a very nice house on top of a solid foundation, i.e. a relatively stable staff of teachers and administrators. Some of these urban divisions don’t have that solid foundation. Until those issues are fixed, i.e. teachers and administrators actually want to work there, they don’t have a snowball’s chance of improving outcomes for their kids.

          This is one thing that a charter can possibly help with. Before I start down this road, it is important to understand that there are successful charters as well as those which produce dreadful results. From what I have read, they average out about even with traditional public schools. IF, you can create a charter school which has a very positive culture in which educators are beating down the doors to get in, it will have the conditions necessary for really good outcomes for students.

          Some of these urban divisions have been struggling for years, and can’t seem to get their foundations shored up. I’ve always heard that the true mark of insanity is to continue to do the same things, and expect different results. Sometimes, a charter can provide the insulation from the systemic failings of the broader division and allow folks on the ground in that building to address their specific issues better. I don’t for a second believe that a charter is a magic wand or an easy button that will ensure success, but at least it can provide some flexibility to get around the systemic failings of the division.

          In fact, Karin Chenoweth writes out such ideas in her most recent book, Districts that Succeed. She highlights how Chicago public schools realized some significant gains by decentralizing leadership from the central office to the schools. Each school had autonomy to hire their own staff, make curriculum decisions, and basically had their own board of parent and community advisors.

          I think the big point is distributed leadership. This can be done in traditional public schools as is widely done in Region VII, or through charters. If the division cannot provide the culture and climate to retain teachers, maybe a dynamic school leadership team untethered from unsuccessful division practices/mandates can.

          Charters can be public or private. If they accept state funding, they’re public, and they’re accountable for the same accreditation standards as any other public school in the state.

          1. LarrytheG Avatar

            “distributed” … “works” if there are standards… IOW… figure out how to operate but produce results.

            What will gain loyalty especially for new teachers is if they are supported and are not fearful of failing…or thrown to the wolves with way too many hard-to-teach than they can handle – IMHO of course.

            I like conversation – appreciate it.

            good stuff!

  4. James Wyatt Whitehead Avatar
    James Wyatt Whitehead

    I always thought it was unfair that the SOL core subject teachers were held to higher standards than say the PE teacher, or the shop teacher, or the band teacher, or the sharpest knife in the drawer who always got a paddy cake non SOL course schedule just because they were golfing pals with principal. How did Region 7 and the CIP consortium utilize the numerous non SOL teachers as assets to accountability?

    1. Matt Hurt Avatar

      As with anything else, it’s done well in some places, and less well in others. Basically, those with it principals look at the needs of their school and compares those with the capabilities of their non-SOL teachers. Whenever a match is made, that becomes how that teacher can pull his/her weight.

      1. LarrytheG Avatar

        how does a principal hold a non-SOL teacher accountable and it’s not seen as subjective and arbitrary with respect to all the non-SOL teachers in a school? What standard ?

        1. Matt Hurt Avatar

          Well, the term in favor these days is the SMART goal (Specific, Measurable, Achievable, Relevant, and Time bound). The principal sits down with the teacher, and they agree upon the goals that will be included in the evaluation. Where you run into a problem is if you have a teacher with an external locus of control who believes that they can’t make a difference and their outcomes are predetermined by some other force in the universe. Obviously, we don’t want folks with this type of mindset in the business, so this will take care of itself.

          1. James Wyatt Whitehead Avatar
            James Wyatt Whitehead

            My brief experience wit SMART before I left in 2020 was not impressive. It seemed to be some made up goals that checked off boxes so the nice administrator I was working with could bet back to the bucket brigade and put out school fires.

          2. Matt Hurt Avatar

            This has not been done well in most places based on my experience. I would bet most evaluations are conducted simply to mark that task off the admin’s to do list.

  5. Don Tabor Avatar

    Pretest/posttest is a faulty concept. It weighs against the student who comes to the new year ahead of his peers.

    In college, I filled a PE requirement by taking archery, figuring that since I have been bowhunting since I was 14 it would be an easy course. First day, we were told to shoot 6 arrows at 20 yards. Showing off, I shot 6 bullseyes. Then we were told we would be shooting at 45 yards at the end of the year and would be graded on improvement. Serves me right but…

    My grandchildren all read well ahead of their age group. How are they going to improve? How will the teacher who gets a class whose parents have been involved be scored against a class with less involved parents?

    1. LarrytheG Avatar

      Appreciate you weighing in. I think that is a legitimate concern… and it puts enormous stress on a teacher who is going to be held accountable for all the kids – those behind as well as the advanced.

      And this can lead to what is known as “tracking” which can doom kids who are behind and essentially do something similar to them that is done to kids who are advanced already and ready to go further and cannot.

      both cases – there are kids ready to advance but the lower kid is more more subject to harm that keeps him from even meeting minimum standards or graduate.

      This is a challenge for all schools but especially public schools who do not have the luxury of rejecting students that don’t meet their standards – and in my mind – a key issue with charter and voucher schools… that are premised on “helping the poor kids” but, in fact, end up catering to kids with advantages who are easier to teach and want a school that helps them go further.

Leave a Reply