ASSESSING TEACHER KNOWLEDGE: USING RESEARCH TO INFORM LEGISLATION AND POLICY
written by Murray Mitchell, University of South Carolina

"There are three kinds of lies: Lies, Damned Lies, and Statistics" - Benjamin Disraeli (n.d.)

Assessment is a high stakes enterprise in education because the consequences of collecting and interpreting data can change lives. For students, assessment can make the difference between passing and failing, or admission to or rejection from a college, university, or other post-high school program. For teachers, principals, and school district superintendents, assessment can result in salary adjustments and keeping or losing a job. Because of the high stakes involved, many fear what statistics may be used to inform important decisions, including the creation of legislation and to make policies.

The assessment of teacher performance is a current issue in many states. For example, there is a bill in the South Carolina legislature (House Bill 3716) designed in part to allow school districts to decide how to provide raises for teachers, and directing the Department of Education to develop an "incentive compensation system based on teacher performance" (South Carolina Legislature, 2011). A concern for any teacher must be tied to exactly how teacher performance will be assessed.

Student Outcomes as a Measure of Teacher Performance

Examining some measure of student outcome data is one possible approach to measuring teacher performance. Most teachers working in affluent schools with no behavior issues, lots of resources, and a supportive parent/teacher environment might be attracted to this kind of proposal. Teachers working in challenged environments, with sparse resources, a transient student population, minimal contact with parents and guardians, violence and substance abuse issues, might be less attracted to this kind of proposal. In spite of these kinds of discrepancies, there is evidence to support the notion that really good teachers can teach regardless of these kinds of obstacles, and that bad teachers still can't teach even with all the support in the world (c.f., Rink & Stewart, 2003).

Content Knowledge as A Measure of Teacher Performance

Another approach to assessing teacher performance is to examine their content knowledge; but, this is not as simple as it might seem. For example, the health-related fitness (HRF) content knowledge, using samples of middle school physical education teachers, inservice teachers, and preservice teachers, has been found to be less than optimal on artificial paper and pencil assessments when measuring content knowledge (Ayers, 2002; Castelli & Williams, 2007; Miller & Housner, 1998). In contrast, Mitchell (2010) used an alternative assessment strategy to assess what he described as "task-relevant knowledge" (p. A-67) and found it to be far more flattering. Rather than an out-of-context paper and pencil assessment, Mitchell analyzed the actual contracts submitted by 2,828 students to 126 teachers.

Contracts were content analyzed in a 6-step process:

• Step 1, Identify the type of activity indicated on the contract, to ensure that the activity would contribute to a health-related fitness variable;
• Step 2, Identify the health-related fitness variables, looking for two details: (1) the HRF variable matches the activity; and (2) the variable identified is one of the accepted HRF variables (cardiovascular endurance, muscular endurance, muscular strength, flexibility and/or body composition);
• Step 3, Confirm that the dates indicated span a minimum of 6 weeks;
• Step 4, Confirm that a minimum of 20 minutes per episode is indicated;
• Step 5, Confirm that a minimum of 3 episodes per week is indicated; and
• Step 6, Identify if the teacher has indicated whether or not the information presented on the contract is or is not appropriate.

In Mitchell’s (2010) study, the mean percent of correctly accepted contracts was 80.22% (SD=25.2: Median =88.35). There were no incorrectly rejected contracts. This is a much different success rate or indicator of teacher performance than was evidenced in the other studies. See Table 1 for a contrast of these assessments of teacher performance.

Table 1: Comparison of Physical Educator Competence Scores

Conceptual Clarity

Content knowledge is an integral part of effective teaching. Identifying what knowledge is necessary for effective instruction, how thoroughly that knowledge must be "known," and how to measure content knowledge are disputed in the literature. Anderson (1995) makes a distinction between declarative knowledge (descriptions of facts) and procedural knowledge (ability to "do").

Landy (1974) made a comparable distinction when he sought objective measures for performance obsolescence in work with engineers. Kelley and Lindsay (1977) tried to make a similar distinction between knowledge obsolescence and performance obsolescence when examining physical educators, when they noted poor performance on a paper and pencil test does not necessarily mean that "physical educators as a group do not have performance capabilities that enable them to fulfill effectively their responsibilities" (p. 473).

In recent work on physical educators, paper and pencil tests have been used to examine content knowledge. Inservice and preservice teachers have been found to be wanting. Such findings have been used to castigate the professionalism of many teachers, and to identify poor content knowledge as a possible contributing factor to the health crisis facing our nation.

In the Mitchell (2010) study, an alternative approach was used to examine knowledge in the context of a real (or authentic) performance task for which teachers are responsible - grading student contracts. Correctly grading the contracts requires knowledge of the HRF content for teachers to make professionally appropriate decisions. Using this authentic approach, much higher competence scores are reported for teachers.

Methods for measuring performance require more theoretical clarity than has been evident in recent studies. It is possible that measurement error, rather than incompetence, may explain some previous findings assailing physical education teacher reputations. So, in preparation for the predictable needs for more effective legislation and policy decisions, it will be important to design research studies that can help to ensure the best teachers are in classrooms, and that they are appropriately rewarded. Equally important will be the need for careful interpretation of data to inform important decisions in this high stakes assessment environment.

click for references

Murray Mitchell is an Associate Professor and Graduate Director for the Department of Physical Education and Athletic Training at the University of South Carolina. He also serves as the High School Assessment Director for the South Carolina Physical Education Assessment Program. Mitchell is also a Fellow in the Research Consortium of AAHPERD and a Fellow in the South Carolina Education Policy Fellowship Program, sponsored by the Institute for Educational Leadership.

(back to pelinks4u homepage)