Performance
People train animals for many reasons; predominantly to help their animals learn and master new skills, learn and master new behaviors, and to replace non-optimal behaviors with better ones. (People also train animals for the enjoyment and bonding experience, but that's not the central focus of this discussion.)
Central to these reasons is the need for feedback — the need to understand whether the animal (and handler) are actually acquiring the intended skills or behaviors. To provide this feedback we measure team performance.
This section discusses Training Corner's model of performance measurement.
TopPrinciples of Performance Measurement
When we think about performance measurement, we tend to apply models and concepts from human endeavors (such as education, business, etc.). However, measuring performance of an animal (or an animal-handler team) has a number of important differences we need to be aware of.
- Subjectivity. In human education there is a blend of objective and subjective evaluations. (What is the capital of Kenya? The square root of 64? versus Describe poetically your feelings from the field trip to the zoo.) In animal training, the facts are rarely clear-cut and even the most objective forms of evaluation have a significant subjective component. (I.e., evaluating animal performance is more like judging figure skating than grading a math test.)
- Observation. If performance measurement is always (at least partly) subjective, then it is important to approach that subjectiveness in a way that minimizes arbitrariness and bias. In Training Corner, this means that formalized observation (with evidence collection) is an integral part of every performance evaluation.
- Evidence. Training Corner has built-in facilities for collecting a variety of evidentiary materials — photos, videos, audio recordings, etc. These materials provide valuable corroboration of observations. Whenever you record performance in Training Corner, you will always have an opportunity to include them.
- Expectations. A key element in the measurement process —
and one which helps address issues of arbitrariness and bias —
is documentation; what you want to measure, how you plan to perform the
measurement and what results (at different levels of achievement)
look like and mean. There are a number of clear advantages to being
explicit up-front with how you will measure perfromance.
- Thinking and planning ahead make for a better process.
- Documenting the process means you can delegate it colleagues and assistants and maintain consistency.
- Most importantly, documenting the process means you can communicate this information with your clients. Not only does this allow them to be prepared (and hence know what to practice), but it also helps them understand what you are looking for — what "correct" means.
- Sharing. If one purpose of a rubric is to help inform (and set expectations) for your client teams, then it is important for this rubric to be shared. There should be no surprises.
We can summarize the performance measurement process as having three parts.
- Development of the rubric, which describes what and how the measurement will take place.
- The actual performance measurement itself, which involves applying the rubric and collecting corroborating evidence.
- The recording of the measurement in Training Corner.
Training Corner identifies five styles of performance measurements. In the sections which follow, we will describe each one; show how to document a measurement of each style in a rubric, and how to record the results.
TopUsing Rubrics and Documenting Performance
Training Corner provides multiple performance assessment opportunities:
- Class Summary — a summary of a team's performance in class.
- Session Summary — a summary of a team's performance in a session.
- Observation Segment — a recording of what you observe during an observation segment of a session.
- Exercise Segment — an evaluation of a team's performance during an exercise segment of a session.
- Test Segment — the results of the test administered during a test segment of a session.
Training Corner provides a number of different styles of measurement; that is, different ways to approach performance assessment.
- Observation: while all assessment styles include an observational component, the observation style is exclusively observation.
- Pass/Fail: a style of assessment where the team is judged to either meet requirements (as specified in the rubric) or not.
- Likert: a style of assessment where performance is grouped into performance categories. We are all familiar with this type of rating system — "on a scale of 1 to 5 …".
- Points: a style of assessment where the team performs actions for which point values are attached. The assessment is the sum of the points awarded.
- Deductions: a style of assessment where the team performs a set program of actions and points are deducted for missed or incorrect elements. The assessment is the sum of the deductions.
You need to match up the goals of your class, the various assessment opportunities that are available, and the styles of assessment that are most appropriate to each opportunity. For example, if you are offering a certification preparation class you will likely make different choices than if you are offering a behavioral modification consultation class.
TopAccessing Performance Data
Accessing rubrics is pretty easy; accessing performance data is a bit more involved because you may have many teams on your roster and (for each team) many performance measurements. So it will take several taps to focus in on the data you want. You have several options:
- You can open the class, choose the Roster section, and tap
Performance.
Tap on the team to open their class performance summary,
and tap on the appropriate performance record (in this case, the
observation segment of session 1).
- Alternately, you can get to the same screen by opening the class,
choosing the Sessions section, tapping on the session
(in this case session 1), and choosing the Roster section of
the session.
Be sure to mark the team present. You can't add performance data to
a team that is absent. Then tap the team's performance button (circled
in red)
and tap on the appropriate row.
Note: During the course of a class, you may have the opportunity to evaluate a team's performance many times, so there is potentially quite a bit of performance data available for each team. That translates to a few more taps to get to the data you want.
For an observation style evaluation, this screen has three parts: a section to enter the observation, a section to add photos, videos, etc. (i.e., Resources) and a section to access the Rubric (just in case you need to look at it.)
TopObservations
An observation style of assessment was created for use in observation segments of a session (typically in a one-on-one consultation environment). However, observation is such a basic component of training that you'll find this style of assessment useful for many other situations as well.
The skills section of the rubric indicates what you are looking for and the criteria section indicates how you will know when you see it. In both cases, you can enter as many criteria as you want (using the Add menu.)
TopPass/Fail
The idea of a pass/fail style of assessment is self-explanatory. It's principal use is in a testing environment (such as in a certification test) where the animal must perform at a minimum required level to qualify for the certification.
The rubric lists the skills being tested. If you wish, you can expand the criteria to provide more detail about what a pass or fail means.
Navigating to the performance screen,
you can see that there are four sections. The new Values section displays the performance elements and criteria you defined in the rubric and offers the opportunity to evaluate the team's performance using them.
The other sections (Observation, Resources and Rubric) are the same as before. This means that in addition to evaluating the team's performance on the specified skills, you can still record your observations and evidentiary photos, videos, etc.
TopLikert Scale
The problem with using a pass/fail rubric is that it doesn't convey the degree of progress that a team is making. Instead, many people prefer a Likert scale. When you choose to use a Likert scale to evaluate performance, you get to pick how many levels to include in the scale as well as defining what each level means.
Using the same list of skills as above, a Likert scale rubric supports multiple levels of performance.
As you can see, we have defined a four-point Likert scale here.
While scales with an odd number of levels (e.g., 3-point or 5-point scales), we would like to advocate for scales with an even number of points, such as the 4-point scale shown here because there is no "middle" value to choose.
Evaluating performance using a Likert scale is just a matter of tapping some radio buttons,
Top
Points
A points-based evaluation is primarily useful when measuring how much a team or animal can do.
As you can see, a points-based rubric can assign values both to levels of performance and bonus points for special behaviors.
Evaluating using a points-based rubric involves recording a point value for each skill.
Here, the team got a bonus on the Long Distance Come for an excellent finish.
TopDeductions
Unlike a points-based rubric, a deductions-based rubric is used when testing a team on a prescribed sequence of exercises. Deductions are taken for errors in performance.
As you can see, in a deduction-based rubric, a value of 0 is awarded for error-free performance, with higher values associated with increasing degree of error.
Deduction-based evaluations are probably best suited to practice testing for competitions.
Top