1. If you call a student (or employee or boss) stupid, the person will probably not evaluate you highly on an anonymous questionnaire.
2. Few institutions use statistically valid evaluation forms--only 7 out of 73 in one survey.
3. Federal standards as outlined in the Uniform Guidelines on Employee Selection Procedures are probably not met when evaluations are based upon SET data.
4. Students are not customers. They are inventory or products. The customers are society, employers, and parents.
5. SET ratings are affected by a number of factors beyond the control of the instructor--sex of instructor, class size, time of day class is taught, sex of students, required course or elective course, level of course, competing instructors, quality of students, and age of instructor, to mention a few.
6. Even defenders of SET admit to a positive correlation from 0.10 to 0.52 between student ratings and expected grades. In the social sciences validity correlations above .70 are unusual, especially when studying complex phenomena (e.g., learning). Thus, correlations between 0.20 and 0.49 are practically useful.
7. Strict grading practices lead to lower SET scores, because students mirror back the evaluations that they receive from instructors. Students will retaliate against instructors through their responses to items on the SET questionnaire. If SETs are valid, why not distribute anonymous questionnaires to all of the football players and basketball players, tabulate the results and fire the coaches based upon this data ? If we did, it's highly unlikely there would ever be a winning season.
8. At Harvard in 1992, 91% of undergraduate grades were B minus or higher. At Stanford only 6% of all grades were C's.
9. Use of SET is a control device, which creates due process and academic freedom issues. Challenge the validity of SET data in the courtroom.
10. An effective teacher adds value to his or her students. Measure the students beforehand, measure the students afterward, and the difference is value created. The more value created, the better the teacher.
11. Challenge a "doubting Thomas" instructor to give average grades at least one letter grade below the norm (i.e., be an outlier). Then watch the decline of the instructor's SET scores over future semesters.
12. If students lie at least 50% of the time to their mothers, how can data from SET questionnaires be valid?
13. Laws highly regulate financial statements to reduce income manipulation and opportunistic behavior, yet there is no regulation of the use of SET. Most administrators blindly accept them as truth. Instructors have a high incentive to manage SET, even more so than managers have the incentive to enhance earnings.
Society for A Return to Academic Standards
Evolution of Teaching Business
Last Updated: 14 November 1996
