In 1994 a faculty member asked his Senators in the School of Business to introduce a proposal to outlaw the publication of student evaluation of teaching (SET) results. Nothing happened from the six senators.
Later in the year the same professor approached another senator in the Department of Speech Communication. This senator wrote a letter to the then speaker of the Senate, noting the dysfunctional impact that published SET scores have on the academic process (i.e., grade inflation). Eventually an ad hoc committee was established [Teaching Effectiveness Committee].
Regrettably seven of the eight members of the committee were either administrators or students. One of the administrators was the close friend of the department head of the complaining faculty member. Meetings of the ad hoc committee were held on February 6, February 27, and April 3, 1995, with communications also conducted by mail. People in the Texas A&M community were consulted for further information [e.g., Liz Miler (Center for Teaching Excellence) and Glenn Ross Johnson (Curriculum and Instruction)]. The complaining faculty member was never invited to speak (or informed of the committee).
The chairman of the ad hoc committee indicated that no minutes were kept and no votes were taken. The committee's proof that SET data is "valid and reliable" was based upon a quotation by H.W. Marsh which appeared in an earlier School of Business report. If the committee had read the entire article, Marsh states: "The appropriate use of SETEs for summative evaluations--particularly if they are used in personnel decisions--is more controversial than their use as formative feedback."
The committee also provided several "Idea" Papers from the Center for Faculty Evaluation & Development (Kansas State). But they failed to submit any information about the impact grades have on evaluations and the massive grade inflation in the U.S. For example, in a 1994 report published by the University of Washington Office of Educational Assessment, Gillmore and Greenwald found that the student ratings' general factor is influenced by three variables: "students perceptions of the ratio of valuable hours to total hours, the challenge of the course, and their grades in the course." These researchers worry about a "cycle of grade inflation--giving higher grades leads to higher ratings and the averages of both slowly creep upward."
Even in the Kansas materials provided by the ad hoc committee, Cashin indicates a positive correlation of .10 to .30 between student ratings and expected grades. [The correlation goes as high as .52]. In the social sciences validity correlations above .70 are unusual, especially when studying complex phenomena (e.g., learning), and correlations between 0.20 and 0.49 are considered practically useful. As early as 1976, Centra and Creech in a study involving 14,023 students, found that "students expecting an A grade gave a mean rating of 3.95; while those expecting a D grade gave a mean rating of 3.02. Few professors in a SET-driven university like TAMU can afford to give Ds and Fs, and probably few Cs.
In reality, SET ratings are affected by a number of factors beyond the control of the instructor--sex of instructor, class size, time of day the class is taught, sex of students, required course or elective course, level of course, competing instructors, quality of students, and age of instructor, to mention a few. Strict grading practices lead to lower SET scores, because students mirror back the evaluations that they receive from instructors. Students will retaliate against instructors through their responses to items on the SET questionnaire.
The resolution was brought before the Senate in July when there are fewer faculty Senators in town. To make a long story short, just like the pied piper leading the rats over the cliff, this slanted material lead the Senate, with a voice vote, to pass a resolution stating that "properly designed student ratings... have been shown to be reliable and valid" [July 10, 1995]. The Senate did not even state for what the SET data was valid and reliable.
To implement this resolution a four-person committee was established, and three of the four members are administrators. Both a senator and nonsenator asked by phone, mail, and fax to attend the meetings under the Freedom of Information Act. The two parties received no response or permission to attend the meetings.
Even ignoring the fact that many research articles agree that the grades given by an instructor have an important impact on an instructor's evaluations, how reliable is the data collected at Texas A&M?
On Thursday, December 7, 1995, the SET questionnaires from across the TAMU campus were stored unsecured on a table in a foyer on the second floor in Bizzell. One professor sat in the foyer for approximately one hour on Thursday reviewing some psychological periodicals while student workers came in and out. The envelopes were already opened, and the professor was never asked to leave.
On Friday a faculty senator and the professor returned to the second floor, and there were still four large boxes of questionnaires on the table. Later in the afternoon, one party took photographs of the unsecured questionnaires. How can SET data be valid and reliable when there is no security? Anyone--students or faculty--could have contaminated the questionnaires. Why are students used to tabulate the SET questionnaires? Such a cavalier attitude with respect to the storage and tabulation of this important data is reprehensible.
Professors have rights just like students. The publication of SET data causes important due process and academic freedom issues, which should not be handled in such a sloppy fashion. For example, in some accounting classes at TAMU as many as one out of five students are zapping male professors on the SET questionnaires.
Professors must realize that the SET questionnaire is a control system used by students and administrators to control teachers. This control device is similar to alcohol to an alcoholic or drugs to an addict. Students and administrators will not give up this control device easily.
Society for a Return to Academic Standards
Suggested Faculty Senate Proposals
Last Updated: 6 March 1997
