Reminder: Online SRTEs Available

It is the time in the semester when work piles up and the last thing anyone wants to do is spend 15 minutes filling out professor evaluation forms. Whether you elect to evaluate your instructors or not, remember that these forms represent the most significant opportunity for students to voice their concerns and express their satisfaction with a course.

The SRTE’s represent an evaluation method designed at Penn State and implemented in 1985, aimed at collating feedback from students about courses and the professors who instruct them. The SRTE’s are administered by the Office of the Vice Provost for Academic Affairs. Every semester, students are encouraged to take the SRTE forms for every class, yet the anonymous nature and opportunity for honest feedback often create challenges.

In a conversation with the Vice Provost for Academic affairs, Dr. Blannie Bowen, Onward State Editor Dave Cole was able to iron out a few of the misconceptions and uncertainties about the SRTE forms we enc0unter as semesters near their conclusion. The following questions attempt to address some of these ambiguities.

OS:What kind of impact do SRTE forms specifically have on professors?

Bowen:”There are two things:

1) The forms influence promotion to full and associate professorship, and impact tenure.

2) The scores are used as faculty evaluations typically by the department heads. The comments are not technically part of the SRTE, but are summarized and used later in evaluation.”

OS: Many students claim that filling in an entire SRTE form with all 0′s or all 7′s will invalidate the form and it will be discarded. Does this occur?

Bowen: “Absolutely not. No forms are thrown out, and there is no way of controlling the forms as the process is all electronic.”

OS: Just how anonymous are the SRTE forms, and can professors have access to the results before final grades are given?

Bowen: “Professors never have access to individual scores, only given access to a summary of scores, which includes the mean scores, and a total number of completed forms.” Deans and department heads are subject to the same evaluation forms by faculty.

OS: The SRTE website answers this question vaguely; can professors offer incentives for participation?

Bowen: “We don’t encourage faculty to offer incentives. Additionally, there are no ways to offer incentives since professors have no way to see who completes the forms.” Now that the forms are online, faculty don’t even have the opportunity to reward students for showing up to class on the day of the SRTE.

OS: How has the transition to online SRTE forms affected participation?

Bowen: “Participation has been a little lower, but not as low as you might think.” Dr. Bowen directed me to a 2010 report by the Senate Committee on Faculty Affairs addressing the participation of forms, addressed two possible concerns about transitioning SRTE forms online.

1) A possible decrease in response rates which could undermine confidence in the results obtained; and, 2) a possible bias in the outcome due to the change in the population of the students providing responses. In addressing these concerns, the report found that; ” Examination of the results obtained thus far via the online SRTE Project suggests that while the second concern may not be warranted, the first concern seems justified. Both the decrease in response rates and absolute rate itself are problematic.

OS: Do professors believe that the system is an accurate way to gauge performance?

Bowen: “Most proffessors feel that the system is accurate. There are multiple ways that performance is gauged, and SRTE’s are just one part.” Dr. Bowen emphasized the importance of written comments and other evaluation methods as numbers alone may not tell the whole story.

OS:What feedback have you heard from faculty about outside evaluation websites such as ratemyprofessor.com?

Bowen: “Most believe that the ratings on these sites fall under two categories; students who love the professor and those who hate the professor, without much middle ground. Whereas on the SRTE results there tends to be a much more diverse and scattered data set.”

OS: Would you consider making the SRTE results available to students who are registering for classes?

Bowen: “This would not be a good idea because the decisions impact promotion, tenure consideration, salary increases, and is a personal decision that is a private matter. The idea has been discussed, but there hasn’t been much movement.”

As you enter the final weeks of the semester and are asked to submit your SRTE scores, remember that they represent a key evaluative measure in ensuring the quality of Penn State University’s faculty, so if you do decide to take them, remember to be honest and fair.

Spencer Malloy

I find it interesting that the senate report felt that there wasn’t a significant bias. It’s hard to imagine some as self-selecting as an online, voluntary form doesn’t draw significantly more bias compared to an in-class form that every student completes. Good interview questions.
Spencer Malloy

I find it interesting that the senate report felt that there wasn’t a significant bias. It’s hard to imagine something as self-selecting as an online & voluntary form doesn’t draw significantly more bias compared to an in-class form that every student completes. Good interview questions.
GTWMA

Very nice interview, Dave.

Spencer, I think the in-class form is also self-selected, since you only complete it if you show up. So, I think it’s just that the different forms of self-selection don’t have too many significant biases.

I certainly can’t speak for all department heads, but I review every faculty member’s numeric results and comments every semester, and recognize those who did well and not. Those results, as Blannie notes, are summarized and used in both evaluatiuons for promotion, tenure, and annual salary increases. I hope students do take the time to complete them.
Chadwick Lynch

I feel like it would be possible to connect the SRTE’s with eLion. If they want full participation, make it obligatory to fill out the SRTE’s in order to see one’s grades. Not that I endorse that idea, but I see the solution to the problem. There is no incentive to do it, so simply make it a requirement. Boom. Lynch ’12, vote for me.
Dave Cole

An interesting suggestion Chad. Forced participation, however, can create a situation where unwilling students carelessly submit answers. You could argue too, though, that you may get a more even distribution assuming everyone answers honestly.
Evan Kalikow

Here’s a question that I have–do the SRTEs mean anything if your professor is also the head of the department? Who watches the Watchmen?
GTWMA

Again, I can’t speak for all colleges, but typically department heads are evaluated annually by their Dean, so it would be the Dean looking at any SRTEs. Department heads and tenured faculty also receive a more complete long-term evaluation every 5 years.

For department heads, and really all faculty, it’s also probably important to mention that teaching may be only a part of their overall evaluation. The typical “baseline” tenured-track faculty appointment might be something like this: 60% teaching/30% research/10% university, professional and community service. A faculty member who is a successful researcher might have only 15% of their time allocated to teaching, because their grants “buy out” their teaching. And that allocation changes from semester to semester, year to year as research grants come and go. A department head might have 60-75% of their time involved in service, including their administrative duties. So, evaluations have to involve both the quality of work done in each area, and the weight that area of performance receives,

So, yes, SRTEs even matter for department heads, but possibly not as much as their other duties.
Angela Linse

This is an excellent story that emphasizes why it is so important that students participate in the SRTEs. SRTEs provide faculty with valuable feedback on how to improve their courses and teaching, as well as evaluation data for administrators.

Faculty and administrators do read and respond to the results. The Senate report is excerpted from a longer report posted on the SRTE website (SRTE Reports). The recent reports focus on:
1) whether the response rates for online SRTEs are lower than paper SRTEs, and
2) whether we see any consequent changes in average scores.
The response rates do decrease about 20%. However, the average scores are very stable when comparing paper to online. When we compare the same course taught by the same faculty member, the paper and online ratings are nearly identical–despite the lower response rates.

That said, higher response rates would help ensure that the SRTE results reflect all (or most) students’ views.

Finally, students have always self-selected to participate, whether in-class or online. This is why faculty teaching evaluations are based on more than just their SRTEs scores. Only with multiple sources of information (students, administrators, other faculty), can we be confident in the overall evaluations of courses and instructors.
Chadwick Lynch

How many people before were giving real answers in class? If anything, despite a lower participation in recent times, I assume that they are more accurate. They could be re-done too, so that automatic answers are eliminated and all responses are open ended. That would take away any mathematical component, but how accurately can a professor be assessed by a mathematical mean anyway?
Dave Cole

Thanks for reading!
Spencer Malloy

GTWMA, perhaps you’re right about self-selection via absenteeism, but I always showed up for my many attendance-inclusive classes that had SRTEs that day
Dave Cole

thanks for reading Angela!
GTWMA

I think both the numbers and the comments serve a purpose. From my experience, the comments are really the most effective in helping faculty understand what they can do to improve. If you see several students making similar comments and suggestions, you know an area to target. And students are often great about suggesting ways other faculty or courses have solved a particular problem, so you often not only get an idea about your weaknesses, you get info on how other people do it better.

From an evaluation point of view, however, the numbers really help. After reading several hundred comments on 20 different faculty, I have no doubt that I would be able to arrange them in rough approximation of their teaching quality, as measured by the numbers. But, it makes it so much easier from an evaluative approach to simply point to the mean of 4.2 and say “Not good enough”.

So, the number gives me an efficient summary for evaluation, while the comments provide both me and the faculty member with more information for development and improvement.

I would tend to agree with dave that the downside of forced participation offsets the additional info to be gained from those 30% of students who fail to complete the numbers.

While I understand Blannie’s comments on making them public, I’d love to find a way to do that. It would certainly be much better than RMP. Dave, I think Blannie may have said that salary, promotion, tenure, etc. are personnel (not personal) decisions, and this limits their release. There are some challenges with public release, which we also see in my field with the release of data on doctors and hospitals. It can create some interesting incentives for both faculty and students, some of which are not beneficial for either.

Students–do you think faculty concerns about SRTE scores lead them to engage in grade inflation?
Eric Collins

Thank you, Onward State for making an article that belongs in the spam of my webmail.
Eli G

hater.
Dave Cole

I certainly cannot speak for all students, but I can certainly imagine a professor recieving much more beneficial scores if they give grade inflation is high. Another challenge of giving prospective students access to SRTE results, is that students may correctly, or incorrectly, interpret positve SRTE scores as an indication of an “easy” class, when the scores should rate the quality of the course and course instructors