A look into end-of-year professor performance rating system
At the end of every semester at the University of Rhode Island, students are asked to take IDEA Surveys, but many do not know just how their survey responses change their classroom.
The Individual Development and Educational Assessment (IDEA) is a survey system that URI implemented to help create a way for students to evaluate their instructors each semester. The anonymous survey is composed of 42 standardized questions that were originally created at Kansas State University in 1968 according to Campus Labs, an online resource for higher education institutions around the world.
The IDEA course evaluation system cannot be altered by the instructor, according to Sean Krueger, URI’s coordinator of course evaluations.
“It’s not a homegrown survey,” Krueger said. “Instructors are not coming up with these questions or departments or anything like that. This is a validated survey. And the reason why URI uses it is because of the formative feedback.”
The questions focus on the classroom environment and the effectiveness of certain teaching methods.
“It asks a lot of questions about teaching methods, not about the individual themselves,” Krueger said. “It’s not a popularity contest. It really gets at the science of teaching and the methods of teaching much better than any homegrown survey.”
He noted that there may be a level of dissatisfaction from students about the IDEA course evaluations and that they may not see immediate improvement.
“There’s a loop and it’s continuous and it’s slow,” Krueger said. “And that might be why students are frustrated. It does take time, but [change] happens.”
Student voices are only one part of a larger, three-part feedback system for instructors, according to Krueger. In addition to the evaluations, feedback is derived from peer review and self reflection from instructors.
“I can say that if students don’t provide any comments, I can only guarantee that there will be no change,” Krueger said. “It’s really up to the instructor. Ultimately, now if term after term there are, you know, areas of improvement, typically, the dean or chair or even the Provost’s Office would encourage them to do so.”
Anna Santucci, the faculty development specialist with the Office for the Advancement of Teaching and Learning (ATL), agreed with Krueger’s sentiments on the IDEA surveys.
“Take the survey,” Santucci said. “This is an opportunity for all of us to be on the same page about why the course works the way it does, and at least complete you know, moment of transparency, a moment of sharing.”
ATL works with hundreds of faculty members, according to Santucci. She explained that instructors are not required to implement changes in the classroom; however, many do and look to ATL for help.
“We do consult with the people that make the decisions, so the Dean’s Office, the Provost Office, are using our same access to what is a research-based good evaluation of teaching,” Santucci said. “We provide input to them so that they can do that evaluative portion of the job as well. So I’m kind of the guide on the side for all the people involved in the process.”
Krueger and Santucci both expressed the importance of student voices being heard and addressed concerns about the nature of the IDEA program providing meaningful feedback for faculty. Santucci highlighted examples of mid-term evaluations and check-ins to create a constant loop of feedback between students and their instructors.
“If there isn’t an open channel for communications, then we just can’t know what each other is thinking, right?” Santucci said. “The faculty member may just be sitting there not really being sure what the students are thinking or feeling. The communications are actually a two-way street.”
Krueger said that not every issue in the classroom can be fixed by the IDEA course evaluations, and offered advice for students who have other issues.
“If a student had a problem with the instructor for something other than teaching than the course itself, then they should really be bringing those concerns to a department chair, or the dean of students or Title IX,” Krueger said. “It’s really important to make that distinction. This is really a place for feedback on teaching in the classroom.”
Vice Provost for Academic and Faculty Initiatives Anne Veeger said that the IDEA surveys are for instructor and class feedback only, including the comment section.
“The comments in the IDEA are a place for a student to communicate directly with a faculty member,” Veeger said. “If a student has a concern that they want to address, at a department chair level, for example, the IDEA is not the appropriate approach to make that communication.”
Instructors at URI go through an annual review process, the frequency of which depends on their rank at the institution. Relatively newer faculty may get reviewed annually, whereas senior faculty may get reviewed only once every three years. IDEA survey results are included in every annual review, however, the comments that students leave are not.
Veeger said that the only person who sees comments left by students at the end of the IDEA survey is the faculty member themself.
“The department chair and the Dean’s Office do not get to see those comments unless a faculty member chooses to include them in their annual review,” Veeger said.
Krueger said that classes being taught remotely due to COVID-19 has affected the IDEA survey feedback rates negatively. Since the pandemic, there has been a drop in response rates, according to him.
The voluntary nature of the surveys, according to Krueger, leads to higher quality feedback, but also comes with the risk of students ignoring the surveys.
Krueger emphasized that the most important part of the IDEA surveys’ success is student participation.
“The more students who participate, the better,” Krueger said. “Then, the more useful this instrument is for everyone.”