Section 5.11 Involving the external evaluator in the curriculum meetings.
¶The external evaluator also commented about the on-going curriculum meetings:
We were invited to every meeting. We brought in to the discussion what we were learning, how the students were doing...I was amazed at how they (paradigms faculty) kept picking at our brain. We had background on different instructional strategies...It is important for the evaluator to go to the meetings and participate, that puts context into the data that you're gathering, not a bias,...provides a framework for how to look at the data that you have.
The evaluator noted the importance of collaborating with the faculty in developing the protocol of questions she and her assistant were asking:
(My assistant) and I put together a protocol of the information we wanted to get from the students; we ran that protocol through the physics faculty at meetings, so it was collaborative.
After her assistant conducted the interviews, he discussed the transcripts with the external evaluator and then they shared their findings with the physics faculty at the meetings:
(He) would do the student interviews, face to face, he would record and transcribe; he and I would read the transcripts, talk about what was going on here; we could take that to the group: this was working, or they (the students) are really struggling with this concept...We were able to give that advice to them and to help them shape how they organized the content and how they did what they did.
The PI commented upon how helpful the external evaluator had been:
...one thing she did was that she arranged for the students to report to her and to (her assistant) weekly in the paradigms, so three times for each paradigm, just open-ended questions about how was the paradigm going, and she was able to pull out of that problems that we were having that we did not detect, just because it was anonymous and because she was experienced, it was both, but that she caught some things that were going on that could have ruined the program, because of the student reaction early on...
The external evaluator's input provided an important way for the faculty to become aware of aspects of the students' experiences that the students might not have communicated directly to the faculty.