International Institute for Engineering Education Assessment

The Global Real-time Assessment Tool for Teaching Enhancement (G-RATE) was developed in an effort to provide multidimensional direct observational feedback to engineering instructors about their instructional interactions in a classroom. The conceptual framework on which this study is based is the “How People Learn” (HPL) framework (Bransford, Brown, and Cocking, 1999). The HPL framework identifies four dimensions which are essential elements of an effective learning environment (1) learner-centeredness, (2) knowledge-centeredness, (3) assessment-centeredness, and (4) community-centeredness. After observing an instructor’s interactions in a classroom, a report is compiled for the instructor to apply to their teaching. With enough reports, certain combinations of instructional dimensions may be established as more effective than others, which could then be used by instructors to change their interactions to be more successful.

Function Description

Administrator	This function allows a lab coordinator to modify the data collection parameters of the G-RATE (e.g., adding the name of observed GTAs; tailoring self-reflective questions for the GTAs; selecting appropriate questions for undergraduate students). This occurs at the beginning of the lab session.
Observer	This function produces several time- or event-sampled codes that reflect observed instructional practices of GTAs during lab sessions. Codes represent elements of the How People Learn framework (Bransford, Brown, & Cocking, 1999)
Student	This function contains open-ended and closed-ended questions to be completed by undergraduate students enrolled in the observed lab session. These questions will be formative and summative and will be answered during the last ten minutes of the lab session.
Instructor	This function contains open-ended items about instructors’ pedagogical practices during lab sessions. The instructor will complete this reflection after the lab session.
Researcher	This function gives a researcher access to data from the other G-RATE functions without overriding modifications made by the lab coordination. This function houses the profiles given to instructors about their pedagogical practices. The researcher accesses these profiles after all other data have been collected.

1. Does G-RATE feedback affect the development of instructors’ (i.e., engineering faculty and GTAs) pedagogical expertise?
2. (2) What is the relationship between this feedback and undergraduate student outcomes (e.g., grades) within observed environments?

This is an ongoing study.

The expected outcomes of this research are to (1) enhance instructors’ pedagogical expertise via the implementation of theoretically-based (i.e., HPL framework) teaching assessments that will be incorporated into first-year engineering and materials science and engineering courses; (2) develop positive relationships between instructors’ teaching and curricular innovations and undergraduate student outcomes (utilizing a mixed-methods approach by integrating both qualitative and quantitative data); and (3) build a support network of community engineering instructors through engagement in open dialogues about teaching and curricular design for undergraduate engineering courses.

Reference

J.D. Bransford, A.L. Brown, and R.R. Cocking, R. R. (Eds.). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press, 2000

Publications

Cox, M.F., Zhu, J., London, J., Hahn, J., & Ahn, B. (2012). Feedback about Graduate Teaching Assistants’ Pedagogical Practices: Content Validation of a Survey Informed from Principles of the “How People Learn” Framework. In Greta Gorsuch (Ed.), Working Theories for Teaching Assistant and International Teaching Assistant Development. Stillwater, OK: New Forums Press.

Cox, M.F., Hahn, J., McNeill, N., Cekic, O., Zhu, J, & London, J. (2011), Enhancing the Quality of Engineering Graduate Teaching Assistants through Multidimensional Feedback, Advances in Engineering Education. 2 (3)

Cox and Cordray,2008 Assessing pedagogy in bioengineering classrooms: Quantifying elements of the “How People Learn” model using the VaNTH Observation System (VOS) Journal of Engineering Education, 97(4), 413-431

Harris, A.H., and M.F. Cox. 2003. Developing an observation system to capture instructional differences in engineering classrooms. Journal of Engineering Education 92 (4): 329–36.

Sambamurthy, N., London, J. S., Hahn, J., Zhu, J., & Cox, M. F. (2013). Reliability of the Global Real-time Assessment Tool for Teaching Enhancement (G-RATE). American Society of Engineering Education (ASEE) Annual Conference and Exposition. Atlanta: ASEE.