Evaluating the effectiveness of professional development can be time consuming, costly, complex, and inconclusive. An analysis of teacher development came from a report from TNTP “The Mirage” (2015) by surveying 10,507 teachers and 566 school leaders. Their conclusion was that “in short, we bombard teachers with help, but most of it is not helpful – to teachers as professionals or to schools seeking better instruction.” The cost of professional development is significant. The school districts participating in this research had an average yearly cost per teacher of 18,000, and yet there was little evidence of the effectiveness of such professional development. The mirage reports that “only 3 in 10 teachers demonstrated substantial improvement” However, “Even when teachers do improve, we were unable to link their growth to any particular development strategy” (p. 2).
ISTE Standard, Performance Indicator 4.5c states: Evaluate the impact of professional learning and continually make improvements in order to meet the schoolwide vision for using technology for high-impact teaching and learning.
What are research-based strategies and elements that need to be included when evaluating professional learning programs?
Addressing the issues with professional development and the difficulty of their evaluation, The Mirage gives recommendations for designing effective professional development. While their research suggests that professional development needs to be highly individualized, they explain how schools and districts can create conditions for successful teacher development. Further recommendations are to redefine, reevaluate and reinvent professional development. For example:
- Define “development” clearly, with observable and measurable goals
- Evaluate the effectiveness of existing development activities
- Reconstruct the teacher’s job
Another recommendation comes from the website from the Office of Educational Technology presents a professional learning toolkit for planning and evaluating professional learning. Future Ready Schools: Empowering Educators through Professional Learning toolkit presents a 5 step approach:
- Determine district readiness
- Assess current professional learning
- Refine professional learning
- Implement professional learning
- Measure refined professional learning
Step #5 – Measure refined professional learning addresses ISTE standard 4.5c.
This site presents an evaluation tool that facilitates the collection, analysis and dissemination of evidence on the impact of professional learning activities have on educators and students. The tool is a 5-level evaluation framework including:
- Immediate: activities and interactions: educators’ experience immediately after their participation
- Potential: knowledge and social capital: knowledge and skill acquired by participants
- Applied: Changes in practice: educators apply knowledge and skill in practice
- Realized: Performance improvement: evidence is gathered from assessments, portfolios, videos, etc.
- Reframing: Changes in what is valued: Educators share successes and the knowledge and skill move beyond their classroom
This toolkit addresses “new and innovative forms for online and blended professional learning.” A possible drawback to this toolkit is explained in the “about” section: “There is still debate among educational researchers about the features of professional learning that generate value, and, to date, there have been a limited number of large-scale experimental studies of professional development and, in particular, the new and innovative forms for online and blended professional learning on which this toolkit focuses. The results of some such studies that have been completed are inconclusive or contradictory.”
One needs to consider that innovation, although necessary for improvement and growth, is not always supported by research.
Yet another report that gives recommendations for professional development is Evaluating Professional Learning. A tool for schools and districts. This resource focuses on the following topics:
- Importance of evaluating professional learning
- Designing or selecting high-quality professional learning
- Developing a logic model
- Developing strong evaluation questions
- Developing a data collection plan
- Making meaning of your data
This toolkit draws from Guskey’s research and findings.
In the article Does It Make a Difference? Evaluating Professional Development (2002) Guskey explains that good evaluations “require thoughtful planning, the ability to ask good questions, and a basic understanding of how to find valid answers” (p. 46). Thoughtful planning is essential. The goals need to be clearly defined at the beginning, following a backward design model. He recommends to define the student learning outcomes (level 5) first. Next, determine the evidence that can be gathered, such as lesson plans, student work and school records. Then, determine research-based instructional strategies that would produce the desired results. This is a good place to include digital tools that would support and transform the instructional strategies.
My conclusion is that the initial planning and design is essential for a successful evaluation of professional development. Following the backward design model when designing professional development provides concrete elements for evaluating its effectiveness.
References
Breslow, N. & Bock, G. (2020). Evaluating Professional Learning. A tool for schools and districts. REL Northeast & Islands. https://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/NE_5.3.5_Evaluation_PD_Brief_12-22-20_accessible.pdf
Guskey, T. R. (2002). Does It Make a Difference? Evaluating Professional Development. Educational, School, and Counseling Psychology Faculty Publications 7. http://uknowledge.uky.edu/edp_facpub/7
ISTE Standards: Coaches. https://www.iste.org/standards/iste-standards-for-coaches
Office of Educational Technology. Future Ready Schools: Empowering Educators through Professional Learning toolkit. https://tech.ed.gov/futureready/professional-learning/
TNTP. (2015). The mirage: Confronting the hard truth about our quest for teacher development. Brooklyn, NY: Author. https://tntp.org/assets/documents/TNTP-Mirage_2015.pdf
Treacy, B, Lleiman, G., & Peterson K. (2002). Successful Online Professional Development. Learning and Leading with technology 30(1). http://courses.edtechleaders.org/about/research/articles/SuccessfulOPD.pdf
