For the fourth and final module for EDTC 6106, we are continuing to explore professional learning. The focus of this research is through the lens of the third indicator from ISTE Coaching Standard 5: Professional Learning Facilitator, indicator 5c. As I began thinking about my inquiry question in relation to this indicator, I had several questions swirling through my mind. How do you effectively evaluate professional learning? What does impactful professional learning look like? How can data-informed decisions be made from professional learning evaluations? What does high-impact teaching and learning look like? How do data-informed decisions improve high-impact teaching and learning? Some of these questions I have already begun to explore in previous blogs, like my blog post Designing, Implementing, and Evaluating Effective Professional Development or my blog Understanding Adult Learning Theories in Reimaging Professional Development. While I am continuously growing in refining my understanding of these topics, I wanted to focus on finding a purposeful, impactful approach that would help guide coaches through the evaluation of professional learning and improvement of high-impact teaching and learning. This focus led me to examine Learning Forward’s “Seven Steps to Evaluating Professional Learning.” I found this approach to be incredibly useful in understanding how coaches can make informed decisions on improvements to professional learning. While this approach was beneficial in many ways, to me one of its flaws seemed to be the large amount of weight put on teachers to be responsible for this evaluation. As I break down the steps of this evaluation process, I worked to layer in advice for coaches on how to partner with teachers throughout these evaluation steps.
Inquiry Question:
How can coaches evaluate the impact of technology on teaching and learning, to make informed decisions about improvements to the professional learning program?
ISTE Standard 5: Professional Learning Facilitator
c. Evaluate the impact of professional learning and continually make improvements in order to meet the schoolwide vision for using technology for high-impact teaching and learning.
Data-Driven Decision-Making
As Craig Mertler (2014) explains, “Data-driven educational decision making refers to the process by which educators examine assessment data to identify student strengths and deficiencies and apply those findings to their practice” (n.p.). Data-driven decision-making helps educators to collect feedback on student learning and make adjustments to meet student needs. Data can be an incredibly powerful tool, but in order for data to be analyzed and utilized to its fullest potential educators must have support in this utilization. “Data in any context isn’t powerful or valuable until we put the tools, processes, and training or supports in place for users to accurately understand it and put it into action,” Joe Siedlecki and Dan Stasiewski (2012) explain. Coaches can support educators in data-driven instruction by guiding them in the selection of educational technology that both meets students needs and helps to improve student outcomes through data collection. This Educational Technology Evaluation Guide from Edmentum is an example of a valuable evaluation tool to support educators in identifying if the tech tools they are using support high-impact learning and produce data that helps to inform instruction. Data-driven decision-making, when partnered with an effective evaluative approach, fosters a school culture grounded in ongoing evaluation and continuous improvement, and a focus on assessing the impact of professional learning on improving student learning outcomes.
The Steps
Figure 1 below is an infographic I created to help visually represent the seven steps to evaluating professional learning provided by Learning Forward (2014) in the article Evaluating professional learning: Measuring educator and student outcomes. Through their seven steps, Learning Forward takes the evaluation process of professional learning and breaks it down into an approach into a clear action plan that highlights the responsibility of teachers and school leaders to partner in this work. As I continued to research for my inquiry question for this module, I wanted to explore professional learning evaluation through the use of this approach and building a further understanding of these steps.
7-Steps-to-Evaluating-Professional-Learning
Information from Learning Forward (2014, March). Evaluating professional learning: Measuring educator and student outcomes. Learning Forward. Retrieved March 4, 2021, from https://learningforward.org/march-2014/evaluating-professional-learning-outcomes/
Step one of this evaluative approach places a focus on analyzing data and providing opportunities for teachers to learn and practice how to analyze and examine data. As referenced in the data-driven decision-making section above, if best practices tell us that it is essential for teachers to use data to evaluate student learning outcomes and drive their decision-making, then coaches need to provide teachers with opportunities in professional learning to strengthen these skills. According to Digital Promise (n.d.) “Districts need evidence not only to validate decisions but also to continuously assess and improve their programs…Not only do districts need accurate and timely data, but teachers need ongoing support in how to use the data to make decisions” (n.p.). The analysis in this first step is critical for coaches to gain clarity around how students and teachers are progressing in reference to the school’s vision and goals, and to use as a reference to plan and implement future professional learning. During step two coaches can support teachers in taking the information they analyzed through step one about identified needs for student learning and professional growth. This support can be provided with the use of coaching plans and goal-setting tools, like the Learning Design Matrix designed by Les Foltos that I reference in my blog post Peer Coaching – A Community Engagement Project. Defined clear, measurable goals help coaches to support educators in the data collection and ongoing evaluation of provided professional learning in relation to those goals.
During steps three and four of this evaluation process, the focus is on establishing benchmarks and implementing a learning plan respectively. When coaches connect with educators during step three of this approach, the goal is to identify a few skills and instructional strategies that can be improved upon to increase student understanding. In this coaching conversation, this is an opportune time for coaches to evaluate the use of technology and use a technology integration framework in partnership with this peer coaching. One example might be utilizing the Technology Integration Matrix (TIMS), which supports educators in enhancing the learning activities they design in different learning environments. Identification of goals from this framework during coaching conversations also provides an opportunity for coaches to think about potential professional learning that could be designed to support this growth. In progression from step three, during step four coaches can continue to provide ongoing learning support for educators by providing options to educators of learning designs that match their growth needs. One such learning design a coach might consider to use is the Learning Design Matrix designed by Les Foltos that I reference in my blog post Peer Coaching – A Community Engagement Project. Selection of a focused quadrant from this design matrix will also guide coaches in supporting individuals in finding meaningful technology to integrate in their instruction.
During steps five and six of Learning Forward’s seven step evaluation of professional learning, educators conduct formative and summative assessment evaluations. In step five when educators are asked to conduct formative assessment evaluations, coaches can support this evaluation by offering strategies on data analysis and revisiting previously selected goals that were identified by the educator in step one. Coaches can suggest tech tools for educators to use during step five’s formative assessment evaluation. This is a wonderful opportunity for educators to explore using a new or familiar tech tool to formatively assess their students on learning outcomes. Coaches might also consider leveraging tech tools that an educator is already familiar with and coaching them on how to utilize it to formatively assess students. For example, if a teacher has used Padlet with students previously, they might look at using this as a way to conduct a formative assessment evaluation. As educators transition to step six, conducting summative assessment evaluations, they now evaluate how successful students were in reaching performance goals. These summative assessments will support the data analysis in the final step, step seven, of this evaluation approach.
Lastly, in step seven educators assess the effectiveness of the professional learning, analyze data in relation to established goals, and consider potential changes to the learning experience in the future. This final step is also a valuable time for learning coaches to get evaluative feedback from educators through a variety of formats to collect data to inform their decisions. This feedback might look like surveys, follow-up visits, observations, small group meetings, or interviews. Teacher Professional Development Evaluation Guide states that “Evaluations that combine an examination of teacher perceptions of all components of the professional development, a comprehensive look at the implementation of the professional development, and professional development outcomes are much more useful in understanding what happened and how professional learning paid off for teachers and their students” (p. 89). Learning Forward also lists several useful questions in step seven for learning coaches educators to consider when conducting an evaluation at the end of a professional learning experience.
How do you effectively evaluate professional learning? What does high-impact teaching and learning look like in your school? How do data-informed decisions improve high-impact teaching and learning for your staff and students? How can data-informed decisions be made from professional learning evaluations? Please share your thoughts and experiences, as well as any feedback or questions you have, in the comment section below.
References
Angevine, C. (2019, October 29). Data-informed instruction isn’t easy, but these educators are working toward it – edsurge news. Retrieved March 03, 2021, from https://www.edsurge.com/news/2019-10-29-data-informed-instruction-isn-t-easy-but-these-educators-are-working-toward-it#:~:text=Prioritizing%20Data-Informed%20Instruction,challenges%20of%20small%20group%20instruction.&text=When%20teachers%20can%20see%20data,respond%20to%20their%20students’%20needs.
Data-informed decision-making. (2020, August 13). Retrieved March 04, 2021, from https://challengemap.digitalpromise.org/systems-change/data-informed-decision-making/
Foltos, L. (2018). Learning Design Matrix. Peer-Ed, Mill Creek
ISTE Standards for Coaches. (n.d.). Retrieved from https://www.iste.org/standards/for-coaches
Karlin, M. (n.d.). K-12 technology leaders: Reported practices of technology professional development planning, implementation, and evaluation. Retrieved March 05, 2021, from https://citejournal.org/volume-18/issue-4-18/current-practice/k-12-technology-leaders-reported-practices-of-technology-professional-development-planning-implementation-and-evaluation/
Learning Forward. (2020, December 21). Evaluating professional learning: Measuring educator and student outcomes. Retrieved March 02, 2021, from https://learningforward.org/march-2014/evaluating-professional-learning-outcomes/
Mertler, C. (2014). Introduction to data-driven educational decision making. Retrieved March 05, 2021, from http://www.ascd.org/publications/books/sf114082/chapters/Introduction_to_Data-Driven_Educational_Decision_Making.aspx
Missouri Department of Elementary and Secondary Education. (2020). Missouri Professional learning guidelines for student success. [PDF]. Retrieved March 05, 2021 from https://dese.mo.gov/sites/default/files/Professional-Learning-Guidelines-section-4-with-cover.pdf
Stasiewski, D., & Siedlecki, J. (2020, May 17). The five building blocks of Data-informed Instruction. Retrieved March 04, 2021, from https://www.dell.org/insight/the-five-building-blocks-of-data-informed-instruction/
