Categories
Success Stories

eXtension NAEPSDP Fellowship Provides Opportunity to Work Differently In Program Evaluation

As long as we are in the realm of ‘I finished my program, now I need to evaluate it’ we are not serving diverse, or really any, audiences as well as we could.

When Julie Huetteman saw the call for applications for a joint eXtension/National Association for Extension Program and Staff Development Professionals (NAEPSDP) fellowship, she was intrigued.

In her role as Strategic Initiatives Coordinator at Purdue University, Julie tracks metrics, reads every impact report, and analyzes the impact of Purdue Extension and how it fits with the University strategic plan. She interacts with people in many different positions, all the way from individual consultations on program evaluation to system-wide reporting. As she puts it, she gets to see both the forest and the trees.

It was also because of these many roles, that she has a unique perspective on the “busy-ness” experienced by most Extension professionals.

The busy-ness of Extension has created a reality in which we ‘add-on’ evaluation. It is something we have to get done. We don’t take the time to engage the stakeholder and fully consider their perspectives.

Huetteman applied for the Fellowship because she saw a moment where she could step forward and focus on the most important part of her job, evaluation. The Fellowship gave her the permission and the time to pursue additional knowledge and skills and focus on something she is passionate about, program evaluation that is responsive and inclusive. She not only saw the opportunity to apply a new approach to her own program but also a platform to influence other Extension professionals’ approaches to evaluation.

As she read literature and connected with colleagues for recommended resources, she soon gravitated toward an emerging approach known as Culturally Responsive Evaluation or CRE. CRE requires engaging the stakeholders at the beginning so that the program evaluation uses culturally appropriate ways to collect, interpret, and share data that is valuable to the audience. It is a way for data and information to serve the culture and not the evaluation itself.

Instead of ‘I’m the evaluation specialist and we need to do this’, we need their perspectives from the beginning to learn what is of value to them and so we can adjust our approach as needed.

This seems self-evident, but it is not how many Extension professionals have traditionally evaluated their programs. During her fellowship, Huetteman served as a key informant for the Diversity & Inclusion Issue Corps (now known as the Impact Collaborative). Through her interactions with different projects, she found herself repeating the same question….”Have you asked them?” She was surprised to hear how often the answer was “No.”

She suspects one reason for this, beyond the busy-ness of Extension, is that Extension has served a fairly traditional audience that is somewhat homogeneous (at a system-scale). This is changing in many areas and has caused some Extension professionals to rethink their approach. She recently toured an Extension office whose 4-H program largely serves a diverse, urban audience. “They are completely changing their way of thinking and considering new ways to serve their audience differently.”

The hard part of CRE, according to Huetteman, is that every program evaluation effort will be different. Every audience, every program, and every change in context requires a different approach.

What’s next? Huetteman plans to use what she learned and created during her Fellowship to help Extension professionals at Purdue approach program evaluation in a new way. She is also part of a network in the North Central. Each person holds a unique program evaluation role at their respective institutions, and by working together they hope to share resources, consult, mentor, and form a critical mass that can advocate for an engaged and responsive approach to program evaluation as the norm for Extension.

Considering the polarized political climate we all live in, taking time to listen and adapt our approaches and appreciate the perspectives of other people is more important than ever.

You can contact Dr. Huetteman at jhuettem@purdue.edu and visit her fellowship page for links to webinars and blog posts developed during her Fellowship.

Learn more about the National Association of Extension Program and Staff Development Professionals (NAEPSDP)

Learn more about eXtension

Categories
Diversity & Inclusion Fellowships Information NAEPSDP

Are you evaluating your program? Ask the stakeholders!

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

My role as 2017 eXtension NAEPSDP Fellow for Program Evaluation was launched with the Diversity and Inclusion Issue Corps (now called the Impact Collaborative) in Cincinnati. Since then, I have attended online sessions for those projects to share progress, challenges, and accomplishments. In addition, I have been included on the Corps evaluation team to learn of feedback from project teams.

A theme in this feedback was “stakeholder” involvement expressed as 1) key to their program goals; 2) instrumental to providing external input, perspective and support for their program; and 3) important in their next steps to move forward in their program planning, implementation and evaluation efforts.

In my online interactions with project teams, I found myself repeating, “Have you asked them?” I reminded many to “keep asking questions” of their stakeholders, audience, participants, and attendees to connect to those perspectives, interests, and insights.

We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

In education, the “expert” typically shares information or content. But do we know what is of interest to attendees? Do stakeholders understand what is being shared? Is the program of value to participants? How did the audience benefit from taking part in activities? Here is the break: We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

A lot is involved in planning, implementing, evaluating and reporting Extension programs, and we want to do the best we can. So, consider asking questions throughout and use feedback to inform your decisions.

  • Are you planning activities that encourage attendees to be active, involved and engaged? Check on current research for best practices, then ask the intended audience: “What activities would you find interesting to do?”
  • When deciding which topics are most important, check the literature, then ask a couple representatives of your future audience: “What topics are important to you?”
  • While planning the evaluation, check on practice guidelines, then ask stakeholders: “What questions might be asked to find out the value of this program?” Alternately, give them draft questions and ask: “Which ones work well to capture the value of the program for you?” followed by “How might you state a question to ask about its benefit to participants?”
  • In your outline or curriculum, schedule specific activities to involve and engage participants like asking verbal questions, posting polls, sharing questions on a slide, and so on. Some examples: “Is there anything that you need to be clarified?” “Was this activity helpful?” “What was most valuable to you?” Also: Keep questions going throughout; don’t wait until the end of the program to ask.

Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience.

Ask your audience to 1) help clarify your planning efforts, 2) give feedback during your implementation, and 3) craft questions for debriefing, or 4) review and express the evaluation results. Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience. Include questions to get their ideas on, the perceived value from, and experience of the program. Key questions to get started might be: “Has the program met your needs?” “Is this activity/program of value to you?” “Is this of interest to you?” “What is important to you?” “How have you benefited from this presentation/program?”

Ask questions, then listen. Audience responses and feedback can guide your next steps for planning and evaluation. Make time to get to know, and connect with, the audience by asking about their thoughts or perceptions. Ask your audience – before, during and after your program – so that their perspective is the focus of your planning, activities, and evaluation.

Julie can be contacted at jhuettem@purdue.edu