Categories
Diversity & Inclusion Fellowships Information NAEPSDP

Are you evaluating your program? Ask the stakeholders!

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

My role as 2017 eXtension NAEPSDP Fellow for Program Evaluation was launched with the Diversity and Inclusion Issue Corps (now called the Impact Collaborative) in Cincinnati. Since then, I have attended online sessions for those projects to share progress, challenges, and accomplishments. In addition, I have been included on the Corps evaluation team to learn of feedback from project teams.

A theme in this feedback was “stakeholder” involvement expressed as 1) key to their program goals; 2) instrumental to providing external input, perspective and support for their program; and 3) important in their next steps to move forward in their program planning, implementation and evaluation efforts.

In my online interactions with project teams, I found myself repeating, “Have you asked them?” I reminded many to “keep asking questions” of their stakeholders, audience, participants, and attendees to connect to those perspectives, interests, and insights.

We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

In education, the “expert” typically shares information or content. But do we know what is of interest to attendees? Do stakeholders understand what is being shared? Is the program of value to participants? How did the audience benefit from taking part in activities? Here is the break: We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

A lot is involved in planning, implementing, evaluating and reporting Extension programs, and we want to do the best we can. So, consider asking questions throughout and use feedback to inform your decisions.

  • Are you planning activities that encourage attendees to be active, involved and engaged? Check on current research for best practices, then ask the intended audience: “What activities would you find interesting to do?”
  • When deciding which topics are most important, check the literature, then ask a couple representatives of your future audience: “What topics are important to you?”
  • While planning the evaluation, check on practice guidelines, then ask stakeholders: “What questions might be asked to find out the value of this program?” Alternately, give them draft questions and ask: “Which ones work well to capture the value of the program for you?” followed by “How might you state a question to ask about its benefit to participants?”
  • In your outline or curriculum, schedule specific activities to involve and engage participants like asking verbal questions, posting polls, sharing questions on a slide, and so on. Some examples: “Is there anything that you need to be clarified?” “Was this activity helpful?” “What was most valuable to you?” Also: Keep questions going throughout; don’t wait until the end of the program to ask.

Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience.

Ask your audience to 1) help clarify your planning efforts, 2) give feedback during your implementation, and 3) craft questions for debriefing, or 4) review and express the evaluation results. Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience. Include questions to get their ideas on, the perceived value from, and experience of the program. Key questions to get started might be: “Has the program met your needs?” “Is this activity/program of value to you?” “Is this of interest to you?” “What is important to you?” “How have you benefited from this presentation/program?”

Ask questions, then listen. Audience responses and feedback can guide your next steps for planning and evaluation. Make time to get to know, and connect with, the audience by asking about their thoughts or perceptions. Ask your audience – before, during and after your program – so that their perspective is the focus of your planning, activities, and evaluation.

Julie can be contacted at jhuettem@purdue.edu

Categories
Fellowships Information

Perspectives: Avoiding Stereotypes in Program Evaluation

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

As the 2017 eXtension NAEPSDP Fellow for Program Evaluation, I have been on a journey to expand my awareness and understanding relating to inclusion, and to look at evaluation from this perspective, since participating in the Diversity and Inclusion Corps in Cincinnati.

Quality versus Quantity

I often ponder the busy-ness of those working in Extension. We wear a lot of hats and have many roles, but in providing education to our county or state residents, we want to be sure we are doing the best we can. To help us think about the quality of programming, not just the quantity, I share these thoughts that put stakeholders first.

Another thoughtful and thought-provoking reading recommendation from my colleague, Dr. Pamala Morris, Assistant Dean/Director of Multicultural Programs at the College of Agriculture at Purdue University, led me to Whistling Vivaldi by Claude M. Steele.

This book on “how stereotypes affect us and what we can do” is based on our human perception of identity. It shares the ways in which stereotyping defines groups and characteristics, how pervasive it is, and how it can influence performance. When individuals experience identity threat from associated restrictive characteristics, their performance is negatively affected. Stereotype threats occur from many perspectives and affect how people perform in education settings, as well as personal and professional situations.

What can we do?

In an education setting, researchers share a two-part explanation:

  • Self-affirmation or sense of competence and worth.
  • Accomplished challenges may create a mindset to interrupt negative restrictions of stereotypes.

For example, think of the message that women are not as good as men in math or science, and the resulting performance by women in STEM. Programming that affirms abilities in science — in combination with instruction and challenging STEM opportunities for accomplishment — can help in addressing the gap in performance associated with the stereotype.

Applying these concepts to our Extension setting, we can be deliberate in efforts to maintain keener awareness of our communities, to explore how we might affirm our stakeholders’ senses of self, and provide quality instruction and challenges to encourage achievement in learning.

This awareness can help direct our program evaluation activities to address the participants’ experience and perspective, not our own as program deliverers. Consider asking stakeholders about their experiences, comforts, barriers, challenges, benefits, values, and accomplishments from participating in programs. Here is where we find the quality in our work!

Thanks again to Pamala Morris for sharing and recommending this book on the human situation we live and face every day.

For More Information

You can contact Julie at jhuettem@purdue.edu

Steele, C.M. (2010). Whistling Vivaldi. New York, NY: W.W. Norton. http://books.wwnorton.com/books/Whistling-Vivaldi/

 

Categories
Diversity & Inclusion Fellowships Information

Perspectives: Overcoming Bias in Program Evaluation

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

My eXtension NAEPSDP Fellowship for Program Evaluation 2017 started with the Diversity and Inclusion Corps in Cincinnati. I have been exploring related resources, opportunities, and associations ever since. Here I share thoughts and reflections more so than a set of instructions. We need space and time to ponder our human experience and learn about other perspectives to incorporate those thoughts as we plan, develop, deliver and report on our Extension work.

Blindspot: Hidden Biases of Good People by Mahzarin R. Banaji and Anthony G. Greenwald is one book recommended to me by Dr. Pamala Morris, Assistant Dean/Director of the Office of Multicultural Programs in the College of Agriculture at Purdue University. The book is about research on our human minds that looks at how our biases develop toward race, ethnicity, gender, age, religion and so on. The researchers share their Implicit Association Test (IAT), which measures how the brain associates people and groups with traits and values. This automatic preference that develops pervades even when egalitarian beliefs are expressed.

A lot of self-reflection about perceptions and openness to others resulted for me. Completing sample tests and activities had me assessing my views, thoughts, and actions about and toward others. This created time and space to think and reflect on our society and our human relations across the personal and professional, local and regional, and global.

We can apply these reflections to program evaluation efforts.

  1. Make sure we make time to reflect on our own hidden biases.
  2. Make opportunities to include our clients/participants in our activities. Invite the perceptions, thoughts, and direction of our stakeholders from the beginning, and throughout, as we work to plan, develop, deliver, and report program activities and evaluation approaches.

The ultimate result is that the opportunities made available are of value and benefit to stakeholders. Given the busy-ness of our jobs, these steps can be easy to overlook, but they are incredibly worthwhile.

I would like to send a special thank you to Pam, for sharing this resource with me at this moment in our society and for a time of reflection on our human interactions.

Julie Huetteman, Ph.D., Coordinator, Extension Strategic Initiatives, Purdue Extension

Banaji, M.R. & Greenwald, A.G. (2016). Blindspot: Hidden Biases of Good People. New York, NY: Bantam Books.