Categories
Success Stories

eXtension NAEPSDP Fellowship Provides Opportunity to Work Differently In Program Evaluation

As long as we are in the realm of ‘I finished my program, now I need to evaluate it’ we are not serving diverse, or really any, audiences as well as we could.

When Julie Huetteman saw the call for applications for a joint eXtension/National Association for Extension Program and Staff Development Professionals (NAEPSDP) fellowship, she was intrigued.

In her role as Strategic Initiatives Coordinator at Purdue University, Julie tracks metrics, reads every impact report, and analyzes the impact of Purdue Extension and how it fits with the University strategic plan. She interacts with people in many different positions, all the way from individual consultations on program evaluation to system-wide reporting. As she puts it, she gets to see both the forest and the trees.

It was also because of these many roles, that she has a unique perspective on the “busy-ness” experienced by most Extension professionals.

The busy-ness of Extension has created a reality in which we ‘add-on’ evaluation. It is something we have to get done. We don’t take the time to engage the stakeholder and fully consider their perspectives.

Huetteman applied for the Fellowship because she saw a moment where she could step forward and focus on the most important part of her job, evaluation. The Fellowship gave her the permission and the time to pursue additional knowledge and skills and focus on something she is passionate about, program evaluation that is responsive and inclusive. She not only saw the opportunity to apply a new approach to her own program but also a platform to influence other Extension professionals’ approaches to evaluation.

As she read literature and connected with colleagues for recommended resources, she soon gravitated toward an emerging approach known as Culturally Responsive Evaluation or CRE. CRE requires engaging the stakeholders at the beginning so that the program evaluation uses culturally appropriate ways to collect, interpret, and share data that is valuable to the audience. It is a way for data and information to serve the culture and not the evaluation itself.

Instead of ‘I’m the evaluation specialist and we need to do this’, we need their perspectives from the beginning to learn what is of value to them and so we can adjust our approach as needed.

This seems self-evident, but it is not how many Extension professionals have traditionally evaluated their programs. During her fellowship, Huetteman served as a key informant for the Diversity & Inclusion Issue Corps (now known as the Impact Collaborative). Through her interactions with different projects, she found herself repeating the same question….”Have you asked them?” She was surprised to hear how often the answer was “No.”

She suspects one reason for this, beyond the busy-ness of Extension, is that Extension has served a fairly traditional audience that is somewhat homogeneous (at a system-scale). This is changing in many areas and has caused some Extension professionals to rethink their approach. She recently toured an Extension office whose 4-H program largely serves a diverse, urban audience. “They are completely changing their way of thinking and considering new ways to serve their audience differently.”

The hard part of CRE, according to Huetteman, is that every program evaluation effort will be different. Every audience, every program, and every change in context requires a different approach.

What’s next? Huetteman plans to use what she learned and created during her Fellowship to help Extension professionals at Purdue approach program evaluation in a new way. She is also part of a network in the North Central. Each person holds a unique program evaluation role at their respective institutions, and by working together they hope to share resources, consult, mentor, and form a critical mass that can advocate for an engaged and responsive approach to program evaluation as the norm for Extension.

Considering the polarized political climate we all live in, taking time to listen and adapt our approaches and appreciate the perspectives of other people is more important than ever.

You can contact Dr. Huetteman at jhuettem@purdue.edu and visit her fellowship page for links to webinars and blog posts developed during her Fellowship.

Learn more about the National Association of Extension Program and Staff Development Professionals (NAEPSDP)

Learn more about eXtension

Categories
Diversity & Inclusion Fellowships Information NAEPSDP

Are you evaluating your program? Ask the stakeholders!

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

My role as 2017 eXtension NAEPSDP Fellow for Program Evaluation was launched with the Diversity and Inclusion Issue Corps (now called the Impact Collaborative) in Cincinnati. Since then, I have attended online sessions for those projects to share progress, challenges, and accomplishments. In addition, I have been included on the Corps evaluation team to learn of feedback from project teams.

A theme in this feedback was “stakeholder” involvement expressed as 1) key to their program goals; 2) instrumental to providing external input, perspective and support for their program; and 3) important in their next steps to move forward in their program planning, implementation and evaluation efforts.

In my online interactions with project teams, I found myself repeating, “Have you asked them?” I reminded many to “keep asking questions” of their stakeholders, audience, participants, and attendees to connect to those perspectives, interests, and insights.

We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

In education, the “expert” typically shares information or content. But do we know what is of interest to attendees? Do stakeholders understand what is being shared? Is the program of value to participants? How did the audience benefit from taking part in activities? Here is the break: We don’t have to have all the answers. Instead, consider asking questions of stakeholders to get those answers.

A lot is involved in planning, implementing, evaluating and reporting Extension programs, and we want to do the best we can. So, consider asking questions throughout and use feedback to inform your decisions.

  • Are you planning activities that encourage attendees to be active, involved and engaged? Check on current research for best practices, then ask the intended audience: “What activities would you find interesting to do?”
  • When deciding which topics are most important, check the literature, then ask a couple representatives of your future audience: “What topics are important to you?”
  • While planning the evaluation, check on practice guidelines, then ask stakeholders: “What questions might be asked to find out the value of this program?” Alternately, give them draft questions and ask: “Which ones work well to capture the value of the program for you?” followed by “How might you state a question to ask about its benefit to participants?”
  • In your outline or curriculum, schedule specific activities to involve and engage participants like asking verbal questions, posting polls, sharing questions on a slide, and so on. Some examples: “Is there anything that you need to be clarified?” “Was this activity helpful?” “What was most valuable to you?” Also: Keep questions going throughout; don’t wait until the end of the program to ask.

Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience.

Ask your audience to 1) help clarify your planning efforts, 2) give feedback during your implementation, and 3) craft questions for debriefing, or 4) review and express the evaluation results. Avoid packing your program with so much content that you forget about — or don’t leave time or space for — getting to know the audience. Include questions to get their ideas on, the perceived value from, and experience of the program. Key questions to get started might be: “Has the program met your needs?” “Is this activity/program of value to you?” “Is this of interest to you?” “What is important to you?” “How have you benefited from this presentation/program?”

Ask questions, then listen. Audience responses and feedback can guide your next steps for planning and evaluation. Make time to get to know, and connect with, the audience by asking about their thoughts or perceptions. Ask your audience – before, during and after your program – so that their perspective is the focus of your planning, activities, and evaluation.

Julie can be contacted at jhuettem@purdue.edu

Categories
Fellowships Information

Perspectives: Avoiding Stereotypes in Program Evaluation

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

As the 2017 eXtension NAEPSDP Fellow for Program Evaluation, I have been on a journey to expand my awareness and understanding relating to inclusion, and to look at evaluation from this perspective, since participating in the Diversity and Inclusion Corps in Cincinnati.

Quality versus Quantity

I often ponder the busy-ness of those working in Extension. We wear a lot of hats and have many roles, but in providing education to our county or state residents, we want to be sure we are doing the best we can. To help us think about the quality of programming, not just the quantity, I share these thoughts that put stakeholders first.

Another thoughtful and thought-provoking reading recommendation from my colleague, Dr. Pamala Morris, Assistant Dean/Director of Multicultural Programs at the College of Agriculture at Purdue University, led me to Whistling Vivaldi by Claude M. Steele.

This book on “how stereotypes affect us and what we can do” is based on our human perception of identity. It shares the ways in which stereotyping defines groups and characteristics, how pervasive it is, and how it can influence performance. When individuals experience identity threat from associated restrictive characteristics, their performance is negatively affected. Stereotype threats occur from many perspectives and affect how people perform in education settings, as well as personal and professional situations.

What can we do?

In an education setting, researchers share a two-part explanation:

  • Self-affirmation or sense of competence and worth.
  • Accomplished challenges may create a mindset to interrupt negative restrictions of stereotypes.

For example, think of the message that women are not as good as men in math or science, and the resulting performance by women in STEM. Programming that affirms abilities in science — in combination with instruction and challenging STEM opportunities for accomplishment — can help in addressing the gap in performance associated with the stereotype.

Applying these concepts to our Extension setting, we can be deliberate in efforts to maintain keener awareness of our communities, to explore how we might affirm our stakeholders’ senses of self, and provide quality instruction and challenges to encourage achievement in learning.

This awareness can help direct our program evaluation activities to address the participants’ experience and perspective, not our own as program deliverers. Consider asking stakeholders about their experiences, comforts, barriers, challenges, benefits, values, and accomplishments from participating in programs. Here is where we find the quality in our work!

Thanks again to Pamala Morris for sharing and recommending this book on the human situation we live and face every day.

For More Information

You can contact Julie at jhuettem@purdue.edu

Steele, C.M. (2010). Whistling Vivaldi. New York, NY: W.W. Norton. http://books.wwnorton.com/books/Whistling-Vivaldi/

 

Categories
Newsroom

News Roundup – October 2017

Ask Your Extension Director if they know how to Kickbox!

At the National Extension Directors and Administrators (NEDA) meeting, Adobe Kickbox was introduced as an innovation toolkit which will be used in the upcoming Designathon One event for the Impact Collaborative.

What are your favorite ways to test ideas or to spur innovation? Let us know by tweeting to @eXtension4u and using the #coopext hashtag.

News Roundup

Designathon One. The Impact Collaborative was officially launched at the National Extension Directors and Administrators (NEDA) meetings last week. Based on feedback and lessons learned from the Issue Corps program, we have made improvements to the process. The first you will notice is the addition of “Designathon One”, which focuses on design thinking and will include Adobe Kickbox innovation kit (and other exciting tools). Eight regional events in January and February 2018 will allow you to attend the one most convenient for you. Learn more or signup to be the first to know when registration opens!

Innovate Extension. You are invited to participate in a panel discussion on innovation at Delaware’s First State Innovate! Join eXtension innovation pioneers Jamie Seger (The Ohio State University) and Paul Hill (Utah State University) as they moderate a panel discussion featuring Bradd Anderson (University of Missouri), Bob Bertsch (North Dakota State), Shane Bradt (University of New Hampshire) Christy Mannering (University of Delaware), Daphne Richards (Texas A&M ) and Michelle Rogers (University of Delaware) and they explore the successes and challenges of working differently and innovation in the workplace. Thursday, October 19, 2017 at 9:45 a.m. EDT. Register for the webinar…

national pesticide safety education center logoPesticide Education Grant. eXtension received formal notification from the EPA of a grant to establish and administer a national sub-award program in support of pesticide applicator education and training for certified applicators of restricted use pesticides. eXtension Foundation was invited in early 2017 to apply for this grant by pesticide safety education coordinators and the National Pesticide Safety Education Center (NPSEC). The goal of the newly-formed NPSEC is to support Extension Pesticide Safety Education Program (PSEP) educators. Read more… | Learn more about the NPSEC…

Success Stories. Grace Peterson (Louisiana State University) and Marlin Bates (Kansas State University) may work in two different program areas, but they both know the power in building a community around an issue they care about.

From the eXtension Blog. We hear the word ‘bias’ a lot these days. Much like beauty, bias is in the eye of the beholder. What if we turn that eye onto ourselves and examine our own biases? Julie Huetteman, Purdue University and eXtension/NAEPSDP Fellow, offers some thoughts on how to uncover our own biases by inviting other perspectives into our programming. Read more…

Julie is presenting a webinar on ”Cullture and context in program evaluation” on October 11, 2017 at 2 pm EDT.

Common Measures. The 4-H Common Measures program will be working with eXtension and using Campus for the delivery of professional development. A competency framework has been created and courses are being set up in Campus.extension.org. More on this coming soon!

ed tech ln logo21st Century Professional. The EdTech Learning Network hosts two “Tweetups” per month. The latest was on “Painting a portrait of the 21st century Extension professional”. The discussion topics include things like the role of Extension program staff and the fear of change. Check out the Tweetcap… (You don’t need to have a Twitter account to read the Tweetcap.)

Pueblo Mini Maker Faire. Extension professionals and volunteers interested in the Maker Movement (one of the Horizon Report topics) and especially how to incorporate it into 4-H programming are invited to the Pueblo Mini Maker Faire.  Among the events are Makers in the Dark, MakerCon Costume Contest and Maker Marketplace with robot and drone challenges. Dorms are available for those who need overnight accommodation. October 27 & 28 in Pueblo, CO. For scholarship or dorm information, contact Jane Crayton, CSU Extension. Faire Website | Call for Makers

eXtension LearnUpcoming Webinars

Check out these (among many more) upcoming professional development events listed on learn.extension.org

Search Recordings on Learn

Visit learn.extension.org anytime and search for topics in your area of expertise or in areas in which you need to get started. Want to know about bugs, financial management, food safety, or military children? There are dozens of recordings being added every month to this valuable resource. Visit learn.extension.org…

Categories
Diversity & Inclusion Fellowships Information

Perspectives: Overcoming Bias in Program Evaluation

Julie Huetteman, Ph.D., is the Strategic Initiatives Coordinator at Purdue Extension. She is serving as the National Association of Extension Program and Staff Development Professionals (NAEPSDP) eXtension Fellow for 2017.

My eXtension NAEPSDP Fellowship for Program Evaluation 2017 started with the Diversity and Inclusion Corps in Cincinnati. I have been exploring related resources, opportunities, and associations ever since. Here I share thoughts and reflections more so than a set of instructions. We need space and time to ponder our human experience and learn about other perspectives to incorporate those thoughts as we plan, develop, deliver and report on our Extension work.

Blindspot: Hidden Biases of Good People by Mahzarin R. Banaji and Anthony G. Greenwald is one book recommended to me by Dr. Pamala Morris, Assistant Dean/Director of the Office of Multicultural Programs in the College of Agriculture at Purdue University. The book is about research on our human minds that looks at how our biases develop toward race, ethnicity, gender, age, religion and so on. The researchers share their Implicit Association Test (IAT), which measures how the brain associates people and groups with traits and values. This automatic preference that develops pervades even when egalitarian beliefs are expressed.

A lot of self-reflection about perceptions and openness to others resulted for me. Completing sample tests and activities had me assessing my views, thoughts, and actions about and toward others. This created time and space to think and reflect on our society and our human relations across the personal and professional, local and regional, and global.

We can apply these reflections to program evaluation efforts.

  1. Make sure we make time to reflect on our own hidden biases.
  2. Make opportunities to include our clients/participants in our activities. Invite the perceptions, thoughts, and direction of our stakeholders from the beginning, and throughout, as we work to plan, develop, deliver, and report program activities and evaluation approaches.

The ultimate result is that the opportunities made available are of value and benefit to stakeholders. Given the busy-ness of our jobs, these steps can be easy to overlook, but they are incredibly worthwhile.

I would like to send a special thank you to Pam, for sharing this resource with me at this moment in our society and for a time of reflection on our human interactions.

Julie Huetteman, Ph.D., Coordinator, Extension Strategic Initiatives, Purdue Extension

Banaji, M.R. & Greenwald, A.G. (2016). Blindspot: Hidden Biases of Good People. New York, NY: Bantam Books.