Tips for young / emerging evaluators

tips The Evaluation for Development blog from Zenda Ofir has been collating tips for young / emerging evaluators – that even experienced evaluators will find interesting. Here are some highlights:
From Zenda herself:
Top Tip 1. Open your mind. Read
Top Tip 2. Be mindful and explicit about what frames and shapes your evaluative judgments.
Top Tip 3. Be open to what constitutes “credible evidence”.
Top Tip 4. Focus a good part of your evaluative activities on “understanding”.
Top Tip 5. Be or become a systems thinker who can also deal with some complexity concepts.
Read more about these tips>>

From Juha Uitto:
Top Tip 1. Think beyond individual interventions and their objectives.
Top Tip 2. Understand, deal with and assess choices and trade-offs made or that should have been made.
Top Tip 3. Methods should not drive evaluations.
Top Tip 4. Think about our interconnected world, and implore others to do the same.
Read more about these tips>>

From Benita Williams:
Top Tip 1. The cruel tyranny of deadlines.
Top Tip 2. Paralysis from juggling competing priorities.
Top Tip 3. Annoyance when you are the messenger who gets shot at
Top Tip 4. Working with an evaluand that affects you emotionally
Top Tip 5. Feeling rejected if you do not land an assignment
Top Tip 6. Feeling demoralized when you work with people who do not understand evaluation
Top Tip 7. Feeling discouraged because of wasted blood sweat and tears
Top Tip 8. Feeling lazy if you try to maintain work-life balance when other consultants seem to work 24/7
Top Tip 9. Feeling overwhelmed by all of the skills and knowledge you should have
Read more about these tips>>

And from Michael Quinn Patton, just one tip:
Top tip 1: Steep yourself in the classics.
Read more about this tip>>

 

Read More

Networking mapping as an evaluation tool

I’ve posted previously about network mapping as an evaluation tool and recently I had the opportunity to use network mapping for an evaluation.

In an evaluation of the Shifting the Power project we were interested to see how local networks of NGOs had grown over the three years of the project. We were lucky that the project had carried out a mapping of NGO networks at the start of the project in 2015 and we then did the same in early 2018; here you can see the results comparing 2015 to 2018 from Bangladesh – interesting data!
network_STP.png

 

You can view the full evaluation report here (pdf)>>

 

 

Read More

Infographics to present evaluation findings

I’ve posted previously about using infographics to summarise evaluation findings; here is a recent example of using an infographic to present research results (click on the image to see larger and complete version); admittedly we packed a lot into this infographic – but still a good summary!
surge - final

Read More

Using Sankey diagrams for data presentation

I’ve always found the Sankey diagram an illustrative way to show transfers from inputs to outputs but have never found a use for them in my own work until now…

The following Sankey diagram shows research reports on crises on the left and the number of challenges identified (for humanitarian surge response) per crisis. The right shows the categories used to group the challenges (“Resource gaps, Policies and systems”, etc). This provides a visual overview of the challenges identified and their volume by crisis and type of challenge.

sankey_digram

I produced this diagram using a free online tool. If interested in the research reports, they can be found here.

 

 

Read More

The global rise and future challenges for evaluation

Very interesting article from the latest edition of the newsletter of the European Evaluation Society “Evaluation between evidence‑based policy and “fake news”: paths to the future” (p. 3).

The article presents the reasons why there has been a global rise in evaluations and four future challenges, summarised here:

Why the global rise of evaluation:
1) In many countries, evaluation is a fixed element in policy-shaping and a management control element
2) The number of national evaluation societies has grown
3) Market for evaluation is continuing to grow
4) The dissemination of evaluation findings has surged
5) Training activities have increased

Future challenges for evaluation:
1) Increasing importance of global issues
2) The challenge of populist movements and “fake news” to evidence-based policy
3) Need for reflexive systems-thinking approaches
4) Increasing demand for participatory evaluation

View the full article (p.3)>>

Read More

New article: Pathways to use of campaigns’ evaluation findings

I’m very happy to share with you an article I co-authored – for those interested in campaigning, evaluation and evaluation use:

Highlights

  • A categorization of six pathways to evaluation use is proposed.
  • Evaluation use occurred slightly more non-linear than linear.
  • Evaluation use was mostly unexpected.
  • Overall meaning assigned to evaluation was linked to frequency of evaluation occurring.
  • Evaluation use contributing to change was often indirect and not simple.

Abstract
This article presents a study on the pathways and processes regarding the use of evaluation findings of communication campaigns from two international organizations, the Office of the High Commissioner for Human Rights (OHCHR) and the International Committee of the Red Cross (ICRC). Several years after the completion of the campaigns and their evaluations, our research identified 28 instances of use and six instances of non-use of the evaluation results, of which the large majority being surprising in nature. Results showed that evaluation use facilitated formal and informal changes at the individual and the organizational level; and, this pattern occurred in a predominantly non-linear fashion, interconnected and overlapping, while gradually decreasing in time and space. Evaluation use was mostly unpredictable, which reflected how meanings are constructed by staff members, as they adjusted and interpreted the findings in opportunistic ways.

View the full article>>

Read More

What to read on communication evaluation?

If you are like me and eager to read anything on evaluation and communications, this will be of interest to you.  AMEC has an exhaustive reading list (pdf) put together by Prof. Jim Macnamara, that  has a top ten “must reads” – many of them available online.

View the list here (pdf)>>

Read More

Presentation in Vienna: Evaluating Advocacy and Communication: Challenges and Solutions

slide1-8Next Wednesday afternoon, 9 May, I am happy to announce that I will be making a presentation for the Vienna Evaluation Network on “Evaluating advocacy and communication: challenges and solutions”. Further information is available here. All interested persons are welcome to attend!

Read More

New e-learning course: managing the politics of evaluation

One of the challenges faced in evaluation is managing the politics involved – notably with the “tricky triangle” between the evaluator, the commissioners and the project being evaluated.  Key is to try and preserve the impartiality and independence of the evaluation while meeting the needs of all stakeholders.

To help tackle these challenges, a new e-learning course is available from TRAASS International – “Managing the Politics of Evaluation“.  The course is taught over 5 modules by Dr  Marlène Laubi Loud, an experienced evaluator and commissioner.  More information is available here>>. 

You can learn more about the course in this video:

Full disclosure: I also teach e-learning modules for TRAASS International on evaluation report writing and how to be a successful evaluation consultant.

 

Read More

Use and value of evaluations

What is the value and use of evaluations is a commonly asked question from many funding or commissioning evaluations. So of interest is that the Hewlett Foundation has just carried out an interesting study on the use and value of the evaluations they carry out.  Some of their key findings were:

  • Evaluation quality has improved over time
  • Spending on evaluation has nearly doubled
  • Higher-quality evaluations cost more
  • Evaluations are being shared more than in the past, but there’s room to improve
  • Evaluations are valuable and used in multiple ways

    On this last finding, they provide the following breakdown on evaluation use- interesting to see that only 3% of evaluations had “no use” – good news for evaluation!

epg1-1024x255

Read more about the study here>>

Together with the Centre for Evaluation Innovation, the Hewlett Foundation is holding a free webinar on the study – April 19, 2018 2:00 PM in Eastern Time (US and Canada) –  Register here>>

 

Read More