Evaluation & Strategic Learning

In a couple of weeks, I will be presenting to a group of health focused funders on the benefits of evaluation and research. Specifically, I'm talking about how our Foundation takes the information we gather from evaluation and how we use it to inform our future grantmaking. If you are interested - here is the website for the event - http://www.gih.org/calendar_url2665/calendar_url_show.htm?doc_id=706939. The title of my presentation is Strategic Learning: Integrating Information and Assessing Your Foundation's Impact and while the presentation will focus on the various avenues we are exploring for impact assessment, there are other important bits of information we have gleaned that have dramatically impacted our grantmaking.

Most people think of evaluation in the framework of value. I mentioned that in an earlier post - if you haven't read it, I find it riveting. [What? I can't be sarcastic in my posts?] But, the statement in this case really does hold. Travel with an evaluator some time on a site visit. A significant portion of the first meeting with a team you are "evaluating" is spent on explaining that "this isn't an audit" and that "we aren't here to play "gotcha"". Like it or not, the term evaluation has become synonymous with the favorite activities of the IRS. [Note to anyone affiliated with the IRS, I really like you guys, you do good work, I'm just talking about your reputation, not my own personal feelings. No need to audit me.] Members of the evaluation community actually have discussed using a different word when bringing up the topic of evaluation and there are a few folks who do. Because of this, the general public doesn't quite understand that there are many more benefits to evaluation other than discovering whether a program or process actually worked. The process of evaluation allows those interested to learn a whole lot more about what is being evaluated than just that. For example...

There is some literature out in the world that suggests that having a network of resources and using it is a strong indicator of the probability of an organization and its programs being sustained for a longer period of time. Curiously, it appears that the network is actually more important than funding. I think this is more of a notion that the two are somewhat interdependent - having a larger network probably is predictive of more opportunities to find funding. But, I didn't start writing this example just to explore that argument further with you - we are actually doing some research with our past grantees to see what bubbles to the top for prediction of sustainability. But, for now, it seems that networks are very important. The reason I can say that is that some of our evaluation work has looked at sustainability of programs (we are a "seed" funder, funding new or expansions of current programs, not a sustaining funder). We want the programs we fund to be sustained past our funding cycle and have oriented some of our evaluation work to look at how this works. Well, some of the interesting information was that networks are important (did you notice that I keep driving this point home), but depending on whether you are a rural or urban nonprofit, the relative importance changes - other things bubble up to the top. It is this discovery that led us to the research on prediction of sustainability. The eventual idea is that our program staff will have a checklist to support their review of grant applications, allowing them to assess the probability of the organization and their programs being sustained after funding. How we use the tool may vary, however the initial intent is to identify issues with a possible grantee from the very beginning and to start providing them with support to help them grow in those areas of weakness.

The concept of networks also resides in some other evaluation work we are conducting. One of our initiatives focuses on funding "non-traditional" programs - basically funding organizations that normally would not be working with us. A large percentage of these organizations are faith based and/or are small community organizations. Given that we recognize that resource networks are important, we wanted to know how these organizations work. Well, what we have discovered is that these small community organizations excel at developing resource networks. What seems to be a difficult concept for our more traditional grantees to work with, these small community organizations live and breathe based upon their resource network. Teasing out how their networks develop and why is a huge "learning" that we can share with our traditionally funded grantees. The information we are gathering has nothing to do with whether their program is "successful", but rather is giving us information as to how they do their business and are successful in areas that seem to be a struggle for others.

The point I'm making here is that we have learned some impact unrelated information that will and is changing how we conduct grantmaking. This and other nuggets of information have changed what support we provide and how we provide it. Mind you, we are interested in impact and I'll address my thoughts on that in another post, but I thought it a good time to remind all of us that there is more to evaluation and strategic learning other than whether our funding "did good work".

As always, I'm open to your comments.

Best regards,

Charles Gasper