THE BLOG
07/05/2012 10:09 am ET | Updated Sep 04, 2012

In Defense of Logic Models

Photo by 707d3k

Photo by 7073k

Last month, my post Creative Placemaking Has an Outcomes Problem generated a lot of discussion about creative placemaking and grantmaking strategy, much of it really great. If you haven't had a chance, please check out these thoughtful and substantive responses by--just to name a few--Richard Layman, Niels Strandskov, Seth Beattie, Lance Olson, Andrew Taylor, Diane Ragsdale, Laura Zabel, and most recently, ArtPlace itself. I'm immensely grateful for the seriousness with which these and other readers have taken my critique, and their questions and suggestions for further reading have been tremendously illuminating.

Now that the talk has subsided a bit, it's a good opportunity to clarify and elaborate upon some of the subtler points that I was trying to make in that piece, which definitely left a bit of room for people to read in their own interpretations.  So, just to be clear: despite the provocative title, I wasn't trying to slam the practice of creative placemaking itself, nor call it into question as a focus area for policy and philanthropy in general. As I wrote in response to one of the comments on the original post, I believe strongly in the power of the arts to have a role in revitalizing communities, and I view the desire to direct resources toward bringing such efforts to life as a very positive impulse on the part of funders and policymakers. Furthermore, although I agree with the point made by John Shibley and others that the arts may not be the best way to foment economic development, no one said that cities and regions can only use one strategy. Economic development is a complex beast, and intuition and common sense would hold that there are most likely some specific situations in which the arts can have a real, irreplaceable, and catalytic impact.

My critique is really about how we don't have much information about what those situations are - nor about how infusions of philanthropic capital can make a difference in those situations. What's more, I am not confident that the tools we're currently developing, as useful as they may be for other purposes, will get us there on their own. My contention is that logic models and their conceptual cousins, theories of change, can be useful tools in filling this gap - by forcing us to articulate our assumptions about the way the world works, and by providing a framework that we can use to test those assumptions. The problem is that most of the logic models that I see aren't worked out to the level of detail that I believe is necessary to gain really useful information about the dynamics of these complex processes. In my post, I provided a couple of examples of theories that, while surely far from perfect, at least attempt to recognize some of the numerous and interlocking assumptions embedded in grantmaking of the kind engaged in by today's funders supporting creative placemaking.

It's clear from some of the responses, however, that not everybody shares my optimism in the utility of logic models. Laura Zabel writes that she "hates" them for being too reductive. Diane Ragsdale, taking a cynical view, is worried that they may be misused by funders in order to make themselves seem smarter than they really are or blame grantees for failed strategies. ArtPlace's response suggests that logic models raise the bar for research too high, and that because proving a causal connection between these investments and the change they produce (or don't) is so difficult, we're better off not trying. While I can sympathize with each of these critiques, I also think that they give logic models a bad rap. I feel that logic models are a tool of tremendous power whose potential is only beginning to be unlocked. It's true that, just like philanthropy and policy, logic models can be done very badly. But that doesn't mean there's no gain for us in trying to do them well.

Before we get into all that, however, I'm guessing that some of you probably could use a refresher course on logic models and the terminology associated with them, which can be quite confusing. So let's start with a little background on what this is all about.

What the Hell Is a Logic Model, Anyway?

Simply put, a logic model is a method of describing and visualizing a strategy. Logic models have their conceptual origin in the "logical framework approach" originally developed for USAID in 1969 by Leon J. Rosenberg of Fry Consultants. Their use was largely concentrated in the international aid arena until 1996, when United Way of America published a manual on program outcome measurement and encouraged its hundreds of local agencies and thousands of grantees to adopt logic models as a matter of course. Since then, large private funders such as the Kellogg and Hewlett Foundations have integrated logic models into their program design and execution, and the concept is commonly taught in graduate programs in public policy, urban planning, and beyond.

Even though logic models have achieved greater adoption over the past several decades, there is little standardization in the content, format, and level of ambition seen in professionally produced logic models for institutions large and small. Worse, everyone seems to want to come up with their own terms to describe features of the logic model, and as a result, you'll notice a lot of variation in language as well. Below, I'll do my best to isolate the elements that most of these efforts have in common.

Nearly all logic models contain the following fundamental elements. In combination, they describe a linear, causal pathway between programs or policy interventions and an aspirational end-state.

  • Activities are actions or strategies undertaken by the organization that is the subject of the logic model. These activities usually take place in the context of ongoing programs, although they can also be one-time projects, special initiatives, or policies such as legislation or regulation.
  • Outputs refer to observable indications that the above activities are being implemented correctly and as designed.
  • Outcomes are the desired short-, medium-, or long-term results of successful program or policy implementation.
  • Impacts (or Goals) represent the highest purpose of the program, policy, or agency that is the subject of the logic model. Sometimes you'll find these lumped in with Outcomes.
Logic model for a bicycle helmet public information campaign

Logic model for a bicycle helmet public information campaign, courtesy RUSH Project

Many realizations of logic models combine these essential elements with additional information that provides contextual background for this causal pathway. Several of these supplemental concepts are listed below in approximate order from most common to most obscure.

  • Measures (or Indicators) for outputs, outcomes, and impacts are concrete, usually quantitative data points that shed light on the degree to which each result has been achieved.
  • Inputs are resources available to the program or organization in accomplishing its goals.
  • Assumptions are preconditions upon which the model rests. If one or more assumptions proves unsound, the integrity of the model may be threatened.
  • Benchmarks extend the concept of measures to incorporate specific target goals (so, not just "# of petitions delivered to Congress" but "50,000 petitions delivered to Congress").
  • Target Population refers to the audience(s) for the activities listed in the logic model.
  • Influential Factors are variables or circumstances that exist in the broader environment and could affect the performance of the strategy as designed (e.g., an upcoming election cycle whose outcome might change the underlying landscape in which the program operates).
What About Theories of Change?

A frequently-employed alternative logic model approach strips out this latter set of contextual elements and instead aims to visualize the linear causal chain at a finer grain of detail. This version of a logic model is typically referred to as a theory of change (or, sometimes, a program theory). A well-executed theory of change diagram "unpacks" the processes and factors that lead to successful outcomes, exposing relationships between isolated variables that can then become the subject of research or evaluation.

Partial theory of change from Project SuperWomen case study (ActKnowledge/Aspen Institute Roundtable on Community Change

Partial theory of change from Project SuperWomen case study (ActKnowledge/Aspen Institute Roundtable on Community Change)

Sometimes logic models and theories of change are presented as distinct concepts, while other times they really refer to the same thing. This is because logic models and theories of change evolved out of distinct communities of practice, but the philanthropic field has not always respected the distinction in the terminology it's adopted to describe these tools. In my own practice I prefer to use theories of change, but for the sake of simplicity and readability, in the rest of this article I'm going to use the term "logic model" inclusively to refer to any diagram that clearly shows some combination of activities and outcomes, regardless of what other elements it may include or the visual approach it takes.

*

OK, now that we have our definitions in order, we can start talking about what makes logic models so awesome.

Awesome #1: Logic Models Describe What's Already Going On in Your Head

So, here's the thing: the core questions involved in creating any logic model--What am I trying to do? Why am I trying to do it? How will I know if I've succeeded or not?--represent the very essence of strategy. As a rabbi might say, "the rest is commentary." If you have a strong sense of what the answers to these questions are, then you have a logic model in your head whether you realize it or not. All the diagram does is make it explicit.

To illustrate this, we can look at a simple example. Let's say I decide I'm done with this whole "arts" thing and I want to go to law school instead. I know, though, that in order to get into a good law school I need to get a good score on the LSAT. So, how can I make sure I get a good score? Intuitively, I decide that taking a test prep class is the way to go.

Why do I think taking a test prep class is a way to increase my score on the LSAT? Well, if my score isn't as high as it could be, it's probably due to some combination of two factors. First, I may not know the material well enough. So, if the class helps me to learn how to answer the test questions better, I'll likely perform better on the test. Second, there may be a psychological factor as well. If I'm someone who gets nervous on tests, then my performance on them may suffer. The practice exams and deep engagement with the material that comes with a class could help me to get more comfortable with the idea of the LSAT and make it seem less intimidating, thus improving my performance.

Seems logical enough, right? And voila, it lends itself quite easily to a logic model:

Sample program theory

The truth is that any decision you make, if it has any element of intentionality at all, can be diagrammed as a logic model. You might hate logic models with every fiber of your being and think they're the stupidest thing ever created, but I'm telling you right now: if you believe in strategy, then you believe in logic models.

Awesome #2: Logic Models Are Incredibly Flexible

Now, there's a difference between having a logic model in your head and having a good logic model in your head. The example above is simple, but it's limited by that simplicity. It doesn't explain why I might have decided to go to law school, or explore other ways that I could get into the school I want besides increasing my test score. In short, it's pretty much just a straight-up mapping of a decision already made.

The best logic models don't do that. Instead, they proceed with the end in mind (what is the goal we want to achieve?) and then methodically work backwards to understand what activities or strategies would be most appropriate to achieve that end. The ultimate outcome of this exercise may be a very different set of strategies than the ones you were originally contemplating or the programs you already have in place! Because of that, the logic model creation process can be great for opening up new ways of thinking about old problems or longstanding dreams, as well as clarifying what's really important to you and/or your organization.

I mentioned earlier that not everyone is a fan of logic models. Here's what Laura Zabel had to say about them in her post responding to mine:

I hate logic models. For me they are, somehow, simultaneously too reductive and too complex. Too simple, too linear for how I think the world works and too dry, too chart-y for how beautiful the world is. They make me irrationally grumpy.

Arlene Goldbard, in a 2010 essay, is similarly grumpy about logic models:

[R]equiring one of these charts as part of a grant proposal bears about as much real relationship to community organizations' work as would asking each to weave a placemat...[T]he task of boiling the answers down to colored bars often wastes days, compressing most of the useful meaning out of the inquiry.

I can't speak to Laura's and Arlene's experiences directly, but I know they are not uncommon. Unfortunately, logic models that are rife with imprecision, questionable assumptions, and inappropriate associations are more frustrating to work with than no logic model at all--and it's not as easy as it looks to create logic models that are free of these flaws. Such problems are magnified when logic models are treated as edicts sent down from on high rather than the learning, living documents that they are intended to be.

In her post, Laura presents an alternative formulation of a logic model that describes her theory of change for creative placemaking: artists + love + authenticity -> creative placemaking. While I'd classify this as more of a definition of creative placemaking than a logic model, it goes a long way toward illustrating my point that we all have latent logic models in our head that are just waiting to be expressed as such. Laura writes, "there's no logic model in the world that can capture how a crazy parade [the annual MayDay parade in Minneapolis] can restore my faith in humanity." I couldn't disagree more - in fact, I made one, relying solely on the way Laura describes the parade in her post. Here you go:

Laura Zabel's Faith in Humanity

For the past few months, I've been researching impact assessment methods used across the social sector in connection with some evaluation work we're doing here at Fractured Atlas. Seemingly every year, someone comes up with a new way of evaluating impact, whether it's for social purpose investing, choosing grants, or measuring externalities. I'm not done yet, but what I've found so far has only reinforced my appreciation for the logic model. The beauty of logic models is that, because they relate so directly to the fundamental elements of strategy, they are endlessly adaptable to almost any situation. I actually find it kind of funny when people call logic models too rigid, given the alternatives - especially considering how much of our lives is ruled by the granddaddy of rigid, one-dimensional success metrics: money.

Awesome #3: Logic Models Are a Victory for Transparency

Hewlett Foundation Performing Arts Program strategic framework

Hewlett Foundation Performing Arts Program strategic framework