Video on Decision-making

Here’s a video I made relating to how to make better decisions. Please let me know what you think.

Flow chart with features 2

Using Knowledge for success,
Dr. Rick Hoefer

3 Things Nonprofits Must Do to Thrive

One of the original purposes of this blog was to be to disseminate research knowledge in short bursts—just a quick summary of some of the important papers that have been published in the past few years.

I’ve written for the past month on the horrid funding environment for nonprofits currently and the managerial problems that is causes. But there are three things that research indicate must be done to prepare to the future.

Julie Steen and Thomas Smith describe three primary areas where agencies can prepare: political connectivity, documentation of agency policies, and comprehensive strategic planning.

This research is based on the view that agencies are affected by their environment but that they also can have an impact on it. This is frequently known as the ecological model.

Political connectivity is the extent to which agency leaders are interacting with the leaders of organizations outside the agency, in its environment. This could include governmental leaders (elected and appointed), foundation leaders, business leaders, even university leaders. All of these leaders have resources that the nonprofit may wish to gain access to.

Documentation of agency policies is said to be helpful to running the agency, of course, but can also be helpful to deal with the ever-present threat of lawsuits.

Strategic planning is important because it represents the application of best thinking relating to the organizations strengths, and weaknesses, and relates them to opportunities and threats in the environment. Such planning can provide guidance to the agency as it chooses among various routes it could take to keep afloat. One of the most distressing results from the study is that “Less than a quarter of the organizations reported having a strategic plan that included attention to funding” (p. 37).

Using knowledge for success,

Dr. Rick Hoefer
SOURCE: Steen, J. & Smith, T. (2007). An assessment of the minimization of risk and maximization of opportunity among private nonprofit agencies in Florida. Administration in Social Work, 31(3), 29-39.

The Selection of Outcome Measures

One of the more important administrative functions of human service organizations is to choose outcome measures that appropriately describe how well a program is working to achieve positive results for clients.
But it isn’t always clear how to do this.

One approach that I like to use is a logic model that has a number of outcomes built-in. Logic models are quite useful, but will be covered in a different post.

In this post, I want to provide thoughts based on an article by John Poertner, Terry Moore (a classmate of mine from the MSW program at the University of Kansas) and Tomas McDonald, called “Managing for Outcomes: The Selection of Sets of Outcome Measures” from Administration in Social Work, 32(4), 2008, pp. 5-22.

The authors provide guidance on choosing outcome measures in the form of 7 criteria, beyond basic validity and reliability. I’ll list each one and provide a commentary of my own on the suggestion.

1. The set of outcome measures should be understandable. (p. 8).

This means that they should be understandable without a lot of extra education and explanation. The authors say that they should “pass the common sense test for consumers of the data” (p 8).

2. The set should contain as few measures as possible. (p. 9)

This is pretty straightforward, except in knowing how many are “as few as possible”. Too many measures dilute your attention. Measures that are not used in any decisions don’t need to be collected. The authors suggest the following question be asked of people using the information: “If you could only have one indicator of X, what do you think is most important?” (p. 10).

3. The costs and benefits of data collection should be considered. (p. 10)

It is also important to estimate the cost of collecting the information, inputting it into a useful form, storing it and retrieving it. Sometimes the cost astounds the person suggesting the measure and persuades decision-makers from adopting it.

4. The set of indicators should count all outcomes that occur as clients move through the program/system. (This is a paraphrasing of their guideline found on p. 12)

If you don’t look at the client’s entire career in the program or system, it is easy to “lose track” of what is going on at certain times in the client’s life. In addition, it can be very helpful to getting a good sense of client outcomes to build in a follow-up measure at 6 months or a year after the client’s involvement in a program is completed. If you do this, you need to build in the staff and other infrastructure to support it. Follow-up is expensive, but the benefits are enormous.

5. Perverse incentives should be avoided. (p. 12).

This sounds pretty obvious–no one is going to say, “Be sure to include perverse incentives.” But the tricky thing with perverse incentives is that it isn’t clear what they might be ahead of time. The authors indicate that this can occur when only one aspect of the client’s life is measured. Staff time will be devoted to achieving this one outcome, perhaps at the expense of other, equally or more important outcomes, that simply are not measured.

An example of a perverse incentive that I read somewhere is as follows. In order to decrease the number of rats on an Army base, the Commanding Officer decreed that a soldier would receive a cash bonus for every complete rat tail turned in. The rat population seemed to decrease for a while, but then there appeared to be no reduction is rats, even though bonuses were being paid out at an accelerating rate. At last, one private told the officer what was happening. Soldiers were raising rats by the hundreds in order to kill them for their tails. The measurement drift became how many rat tails were being collected rather than how many rats were running around on the base.

6. A set of measures needs to capture real changes and avoid being susceptible to administrative manipulation (gaming the system) (p. 13).

While there is some overlap between guideline 6 and guideline 5, it is important to note that vagueness is often the enemy here. Almost any vagueness can be used to advantage if the program desires to game the system. The authors provide the example of “placement stability” in a child welfare context. Placement stability is defined as “the number of places that the child has lived during the current removal episode” (p. 13).

But what does “lived” mean? Does it include a short hospital stay? Or a trip to see a relative for a short time?

Poertner, Moore and McDonald also suggest developing a culture in the organization that “rewards openness and positive performance” (p 14). This will do much to prevent efforts to “game the system”.

7. The set should include indicators that counterbalance each other (p. 15).

This guideline repeats some of the prior ideas, but is interesting. Opposite measures are useful so that one doesn’t assume that a decrease in one problem area is a totally good outcome. It could be the effort to reduce one (measured) problem can lead to an increase in another (unmeasured) problem area.

Taking these suggestions as a whole can lead managers and executives to choosing better outcome measures and trusting the measures more. Perhaps managers can begin to make decisions based on the data collected and analyzed.

It is only by having clear outcomes, and adequate measures of those clear outcomes that we can know what is happening to clients and recipients of human service interventions. It is up to all of us to move forward as rapidly as possible, with sufficient detail paid to adequate outcome measures so that we can know what we’re doing for our clients.

Using knowledge for success,

Dr. Rick Hoefer

Evaluating Capacity Building, Part 2

NOTE: This post was supposed to be published BEFORE part 3, but I goofed. Here it is, later than expected, but still good..

My previous post introduced a new report called Evaluating Foundation-supported Capacity Building: Lessons Learned, by Thomas E. Backer, Jane Ellen Bleeg and Kathryn Groves, from the Human Interaction Research Institute. That post listed the five key conditions for effective capacity building.

This post will discuss the ten good practices for effective capacity building that are identified in the report (my paraphrasing below).

1. A good needs assessment should be done first in order to know what is really needed by nonprofits in the way of capacity building.

2. Capacity building efforts need to be planned with assessment built in.

3. Evaluation results need to be used to improve the capacity building efforts.

4. Capacity building consultants need to be trained and then matched to nonprofits who can most benefit from their expertise.

5. Nonprofits need to be linked to resources to support the capacity building. These resources can be from inside of the nonprofit or from the community.

6. Trainings need to be more than didactic–and must be specific to the nonprofit receiving the training.

7. Learning from peers is an effective strategy to enhance learning, professional development, and cross-agency collaboration.

8. Training should include capacity building in the area of leadership for staff and volunteers.

9. Capacity building efforts work better when recipients are clustered into specific cohorts.

10. Along the lines of number 3, data from specific research efforts on capacity building should be used to improve the training.

So what can we say about these ten “good practices”?

First, there is a lot of emphasis on getting data to improve capacity building practice. That sounds very evidence-based to me!

Second, we need to eliminate the idea that one-sized training fits all. Capacity training must be fit to the individual nonprofit organization. You can’t just hire some person off the street to give the same training seminar to every organization in town.

Third, if you let practitioners do some of the training and other work (peer-to-peer efforts) you may get better results than if you just use professional consutants and/or academics to do the training. This may have the benefit of professional development for the peer leaders, as well.

Also, if organizations are in cohorts, you can use an earlier cohort to help out more recent cohorts. After all, to a third grader, most 5th graders are geniuses!

In the next post, we’ll continue looking at this report and learn about eight barriers to doing capacity building well.

Using knowledge for success,

Dr. Rick Hoefer

Evaluating capacity building, Part 1

Every now and then I find a report or study that I think has really useful information, and supports what I’m trying to do with this blog: promote evidence-based social work management practice.

So here’s what I’ve found this time: “Evaluating Foundation-Supported Capacity Building: Lessons Learned” by Thomas E. Backer, Jane Ellen Bleeg and Kathryn Groves of the Human Interaction Research Institute (www.humaninteract.org), published last month.

Without going into the full report, let’s cut to the chase: what are the lessons learned?

Here are five key conditions for effective capacity building (paraphrased):
1. Foundations supporting capacity building must communicate well and do so in an on-going manner with the grantees.
2. Grantees must be ready to change, that is, to have their capacity increased.
3. The top people in the grantee organization have to be strongly supportive of capacity building for their organization.
4. BOTH the funder and and the recipitent organization have to be flexible in implementing the capacity building.
5. There has to be enough time for real and lasting change to occur.

So, to put this into even fewer words: Before change can occur, the grantee organization has to be ready and willing, the funder has to communicate clearly the goals desired, but be flexible in how the capacity building occurs, and everybody has to realize it may take a long time to see results.

Next time, we’ll look at the next section of the report, examining good practices for effective capacity building.

Using knowledge for success,

Dr. Rick Hoefer

Social Justice Defined? Part 3

The last two postings have been about the idea of social justice. I make the point that social workers (and others) don’t really know what it is, not even when they see it. So I am examining the idea of social justice from the viewpoint of John Rawls (last time) and Robert Nozick (this post). The last posting on the topic will compare the two views and relate them to social work management and practice.

Robert Nozick’s Views on Distributive Justice

A very different interpretation of distributive justice is set forth by Robert Nozick in Anarchy, State and Utopia (1974). He argues that Rawls (and others) who focus on “end-states” or “patterns” of a distributive process are wrong. In order to maintain a “fair” distribution of resources, a central distribution mechanism would have to exist, and it does not. In other words, the end-state of a Rawlsian system is a rather equal distribution of economic goods that is constantly being made less equal due to the unequal amounts of effort and skill people bring to the task of making a living. The only way to keep inequality from taking place is for government to redistribute wealth constantly.

In a free society, diverse persons control different resources and new holdings arise out of the voluntary exchanges and actions of persons. There is no more a distributing or distribution of shares than there is a distributing of mates in a society in which persons choose whom they shall marry. The total result is the product of many individual decisions which the different individuals involved are entitled to make (Nozick, 1974: 149-150).

Nozick then proposes a procedural approach to distributive justice in which “a distribution is just if everyone is entitled to the holdings he possesses under the distribution” (p. 151). To simplify his theory, Nozick makes a slogan out of it: “From each as they choose, to each as they are chosen” (p. 160).

An example can illustrate his meaning clearly. An “end-state” theorist might object to a distribution of income that left many people with little and a few (including sports stars) with much. But suppose that “the many” choose to buy tickets to football games where the stars play. The football team makes a large profit and pays the players quite well. Nozick argues that this voluntary transfer of holdings (income) from the many to the few is completely just and that any move to redistribute it through governmental action (coercion) is unjust. He makes this last point very strongly when he states: “Taxation of earnings from labor is on a par with forced labor” (p. 169).

The main principle to ensure social justice for Nozick, then, is to set up a way for fair, voluntary, exchanges to take place. This market should be as unfettered as possible. Once the rules are set and followed any end result, no matter how unequal, that occurs is socially just. Government’s major duty is to ensure that fair rules are followed because this leads to a just outcome. The idea is similar to political freedom. As long as the rules of one-person, one-vote, are followed in an election, and everyone has a chance to vote, then the result of such a free election is just and fair. Nozick argues that it is not just to determine who should win an election ahead of time in order to have a “fair” distribution of elected positions given to different types of people. Similarly, it is not just to determine if the outcome of an economic distribution is fair by looking at the amount of inequality that ensues. As long as fair rules are followed in the marketplace, the distribution of money that results is just.

Using knowledge for succcess,

Dr. Rick Hoefer

Researching Policy Practice and Advocacy

I just finished editing a new issue of my journal, the Journal of Policy Practice. That always feels like an accomplishment. But it brings to mind the difficulty of actually defining and then doing research on policy practice and other topics, such as advocacy.

Sure, you can find definitions of policy practice, or advocacy, or most other social work terms, but that doesn’t mean that the definitions are particularly easy to use in a research perspective.

Policy practice is pretty much “doing” advocacy, but it also includes a lot of other things, such as engaging, assessing, planning, and evaluating, as my students hear umpteen times during any course I teach. But that is such a broad definition, that it is not so useful to narrow things down, which has to be one of the purposes of a definition.

If we can’t define well what we’re talking or writing about, we clearly can’t more forward with evidence-based work, because it will not ever be clear what we’re researching or even what we want to talk about.

This entry is just the starting point of seveal related postings about how we need to have stronger definitions, if you are going to have stronger research, on policy practice, advocacy or just a very large number of other things.

I hope you find this series of interest.

Using knowledge for success,

Dr. Rick Hoefer

Advocacy Evaluation

A few weeks ago I posted some information regarding advocacy evaluation, and Ms. Julia Coffman, one of the people bringing new ideas into the field on this topic.

I am happy to say that one of my students in a Program Evaluation class took on the challenge of using this new information to create an evaluation plan for an advocacy organization. Because of the brevity of the class (only 4 months long) this student couldn’t actually conduct the evaluation she planned, but she did a great job in terms of laying out what could be done, including both evaluating the process of the advocacy training program and the possible results of advocacy efforts.

Potential process outcomes she pinpointed include: # of volunteers trained, number of contacts with elected officials, and # of coalition partners.

The short-term outcomes she described include knowledge level of volunteer advocates, level of knowledge on part of policy makers, amount of media coverage (in local media where chapters are).

Longer term outcomes include increased appropriations and amendments to current law.

I’d love to mention my student by name, because it is an outstanding paper, but I don’t feel comfortable doing so without her permission. At any rate, the paper demonstrates that the work Julia Coffman is doing can be transferred from “theoretical” to “practical” by people in the field.

Using knowledge for success,

Dr. Rick Hoefer

Reference:
Coffman, J. (2009). A user’s guide to advocacy evaluation planning. Harvard Family Researach Project, 1-22, Cambridge: MA.

Better Late Than Never

Evidence-based Nonprofit Management: Is it Possible? is the title of the paper I am presenting early Thursday morning at the Association for Research on Nonprofits and Voluntary Action in Cleveland.

I’ll have 10 minutes to discuss this topic from my point of view, after having listened to 3 other presenters give information on their papers, none of which have much to do with evidence-based management, at least judging from their titles. So, we’ll see how it goes.

But to answer my own question, “Yes, evidence-based nonprofit management IS possible. Because EBMgmt begins with a process and never has an end point. We always start by wanting to answer a question, and then proceed to determine what information is possible.

My current research project is trying to bring together what is known about management in social work, but I can’t say that anyone should look at what I have found and synthesized and say, “OK, that’s all I need to know about that subject. Next question.”

I’ll provide more detail later. One thing I’ve found out these last couple of over-worked weeks, is that it is extremely easy to get behind on the blog writing business. One reason is that I want to provide a lot in each blog entry. So it is probably better to write more consistently but with less content, than to write only sporadically but with a lot of content.

Using knowledge (including self-knowledge) for success,

Rick

EB Social Work Management in Administration in Social Work

I’ve just uploaded a paper that I’ll present at a conference in a couple of weeks. While this paper only scratches the surface of the data I collected during my sabbatical last spring, some interesting findings are already evident. For this project, I read 15 years worth of articles in Administration in Social Work.

First, it is somewhat surprising the percentage of articles that had full professors as the first author (nearly a third of all articles). It was also surprising that when full professors are the lead author, their manuscripts are among the least likely to include practice principles. As expected, when practitioners are the lead author, the articles are most likely to contain practice principles.

While ASW has published more empirical articles than literature reviews or conceptual manuscripts by far, it is clear that these empirical articles are using very low level research designs, in general. No experimental comparison group designs were published during this 15-year period, for example, and the most common type of design over all types of empirical research is the case study, followed by cross-sectional survey with statistical controls.

While the EBP model indicates a search for the best evidence available, it is clear that the level of rigor in these years has been fairly low, leading one to question how much faith to put in the results of the studies.

Another finding of the research is that there is a considerable gap between what is considered to be important for social work managers to know (as judged by the social work managers themselves in the National Network for Social Work Managers) and what exists in terms of evidence-based practices.

If some topics can only be learned from practice wisdom and “on-the-job training” then the call for evidence-based practice in social work administration (and perhaps nonprofit administration in general) rests on a shaky foundation.

An obvious limitation of this paper is that the information presented comes from only one social work journal and EBP ideas should come from a larger variety of sources, such as Nonprofit Management and Leadership, Public Administration Review and various other administration journals, including some in business.

Still, we should be able to determine a great deal about the state of the nonprofit social work administration literature by reviewing all articles in the most apt social work journal devoted to administration as a good starting point.

There is a clear need to conduct further research into other journals, both in social work and in other disciplines, to determine if the same lack of research support for current practice exists or if other fields more fully cover the breadth of their field with empirically-derived recommendations for practice. Pfeffer and Sutton (2006) shows one effort to pull together research to inform managers as to what might work, and what has little or no research support.

Future Steps and Conclusion

Because this paper does not permit a full examination of all the data and practice principles I collected, I am providing a beginning list of practice principles uncovered in the Appendix. It is too bulky to put here on my blog, but I am asking the folks at the conference to look them over. Judging from a quick view of these practice principles, one can determine that many of the questions that might come up in a program manager’s life could be at least addressed in some way, with one or more of the findings. It is important to note that, of course, there are a host of other questions that might be important that no information seems to touch on.

Further, to what extent would you like to have managers relying on these findings to determine future behavior? If you examine the material and believe that there is much to be desired, is it possible that you can alter your behavior and research agenda to make the world an easier place for managers to determine how best to run their agencies?

We must remember that is it not enough to develop catalogs of “what works,” or what is “best practice” if we wish to have social work and nonprofit management be evidence-based. EBM is a process that must be continually practiced, lest it become another list of “authoritative” guidelines that unknowledgeable managers look at and try to use regardless of real applicability.

Despite this caution, it is also important to remember that overworked managers can certainly use some help in doing their jobs better, and one way of doing this is for academics to assist in the translation of abstract research findings into the language of practice, all the better to assist managers to run their agencies more effectively. The final beneficiary, the client, will be helped the most.

Using knowledge for success,

Dr. Rick Hoefer