By Keith Morris, President, Elder Law of Michigan, and Director, Center for Elder Rights Advocacy
It can be a daunting task to capture outcomes from legal services. The Legal Services Corporation, which funds any of the legal service providers involved in the Model Approaches to Senior Legal Services, tried to recognize best practices among its grantees and developed the Civil Legal Outcomes Toolkit. The basic premise from their work is that you figure out what will work in your service area given the resources that you have available.
The National Legal Aid and Defenders Association compiled the efforts of many programs across the country in their 2013 Measuring Outcomes – Overview of Some (of Many) Efforts. A quick scan through this shows just how varied the efforts are to try and put a measurable outcome to legal services. There are several other toolkits and summaries related to outcomes and legal aid that you can find with a quick search online.
Generally, outcomes are recorded for three reasons: measure effectiveness, justify funding, or promote the service. Which one(s) of these will you use any outcome reports for? Take the time to do some research to find out what will be most effective in accomplishing your goal. For example, do you need monetary impact (costs avoided, money recovered, etc.)? Or do you need legal status change (no longer facing foreclosure, has health insurance again, etc.) The data needed for these two types of outcomes is somewhat different.
Another important fact in this process is the outcomes are different at different points in time: immediate outcome, intermediate outcome, and long-term outcome. For example, a client who calls the hotline about being evicted and receives advice has an immediate outcome of a better understanding of her legal situation and the options available for her to pursue. An intermediate outcome would be when she took the information she received and followed the attorney’s advice by going to court and winning her case, so her outcome is no longer just her understanding of the situation improving, but that she could avoid eviction, remain in home, and avoid having to pay additional fees, etc. (You can find several webinars and presentations at www.legalhotlines.org/measuring-outcomes that explain the differences.)
The long-term outcome of the legal services intervention is how did this affect her overall trajectory in life? If possible, it would be great reporting to say that because we helped someone with a power of attorney for finances that they avoided a conservatorship. As you probably see, this is not an easy statement to support because it uses a “but for” analysis and assumes that nothing else in her course of life could have caused things to change.
Several senior legal helplines have been conducting a study over the past two years on capturing the immediate outcome based on the client’s perception. One of the strongest findings so far is that the legal advocates are very skeptical at recording anything that declares with certainty that because of their conversation with the client, the legal situation is improved in a specific way. They were comfortable declaring that the client reported understanding the situation better, feeling better about their matter, etc. It proved quite challenging to provide quantitative measures like costs saved, based on the stage in the process.
Another interesting thing that we found with this study is that the legal advocates found it very difficult to select just one outcome and appreciated being able to close the case with two outcomes. In a survey we conducted, we found that the advocates appreciated the opportunity to ask the client about this because it brought closure to the matter.
I encourage you to look at the work done by the Georgia Senior Legal Hotline a couple of years ago on capturing outcomes. (You can find the documentation at legalhotlines.org/measuringoutcomes.) They were fortunate to have a Borchard Fellow with them for a year who focused on capturing outcomes. The process that they used was to identify cases of a legal subject matter, for which they reached out to the client by phone and letter to find out what happened in their situation. Their efforts for capturing these intermediate outcomes have become what other senior legal helplines measure their efforts against.
Jan May from the Legal Counsel for the Elderly, wrote an article for the Management Information Exchange magazine, Tempering the Need to Measure Outcomes with Common Sense: Hotlines as a Case in Point, that highlights the need to look at the cost versus the benefits received. While his focus was outcomes for senior legal hotlines, the concepts are easily transferable to the larger senior legal services programs.
When designing your system, you must make sure you can justify the added time it will take to find out about the outcome, record them in the case management system, aggregate them for reporting, and interpreting them to make sure the data is correct. For many legal services programs, the money received through OAA Title III-B is usually less than 10% of their overall budget. If they are already capturing a set of outcomes for another funder, they may have to pay to have additional fields added to the case management system just to capture another set of outcomes. So, make sure that you are asking them to collect what they will see you use and get buy-in repeatedly in the development process.
Good luck with your efforts to measure outcomes. Remember, don’t reinvent the wheel. It is very likely that another state has gone through a similar problem that you may face in this endeavor. Please give me a call if the Center for Elder Rights Advocacy Staff can be of assistance.