With companies seeking to cut litigation costs, some attorneys have begun trying to conduct their own jury research without the aid of a litigation consulting firm. Though you may think this will save a lot, while still providing valuable feedback about your case, attorneys who conduct their own research risk making key strategic decisions on results that may not be methodologically sound. That is, there are reasons why attorneys should focus on advocacy and leave the research to the social science researchers. Below, we discuss the reasons that attorney-conducted research can produce faulty or false positive results, despite the attorneys’ best efforts.
Introducing Bias: Drinking Your Own Whiskey
One of the biggest problems with attorney-conducted research is bias. Lawyers, by design, are zealous advocates for their clients. No matter how hard they try, those who have been invested in a case for months or years will inevitably be biased in presenting the case, eliciting juror feedback and interpreting the results. When preparing for research projects, we continually have to play devil’s advocate and remind our defense clients to incorporate the pro-plaintiff facts and arguments, or to make sure to include plaintiff-friendly jury instructions or verdict questions. Without a neutral third party reviewing the presentations and discussion questions for parity, the research may be tainted by experimenter bias. This is particularly true when an attorney conducts a focus group or moderated discussion. As humans, we often don’t realize the verbal and nonverbal cues that we produce when speaking, and an attorney will undoubtedly pose questions, produce inflections, and make facial and body reactions to juror responses that will reveal which side he or she favors. This causes jurors to mold their responses to what they think the moderator wants to hear, and it can compromise not only the confidentiality of the research, but also the results.
Bias also emerges when interpreting the results. Attorneys will often highlight the positive findings and minimize or try to explain away the negative. A consultant who has less familiarity with the case is better situated to provide an accurate evaluation of the case based on experience and juror feedback. Jury consultants also have the benefit of talking to thousands of jurors and participating in dozens of trials each year of varying case types. They are therefore better suited to spot unusual findings, extrapolate to actual trial results, and compare your case and research results to other cases that were successes or failures at trial.
Another benefit of using a jury consulting firm is access to the types of people who are likely to be jurors in your case. Years of research and experience have shown that several people can hear the same exact facts and arguments, yet come to very different opinions and conclusions. Often, these differences in perception are due to the diversity of backgrounds, experiences and beliefs that cause jurors to filter and characterize the evidence. Attorneys who conduct their own research usually don’t have access to the diversity of people needed to get accurate feedback about their case. They often use their friends, relatives and co-workers – or referrals from their friends, relatives and co-workers – as mock jurors in the research. There are two fundamental problems with this. First, these individuals will undoubtedly be biased in favor of the side that recruited them. No matter how helpful they are trying to be, they will often say things in attempt to please the lawyers who recruited them, or subconsciously tell them what they think they want to hear. Second, these people are not representative of the types of people who will end up on your jury. They will likely be more educated, more affluent, more conservative and more knowledgeable about the legal process than the typical jury pool. This is particularly problematic given the correlation between all of those characteristics and a defense leaning. Using employees of a law firm – or even their friends and relatives – is likely to provide more optimistic feedback about your case than is warranted.
Even if an attorney attempts to recruit jurors from “off the street” – via the phone book, newspaper or ads on craigslist – the mock jurors recruited are not likely to be representative of your jury pool. Often, these recruits are unemployed, and basing recruit quotas on the census data alone will not provide accurate representation. While census data will provide an overview of the population, it does not accurately describe the people who show up for jury duty. For instance, census data includes undocumented workers, people without drivers’ licenses or voter registrations, and people who just don’t show up for jury duty or rarely, if ever, are selected to serve. Jury consulting firms have the benefit of seeing dozens of jury pools in many jurisdictions, and in some instances, have records of the jurors who showed up for jury duty and were selected to serve. Jury consultants also have the experience to collaborate with local counsel in determining what jury pools in your trial venue actually look like, and we are familiar with the scientifically sound methodologies for contacting these people, screening them and obtaining their consent to participate in the research.
Facility, Equipment and Logistics
In addition to juror recruiting, litigation consulting firms also have the means to conduct the research optimally. This includes access to research facilities with two-way mirrors that allow attorneys and clients to unobtrusively observe the research presentations, focus groups or mock deliberations. We work with research facilities frequently and are often able to get the best rates for renting these rooms. In jurisdictions where none of these facilities are available, many consulting firms also have preferred audio-visual teams that have the equipment for filming the research and allowing observers to view and listen to simultaneous focus groups or deliberations in real time. Attorneys who rent this kind of equipment and attempt to record the research themselves are often disappointed in both of the cost of the rental equipment and the quality of the recordings. Most importantly, attorneys who attempt to conduct research at a law firm location are just asking for problems. Not only does this create a bias in how jurors will respond, it also creates a host of confidentiality concerns (now the jurors know who is sponsoring the research, how to contact the attorneys involved and, with a quick Google search, can determine who is actually representing your opponent and how to contact them).
Executing a research project may look simple to an outsider, but there is a great deal of logistical work behind the scenes that trial consulting firms are skilled to handle (e.g., managing the recruit, food catering, dealing with difficult mock jurors, special dietary needs, no-shows and late arrivals, confidentiality concerns – just to name a few). Attorneys who conduct their own research are often unprepared for the logistical problems that sometimes arise, and poor management of these problems can easily disrupt the research.
Analyzing the Results
Even if the research goes smoothly and yields a wealth of information, attorneys often don’t have the means or know-how to cull that information into meaningful results. For example, statistical software to analyze the data can run in the thousands of dollars for a single subscription, and few attorneys have been trained to use this kind of software. Most jury consultants have advanced training in research methods and statistics. When a case goes to trial, one of the most useful outcomes of jury research can be the development of juror distinguishers (also known as juror profiles). It can be very dangerous to base these profiles on results that are not statistically valid or reliable. Moreover, consultants are trained to avoid, minimize and compensate for various impediments to reliable research results, such as response bias, confirmation bias, order and commitment effects and anchoring. If an attorney cannot explain these concepts, chances are the research will be fraught with design defects that render the results practically meaningless – or worse – misleading.
Can attorneys conduct their own jury research to save costs? Sure, but for the reasons outlined above, the results can be more harmful than helpful to your case. While some people may argue that any feedback from mock jurors is better than no feedback, we respectfully disagree, as the danger of misleading results is too great. However, that doesn’t mean that those on a tight budget can’t obtain meaningful feedback from mock jurors.
There are other ways to save costs that won’t compromise the integrity of your research. For example, consider asking for an oral strategy session following the research as opposed to a written analysis, or limit the scope of the research to the most important issues. A smaller research project can elicit helpful feedback, so long as a consultant can explain the caveats and limitations of such research. Before deciding to launch your own research endeavor, we strongly suggest speaking to a qualified consultant to determine if they can offer assistance that meets your budgetary needs.
By: Christina Marinakis, J.D., Psy.D. – Director, Jury Research