It seems that no one is exempt from the effects of today’s economy. From the biggest corporations to the smallest law firms, all the way down to the 12 members of the jury hearing a case, everyone is feeling the pinch of shrinking budgets and tightening wallets. However, this has not necessarily changed the number of cases headed to trial, nor has it changed the complexity of such cases. In fact, the current economic situation adds challenges to some lawsuits, increasing the ambiguity of jurors’ decision-making processes, thus comporting an even greater need for jury research and related services.
Recently there have been blog discussions and articles advocating jury research “on a dime.” While jury research projects can vary in scope and cost, the old adage tends to apply with lower-priced projects – you get what you pay for. There are empirically sound ways to conduct research on a budget, but there are also inappropriate ways of cutting corners that could drastically compromise the methodological reliability of the results. Improperly conducted research can damage outcomes, resulting in false positives and inappropriate generalizations, leading counsel to make critical strategic trial decisions based on unreliable results. In addition to ensuring proper research methods are followed, the right consultant can actually save you time, get you more “bang for your buck” and, most importantly, ensure proper methodology is followed.
In this issue of Insights, we focus on key methodological considerations one needs to follow in order to minimize the chance of a false positive and to increase the generalizability of your results while conducting research on a budget. We then turn to a review of the various jury research alternatives available for all clients and all budgets, while tailoring jury research to your goals.
Some examples of the risks of cutting corners that can adversely affect your results include:
- Cutting the wrong costs. Inevitably, all research has some costs involved and skimping on such costs is not always a good idea. For example, while it may seem excessive to pay jurors $200 for one day of work, reducing the juror honorarium may mean a higher likelihood of only having unemployed persons and homemakers participating in the study since employed persons are not likely to miss work unless they have the incentive to do so. This would leave you a skewed distribution of panelists not representative of the trial venire. A key methodological tenet, the more dissimilar the mock jurors are from being representative of the trial venue, the less one can rely on the research results.Skimping on audio visual equipment costs would mean you may not have videotape to refer to after the research, which can lead to reliance on several memories for the results. Relying on memory is not as reliable as memorializing deliberations on video such that nuances not originally heard could be later teased out. Conversely, having good audio visual equipment might mean less travel for your team to observe research.
- Juror recruitment. Tied closely with cutting the wrong costs, it is important to secure a well-principled participant recruiter. Not all participant recruiters play by the rules. Many try to save money by choosing inexpensive and easy ways to find mock jurors (e.g., ads on Craigslist, fax blasts, temp agencies, phonebook or existing focus group databases). By utilizing these methods, the recruiters are creating a skewed distribution for the sample of jurors. Instead, it is important to receive a cross section of your sample. For example, if jurors are recruited by a Craigslist ad or a wanted pages newspaper ad, the jurors who show up are only those folks who had access to read the ad rather than a cross section of those who read and don’t read the Internet/newspaper ad. Therefore, it is important to ensure the recruiter is using an appropriate method that will provide a representative sample of participants. A constant goal of jury research firms is partnering with good recruiting firms who can expeditiously and accurately recruit jurors to match a demographic sample. Jury research firms do enough research to weed out the recruiters who don’t play by the rules and create standing relationships with reliable recruiters who have experience in matching the sample jurors to what you would expect to see at trial. The recruitment of participants is both important to the validity of the research and also to the generalizability of study results, which becomes even more important when trying to infer and apply results of the research to your trial venue.
- Be wary of conducting research online. In conducting research via the internet, you lose the natural, face-to-face, personal communication that is always found in jury research and that you would have with your jury at trial. Research has shown that face-to-face impressions can change the entire dynamic of group deliberations. Another drawback to this type of research is losing control of who is actually participating at the computer; even with webcams, there is no way to really ensure that the person who signed up is the person sitting at the computer. While internet research can still be effective under careful circumstances and with proper methodologies, it will always be difficult for it to match the effectiveness of in-person research.
Many Alternatives to the Traditional Mock Trial
When the words “jury research” are mentioned, for some, it brings to mind high-dollar, multi-phase mock trials involving a myriad of issues. It is no secret that proper market research takes time, requires professional experience and can be costly. However, while unmatched in terms of the information gathered and its usefulness, a full-blown mock trial is certainly not for every case, or every budget. Outside of conducting a mock trial, there are many different ways to utilize trial consultants and jury research to yield valid, meaningful feedback to support any case.
Consider these tips next time you are planning to elicit help from a trial consultant:
- Inquire about a la carte services. There are many areas of jury research outside of mock trials and focus groups. If a few weak witnesses are hindering your case, you could benefit from witness preparation. Perhaps you want to find out what evidence, arguments, documents, etc., persuaded your trial jurors to vote the way they did – you could benefit from bringing in a consultant to conduct post-trial interviews (i.e., contact a number of impaneled jurors, discuss specific trial reactions with them and provide a report of the results). Or you might just need a brainstorming session with trial consultants who bring a variety of experience from working on a range of cases and can offer insights into an impending trial. Many litigation consulting firms offer specific a la carte services that can be tailored to your specific goals.
- Shorten the length of the study. As previously mentioned, it is not always necessary to conduct a full-blown mock trial. Perhaps a half-day, or even a two-hour, Issues Focus Group (i.e., testing a few key issues/themes that the trial team is particularly uncertain about) would garner significant feedback from a pool of potential jurors (who would need to be matched to the trial venire’s demographic specifications). If you have more than a few issues you need feedback on, sometimes conducting two back-to-back Focus Groups can produce the feedback you need while providing significant economies of scale (e.g., cutting down on facility, catering, juror time and consultant costs). Such research is designed to specifically target the issues about which the trial team feels most uncertain.
- Use what you’ve got. Does your law firm have a courtroom area in-house? A meeting room space? If so, you can use these resources in order to avoid the more costly hotel and facility rental fees. A caveat to this is that jurors may assume that if the project is being held at ABC’s law firm, ABC must be sponsoring the research. From a social science prospective, it is very important to minimize mock jurors’ knowledge of who is sponsoring the research, as it tends to create a response bias – i.e., when jurors believe they know who is sponsoring the research, the client then receives answers it “wants” to hear rather than receiving answers it “needs” to hear.
- Choose a flexible consultant, and be flexible with your consultant. If you are on a tight budget, simply let your consultant know; a good consultant will want to work with you and your budget. Tell them your goals and ask what your budget can provide you regarding feedback. It is easier to propose the right research program if all parties are on the same page, realizing the budget constraints of a project. Here are some good questions to ask consultants when obtaining bids for jury research:
- Does the bid include both consultant time (to design, implement and analyze the results) and project expenses? Some consultants price their work as “time plus expenses” and do not offer a number for the expenses, which can be a substantial cost. When comparing bids for time and project expenses, be sure you are comparing apples to apples.
- What methods are being used to recruit jurors (examples include: database, newspaper ad, Craigslist posting, fax blast, phonebook, or random digit dialing)? The latter method is the most expensive, but it also yields the most representative cross section of mock jurors, which is essential for obtaining the most reliable feedback.
- What are the methodological caveats to the design selected? If your goal is to learn the range of damages that jurors may or may not award, an inductive, Issues Focus Group is not the correct design and would come with several caveats if used for this purpose. To ensure you are receiving reliable, usable feedback, be sure to ask about the caveats to the design being used and how they will contribute to/detract from your research goals.
- What type of written report will you receive? Will you receive descriptive statistics shown via a series of graphics or a written report providing a synthesis of the research results? The former method is less expensive because a computer can generate the charts and graphs, whereas synthesizing the findings and recommendations into a written document takes considerable time for thoughtful analysis.
- Finally, when it comes to reporting, what type of report is right for you and the trial team? Sometimes the full-bound, written report is more than what is needed if your trial date is fast approaching. Perhaps an executive summary followed by a conference call would suffice. There are many economical options to receive the feedback you are looking for without the price tag of a full report.
Ultimately, ensuring good research and good methodology, at whatever cost is affordable, is worth it in the long run. According to author George V. Higgins, the goal of research is to “distinguish the dilettante from the artist.” However, to accomplish this goal, one does not have to break the bank! There are many cost-effective ways to incorporate sound research into any litigation. The trick is to build a goal-specific, cost-effective method to conduct jury research, and to find the right consultant, with a foundation in proper methodological principles, to help you achieve your research objectives without compromising the reliability of the results.
By: Jessica Baer, M.A., Consultant