Southern District of New York Magistrate Judge Approves Use Of Predictive Coding
In what appears to be the first of its kind, Magistrate Judge Andrew J. Peck of the Southern District of New York, in the case Da Silva Moore v. Publicis Groupe, No. 11 Civ. 1279 (S.D.N.Y. Feb. 8, 2012), has approved a draft protocol for the use of “predictive coding” to assist in responding to demands for electronic discovery. Predictive coding is likely the next evolution of electronic discovery; however, it is not right for every case. It is critical that litigants and the counsel have a firm understanding of what it entails so that they can make an informed decision about whether to enjoy its advantages and accept it disadvantages.
WHAT IS PREDICTIVE CODING?
Traditional discovery protocols utilize linear document review, where human reviewers manually code documents for responsiveness or relevance to case issues. This review process is “linear” because attorneys review large sets of documents organized by date, custodian, keyword, or some other simple category. Linear review can be costly and time consuming because attorneys inevitably review many irrelevant documents.
For the right case, predictive coding, on the other hand, may greatly automate the document review process. Predictive coding typically begins with an experienced attorney reviewing relatively small sample sets of potentially responsive documents and coding those documents as “responsive,” “nonresponsive,” or “privileged.” These designations are then inputted into a sophisticated computer algorithm that extrapolates that categorization into the remaining universe of documents, coding them as “responsive,” “nonresponsive,” or “privileged.”
After the algorithm categorizes the document population, the attorney conducts secondary reviews of samples from each category to ensure that the algorithm is coding the documents properly. Based on those reviews, the attorney determines whether any adjustments need to be made in order to improve the accuracy of the coding algorithm. The attorney may go through several iterations of this process until satisfied with the accuracy of the results and, in some cases, opposing counsel participates in later iterations of subset reviews of relevant and non-relevant documents to further ensure accuracy of the coding algorithm. One of the main cost-savers lies in the fact that parties have accepted that, outside of the sample sets, an attorney will not be reviewing any of the documents the computer deems to be non-responsive.
WHAT ARE SOME OF THE ADVANTAGES AND DISADVANTAGES OF PREDICTIVE CODING?
Predictive coding, as a result of cases like Da Silva Moore and others, should spark litigants’ interest because it can be cost effective. For example, a single search term in Da Silva Moore returned over 165,000 documents from a universe of around 3 million documents. Traditional linear review of these documents would be costly. As part of the agreed protocol in that case, the parties agreed to initial sample sets of 2,399 documents (95 percent confidence level; plus or minus 2 percent variance). In effect, the parties will review roughly 15,000 to instruct the algorithm on how to code the 3 million documents.
Not only can predictive coding more efficient, it may also be effective. Some statistical studies have shown that predictive coding is more accurate in identifying responsive, non-responsive, and privileged documents than traditional linear review methods. In fact, Judge Peck stated his view that predictive coding is capable of identifying responsive documents “significantly better than” alternative methods.
Additionally, predictive coding can make substantive review of documents more efficient. For example, the algorithm can assign each document a specific numerical score on a scale from 100 to 0. With this scale, the algorithm identifies a document assigned the number 100 as most responsive (i.e. a “hot” document) and a document assigned the number 0 as least responsive. This scale allows attorney reviewers to analyze the most relevant documents first, leading to earlier and faster identification of key documents.
One of the most significant disadvantages to predictive coding is that the parties are relying upon a computer algorithm to determine what is and is not responsive. Therefore, it is possible that, depending upon how it is written, a so-called “smoking gun” document could be coded by the algorithm as “non-responsive” and never seen by the opposing party.
HOW DOES JUDGE PECK’S APPROVAL IMPACT PREDICTIVE CODING?
Although the benefits of predictive coding technology have been available for several years, litigants and courts have been slow to adopt it due to their skepticism that a mathematical algorithm can actually do as good or a better job of identifying responsive and privileged documents than attorney reviewers. Moreover, even litigants who have shown some interest in using predictive coding have nonetheless opted not to use it for fear that courts and opposing parties will be unwilling to accept the results of a complex predictive coding algorithm over a traditional linear review process that is much easier to understand.
Ultimately, predictive coding will likely be used more frequently in litigation once attorneys and courts become more familiar with the process and once judges accept its use in litigations. The Da Silva Moore action– the first where a court has not only accepted litigants’ decision to employ predictive coding technology but also has addressed specific issues as to how the technology should be employed – is an initial step in this process.
Although the Da Silva Moore case is unlikely to usher in an immediate wave of widespread acceptance of predictive coding, litigants need to be prepared to address its use. Parties who wish to use predictive coding can now cite this action to respond to concerns about predictive coding and bolster their argument that courts should allow use of the technology to identify relevant and privileged materials. Thus, with this case, litigants who are faced with producing or reviewing hundreds of thousands (or even millions) of potentially responsive electronic documents should consider predictive coding as a means to reduce discovery costs significantly.
WHEN TO USE PREDICTIVE CODING?
Predictive coding makes the most sense in cases where both sides possess and will be required to review voluminous document databases.
In cases where one side possesses the vast majority of documents (e.g., certain class action cases), only one party may want to use predictive coding. In that situation, the other party may have to convince the other side and the court that it is appropriate and may even attempt to argue for the shifting of some of the costs if the other side refuses.
In addition, a party who receives a voluminous document production may decide to use predictive coding to expedite their own internal review. In that situation, the party does not, of course, need the permission of the other side or the court, but does need to understand the advantages and disadvantages.
The following are some practical considerations:
- The volume of documents to be reviewed should warrant the use of predictive coding.
- There can be significant litigation support costs associated with predictive coding.
- The cost-savings of using predictive coding must outweigh the possibility (as limited as it may be) that some critical documents are never seen.
- The parties and the court must agree to predictive coding and the protocols to be used.
- The attorneys involved must understand how to manage the technology and associated costs to get the most from predictive coding.
- Given that the sample sets set the stage for the entire production, it is critical to use attorneys who are well versed in this type of document review so that the sample sets are accurately coded.