Document Type

Article

Publication

Legal Reference Services Quarterly

Year

2013

DOI

10.1080/0270319X.2013.759036

Abstract

Humans and machines are both involved in the creation of legal research resources. For legal information retrieval systems, the human-curated finding aid is being overtaken by the computer algorithm. But human-curated finding aids still exist. One of them is the West Key Number system. The Key Number system’s headnote classification of case law, started back in the nineteenth century, was and is the creation of humans. The retrospective headnote classification of the cases in Lexis’s case databases, started in 1999, was created primarily - although not exclusively - with computer algorithms. So how do these two very different systems deal with a similar headnote from the same case, when they link the headnote to the digest and citator functions in their respective databases? This paper continues the author’s investigation into this question, looking at the relevance of results from digest and citator searches run on matching headnotes in Westlaw and Lexis for ninety important federal and state cases, to see how each system performs. For digests, where the results are curated where a human has made a judgment about the meaning of a case and placed it in a classification system humans still have an advantage. For citators, where algorithm is battling algorithm to find relevant results, it is a matter of the better algorithm winning. But neither algorithm is doing a very good job of finding all the relevant results; the overlap between the two citator systems is not that large. The lesson for researchers: know how your legal research system was created, what involvement, if any, humans had in the curation of the system, and what a researcher can and cannot expect from the system being used.

Comments

The author's preprint version is available at http://scholar.law.colorado.edu/articles/102/.

Analysis of the empirical data supporting the conclusions reached in this article is available at Jeffrey T. Luftig, Appendix A: Statistical Analysis of the Data, Susan Nevelow Mart Study of Search Functions in Lexis and Westlaw, (2012), http://scholar.law.colorado.edu/research-data/3.

Share

COinS