Research Algorithms Have A Point of View: The Effect of Human Decision Making On Your Search Results, 1117 COBJ, Vol. 46, No. 10 Pg. 10
| Author | SUSAN NEVELOW MART, J. |
Legal Research Corner
SUSAN NEVELOW MART, J.
You might not think that the algorithm returning your search results from a keyword search in a legal database has its own opinions about the results you receive. After all, algorithms don't think, so how can they have opinions? However, algorithms are created by teams of humans. Those teams of humans made choices about how the algorithm would work that became the rules of the game long before you sat down at your computer. When the algorithm follows those rules, as it must, the rules govern what you see in the results. And every team of humans has implemented its own unique opinions about how to filter and sort the search results.
Let's say a lawyer is searching for cases in the Tenth Circuit that discuss the scope of discretion school boards have to remove, retain, or purchase books or other library materials. She opens a legal database, limits the search to the Tenth Circuit, and types in school board discretion remove retain library material. No matter what database she uses, she would expect to get relevant cases in the top 10 results. Regardless of the choices those teams of humans made when they set up the rules for the search algorithm, it seems a reasonable assumption that the same relevant cases will show up. That assumption is not correct.
The top 10 results in each database will differ wildly. In a recent study I conducted of 50 jurisdictional limited searches in Casetext, Fastcase, Google Scholar, Lexis Advance, Ravel, and Westlaw, each database returned about 40% unique cases.1 And only 7% of the cases showed up in five or six databases.2 The human factor in algorithm creation means that every algorithm is solving the same problem in a unique way. Each database has algorithms programmed to make unique choices about how to process search terms.
The Human-Computer Interaction: What Happens When Lawyers Search?
The age-old problem computer scientists have been trying to solve is how to make the connection between human input (the search terms lawyers use to reflect the concepts in their legal problems) and output (the documents that will be relevant to the legal problem you are trying to solve). This is a lot more complex than trying to match your need for a local pizza parlor with gluten-free options to the available restaurants in your area. Legal research involves complex cognitive concepts. The toolkit that engineers use to solve the problem for keyword searching includes such decisions as:
■ how to treat the number of terms in the search (i.e., if the search phrase has four terms and a document has three of them, will the document show up in the search results?);
■ how close the terms have to be to each other (proximity);
■ whether terms are stemmed;
■ whether legal phrases can be recognized without quotes;
■ whether additional concepts are added to the search;
■ how...
To continue reading
Request your trial