“The signature of counsel on a pleading certifies that a reasonable investigation of the facts and a normally competent level of legal research support the presentation.” Lieb v. Topstone Indus., 788 F.2d 151, 157 (3d Cir. 1986).
I. Introduction
Legal research skills are fundamental to the practice of law.[1] In the December 2004 Report on the American Bar Association’s Hearings on the Right to Counsel in Criminal Proceedings, titled Gideon’s Broken Promise: America’s Continuing Quest for Equal Justice, written by the American Bar Association Standing Committee on Legal Aid and Indigent Defendants, the Standing Committee identified the lack of “Investigation, Research, and Zealous Advocacy”[2] as one of the issues contributing to the inadequate legal representation of indigent defendants across our nation, which has resulted in “unequal justice for the poor.”[3] The Report provides many examples revealing the ability to, and the conducting of, adequate and competent legal research can have life and death consequences.[4] One tragic example, cited in the Report, was of a Georgia man being sentenced to death after a one-and-a-half-day capital trial where the defendant’s “defense lawyers did not make a single objection and filed three boilerplate motions of one page each.”[5] The use of the term “boilerplate” in the case cited by the Report telegraphs the court’s belief that the motion contained no new ideas and definitely no original or relevant legal research.[6]
However, the effects of deficiencies in legal research skills are not limited to only criminal cases; they can be found in all cases. A recent article from The National Law Journal titled, ‘Height of Recklessness’: Judge Sanctions Attorneys Behind Election Lawsuit Over ‘Woeful Lack of Investigation,’ discusses a 2020 election lawsuit case where attorneys were sanctioned by the judge for a lack of investigation and research.[7] The case the article discussed was O’Rouke v. Dominon Voting Systems.[8] The judge explained in his Order Granting Defendants’ Motions For Sanctions that the attorney had no rational answer for not conducting research originally, and then after “supposedly” conducting research, voluntary dismissing several named out-of-state plaintiffs.[9] When questioned about the later voluntary dismissal of some plaintiff’s, the attorney stated, “Well, when they filed the motions [to dismiss] we researched it, we looked into it, we could not feel with certainty that we could establish personal jurisdiction.”[10] The judge, however, remarked, “Not that any research should have been required to figure out what is obvious—a federal court in Colorado would not have personal jurisdiction over claims against other states’ governors and secretaries of state for actions taken in those other states with respect to the administration of those states’ elections.”[11] As is made clear by this case, the need for competent legal research starts even before filing a lawsuit.
Unfortunately, for nearly thirty years law firm partners, legal scholars, and governing bodies in the legal field, have acknowledged that there is a problem of deficient legal research skills for recent law school graduates and new associates.[12] In 1992, the MacCrate Report, indicated a decline in recent law school graduates’ and new attorneys’ legal research skills.[13] Since then, there have been numerous conferences, statements, and reports about legal research education.[14]
Sadly, although there has been an awareness of the problem, there has been no solution; the problem persists to the present day. Reports and surveys continue to show persistent deficiencies in the legal research skills of recent graduates and new attorneys.[15] Moreover, although legal research instruction has changed since 1992; overall, little has changed in how legal research education is prioritized by law schools and taught by legal research and writing (LRW) professors and/or professors of advanced legal research (ALR) courses.[16] Nothing has remediated the deficiencies in legal researching skills of recent law school graduates and new attorneys.
This article identifies why legal research skills of recent law school graduates and new associates have remained deficient to the present day and suggests one possible solution to remedy this problem. Part One highlights the importance of legal research skills to new attorneys. Part Two examines why this problem has persisted, despite an awareness of the problem. Part Three proposes one solution to this problem that would dramatically improve the legal research skills of recent graduates, at least for those graduates taking an LRW or ALR course using this method of research instruction. Additionally, Part Three suggests a method for testing the proposed solution to garner empirical evidence of whether the proposed solution provides an identifiable measurable increase in legal research skills of those students who have taken courses using this method.
II. Legal research skills are a fundamental lawyering skill and affect, not only the individual attorney’s case, but also, over time, the entire body of law.
Legal research skills are a fundamental skill in the practice of law.[17] Legal research skills are a combination of understanding legal doctrine and principles, applying that knowledge to a specific legal problem, and knowing where and how to find the answer in the law: cases, statutes, regulations, administrative decisions, etc. For example, think about a case involving a criminal defendant who was arrested by police for driving under the influence of alcohol where the defendant was sleeping behind the wheel of a running vehicle, but where the vehicle lacked a transmission and drive shaft, making the vehicle immovable. The attorney researching such an issue would need to understand criminal law doctrines, as well as the specifics of the law regarding actual physical control of a vehicle for his or her jurisdiction. Additionally, the attorney would need to know the resources to look for to find the statutes and case law applying the statutes. The attorney would need to know how to create search strings to garner relevant search results. Finally, the attorney, would need to have the ability to sort through search results, find cases, and select cases dealing with the legal issue the attorney was researching.
The fundamental nature of legal research skills is also confirmed by the American Bar Association,[18] and by feedback from practicing attorneys.[19] Practicing attorneys rank legal research skills higher than writing and drafting skills for new associates.[20] In a Lexis Nexis Hiring Partners survey, nearly ninety percent of attorneys considered legal research a “highly important” skill.[21] It is an essential skill according to the MacCrate Report.[22] Even anecdotally, students in Advanced Legal Research courses say they feel they would have been more successful in law school if they had been taught more legal research skills earlier, with many wishing they would have been taught more legal research skills in their first year of law school.
Legal research is often the most time-consuming part of preparing a case.[23] For new attorneys, a substantial portion of their work week is consumed in conducting legal research.[24] Legal research may require learning the general principles of the law on a particular legal topic, as well as, researching and finding case law and statutory law for a specific legal issue. Rules of procedure have been interpreted as requiring attorneys to have an ethical duty to be able to conduct competent and cost-effective legal research.[25] Clients expect that their attorneys have these skills as well, whether the attorney is newly licensed or a senior partner in the firm.[26] Big law firms have clients that say they do not want to pay researching costs for their case that amounts to paying for teaching new associates researching skills while working on their case.[27]
With legal research skills holding such an important place in the activities of an attorney, one would think that they would be a priority in legal education and those graduating from law school would be well prepared in this skill. Unfortunately, as Part Two of this article reveals, that has not been and still is not the case for many recent graduates of law school and new associates. Reports and studies show that although legal research is a fundamental skill, recent graduates are not learning it, at least not to a satisfactory level for law firms and those hiring new associates.[28]
III. Post-MacCrate there have been more reports, conferences, seminars, and activity in legal education and accreditation, but legal research deficiencies remain mostly for two reasons: (1) lack of law school prioritization and (2) lack of practice research opportunities.
It has been nearly thirty years since the MacCrate report was published in 1992, identifying deficiencies in the legal research skills of law students and new attorneys. At the time, the internet was in its infancy and had not garnered widespread use, nor had online legal research become the primary means of conducting legal research.[29] Legal research was conducted mostly in print resources.[30] Westlaw and LexisNexis were archaic versions of what they are today.[31] Westlaw originally used dial-up modems and lacked a personal computer interface.[32] Later, with the development of the internet, Westlaw built interfaces for its platform and added federated searching of its databases.[33] Today, Westlaw uses artificial intelligence (AI) to assist in searching.[34] LexisNexis went through similar changes as Westlaw over the same period.[35] Since the 1990s, the internet has become common and garnered global daily use.[36] Law students today have grown up as “digital natives” knowing of the internet from their first memories.[37] Westlaw, LexisNexis, and other legal research databases have exploded in content and capability.[38] We have been told by these legal research database companies and the developers of internet search engines that their products make researching easier.[39] The databases are even using AI to assist in researching.[40] Yet, to this day, deficiencies remain in the legal research skills of recent law school graduates and new attorneys.[41]
Fifteen years after the MacCrate Report, the Carnegie Foundation, in 2007, published Educating Lawyers: Preparation for the Profession of Law, also known as the Carnegie Report.[42] Like the MacCrate Report, it noted the deficiencies in legal research skills.[43] It also recommended legal educators use formative assessments as a primary means of assessment, rather than summative assessments.[44] Further, the Carnegie Report suggested an integrated curriculum in law schools that “moves back and forth between understanding and enactment, experience and analysis. . . . bridg[ing] the gap between analytical and practical knowledge.” [45] The Carnegie Report also suggested that “[l]egal education should seek to unite the two sides of legal knowledge: formal knowledge and experience of practice.”
One year later, in 2008, the ABA Legal Outcomes Report noted, in reviewing two previous reports (Carnegie Report 2007 and Best Practices for Legal Education: A Vision and a Road Map/ Roy Stuckey and Others (Clinical Legal Education Association 2007)), a “theme [emerged] of the necessity to infuse the ‘practical’ into essential lawyering attributes, such as practical wisdom, practical reasoning, and practical judgment.”[46] The ABA Legal Outcomes Report went on to say that the two reports it reviewed, “concur that legal education focuses disproportionately on developing the academic knowledge base (what Best Practices calls ‘content’) to the exclusion of developing necessary practical skills and professionalism.”[47]
The ABA Legal Outcomes Report explained that a conclusion of the Best Practices Report was that implementation of learning outcomes is often ad hoc and “law schools should demand that faculty be able to articulate clearly what each course demands.”[48] The ABA Legal Outcomes Report identifies seven principles for developing outcome assessments: faculty should formulate outcomes in collaboration with the bench and bar; outcomes should serve the law school’s mission; outcomes should be adopted only after consensus is reached; outcomes should be measurable; outcomes should be clear, straightforward; faculty should choose a reasonable amount of outcomes in terms of resources available; outcomes should be reasonable in light of the abilities of the students and faculty.[49]
These Reports—MacCrate, Carnegie, and ABA Legal Outcomes—identify deficiencies in legal research skills as a problem in legal education affecting recent graduates and new attorneys.[50] The reports also suggest changes in law school education and best practices that can be implemented.[51] However, the reports do not lay out the solution to the problem of deficient legal research skills, so other organizations and groups have been left to implement changes and develop frameworks for improving legal research education.
With an awareness of the MacCrate Report, the Carnegie Report, and ABA reports, organizations like AALL and the Boulder Conferences have worked to improve the legal research competencies of law students. These organizations and groups are at least partially comprised of legal research instructors and have a familiarity with the legal research needs and instruction of law students. They have worked to improve legal research skills through scholarship on pedagogy, creating resources for teaching legal research, and creating standards of research competency.[52]
The Boulder Conference on Legal Information: Scholarship and Teaching, have worked to build off the Carnegie Report’s suggestions.[53] In 2009, after meeting at the University of Colorado in Boulder, Colorado, the Conference attendees issued a statement, known as the Boulder Statement on Legal Research Education, that created the foundation of a signature pedagogy for legal research education.[54] In 2010, a second conference expanded the 2009 Boulder Statement.[55] In subsequent conferences, in 2011 and 2012, participants created a template for legal educators using the acronym COACh, providing a framework for legal educators to lesson plan and conduct research activities.[56] The COACh acronym stands for Context and Objectives; Activity; Checklists for teacher notes and reflections.[57] The Boulder Conferences continue adding scholarship and resources to the area of legal research instruction.[58]
The American Association of Law Libraries (AALL) has also worked to improve the legal research capabilities of law students and new attorneys.[59] C In 2013, the AALL created and published the AALL Principles and Standards for Legal Research Competency, inviting law schools and legal research educators to adopt the principles and standards in order to produce more competent legal researchers.[60] Based on the principles for legal research competency identified by AALL, the authors created standards for each principle and a list of competencies for each standard.[61] Additionally, the AALL has created the Body of Knowledge (BoK), published in 2018, which includes domains in research, analysis, teaching, and training that can be used by information professionals in helping educate students and new attorneys.[62]
In 2015, the Managing Director for the ABA Legal Education and Admissions to the Bar issued a Guidance Memo regarding new and amended Standards 301, 302, 314, and 315, which came from the Guidance Memorandum titled, “Transition to and Implementation of the New Standards and Rules of Procedure for Approval of Law Schools” dated August 13, 2015, with implementation beginning in the 2016-2017 academic year.[63] The 2015 version and the current version of these Standards have applicability to legal research instruction and skills.[64] Standard 302(b-d) requires:
A law school shall establish learning outcomes that shall, at a minimum, include competency in the following: … (b) Legal analysis and reasoning, legal research, problem-solving, and written and oral communication in the legal context; (c) Exercise of proper professional and ethical responsibilities to clients and the legal system; and (d) Other professional skills needed for competent and ethical participation as a member of the legal profession.[65]
Standard 314 requires, “A law school shall utilize both formative and summative assessment methods in its curriculum to measure and improve student learning and provide meaningful feedback to students.”[66] Additionally, Standard 304 is relevant to legal research instruction, as it covers experiential learning courses, and experiential courses are a sub-category is simulation courses.[67]
ABA Standards for legal education 302 states “[a] law school shall require that each student receive substantial instruction” and 302(a)(2) discusses lawyering skills which are “legal analysis and reasoning, legal research, problem solving, and oral communication”.[68] CUnfortunately, the current legal research instruction is woefully lacking in quantity and, sometimes, in quality as well. One reason legal research instruction is woefully lacking is that most law schools fail to provide enough legal research instruction during the first year, and beyond that many law students never receive any additional instruction in legal research throughout their law school education.
Despite an awareness by governing bodies, law schools, and law firm hiring partners of the deficiencies in the legal research skills of recent law school graduates as identified in the aforementioned reports, and despite the creation of statements, competencies, and standards, recent surveys suggest that there remains a problem with deficiencies in the legal research skills of recent graduates and new associates. There are primarily two reasons why deficiencies in the legal research skills of recent graduates and new associates have persisted since the MacCrate report was published almost thirty years ago. First, law schools have not prioritized legal research skills like they have other legal skills and abilities such as legal writing. Second, because the way legal research is often taught in legal writing courses, students do not get the practice necessary to become proficient in legal research skills in these courses, and many students never receive any legal research instruction beyond their first-year legal writing course while in law school.
Law schools have not prioritized legal research instruction as they have other topics such as legal writing and those doctrinal courses tested on the bar. Only a small minority of law schools make the legal research component a separate graded credit of the first year Legal Research and Writing course. Generally, nearly all of the legal research instruction law students receive comes as a portion of first-year law students’ Legal Research and Writing courses. This legal research instruction may be from legal-research-database (vendor) representatives, law librarians, or Legal Research and Writing instructors, and typically consists of one-off sessions on the features of a database, if done by the vendor representative, or one or two classes on finding primary and secondary sources, if done by law librarians. Even if the legal research instruction is done by the students’ Legal Research and Writing instructor, the amount of time spent on legal research is about the same as if done by a law librarian or vendor.
Some students—less than fifty percent of each graduating class—may get additional legal research instruction during their law school career if the students choose to take an upper-level elective course in legal research.[69] These upper-level-elective-research courses are generally capped at twenty-five students or less, many having caps below twenty students. Thus, only a limited number of graduating law students receive any additional legal research instruction beyond the few sessions they received in their first year of law school, especially if the law school only offers upper-level-elective-research courses once or twice a year. Recent ABA data shows the average 1L class size is slightly less than two hundred students.[70] If a law school had an incoming class of two hundred students, and only offered an Advanced Legal Research (ALR) course twice a year, only fifty percent of the class could ever take the ALR course. Even, if the ALR course was offered every semester, students would only have at most six opportunities to take the course, meaning only seventy-five percent of the students would have an opportunity to take an ALR course based on the same hypothetical class size. Moreover, the above examples do not take into consideration additional limiting factors such as, competition for seats from a second class (such as 3Ls for a 2L wanting to take the ALR course) or scheduling conflicts with other upper-level courses.
The lack of prioritization of legal research skills or even the availability of such courses by law schools is in stark contrast to the prioritization of legal writing skills where most law schools require two semesters in the students’ first year. Unfortunately, changing law schools’ desires to prioritize the teaching of legal research would require much work, including influencing governing bodies such as the ABA. It is not that law schools do not want to produce competent legal researchers; the problem most likely is finding credit hours for more required courses. Finding credit hours for required courses is difficult for most law schools because they generally want to provide as many electives as possible to prospective students. Additionally, law professors generally do not want to give up their preferred courses and credit hours to make way for a new required course. Because of these internal factors within law schools that are resistant to add required courses, any push for changes in legal research requirements must come from outside the schools. Changes to law schools’ Juris Doctor program requirements, required courses, and faculty teaching loads, will likely be timely and difficult, which explains why these changes have not been made, or why any change that has occurred has been slow and insufficient.
The second reason for the persistent deficiencies in legal research skills of recent graduates is the manner and frequency in which legal research is taught in law school research courses, whether that be a Legal Research and Writing (LRW) course in the first year or an upper-level Advanced Legal Research course. Legal research instruction, if required in the first year, as separate from the Legal Writing course, is often only a one credit course. In most law schools, legal research instruction is simply a part of the required first year Legal Research and Writing course. LRW courses typically focus primarily on the writing aspect of the course. Librarians, legal research database representatives, or the LRW professor may speak to the students about conducting legal research in an LRW course, but this “instruction” usually amounts to the equivalent of about three hours or less of training or two entire class periods for a three credit LRW course—less than ten percent of in-class instruction time.
Most law schools’ LRW programs combine the research and writing into one course, and thus, the legal research component is not given separate credit or a separate grade as part of the LRW course.[71] Law Librarians who teach legal research know that this creates an unspoken attitude in students that the legal research is less important because the grade is primarily on the writing portion and that is the main focus of the instruction as well.[72] As previously mentioned, in some cases less than ten percent of class time in the entire LRW course is spent on learning how to conduct legal research.[73] Additionally, because LRW students are generally only faced with one to three research scenarios per semester (two to six research scenarios for two semesters of LRW), students may go weeks or months between conducting any legal research at all.[74] The lack of legal research instruction and the lack of practice using a newly acquired skill are deleterious to learning and retention of any research skills that students may have learned.[75]
The use of combined writing and research courses (LRW courses) to teach legal research, as well as, only law students in those courses being assigned a few research problems, poses two problems: (1) students assume, based on the fact that the majority of the class time and their final grade is focused on writing rather than research, that research is not as important as writing, and (2) because there are only a few research scenarios for each semester, students do not get enough practice to develop their research skills. Furthermore, as Caroline Osborne observes, many “research assignments” are actually assessed in a way that is “consistent with that of a writing class,” such as compiling research logs and drafting research plans and reports.[76] Therefore, some students are actually not even being taught the type of legal research skills that would be required of them for researching a legal issue in a real-world setting.
LRW courses generally have one or two major writing and research projects each semester. Thus, students may get two to four research opportunities in a year of LRW. In the first semester of LRW courses, students typically have one or two major writing projects, but only one of them—an open memo in the second part of the course—requires students to conduct their own research for legal authority. The students’ first writing project is generally a closed memo, where students are provided the legal authority and no actual legal research is required. In the second semester LRW course, students are often assigned a persuasive memo and appellate brief.
An example of a recent appellate brief assignment in a second semester LRW course involved an appeal to the United States Court of Appeals for the Seventh Circuit regarding an inmate’s claim that a Correctional Officer violated the inmate’s Eighth Amendment rights. The inmate argued that the Correctional Officer was aware of underlying facts from which the inference could be drawn that the inmate was at risk of retaliation from his former prison gang, that the Correctional Officer actually drew that inference, and was deliberately indifferent to the risk of harm to the inmate. Another example, of a second semester LRW problem dealt with whether non-consumers could claim damages under the Florida Deceptive and Unfair Trade Practices Act (FDUTPA). These are the types of assignments students must research in the second semester of LRW; they are appellate level cases, requiring students to write persuasive memos or appellate briefs for these assignments.
The problem with the persuasive memo and appellate brief assignments is twofold. First, these assignments generally require significant writing, and the focus of the research is only on a couple of issues. The writing tends to be the major portion of the grade and comprises the majority of the LRW professor’s instruction during class time. Second, most recent law school graduates will not enter the workforce doing appeals or researching for them. The majority of recent law school graduates will enter smaller law firms.[77] As such, recent law school graduates in smaller law firms typically encounter a wide variety of legal issues. Additionally, most recent graduates or new associates will start out on smaller cases and legal issues, demonstrating that practice research hypotheticals designed for those types of quick research issues are better than long drawn-out hypotheticals that require researching an issue in detail across multiple jurisdictions to find persuasive arguments.
The infrequent legal research combined with a focus on legal writing in LRW courses results in first-year law students entering their second year woefully unprepared and incompetent to conduct legal research. Students in their second year of law school that do choose to take an ALR course, often enter the course having very little research skills, knowledge of resources, or even a basic ability to identify the main sources of primary law or essential secondary resources. Thus, ALR courses often must start by re-teaching basic legal research skills and laying a foundation to then move forward with building more advanced legal research skills.
IV. Providing students a research process and numerous research opportunities using the Practice and Process method is the most effective way to improve legal research competency.
Though the problem of deficient legal research has been identified—repeatedly—and suggestions have been made for improvement, the problem has persisted because significant changes in the classroom have not been made. The previously discussed reports, conferences, and organizational bodies, identified the problem, created standards and competencies, and added scholarship and teaching resources, as well as frameworks to assess legal research skills, but the missing component has been the application within the classroom in a way that causes those skills to be learned. The way to solve this problem is the Process and Practice Method discussed in this section.
The lack of law school prioritization of legal research skills, discussed earlier, and the lack of law students’ legal research opportunities, both must be addressed to cure the research deficiencies of recent graduates and new associates. The focus of this article, however, is on fixing the lack of practice research opportunities. Even this solution, though, has its limitations because the suggested changes can only be implemented fully in an ALR course. Thus, only a small portion of the law students (those able to enroll in an ALR course while in law school), will receive the benefits of Process and Practice Method. Nevertheless, implementing the Process and Practice Method will dramatically improve the legal research skills and outcomes for those students who do have the opportunity to take an ALR course taught in the Process and Practice Method. Additionally, aspects of the Process and Practice Method can be modified to be included in both LRW and doctrinal courses, which would improve all law students’ research skills using the Process and Practice Method.
Learning legal research differs from learning doctrine. Primarily, all doctrinal courses require that students learn specific substantive rules and their application to relevant facts. In contrast, legal research differs in both structure and application because nearly every legal research task differs depending upon the substantive law, the research tools available and the sources to be researched. In short, learning legal research requires more: an understanding of the substantive law and its exceptions; acquiring the technical skills necessary to navigate and properly use research databases or print research resources; the ability to pull key words (based on the law, the specific facts, and legal issues) out of specific fact patterns; and using those key words to build quality search strings. Legal research also involves learning how to locate the relevant rules of law in areas where the student lacks familiarity with the substantive law—an ability to make oneself knowledgeable in an unfamiliar area of law. The ability to conduct legal research requires a mixture of both legal knowledge and research skills, making legal research challenging for law students. It is not passive learning; students cannot simply memorize the law and apply it to the fact pattern.[78] They must know the law, apply it to the fact pattern, know where and how to find the law, know how to build search strings that will return relevant results, and finally, complete the daunting task of sifting through many search results to locate analogous case law or other information that meets the narrow research needed, considering the information they have and making the appropriate conclusions.[79]
Traditionally, Advanced Legal Research (ALR) courses have been taught through dry lectures describing the various databases available, combined with two to five graded research projects and one larger semester long research project such as a bibliography and/or a presentation. However, this traditional approach, like LRW courses, does not give the students enough practice at researching legal issues, nor does it adequately teach students the type of legal research they would likely conduct in a law firm setting. Additionally, the traditional method fails to provide students with a process or framework they could use for conducting any legal research they may encounter and fails to provide them with enough practice to become comfortable and confident in using that process, especially in ever changing legal research databases.
Adjustments to the traditional ALR teaching approach can make the law school researching more closely resemble the assignments that law students and recent graduates encounter in a law firm setting. This newly conceptualized approach, named the Process and Practice Method, eliminates the antiquated and unnecessary bibliography assignment, and reorganizes the ALR syllabus.[80] The Process and Practice Method teaches students a research process applicable to any research task and provides ample practice to learn the process as well as how to create search strings, use research databases and other resources, and analyze search results. Part of this change from the traditional approach requires adding new topics to the ALR course. These new course topics (1) focus on understanding the research process and providing a seven-step process to use for any legal research project; (2) teach the manner in which computers retrieve data and how to build advanced search strings; and (3) teach how to spot key terms in fact patterns and client interviews so they can be used in building relevant search strings.[81]
The Process and Practice Method is a dynamic approach that requires on-going re-evaluation.[82] Under this method, ALR professors critically assess teaching goals and assignments with the expectation that research classes will change every semester. ALR professors should note which skills the students did not master and create more effective lessons in those areas. Examples of changes from semester to semester include originally requiring students to provide only the answer and the time required to conduct the research for a graded research assignment, to requiring the students to provide the jurisdiction, the clearly stated legal issue, the type of information being requested, the resource where they would find that information, their search trail, as well as, the time needed to conduct the research, and then the answer. More changes include, adding the requirement of having students provide as least five relevant terms or key words from the hypothetical and providing at least three advanced search strings using a minimum of one Boolean term or connector and at least one other advanced search technique, along with all the previous requirements mentioned above. Other significant changes made from assessing previous courses include adding an additional class topic on helping students identify key terms by teaching the acronym TRAPP[83][84] and helping students understand how attorneys’ informational needs change depending on the type of attorney they are (transactional or litigation) and depending on where the attorney is in the process of a case or in analyzing a legal problem.
With the Process and Practice Method, students learn skills applicable to any legal research need, while also receiving abundant and continuous opportunities to practice the process of conducting legal research, enabling them to become competent legal researchers. This teaching method focuses on providing as many practice opportunities for research throughout the course as possible. Without practice the students will not learn the research process, nor gain the necessary skills in using the research databases or print resources to be competent with those databases or print resources. As such, lectures in the revamped ALR course are relatively short, and most of class time is spent going through assigned in-class-practice-research problems with the students. Each class consists of a brief lecture lasting between fifteen and forty-five minutes for a ninety-five-minute class, followed by between five and ten ungraded in-class practice-research problems. The in-class-practice-research problems are hypotheticals that are similar to what the students might be asked to conduct in a law firm setting. Many of the hypotheticals were developed from actual, real-life cases. Students are required to conduct their research in both Westlaw and LexisNexis. As a result, students end up conducting research between ten and twenty times for each class period. Additionally, there is a graded research assignment for each class based on a hypothetical like the ones used for the in-class-practice-research problems. Over the course of a semester, with the in-class-practice-research problems and the graded research assignments, students end up conducting legal research between three and four hundred times, with over twenty of those being on graded research assignments—nearly a hundred times as many research opportunities students get in two semesters of LRW.
It is common knowledge among both LRW professors, law librarians, and legal research professors that many students enter law school with an overconfidence in their ability to conduct effective research. Most students entering law school have never conducted legal research, and therefore, do not understand the complexities of legal research, nor do they understand nuanced legal terms or how to use the intricate resources often found in the legal field. Fortunately, by the time most students enter the ALR course, they have come to the realization that their legal research skills are lacking. Students entering ALR will often admit that they realize researching is important, that they are not proficient at researching, and that they lack confidence in their legal research abilities. The following is a typical response from a student answering a pre-assessment question given to students entering the ALR course on why they took the ALR course, “Legal research is clearly very important to the practice of law, and I want to learn more in order to feel more confident doing it. I remember very little from LRW.” The students want to learn how to research, but their confidence is often shaken by their first-year appellate brief experience and the scarce training and opportunity to practice researching during the first-year LRW courses. Students know they are not getting enough legal research training in law school, and they want the additional training because they know law firms, and attorneys, they will work for expect it.[85]
Students are often intimidated when they view the syllabus for the ALR course using the Process and Practice Method because they see so many graded research assignments and see that it comprises most of their grade, about seventy percent, for the course. Some students have even admitted that they almost withdrew from the ALR course once they saw how much research it entailed. However, after completing the ALR course, students always heap praise on the course and share how thankful they are that they took the course. Here is an example of such praise for the course made by a student in their final reflective paper for the ALR course:
The seven-step process not only helps me to narrow down the issue and identify the type of resource I need, but also helps me to craft appropriate search terms in order to quickly find what I am looking for. Practicing this process has greatly enhanced my research skills and has cut back on the stress and time that I previously experienced when researching.[86]
Comments like this one and many others from students taking the ALR course indicate anecdotally that the ALR course has improved their legal research skills.
The Process and Practice Method employs many of the standards, best practices, and competencies discussed earlier in this article. Using the Process and Practice Method aligns with the American Bar Association’s best practices principles discussed in both the ABA Legal Outcomes Report and the Best Practices Report.[87] The Process and Practice Method uses the research competencies developed by the AALL.[88] The ALR course taught using the Process and Practice Method uses both formative and summative assessments as suggested by the ABA.[89] Additionally, student learning outcomes are measurable, clear, straightforward, and are included in the ALR course syllabus. The outcomes are also reasonable in number and in light of the students’ abilities.
In the six tasks the ABA Legal Outcomes Report identifies as preparing professionals, the ALR course incorporates at least three, and arguably four, of the tasks: developing an academic knowledge base; providing students with the capacity to engage in complex practice; enabling students to learn to make judgments; and teaching students how to learn from experience.[90] The ALR course requires students to learn how to use various databases and print resources for conducting legal research, fulfilling the requirement to develop the academic knowledge base. The ALR course forces students to engage in complex practice by researching practice hypotheticals based on actual cases or case scenarios like real cases. Students must then make judgments based on the information the students garner from their research. Finally, the ALR course enables students to learn from experience by getting immediate feedback from the professor during class time while conducting research on practice hypotheticals.
The development of “practical skills” discussed in the ABA Legal Outcomes Report is the focus of the ALR course using the Process and Practice Method, by inundating the students with research opportunities using real world, or as close as possible to real world, hypotheticals. Students taking the ALR course are tasked with researching numerous items, in-class, based on real world hypotheticals, providing the opportunity to see real world legal research problems; work under a time constraint; and get immediate feedback on their search strategies. The students’ work in researching real world hypotheticals is a both practical and essential skill, considering all lawyers must conduct research to support their work.
V. The Process and Practice Method applied.
The ‘process’ portion of the Process and Practice Method is taught starting in the very first class of the ALR course and repeated in every subsequent class. Students also learn the seven-step process by using it to complete their practice research hypotheticals, their graded research assignments, and in presenting their practice research hypotheticals to the class during each class session. The process consists of seven steps identified as follows: 1) Identify the controlling jurisdiction; 2) Clearly articulate the legal issue needing to be researched; 3) Identify the type of information being requested; 4) Identify the resource containing the information and understand how to use the resource; 5) Know how much time you have to conduct the research; 6) Use the above information to create a written research plan; and 7) Keep track of your research in writing. The seven-step process is not something that this author has created wholly by himself. Many legal research instructors and scholars have written about steps in conducting legal research and have organized those steps in various ways and included many or most of the seven steps in their own teaching of legal research. Additionally, many articles and books have discussed using the various combinations of the seven steps identified in this process. What this author believes is the ‘secret sauce’ that makes the process really accomplish the goal for making students competent legal researchers is combining the process with abundant and continuous research practice.
Before getting into a deeper discussion of practice, let us examine the seven-step process more closely and see why these steps are important. Again, these seven steps are a part of every class throughout the course and are included in every class PowerPoint and lecture. Additionally, the students explain these steps as they go through the in-class research practice. Moreover, for each classes’ graded research assignment, the students must provide a response for each step in the process because the graded assignment questions are formulated to cover the steps in the process. As a result, by the end of an ALR course, students have heard, seen, and responded to the process hundreds of times. This abundance of practice over the course of a semester enables the students to learn the process and gives the students a sense of confidence and peace of mind that they are not missing something, as they conduct legal research. The process is like a checklist that students can go back to for reassurance. On an ALR student’s final reflective paper, the student commented, “Using the seven steps for starting my research is a great tool that helps me get myself grounded before I get lost in a sea of information.” This type of response from the students regarding the seven-step process is common.
Step one of the seven-step process is to identify the controlling jurisdiction. This seems easy enough, but oftentimes students do not really learn enough about jurisdiction in LRW. Often, students enter the ALR course with only a general understanding of jurisdiction and have very little practice in determining the jurisdiction for researching an issue on their own. In LRW, students are typically told the jurisdiction for their assignments, but in the ALR course this author does not tell students the jurisdiction. Each research hypothetical contains the necessary facts to identify the proper jurisdiction, and students must be able to identify the jurisdiction on their own. Often, students are unable to do this early in the ALR course. For example, when students first enter the ALR course and are presented with a research hypothetical about a civil case involving damages of $500,000.00 in Orlando, Florida, where the student is asked to file a complaint, the students often respond to the question of what the proper jurisdiction is by simply saying Florida, instead of the 9th Judicial Circuit Court in and for Orange County, Florida. However, by completing the many in-class research hypotheticals and graded research assignments, ALR students learn how to determine the proper court for conducting legal research and filing documents with the court.
Step two in the seven-step process is to clearly articulate the legal issue needing to be researched. The reason this step is so important is two-fold. First, if students do not know what they are looking for they almost certainly will not find it. If students cannot clearly identify what the legal issue needing to be researched is, they will not be able to find the needed information. Second, the clearly stated legal issue contains the descriptive words that are used for building search strings students will use to conduct research in the legal databases or print resources. However, being able to clearly articulate the legal issue needing to be researched is often confusing to students starting in the ALR course because the students do not realize that the legal issue of the case is not necessarily the legal issue needing to be researched. For example, a research hypothetical might say Jim was driving for Orange State Trucking and, while talking on his cell phone, Jim rear ended the car of our client, Trisha. Trisha suffered a broken back, neck injuries, and a traumatic brain injury from the accident. Trisha has nearly $1,000,000.00 in medical bills. The student represents Trisha who filed a lawsuit against Jim and the trucking company claiming negligence and seeking damages for the injuries. The case is currently in the discovery stage. The hypothetical asks the student to find the rule and two cases supporting a motion to compel because the trucking company has refused to respond to requests for production of the vehicle records. Often, when students are asked what the legal issue is for this hypothetical, they will say whether Jim is liable for the damages resulting from injuries to Trisha due to Jim’s negligence in rear ending Trisha’s car, but the legal issue needing to be researched is whether the trucking company can be compelled to respond to a request to produce the vehicle records. Although the difference between the legal issue of the case and the legal issue needing to be researched may be clear to more experienced legal researchers, many students entering the ALR course have difficulty discerning the difference.
Step three of the seven-step process is to identify the type of information needing to be researched. Similar to the struggles students have with steps one and two, students entering the ALR course have difficulty with step three as well. Because students in LRW research mostly case law, the first thought of many students entering ALR is to immediately want to search for case law. Students really do not consider that legal research includes a plethora of things besides finding cases such as, finding a checklist for handling a divorce; finding a form for a subpoena duces tecum; locating the address of the registered agent for a business; finding jury instructions; finding legislative history; or finding out historical information on an expert witness. Step three, though, helps students break their tendency for immediate case research and really think about what information needs to be researched. By doing step three and knowing what type of information is needed, students are able to properly focus their search. Step three also helps students with step four of the process.
Step four requires students to identify the resource containing the information needing to be researched, and it requires students to understand how to use the resource. The reason for this step is straightforward and obvious—we cannot find the information we are looking for if we do not know what resource it is located within. Legal information in print form is often in multi-volume sets. Legal research databases such as Westlaw and LexisNexis contain tens of thousands of individual databases and resources (practice guides, journals, newspapers, legal encyclopedias) within the overall research platform. Besides the obvious reason for step four, identifying the resource containing the information needing researched is important because it enables students to limit their search to only the resource containing the information needing researched. The benefit of students limiting their search to only the resource containing the information needing researched is that it also limits the number of results students must sort through, saving students invaluable time. The ALR course focuses on training the students to go to the specific resource or database containing the information needed. Thus, instead of searching in Westlaw’s or LexisNexis’s main search bar to find a Florida Statute on dog bites, students are instructed to go directly for the Florida materials and select the Florida Statutes to run their search in only the Florida Statutes to find statutes on dog bites.
Most students enter the ALR course having done almost no searching in legal research databases using advanced search techniques or limiting their searches to specific resources within those databases. Most first-year law students tend to search legal research databases such as Westlaw or LexisNexis by entering natural language searches into the main search bar. After running a search by entering natural language searches into the main search bar, a student must often search through tens of thousands of results, many of the results are from resources that are irrelevant to the resource containing the information the student needs. This is because the student did not identify the specific resource for containing the information they need and limit their search only to that resource. Entering ALR students must be taught that such a search is like going to the law library to find a specific Florida Statute on liability for personal injuries resulting from a dog bite and being told that the Florida Statutes are on shelf one on the first floor of the library, but the student chooses to go to the catalog and search for liability for a dog bite. The student’s search in the catalog would return thousands of irrelevant results (irrelevant in that the student is looking specifically for a statute on dog bites, not other information on dog bites) from treatises on tort law, treatises on animal law, personal injury practice guides, etc. The Florida Statutes consist of six volumes, but the law library contains hundreds of thousands of volumes and key words like dog, bite, and liability will show up in thousands, maybe even tens of thousands of books, newspapers, and journals, within the library. This scenario helps step four become clear to students. The students realize that searching through tens of thousands of results from irrelevant resources is silly when they know exactly what resource contains the information they need.
Step five of the seven step process is to determine the amount of time available to conduct the research. This step is important for students to think about for two reasons. First, legal work is often done using hourly billing. Thus, the time spent researching will have to be calculated to properly bill the client. Second, the amount of time available to conduct research determines the length and, to an extent, the depth of research that can be done. If an attorney needs a case to support a motion for a continuation that must be filed by four o’clock and it is currently two o’clock, the attorney cannot spend hours reading practice guides, legal encyclopedias, or American Law Reports about a motion for continuation, they need to go to their specific jurisdiction and find a case supporting their motion. However, if the same attorney is drafting an appeal the attorney may have one or two months to conduct research, allowing the attorney to go much more in-depth with their research.
Step six is to create a written research plan using the information from steps one through five. The research plan is simply a strategy for tackling a specific research goal. Using the clearly stated legal issue in step two, students can pull key words to build search strings. Then using steps one, three, and four, students can determine where specifically to search in legal research databases or print resources. Step five lets the student know how much time they have to conduct the research. In the ALR course, students are taught to create a plan that works for them. The research plan can be as simple as saying; I need to find Florida cases on dog bites where provocation of the dog was discussed. I will search in Westlaw under Florida Appellate and Supreme Court cases using the search terms dog, bite, and provocation.
Finally, step seven is to keep track of the legal research conducted in writing. The requirement of tracking one’s research in writing means noting the specific databases or print resources searched and the key words and search strings used in those searches as well as any information regarding the results that the student deems important. The benefits of step seven are two-fold. First, by keeping track of the research conducted in writing, students can see what they have already searched and avoid rerunning the same searches over and over and thus wasting time. Second, it gives students (or new attorneys) information to provide to their supervisors if they must turn the research over to someone else to complete, enabling the new researcher to know what searches were run and in what databases or resources.
As students learn this seven-step process, it becomes a framework for them. The students gain confidence that they are not missing something when they conduct research. Students also see better search results by knowing what information they are looking for and in what resource they need to look to find the information. These seven steps were incorporated into, and formed the framework for, the ALR course’s graded research assignments. Initially the graded research assignments only incorporated the seven~~-~~steps, but the graded research assignments have evolved to incorporate two additional requirements and two additional class topics were added to the ALR course to remedy student difficulties with identifying relevant search terms from research hypotheticals and creating good search strings. Thus, the graded research assignment in its current form requires students to answer nine questions, with a few exceptions based on the subject matter being researched and how the assignment is presented.
The first three questions of the graded research assignment parallel the first three steps of the seven-step process. Students are required to provide the jurisdiction, clearly articulate the legal issue needing researched, and state what type of information is being requested. Questions four and five of the graded research assignment are two questions that were added later due to student difficulties in finding relevant search terms and creating good search strings. Question six of the graded research assignment asks students to identify the resource containing the information needed and understand how to use the resource, and question six corresponds to step four of the seven-step process. Question seven of the graded research assignment corresponds to step seven of the seven-step process and requires students to provide the research trail the student used in conducting the research for the assignment. Question eight on the graded research assignment asks the students to provide the actual amount of time it took to complete the research assignment, making it similar to step five of the seven-step process. The last question of the graded research assignment, question nine, asks students to provide the answer that was requested in the research hypothetical, which might be a case, statute, rule, jury instruction, legislative history, form, or even something else. The combination of steps one through six on the graded research assignment are the essential elements of a basic research plan, and thus, fulfill step six of the seven-step process.
Below is an example of a graded research assignment for a class on federal regulatory law research showing the nine questions for each assignment and how the seven-step process is incorporated into the graded research assignment.
From: Supervising Attorney
To: New Associate
Our Client, Farm America Today Television (FATT), is a local non-commercial educational television station located in Ocala, Florida, dedicated to providing local non-commercial educational programming relating to farming and the environment, operating locally as channel 56. FATT Channel 56 is owned by The Farm America Organization, an IRS approved non-profit organization dedicated to building a better environment and better farming practices in Florida. FATT has come to our office for help because Spectrum Cable said they will not carry FATT in their channel lineup offered to cable subscribers. Spectrum Cable is a licensed provider of cable service to the central Florida and Ocala areas. The president of FATT Channel 56 says that he heard from a local PBS station manager and friend that Spectrum, as a cable system operator, is obligated by law to carry FATT in their channel lineup that includes over 40 basic cable channels and more than 300 channels for their premium programming because FATT is a non-profit educational channel. When FATT’s president spoke with Spectrum, they said they already carry two local PBS stations in their channel lineup and don’t have to carry any additional channels. Your supervisor really needs your help. She asked you to look up the law on this issue. She wants to know (1) if any regulation requires Spectrum to carry FATT and (2) if there is a regulation identifying what type of television station FATT is. Provide her with the two requested regulations if they exist AND what (3) federal agency the regulations fall under.
DUE DATE: This assignment is due at the start of class on Tuesday, June 23rd.
This assignment is worth ten points.
1. Jurisdiction:
2. Clearly state the legal issue(s) needing researched:
3. What type of information am I being asked to find (e.g., a case, a hearing, a statute, a regulation, a form, legislative history, property information):
4. Identify at least five relevant search terms for this hypothetical issue:
1-_________ 2-___________ 3- ____________ 4-_____________ 5-_____________
5. Create at least three different advanced search strings for this hypothetical, using a minimum of one Boolean term or connector and at least one other advanced search technique (e.g., proximity locator, root expander, universal character, quotations, truncation):
1-___________________ 2-___________________ 3-_____________________
6. In what specific resource (not simply Westlaw or LexisNexis) would I find this information (e.g., Code of Federal Regulations, Florida Case Law, Florida Jurisprudence 2nd, United States Code, County Property Appraisers’ Website, Florida Senate and Law of Florida websites):
7. Provide your search trail (e.g., Westlaw > Florida Materials > Secondary Sources > Florida Jurisprudence 2nd > search // “premises liability” & negligen! & stairway //):
8. Provide the (ACTUAL) amount of time it took you to complete this assignment:
9. (2 points) Provide the necessary information to answer and/or fulfill to your supervisor’s question or request in the assignment (e.g., a case on point, the relevant statute or regulation, the proper form, the proper encyclopedia entry or journal article), and provide any other information if it is requested (e.g., your opinion–yay or nay, or filing dates):
As discussed above, questions four and five were not part of the initial graded research assignment but were added later. Question four requires students to provide five relevant search terms based on the hypothetical provided. This question was added because many students entering the ALR course have difficulty selecting good search terms. For example, one of the first research hypotheticals students must complete is about a person getting an oil change and transmission flush at a Jiffy Lube. The customer getting the oil change can watch the work being done through the window and sees that the mechanic does not remove certain hoses and does not complete the transmission flush on his vehicle. In the hypothetical, the customer is the student’s client and wants to sue Jiffy Lube. The research hypothetical contains every word needed to research the issue and provide the answer. However, students entering ALR often will select ineffective and generic search terms such as, client, tort, hoses, work, and sue.
To remedy the problem of students selecting ineffective and/or generic search terms, two things were added to the ALR course. A new topic was added to class two of the ALR course dedicated to teaching the acronym TRAPP.[91] The added section of class covering TRAPP helps students understand and develop better search terms and key words to build search strings. TRAPP stands for Things, Remedies and Relief, causes of Action and Defenses, People or Parties involved, and Places involved. In the ALR class, students are instructed to do a TRAPP analysis on every research hypothetical they are provided. Students are asked to write down the relevant words under each letter in TRAPP and, as needed, to find synonyms for these words. For instance, under “Thing” a student might put the word dog from a research hypothetical and then a synonym such as K-9, or a specific breed, if necessary, like German Shepard. Since adding the section teaching the TRAPP acronym, students have done much better at selecting relevant and effective search terms or key words from hypotheticals. Additionally, the use of TRAPP along with step two of the seven-step process have improved students’ abilities to spot legal issues, which benefits them in all courses and in taking the bar. To reinforce the concepts taught in the TRAPP class and help the students develop better skills in selecting relevant and effective search terms, question four was added to the graded research assignment requiring students to provide at least five relevant search terms from the hypothetical.
Question five of the graded research assignment was added to remedy two problems. First, students entering ALR often searched only using natural language searches with little to no familiarity with or ability to conduct advanced searches. Second, students entering ALR, generally do not know how to build advanced search strings. To fix these two problems, a class topic was added to the ALR course focusing on teaching advanced search techniques, including building search strings. Unfortunately, even after adding the class section on advanced search techniques (including how to expand and narrow searches using advanced terms and connectors) and building advanced search strings, students were not using advanced search strings to conduct their research. Thus, to ensure students learned and applied the concepts in the advanced search techniques class and to reinforce those concepts, question five was added to the graded research assignment, requiring students to provide three search strings based on the research hypothetical with each search string containing at least one Boolean term and connector and at least one other advanced search technique such as proximity locators, quotes, truncation, or wildcards. This addition to the graded research assignment dramatically improved the students’ abilities to create advanced search strings and the use of advanced search strings in their research.
Like the improvements in students’ ability to select relevant search terms after the TRAPP class, students make noticeable improvements in their ability to create effective advanced search strings after the advanced search techniques class. For example, one research hypothetical asks students whether a golfer who hit an errant golf shot onto another fairway that struck and injured a person is liable for the injuries when the golfer also failed to yell the warning, “fore” to the other person. Search strings prior to the advanced search techniques class typically looked like [golf shot negligent golfer liable gave no warning], but after the advanced search techniques class the search generally looked like [Negligen! AND golf AND (hit OR struck) AND warn!]. The students continue to improve their skills in building advanced search strings throughout the semester as they complete the graded research assignments.
In addition to teaching the seven-step process in each class by including it in each PowerPoint presentation, lecture, and each graded research assignment, students must go through the seven-step process during class while completing their in-class research. Prior to each class, students are assigned between five and ten practice research hypotheticals as homework that will be presented and discussed in the upcoming class. The students are instructed to complete the hypotheticals in both Westlaw and LexisNexis, which enables the students to see and learn how to use both databases and also to see the differences in search results in each database. Students are assigned one research hypothetical to share with the class. The student will go through the seven-step process and show the class how the student did the research by using the classroom computer and display screen or by sharing their screen with the class if the class is online.
Going through the practice research hypotheticals in class and requiring students to explain the seven-step process the students used for researching has numerous benefits. Having students present their research process to the class and explain how they found their answer reinforces their own understanding the seven-step process in what is known as the learning by teaching method or the protégé effect.[92] Additionally, students are able to practice both their speaking and presentation skills, enhancing these critical skills for practicing attorneys. Beyond these benefits, the in-class student presentations of their research processes using the practice research hypotheticals create serendipitous learning opportunities. Students can sometimes learn something new or be reminded of something they already knew but had not encountered in a while. Some examples of such serendipitous learning are when a student creates an advanced search and uses quotes around an exact phrase but misspells a word and retrieves zero results, leaving the student confused, until they realize they simply misspelled a word. Another example is when two students use different sets of search terms and they both get similar results that are on point for answering their research question. The in-class student presentations of their research processes create great learning environment because students sometimes see that the entire class is struggling with a particular research hypothetical, or students see how making a simple change in search terms can be the difference in getting thousands of results and getting thirty results on a topic. Students are also able to get immediate feedback from the professor on their research process and results. If a student is off track with their research process, the professor can call upon another student or students to help their peer. The professor can also explain important points and insights the professor may want students to know or think about as students present their research process. This enables students to understand how to alter or correct their search strategy if necessary.
Besides reinforcing learning the seven-step process and the serendipitous learning moments provided by doing in-class-practice-research hypotheticals, conducting hundreds of searches throughout the semester causes students to become confident in their ability to create good search strings and conduct searches that will retrieve good search results. Once students know, from the feedback the students receive in class, that they are retrieving relevant results from their searches, the students’ confidence soars and research becomes a far less frightening endeavor for the students. Additionally, as students repeatedly access the legal research databases and use the resources while conducting research, they learn how the databases organize information and how to use the databases better. Finally, unlike long lectures on research databases, how to use them, and their contents, which tend to bore students and make them lose focus, conducting in-class-practice-research hypotheticals gets students’ attention and keeps them involved in learning and practicing those concepts, making class time more engaging.
The seven-step process portion of the Process and Practice Method is vital for students to learn if students want to become competent legal researchers but learning the seven-step process alone is not enough. The key to the Process and Practice Method is the practice portion—specifically the amounts of practice researching the students are able to conduct during the ALR course. Without the substantial amount of research practice students get in each class and throughout the semester of the ALR course, the students would not become competent legal researchers. It is the repeated practice throughout the semester that enables the students to remember and recall the seven-step process and understand how to apply that process to researching a variety of legal issues.
This repeated practice spaced apart from class to class and from week to week employs what educational psychologists’ term as “spaced repetition learning theory.”[93] Spaced repetition learning theory asserts that students learn material when the material is presented repeatedly at short intervals, which produces enhanced long-term memory or recall of the material.[94] Some studies show this long term memory is enhanced even more when tests are interwoven into the spaced repetition learning.[95] However, merely repeating the material multiple times in one setting or learning instance does not have the benefit for long term learning and recall that repeating the material again and again over spaced periods of time has for long term learning and recall.[96] The spaced repetition learning is what makes the process practice method so effective. Students are reminded of, forced to recall, and required to employ the seven-step process to conduct legal research from class to class and from week to week for straight fourteen weeks in a typical semester and for at least seven weeks in a summer semester.
Law librarians and legal research professors know that one-off research instruction, though they are often called upon to do it, does not really benefit students and the students tend to not retain what was taught. Spaced repetition learning theory supports this belief by law librarians and legal research professors that students tend to not retain what is taught in one-off research sessions. Students in one-off research sessions quickly forget the material, especially if the teaching session is not connected to a graded assignment and the instruction is not repeated or practiced. Even when instruction is connected to an assignment, one-off teaching sessions on legal research are not very meaningful for students unless the concepts are repeatedly practiced.
Law students’ research instruction and practice experiences in one-off sessions, LRW courses, and traditional ALR courses, typically consist of learning a research concept, practicing it once or even a few times in class, and maybe even completing an assignment on the concept, but then not practicing that concept anymore for weeks or even the entire the semester. In stark contrast, students using the Process and Practice Method in an ALR course learn the seven-step research process, then repeat that process again and again throughout the remainder of the course during each class, resulting in hundreds of instances where the students are forced to recall the seven-step process and put it and their researching skills into practice. This results in students repeating the seven-step process and researching in databases three to four hundred times on many different legal issues during the semester. This enormous amount of spaced repetition of the concepts and skills produces vastly more competent legal researchers.
Anecdotal evidence supports the effectiveness of the Process and Practice Method in creating competent legal researchers. Students in eight ALR courses taught using the Process and Practice Method over the last five years have consistently commented in their final reflective papers, and in their anonymous course and professor evaluations, that they have found the ALR course the most practical and valuable course they have taken in law school. Many students decry the fact that the ALR course is not required in the first year of law school. Students’ comments also reveal the ALR course is helping them in their jobs as well. For example, one student commented in their final reflective paper, “In all seriousness, ALR has been the class that I have most quickly seen help me improve in my ability and confidence to do the work that I am assigned [at the law firm they work at], and I can definitely see how these skills directly correlate to being a better lawyer.” Another student noted in their final reflective paper:
I had a great opportunity to intern during this summer, so I was fortunate to have many occasions to practice all the skills I learned in class. After my boss saw my research skills, he had me researching all sorts of things for him and often had me find cases to match whatever he was working on.
These anecdotal accounts indicate the students are learning how to conduct legal research and that they find the ALR course taught using the Process and Practice Method a valuable part of their legal education.
With discussions in recent years of adding a legal research component to the bar exam, the need to develop legal research instruction that improves the legal research skills and competency of law students and recent graduates takes on added importance and urgency. The Process and Practice Method appears anecdotally to be an educational solution to the long-standing problem of deficient legal research skills of recent law school graduates. The anecdotal evidence of the effectiveness of the ALR course using the Process and Practice Method is testable though, and the ALR course can be evaluated empirically to determine if students taking an ALR course using the Process and Practice Method really are more competent legal researchers than students having taken only LRW courses or students taking LRW courses and a traditional style ALR course, or an ALR course taught in any other manner. Testing the effectiveness of ALR courses taught using the Process and Practice Method in an empirical manner would be a good next step for those interested legal research instruction and education.
VI. Conclusion
Legal research skills are fundamental to the practice of law. Even though legal research is a fundamental skill for lawyers, surveys of law firms reveal that partners consistently believe recent law school graduates and new associates are deficient in their legal research skills. This problem of recent law school graduates and new associates having deficient legal research skills is not new. It was recognized decades ago in MacCrate Report and reiterated as a problem in subsequent reports and surveys. Unfortunately, the deficiencies of legal research skills of recent law school graduates and new associates remain a problem to this day.
The reason the legal research skills of recent graduates and new associates has persisted is two-fold. The two primary issues for this persistence are a lack of law school prioritization of legal research and the lack of research opportunities throughout students’ law school careers. The lack of law school prioritization is the more difficult of the two issues to fix because of the many competing interests of stakeholders involved. Thus, the easier solution is to increase the research opportunities of law students during their law school career. However, LRW courses and ALR courses taught in the traditional manner do not provide enough opportunities to practice research. As such, there is a need for new approach to teaching legal research that provides more researching opportunities.
The Process and Practice Method is a new and effective approach to teaching legal research and anecdotally has proven to produce competent legal researchers. The ALR course incorporates the standards, principles, and competencies identified in ABA and other reports as well as those identified by groups and organizations such as the Boulder Conferences and AALL. The ALR course also employs the principles of spaced repetition learning theory through the ALR course’s repeated use of the seven-step process and repeated research practice throughout the course. Law students taking an ALR course employing the Process and Practice method receive nearly a hundred times more research opportunities than students taking only LRW courses or LRW courses and ALR courses using the traditional method of teaching legal research.
Due to the fact that legal research skills are fundamental to lawyering, it is imperative that the problem of deficient legal research skills of recent graduates and new associates be addressed. The Process and Practice method offers a new approach to teaching legal research to law students. Anecdotally, at least, it appears to produce competent legal researchers. A next step for legal research educators should be to test and evaluate whether an ALR course using the Process and Practice method actually does produce more competent legal researchers than ALR courses taught using the traditional method or other methods.
ABA Task Force On Law Sch. & The Profession, Legal Education And Professional Development – An Educational Continuum 135 (1992) [hereinafter MacCrate Report] (identifying legal research as a fundamental lawyering skill).
The Am. Bar Assoc. Standing Comm. on Legal Aid and Indigent Defendants, Gideon’s Broken Promise: America’s Continuing Quest for Equal Justice 19 (2004), https://www.in.gov/publicdefender/files/ABAGideonsBrokenPromise.pdf.
Id. at 39.
Id. at 19.
Id.
Id.
‘Height of Recklessness’: Judge Sanctions Attorneys Behind Election Lawsuit Over ‘Woeful Lack of Investigation’, The Nat’l L. J. (Aug. 4, 2021, at 11:57 AM), https://www.law.com/nationallawjournal/2021/08/04/height-of-recklessness-judge-sanctions-attorneys-behind-election-lawsuit-over-woeful-lack-of-investigation/.
O’Rouke v. Dominon Voting Systems, No. 20-cv-03747-NRN, 2021 U.S. Dist. WL 3400671, at *1 (D. Colo. Aug. 3, 2021).
Id.
Id.
Id. at *52.
Lexis Nexis, Hiring Partners Reveal New Attorney Readiness For Real World Practice 3, 3 (2015) [hereinafter New Attorney Readiness], https://www.lexisnexis.com/documents/pdf/20150325064926_large.pdf (discussing how the need to fill the gap in practical skills of newly graduates lawyers, which includes legal research, is costly to law firms); Westlaw, Research Skills for Lawyers and Law Students 3 (2007) (discussing the law firm partners perceived views of new associates’ deficiencies in legal research skills).
See MacCrate Report, supra note 1, at 26.
See generally William M. Sullivan et al., Educating Lawyers: Preparing for the Profession of Law (2007) [hereinafter Carnegie Report] (discussing legal research and skills training); see also Boulder Conferences on Legal Information: Scholarship and Teaching, Univ. of Colo. Law Sch. (2009) [hereinafter Boulder Conferences], https://lawlibrary.colorado.edu/sites/default/files/images/docs/2009_boulder_statement_on_legal_research_education.pdf (starting in 2009 and continuing to the present day).
See MacCrate Report, supra note 1, at 26.
Id. at 14.
Id. at 57.
Id. at 17.
Matthew E. Flyntz, Not Who - but How Much - Prioritizing Legal Research Instruction in First-Year Legal Writing Courses, 30 Second Draft 42, 42-43 (2017) (“Given that the results of the surveys discussed herein clearly demonstrate the importance of legal research in the “real world,” this is unacceptable. The data demonstrates that legal research is the most commonly used and the most important skill for new attorneys to develop.”).
Id. (citing Inst. For The Advancement Of The Am. Legal Sys., Foundations For Practice: The Whole Lawyer And The Character Quotient 11-12 (2016), https://iaals.du.edu/publications/foundations-practice-whole-lawyer-and-character-quotient).
Michele Bradley, Emphasizing the R in LRW: Customizing Instruction for Real-World
Practices, 30 Second Draft 3 (2017) (citing New Attorney Readiness, supra note 12).
Steven M. Barkan, Should Legal Research Be Included on the Bar Exam? An Exploration of the Question, 99 L. Libr. J. 403, 403 (2007) (citing MacCrate Report, supra note 1, at 138-40).
New Attorney Readiness, supra note 12
Id.
ABA House of Delegates, ABA Model Rules of Professional Conduct (1983) (providing in Rule 1.1 that attorneys possess the legal knowledge, skill, thoroughness, and preparation reasonably necessary for the representation, and Rule 3.3 that attorneys have a duty to disclose controlling authority that is directly adverse to the client’s position to the court).
Staci Zaretsky, Biglaw Firm Figures Out A Way For Clients Not To Pay For First-Year Associates, Above The Law (Sept. 16, 2014, 1:06 PM), https://abovethelaw.com/2014/09/biglaw-firm-figures-out-a-way-for-clients-not-to-pay-for-first-year-associates/.
Id.
WestLaw, Research Skills for Lawyers and Law Students 3 (2007).
Eric C. Newburger, Home Computers and Internet Use in the United States: August 2000, United States Census Bureau (Sept. 2001), https://www.census.gov/prod/2001pubs/p23-207.pdf;
A Brief History of NSF and the Internet, National Science Foundation (Aug. 13, 2003), https://www.nsf.gov/news/news_summ.jsp?cntn_id=103050.
Westlaw, Wikipedia, https://en.wikipedia.org/wiki/Westlaw.
Celebrating Innovation . . . and 30 years of Online Legal Research, LexisNexis, http://www.lexisnexis.com/anniversary/30th_timeline_fulltxt.pdf (last visited Aug. 27, 2021)
[Hereinafter LexisNexis Growth]; Robert Ambrogi, Westlaw’s Days Are Numbered, LawSites (May 26, 2015), https://www.lawsitesblog.com/2015/05/westlaws-days-are-numbered.html.
LexisNexis Growth, supra note 31; Ambrogi, supra note 31.
LexisNexis Growth, supra note 31; Ambrogi, supra note 31.
Compare Westlaw Edge, Thomson Reuters, https://www.internetworldstats.com/emarketing.htm (presenting stats on the growth of Westlaw), with Dan Connolly, A Little History of the World Wide Web, W3C (2000), https://www.w3.org/History.html (discussing growth of internet usage generally). Westlaw AI combines state-of-the-art artificial intelligence with that history and human element to create brand new tools that allow attorneys to complete legal research faster and with greater confidence.
LexisNexis Growth, supra note 31.
Compare Internet Growth Stats: Today’s road to e-Commerce and Global Trade Internet Technology Reports, Internet World Stats, https://www.internetworldstats.com/emarketing.htm stats on growth, with Dan Connolly, A Little History of the World Wide Web, W3C (2000), https://www.w3.org/History.html. Each of these websites provide statistics on the change of internet use from 1990s to today.
Marc Prensky, Digital Natives, Digital Immigrants, On the Horizon 1-2 (Oct. 2001), https://www.marcprensky.com/writing/Prensky - Digital Natives, Digital Immigrants - Part1.pdf.
LexisNexis Growth, supra note 31, at 13. Ambrogi, Numbered, supra note 31.
Lyle Moran, LexisNexis unveils new legal research product with better insights, more modern visual design, ABA Journal (July 8, 2020, 3:55 PM), https://www.abajournal.com/news/article/lexis-nexis-unveils-new-legal-research-product. Research databases say the improvements made researching easier. Richard Best, Best Legal Research Databases, Investopedia (July 19, 2021), https://www.investopedia.com/best-legal-research-databases-5192026.
Thomson Reuters, supra note 34; The Power of Artificial Intelligence in Legal Research, LexisNexis (Oct. 9, 2020), https://www.lexisnexis.com/community/insights/legal/b/thought-leadership/posts/the-power-of-artificial-intelligence-in-legal-research..
New Attorney Readiness, supra note 12, at 8. Young associates in litigation practice most often conduct legal research but lack advanced skills in this area.
See generally Carnegie Report, supra note 14 (providing an overview of the American Bar Association’s mandate of skills training).
Id. at 8 (Noting deficient legal research skills).
Id. at 7 (Formative versus summative assessments).
Id. at 4 (Back and forth bridging the gap).
ABA Section of Legal Education and Admissions to the Bar, Rep. of the Outcomes Committee 8 (2008) [hereinafter Outcomes Report] (discussing a theme of necessity based on the best practices discussed by Roy Stuckey Et Al., Best Practices For Legal Education (2007)).
Id. (discussing the exclusion of practical skills).
Id. at 10.
Id.
See MacCrate Report, supra note 1, at 5; see also Carnegie Report, supra note 14, at 8; see also Outcomes Rep., supra note 46.
See MacCrate Report, supra note 1, at 5; see also Carnegie Report, supra note 14, at 10; see also Outcomes Report, supra note 46.
Body of Knowledge [BOK], AALL, https://www.aallnet.org/education-training/bok/ (AALL Created the Body of Knowledge (BOK) with information on teaching and training legal research); Boulder Conferences, supra note 14; Jennifer Dubetz, CFR: Twelfth Annual Boulder Conference on Legal Information: Scholarship & Teaching – New Orleans, LA, Legal Scholarship Blog (Jan. 28, 2020), http://legalscholarshipblog.com/2020/01/28/cfp-twelfth-annual-boulder-conference-on-legal-information-scholarship-teaching-new-orleans-la/; ABA Section of Legal Education and Admissions to the Bar, Managing Director’s Guidance Memo 4 (2015).
Boulder Conferences, supra note 14.
Id.
Boulder Statement on Legal Research Education: Signature Pedagogy Statement, Univ. of Colo. Law Sch. (2010), https://lawlibrary.colorado.edu/sites/default/files/images/docs/2010_boulder_statement_signature_pedagogy.pdf.
Boulder Statement on Best Practices for Teaching: Coach Template and Illustrations: I, II, III, IV, Univ. of Colo. Law Sch. (2012), https://lawlibrary.colorado.edu/sites/default/files/images/docs/conference_on_legal_information_coach_template.pdf.
Susan Nevelow Mart, The Boulder Statements on Legal Research Education: The Intersection of Intellectual and Practical Skills 261 app. C (Susan Nevelow Mart ed. 2014).
HeinOnline, The COACh Template and Illustrations 261..
Boulder Conferences, supra note 14.
AALL, supra note 52.
AALL, PRINCIPLES AND STANDARDS FOR LEGAL RESEARCH COMPETENCY 9 (2013).
Id. at 4-9.
AALL, supra note 52.
Managing Director’s Guidance Memo, supra note 52, at 4.
Id. (Requires legal research skills and ethical responsibilities to client which include competent legal research of client issues).
Council of the ABA Section of Legal Education and Admissions to the Bar, ABA Standards and Rules of Procedure for Approval of Law Schools 2020-2021 17 (2020), https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/standards/2020-2021/2020-21-aba-standards-and-rules-for-approval-of-law-schools.pdf.
Id. at 24.
Id. at 19.
Id.
Enrollment Data 2018-2020, ABA Section of Legal Education and Admissions to the Bar (last visited Aug. 24, 2021), https://www.americanbar.org/groups/legal_education/resources/statistics/ (Based on average law school incoming class size (ABA LAW SCHOOL DATA: JD Applicant and Enrollee Data, Fall 2020 = avg. 195 students) and maximum Advanced Legal research class sizes of 25 students).
Id.
Caroline L. Osborne, The State of Legal Research Education: A Survey of First-Year Legal Research Programs, or Why Johnny and Jane Cannot Research, 108 Law Libr. J. 403, 408 (2016).
Id. at 409.
Id. at 414.
Id.
Id.
Id.
Market for New Law Graduates at Highest Level in 20 Years, Approaching 92%, NALP (July 24, 2008), https://www.nalp.org/marketfornewlawgraduates.
Joni Larson, To Develop Critical Thinking Skills and Allow Students to Be Practice-Ready, We Must Move Well beyond the Lecture Format, 8 Elon L. Rev. 443, 447 (2016).
Id. at 449-50.
This method is Professor Murray’s new conceptual method for teaching legal research using the seven-step process and substantial practice of researching throughout the semester.
New classes were added to Professor Murray’s ALR course as part of the process and practice method such as TRAPP and advanced searching techniques.
Post course assessment of student learning by Professor Murray enables the professor to see what changes need to be made for the next course.
See supra note 82 and accompanying text.
Research Strategy and Documentation, Univ. of Cin. Libr. (Jan. 8, 2021, 8:50 AM), https://guides.libraries.uc.edu/c.php?g=222582&p=1472975. (TRAPP stands for Things, Remedies and Relief, Causes of Action and Defenses, People and Parties Involved, and Placed Involved).
Alyson M. Drake, The Need for Experiential Legal Research Education, 108 Law Libr. J. 511, 514 (2016).
Anonymous student comments from the course final reflective paper.
Roy Stuckey et al., Best Practices For Legal Education 121-22 (2007), https://www.cleaweb.org/Resources/Documents/best_practices-full.pdf. (Students perform law related tasks in hypothetical situations as part of an experiential course as a sim).
Principles & Standards For Legal Research Competency, AALL (July 11, 2013), https://www.aallnet.org/advocacy/legal-research-competency/principles-and-standards-for-legal-research-competency/ (The five principles identified by AALL are incorporated into the syllabus for the ALR course using the practice and process method).
Managing Director’s Guidance Memo, supra note 52, at 4 (Formative throughout course providing feedback and summative providing final reflection and research homework assignments).
Outcomes Report, supra note 46.
Univ. of Cin. Libr., supra note 84.
Reed Rawlings, Mastering the Protege Effect, Medium (June 18, 2019), https://medium.com/age-of-awareness/mastering-the-protege-effect-1a49c62f7be5; Learning by Teaching, Top Hat, https://tophat.com/glossary/l/learning-by-teaching.
Jeffrey D. Karpicke, A Powerful Way to Improve Learning and Memory. Practicing Retrieval Enhances Long-Term, Meaningful Learning, APA (June 2016), https://www.apa.org/science/about/psa/2016/06/learning-memory (Educational psychologists term as spaced repetition learning theory).
Id. (Short intervals produces enhanced long-term memory or recall of the mate).
Id. (Enhanced with testing).
Id. (All in one setting not as good as spaced).