Podcast Episode 37: The Stanford AI Legal Research Paper: Analysis and Implications with Richard Robbins and Brian Blaho

Posted by on | Featured

Richard Robbins and Brian Blaho

In this episode, Ted welcomes back Richard Robbins and is joined by Brian Blaho from Reed Smith to explore the challenges of integrating AI into legal research. The discussion draws on insights from a recent Stanford study, which underscores the limitations of current AI tools, particularly in legal reasoning. For those considering AI solutions for their firm, this conversation emphasizes the importance of transparency, collaboration, and strategic implementation in driving innovation.

Richard and Brian delve into key topics, including how to:

  • Evaluate AI systems for legal research
  • Understand the limitations of retrieval-augmented generation (RAG) in legal AI
  • Identify potential biases in academic studies on legal AI tools
  • Effectively benchmark AI performance in legal technology
  • Seamlessly integrate generative AI into law firm workflows

Key takeaways:

  • The Stanford study highlights significant limitations in the accuracy and legal understanding of current AI research tools from major vendors.
  • AI models face challenges in grasping the complexities of legal precedent and authority, making legal research more difficult than initially thought.
  • While vendors are working to improve AI tools, progress relies on transparency and collaboration in benchmarking.
  • A community-driven approach, rather than vendor-led efforts, is crucial for developing accurate benchmarking standards in legal AI.

About the guests, Richard Robbins and Brian Blaho:

Richard Robbins is the Director of Applied AI at Reed Smith LLP, a leading international law firm with more than 1,400 lawyers across 30 offices. He leads the firm’s applied artificial intelligence and data science initiatives.  He is a veteran lawyer who was both a partner at a major law firm and the general counsel of a US publicly traded company for a decade.  He is also an accomplished legal technologist who, among other things, has spent the last few years focused on data science, machine learning and generative artificial intelligence.  He holds a Bachelor’s Degree and a Master’s Degree in Electrical Engineering and Computer Science each from the Massachusetts Institute of Technology, a Juris Doctor Degree with Honors from The University of Chicago Law School, and a Master of Information and Data Science from UC Berkeley, where he currently supports its graduate class on generative artificial intelligence.  He frequently writes and speaks on topics related to his work.

Brian Blaho is the Global Director of Library Services at Reed Smith, a leading international law firm with more than 1,400 lawyers across 30 offices. In this role, he oversees the strategic direction, operations, and budget of the firm’s library and research services, as well as the development and delivery of innovative solutions to support the firm’s practice groups, business development, and client service. Brian has a J.D. from Rutgers Law School and over 20 years experience in legal research and information management. He has extensive expertise in vetting, rolling out, and getting value from legal tech products in law firms, including artificial intelligence, analytics, and competitive intelligence tools. He also leads the firm’s library and research team in providing high-quality research, analysis, and training to lawyers and staff across the firm’s global network. Brian is an active member of the American Association of Law Libraries, the International Legal Technology Association, and the Law Library Association of Greater New York. He frequently speaks and writes on topics related to legal research, legal tech, and library management.

“It’ll be up to the vendors who build on top of the [legal AI] models to add the other workflows, the agentic systems, and other elements to get us there.”– Richard Robbins

“Generative AI can potentially help us get much better results so that we have to either spend far less time reading through and curating which ones we’re going to pass on or no time at all.– Brian Blaho

Connect with Richard and Brian:

Connect with Ted Theodoropoulos:

Subscribe to Legal Innovation Spotlight:

Comments are closed.