I attended Inside Practice’s AI x KM event this week in NYC put on by Kevin Klein and Trey Shalley and wanted to share my thoughts.
Event Introduction
Patrick DiDomenico who was the master of ceremonies did a great job facilitating the discussion on how generative AI and emerging technologies are transforming Knowledge Management and Innovation (KM&I) functions within law firms. The sessions highlighted the strategic importance of knowledge professionals as firms assess their needs and define an AI strategy. Several sessions were case studies that demonstrated how generative AI is automating repetitive tasks and synthesizing insights from large data sets, thereby acting as a force multiplier within the industry.
Paul Weiss
The event was hosted by Paul Weiss and their conference facilities were very comfortable and their staff did a great job accommodating everyone. That part of Midtown isn’t my favorite (I think because of LegalWeek) but I’m not sure I left a two block radius during the 3 days I was there which was very convenient. The only time I had to venture outside of that radius was go get water because a lot of the Marriott properties in NYC no longer provide bottles of water for their guests (boo!). That’s super annoying but a conversation for another day.
Audience and Panelists
The audience and panelists were the usual suspects from the KM&I community (which is a good thing). Anecdotally I would say that the audience and panelists were slightly more senior than what I typically see at these events. It was a modest but noticeable difference, at least from my perspective. I’m not sure if this had an impact on the content but overall, it was very good. Everyone has different preferences, but I like real-world tangible examples of how technology is being leveraged to solve real problems in the business and there was a lot of that. When sessions stay too high level, I tend to lose focus. I took notes during a few of the sessions, and I’ll share some specifics from each of them below.
I am going to have to call something out here that disappointed me. The organizers allowed some of the sponsors to moderate some sessions and a few others to make short 10-minute presentations. Almost all the sponsors did a fantastic job being respectful of the audience and organizers by not taking advantage of that by hammering hard on the sales front. Unfortunately, there was one exception, and I don’t think anyone appreciated it…not the organizers, not the audience and not the other sponsors.
There’s a tremendous amount of trust that conference organizers put in us as sponsors when they allow us to go on stage. When someone violates that trust it makes all of us look bad and those organizers may hesitate the next time. That’s unfortunate because almost everyone was so respectful of the purpose and spirit of the event. I thought ALMOST every vendor that was allowed to get on stage contributed to the quality of the content significantly.
Okay, enough about that. Let’s talk about a few of the sessions!
Keynote Session: Happy. Sad. AI, Bad?
Board President – CLOC (Corporate Legal Operations Consortium)
Former Director of Legal Ops & Tech – Netflix
This was my first time seeing Jenn speak and all I can say is wow. She is funny, articulate, and full of energy. In her keynote, Jenn shared some experiences from her former role as director of legal operations for Netflix. She discussed her experience rolling out Evisort, a Contract Lifecycle Management (CLM) solution, at Netflix just this year.
The implementation involved managing a vast 9-million-point metadata repository across all enterprise contracts, supported by 2,000 machine learning models. Jenn discussed the revolutionary GenAI features within the solution. Netflix ultimately decided to delay those features but gave us an idea of their capabilities. One such example was a chat interface that enabled Q&A on documents within their repository. Examples of the types of questions the platform could answer are:
- Of all the contracts for film originals in the US, show me the deal financials range for above the line talent from 12 months before and after the Actors Strike
- Of all procurement contracts, make me a report of all contracts expiring for the next two quarters
These innovations posed internal engineering challenges, but her team managed them by fostering a collaborative environment and leveraging the latitude for creative solutions they were given from management. She highlighted that this approach allowed her team to achieve in five years what would typically take ten, demonstrating the power of embracing change and upscaling individuals to work on higher-value tasks.
McCarron addressed the impact of GenAI on internal stakeholders, emphasizing that the technology’s purpose isn’t to replace people but to enable them to focus on more strategic work. She shared examples of where GenAI was thriving at Netflix, from Hollywood deal-making and sports leagues to investor relations, indicating that AI’s influence is broad and far-reaching.
The session also covered what inside counsel seeks from law firms, with a clear message to embrace technology rather than ban it, operate efficiently, and continually innovate. McCarron closed with a perspective on thought leadership, advocating for impact driven by a combination of scientific method, courage, and flexibility to adapt. Overall, her session painted an optimistic view of AI’s role in transforming legal operations and Knowledge Management.
How KM Teams are Paving the Way for AI in Legal Practice: New Roles, New Tools, New Strategies.
Iris Skornicki – Director of Knowledge Solutions, Paul, Weiss, Rifkind, Wharton & Garrison LLP
Elizabeth Wilkinson – Director of Knowledge Solutions, Paul, Weiss, Rifkind, Wharton & Garrison LLP
Gina Lynch – Chief Knowledge and Innovation Officer, Paul, Weiss, Rifkind, Wharton & Garrison LLP
In this session the Paul Weiss team drew parallels between the rise of AI and the early days of Google, noting that initial fears about the impact of search engines on librarians proved to be unfounded. Instead of becoming obsolete, librarians became validators of information emerging from these new data sources.
This comparison was used to highlight how Knowledge Management (KM) lawyers could also adapt to the rise of AI by becoming experts in validating and managing AI-generated information within a legal context. The panel mentioned that in the early days of legal tech, there was a clear split between those who embraced technology and those who resisted it.
The session emphasized that KM lawyers need a diverse set of skills, with a deep understanding of technology as a central component. The speakers noted that there’s a specific role, Practice Intelligence Specialist, dedicated to writing stories about AI and spotlighting relevant use cases in legal environments. This role sounds like the coolest job ever and underscores the importance of not just adopting technology but also communicating its value within the firm.
Additionally, the session featured a graph depicting the growth of the legal tech market, which began to accelerate rapidly around 2009, indicating that technology’s role in legal practices is not only growing but that growth is accelerating rapidly.
Legal Knowledge in the Digital Age: The Productization of KM
David Boland – Chief Knowledge and Innovation Officer, Ogletree Deakins
Tim Fox – Director of Practice Intelligence and Analytics, Ogletree Deakins
Susan Sommers – Director of KM Innovation and Solutions, Ogletree Deakins
The “Productization of KM” session highlighted the possibilities of KM as a revenue generating activity. The Ogletree KM team has productized some of their activities such that clients are paying the firm subscription fees to receive the output. Tim Fox understandably kept the exact number to himself but did say that the revenue generated is “in the millions” showing that it’s not a trivial contribution to the firm’s top line.
The panel emphasized the importance of operating KM as “one team” with shared values and that all KM activities align with the firm’s strategic priorities. This ensures that the KM function supports the firm’s overall goals. Data analytics were identified as a key revenue-generating activity that clients are willing to pay for, showcasing the value that KM can bring to a law firm’s business model. The data analytics team’s success allowed the team to expand from 2 to 40 people over the last four years which is astonishing growth.
The Art of the Possible: Knowledge Engineering and Next Frontier of Legal AI in Knowledge Management
Ilona Logvinova – Director of Practice Innovation, Cleary Gottlieb Steen & Hamilton LLP
Patrick Dundas – Partner, Akin Gump Strauss Hauer & Feld LLP
Bryan Gilbert Davis – Founder and CTO, Centari
This “Art of the Possible” session delved into how Knowledge Engineering and AI-driven tools are reshaping the legal landscape, especially in terms of training and learning management for associates. The session discussed the profound impact that leveraging AI models using structured data can have on a profession rooted in heuristic thinking and intuitive reasoning. It explored the new possibilities for law firms that can extract and leverage unstructured data from various sources, creating innovative knowledge management systems that push the boundaries of AI integration.
The session also addressed the shortcomings and challenges that come with this rapid technological shift. DMS (Document Management System) vendors were called out for not providing AI tools that automatically classify documents and extract relevant metadata. The need for more advanced risk management tools was discussed, such as virtual personal assistants, to ensure that lawyers don’t overlook critical tasks during the execution or closing of legal matters.
Additionally, the session noted that AI tools like GitHub Copilot have the potential to transform legal practice by enabling the 100x engineer instead of just the 10x engineer, indicating the immense productivity gains that AI technology could bring to the legal profession.
Rethinking Organizational Structure in the Age of AI and Digital Transformation
Peter Geovanes – Chief Innovation & AI Officer, McGuireWoods LLP
Meredith Williams-Range – Chief Legal Operations Officer, Gibson Dunn
Christy Bentz – Chief Client Value and Innovation Officer, US, Norton Rose Fulbright
Ted Theodoropoulos – CEO, Infodash
I was fortune enough to get the opportunity to moderate this panel of C suite execs from Am Law 50 firms to discuss how AI will impact organizational design in law firms. I have moderated quite a few panels over the years and this was definitely one of the most esteemed. Peter, Meredith and Christy shared fantastic insights on the topic and each emphasized different aspects which will ultimately impact where AI functions best align in a firm’s organizational structure.
One thing all of them agreed on, however, was the huge role that firm culture will play in the process. Some firms embrace and value KM but may view innovation as a small subset of the KM picture. Other firms may have a standalone innovation function and reporting structure where AI strategy should likely live. Understanding the underlying dynamics that gives one functional area more influence than another will go a long way in defining the right org structure for that particular firm.
The panel first explored the “what” by defining what functions are needed to design and execute an AI strategy and then answered the following questions to explore the “where” and the “how”:
- What are the fundamental differences between KM and innovation and how those functions are measured differ?
- How do these differences impact ideal alignment with AI (i.e. risk taking, tolerance for failure, etc.)?
- How are emerging tools and tech blurring the lines between KM and Innovation?
- Are there ever situations where a standalone Chief AI Officer makes sense in legal?
- How important are cross functional teams managing the strategy and implementation of these new technologies?
- What role does change management play in the strategic planning process?
The AI-Driven DMS – From Document Repositories to Knowledge Engines
Joe Green – Chief Innovation Officer, Gunderson Dettmer
Sara Miro – Director of Knowledge Solutions, Sullivan & Cromwell LLP
Elizabeth Wilkinson- Director of Knowledge Solutions, Paul, Weiss, Rifkind, Wharton & Garrison LLP
James Ding – CEO, DraftWise
“The AI-Driven DMS” session explored how AI-enhanced Document Management Systems (DMS) can revolutionize traditional document handling within law firms by automating manual tasks such as metadata tagging and classification. The session highlighted that these AI-based DMS could theoretically transform document repositories into engines of intelligence by using intuitive semantic search to surface hidden insights, providing a more efficient way to extract and connect knowledge across documents.
As these AI tools improve in their ability to mine data from legal agreements, expectations will shift as lawyers rely more on KM teams to package and deliver these insights on demand, leading to more predictive insights, risk assessments, and trend analyses.
However, the session also identified ongoing challenges and new perspectives on AI-driven DMS. Joe Green mentioned that he doesn’t care who provides these capabilities, emphasizing that the goal is to meet the long-term unmet needs in DMS search technologies. He also described a search approach akin to Google Image search, where contracts are used to search for other similar contracts within the DMS.
The discussion also touched upon user experience, with Sara expressing her frustration with having to use multiple prompts to get answers in AI. The metaphor of lawyers wanting “windmills, not treadmills,” highlighting the demand for efficient solutions was discussed. The session concluded with a candid discussion on error rates of AI tools, pointing out that humans also make mistakes, underlining the need for a balanced approach in adopting AI-enhanced DMS capabilities.
Hopefully that provides an idea of the themes of the conference. It was definitely worth the price of admission and I look forward to participating with the folks from Inside Practice LLC in the future. That’s all for now as I repack my bags for ILTA’s Evolve conference in Charlotte!