Revenge of the Rankings: How AI Uses Awards and Directories in Lawyer Searches

Best Lawyers, Super Lawyers, Chambers, Martindale: Law firm rankings have a new relevance as prospective clients increase their use of AI tools like ChatGPT.

Indeed, as Clio reports, more than one in three potential clients start their attorney searches online. What’s notable is how they are searching – or more specifically, where.

Google remains dominant, but it’s losing market share: A survey released this week showed that 78 percent of respondents say it is their first choice for searching, down from 80 percent in June; meanwhile, ChatGPT increased its standings from 1 percent in June to 5 percent in December.

It may be 5 percent now, but Google shouldn’t get comfortable: Gartner predicts that search engine volume will fall by 25 percent by 2026 due to generative AI solutions.

Given that 2026 is – shockingly – just a year away, it’s imperative that law firms understand how ChatGPT (and its compatriots) are delivering information to their potential clients. Let’s take a look.

Where We Started: ChatGPT A Year Ago

In November 2023, in an experiment on Medium, a ChatGPT user (and legal tech CEO) asked various platforms to find the best personal injury lawyers in Philadelphia. All were different:

• The free version of ChatGPT told him to do his own research.
• The paid version of ChatGPT gave a list of lawyers and law firms, all of which were recognized by Best Lawyers in America and Super Lawyers. Subsequent searches brought forth lawyers from Avvo and FindLaw, directories that allow clients to rate their attorneys.
• In a Google search, the “front-page” results were dominated by sponsored firms – firms that bought Google ads for the keyword in question.

As I wrote last year, this experiment showed that ChatGPT and its compatriots are being trained on lawyer rankings and client reviews. What about now?

Google vs. ChatGPT in 2024

I conducted my own experiment in December 2024. Because personal injury is so contentiously competitive in both paid and organic search, I went for a more B2B approach: “commercial litigation lawyers in Cleveland.” (This is also a market where I have no clients or vested interest.)

From the free version of ChatGPT:
“Help me find commercial litigation lawyers in Cleveland. Please include your source.”

1. Squire Patton Boggs
2. Thompson Hine
3. Benesch Friedlander Coplan & Aronoff
4. BakerHostetler

It’s a safe list, to be sure; all are AmLaw stalwarts, and three of the four are headquartered in Cleveland. As for sourcing, ChatGPT told me: “The sourcing for the list of commercial litigation lawyers in Cleveland was primarily based on a combination of law firm websites, legal directories such as Martindale-Hubbell, and local business journals that cover legal industry news and rankings. These sources provide insights into the firms’ areas of expertise, professional accolades, and client reviews, ensuring the recommendations are relevant and reliable.”

When pressed further, ChatGPT said the “local business journals” referred to Crain’s Cleveland Business, where the first firm on the list, Squire Patton Boggs, has been included in eight articles this year.

Key takeaways: Free ChatGPT may favor big firms that are more likely to have larger body counts on sites like Martindale; here, for instance, Squire Patton Boggs has 113 attorneys in Cleveland.

Moreover, earned media matters – and may be a more feasible tactic than adding 100 lawyers.

From the paid version of ChatGPT:

“Help me find commercial litigation lawyers in Cleveland. Please include your source.”

Interestingly, there was no overlap between the list supplied by the free version and the paid version of ChatGPT. The paid version supplied 10 firms, nine of which were attributed to attorney rankings:

1. Hahn Loeser & Parks (Best Lawyers)
2. Frantz Ward (Best Lawyers)
3. Taft Stettinius & Hollister (Best Lawyers)
4. Buckley King (Best Lawyers)
5. Garson Johnson (Garson Johnson)
6. Sutter O’Connell Co. (Lawyers.com)
7. The Chandra Law Firm (Super Lawyers)
8. Jeckering & Associates (Super Lawyers)
9. Ritzer, Coughlin & Paglia Ltd. (Super Lawyers)
10. Cavell Law, LLC (Super Lawyers)

This confirms the 2023 experiment’s hypothesis that the paid version is being trained on rankings and credentials. With the exception of Taft, this list is skewed toward small and midsize firms, ranging from 1 lawyer (Cavell Law) to 130 (Hahn Loeser & Parks).

Key takeaways: Best Lawyers supplied the top four spots on the list. While Best Lawyers rankings are dependent on peer rankings, law firms (and their marketers) should stay vigilant about nominating their qualified lawyers.

As for the firm names supplied by Super Lawyers, I have no firsthand knowledge of these firms or their marketing plans, but at a glance it appears that all four have a paid profile for at least one lawyer. This merits more investigation, but there could be a new argument for investing in Super Lawyers positioning.

From Google:

“commercial litigation lawyers in Cleveland”

The third search method produced yet another list of firms with no repeats from the Chat GPT lists:

1. Axiom Law (sponsored)
2. Super Lawyers
3. Calfee, Halter & Griswold
4. Dworken and Bernstein
5. Best Lawyers
6. Gertsburg Licata
7. Shumaker

The only overlap: Super Lawyers and Best Lawyers.

Google stays true to its algorithm, delivering first a paid ad. The list includes two sites with high domain authority – Super Lawyers and Best Lawyers – as well as a number of firms with high relevance for the location-based query:

• Calfee, Halter & Griswold has an Ohio-specific positioning: “From our offices in Columbus, Cleveland, and Cincinnati, we provide comprehensive and efficient legal and government relations services.” Its commitment to the Buckeye State is evidenced by its 100-page e-book, Guide to Doing Business in Ohio.
• Dworken & Bernstein, “serving all of Northeast Ohio,” has a commercial litigation landing page that adheres to many SEO best practices, including the use of questions and key phrases.
• Gertsburg Licata – “Headquartered in Cleveland. Serving the Globe” – may appeal to the Google algorithm’s rumored “freshness factor” with blog posts like “What Every Business Owner and CEO Should Do Before December 31.”

Key takeaway: It’s little surprise, but classic SEO techniques reign supreme in Google searches. Given that Google still controls 78 percent of the search market, law firms that value search traffic should continue working to appeal to the algorithm.

What to Do Now

While this is a limited experiment in a wildly evolving landscape, there are a few actions for legal marketers to take:

• Start by running your own experiments. How does your firm appear in logical, likely-to-actually-be-used queries on AI platforms? If you are not ranking where you believe you should, start identifying the factors that may be working against you. (Don’t be afraid to ask the robots to show their math.)
• Refer to strategy. Before making any adjustments to your marketing mix, make sure you’re working purposefully. Who is your ideal client? Are you focusing on local prospects or national companies who happen to have local needs? Are they likely to be using Google or ChatGPT? (More than 56 percent of ChatGPT users are 18 to 34 years old; only 5 percent are older than 65, according to DemandSage.)
• Reconsider directories and rankings. There’s no denying that rankings factor into the fastest-growing search method – and the one used by nearly half of Gen Z. The free version of ChatGPT referred to rankings for nine out of 10 recommendations in our scenario, while the paid version consulted Martindale-Hubbell. Again, run your own experiments, but take a close look at your competitors’ ranking strategies (and use of paid or premium profiles) as they relate to the results.

There’s an old marketing adage that it’s not what you know; it’s who knows what you know. In this brave new world, law firms must work to ensure ChatGPT can find them – and in a funny twist of fate, that may mean reinvesting in some old-school tactics.