Wall Street’s top tech priority: building internal search engines

If data is the new oil, then banks like Goldman Sachs are in good business.

For decades, Goldman hoovered up information about the clients it did business with, trades executed, every dollar invested, and each loan financed. Coupled with external data from Bloomberg and Nasdaq, the hope was to supercharge the bank’s analytics engine and give its investment bankers, traders, and salespeople an edge.

But that fuel is only useful if you can access it.

For banks, much of that data got stowed away, usually only to be found if an employee knew exactly what they were looking for and where to find it.

That could all change as Goldman Sachs rolls out a generative AI chat interface on its firmwide data platform, Chief Data Officer Neema Raphael told B-17.

Goldman employees can ask a question in plain English, and let the AI do the digging. In answering questions, the tool, called Legend AI Query, could pull information that even its users didn’t know existed.

The chat interface, combined with the bank’s data stockpiles, “gives you this sort of information super intelligence to help the human build a better mental model faster and quicker with more sources,” Raphael said.

It’s the latest development in Wall Street’s efforts to crack the code on search.

From Goldman Sachs to Blackstone, the biggest finance firms are using generative AI to make better use of their mountains of data. Even though it’s been decades since Google introduced the world to effective search to use in everyday life, only recently have financial firms started putting resources behind improving how employees tap their internal data. They’re trying to turn the wonky and sometimes impossible task of searching for information into a seamless process that will supercharge employee productivity. Perfecting search, down the line, could lead to more automation and more complex generative AI tools.

Search is just the beginning

Goldman’s peers across the Street have their own search-related initiatives underway. JPMorgan’s private bank AI copilot helps advisors track down information in real-time. Bank of America’s Banker Assist aggregates internal and third-party info to give employees insights. Morgan Stanley’s AIMS helps advisors search the bank’s internal content.

While enabling employees to quickly get answers hidden in mountains of data is going to supercharge worker productivity, there’s likely an even bigger ambition behind these efforts.

Pulling the right information and having some contextual understanding is the first step in tapping into more complex use cases, Keri Smith told B-17. Smith helps financial firms strategize and execute on their data and generative AI initiatives at Accenture.

“The power of enterprise search lies in its ability to save time so that humans can innovate and interact,” Jeff McMillan, Morgan Stanley’s head of firmwide AI, told B-17. “Further, it lowers the barriers for employees to access robust intellectual capital quickly from the firm’s top experts, essentially arming them with knowledge firepower for meetings and discussions,” he added.

A new class of fintechs is starting to emerge to sell Wall Street on cutting out simple, but time-consuming tasks, like perfecting company logos on an investment banking deck and prepping execs ahead of client meetings.

Rogo is one such startup that offers a junior-banker level assistant. It has already onboarded some-25 Wall Street firms onto its generative AI platform.

“Firms are realizing the value of enterprise search is not just a typical search engine, but for all the downstream applications you’re going to be able to build on top of it,” Rogo cofounder and CEO Gabe Stengel told B-17.

Meanwhile, two Stanford grads came together to build Mako, a generative AI associate for the private-equity industry. The startup, which aims to help employees search institutional data, recently raised $1.55 million from the same VC that was an early backer in OpenAI.

Why search is hard

There’s a reason Google is the go-to search engine for the internet.

“It’s actually just fundamentally a difficult problem to suck out and then rank what might be useful” to a specific user, Raphael said of search. Not only is search a hard computer-science problem, but personalizing relevance and dealing with account permissions (who is authorized to see given data) is no easy feat, he added.

The latter is something Blackstone the majority of 10 months figuring out when it recently built its own internal AI-powered search engine.

Add in the fact that Wall Street lingo is complex and nuanced — words like hedge, ticker, and options have completely different meanings outside a financial context — and it presents another hurdle for financial firms using off-the-shelf products, like OpenAI’s ChatGPT.

In March, Balyasny Asset Management hired Peter Anderson, a former AI scientist at Google, to help the hedge fund level up its back-end system that pulls information from millions of documents to answer complex research questions. Familiarizing OpenAI’s models to financial jargon meant Balyasny’s internal version of ChatGPT surfaced the most helpful document 60% more frequently than without this training.

Still, generative AI is bringing companies one step closer to solving for search.

“People are building knowledge bases, they’re letting this genAI crawl and to be able to either search or summarize, I think this is maybe a stepping stone to crack the problem,” Raphael said.

Where Goldman is throwing its generative-AI weight

Legend AI Query is just the beginning for Goldman Sachs.

The search tool is the bank’s second such generative AI tool, the first being a generative AI developer co-pilot that helps software engineers code more efficiently. The effort has resulted in a roughly 20% increase in efficiency depending on the use case.

At the same time that Goldman’s AI/ML engineers were busy trying to crack the code on search, they also built another generative AI tool that aims to help data engineers — the developers who handle the bank’s data and make sure it is vetted, organized, and structured — do their jobs better. Legend Copilot is another tool the bank launched this month, which is designed to make it easier to get more data onto Legend and manage it in a methodical way.

Raphael said he is focused on “really helping both engineers and non-engineers find and discover the right data for their use case.”