Ex-Bank of America quant finds that agentic LLM traders don't like to trade

The oft-used metaphor for implementing agentic AI into your workflow is that you've gained a super-capable intern to do the grunt work for you. In a trading context, you might find yourself repeatedly telling this intern to do basic things, only to have it talk to you about something completely different.

Last week, the University of New South Wales recently released the PhD thesis of Alicia Vidler, an ex-Deutsche Bank and Merrill Lynch trader exploring agentic AI in finance. Vidler found that, when integrating large language models (LLMs) into an agentic trading system, "it proved difficult for LLM agents to make a decision to buy or sell, even when prompted to do so." LLM agents would ignore prompts that explicitly told them what to trade, and would "analyse and discuss rather than making concrete decisions."

Not blindly rushing into trades and taking the time to analyze them is all well and good, but Vidler said the models "tend to have excessive dialogue and can become stuck in a loop rather than concluding the conversation." It's not all the LLM's fault, however; Vidler said that "the engineering and implementation issues surrounding the use of APIs of LLMs remain a key hurdle to their use."

Fintechs developing agentic AI platforms are, naturally, more bullish on their potential. Bin Ren, CEO of SigTech (spun out of hedge fund Brevan Howard) said last year that LLMs will spark a "quantum leap in productivity." Speaking on the intern analogy, he said LLMs can be better as they're "infinitely patient." In Vidler's tests, however it seems that the traders are the ones having their patience tested.

A key positive observation from Vidler is that, the smaller the model, the better the performance. Vidler said there were "significant performance variations observed between GPT model versions and their subvariants" with GPT-4oMini being the best of the models used. The future of agentic trading is supposedly brighter due to the "rapid development of smaller foundational language models." These less noisy models could offer better performance and efficiency. 

Have a confidential story, tip, or comment you’d like to share? Contact: Telegram: @AlexMcMurray, Signal: @AlexMcMurrayEFC.88 Click here to fill in our anonymous form, or email editortips@efinancialcareers.com.

Bear with us if you leave a comment at the bottom of this article: all our comments are moderated by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libellous (in which case it won’t.)