For 2025, on the AI front, the big theme for us at Robotic Online Intelligence (ROI) remains 'making LLMs work in practice' against the backdrop of rapid progress and real opportunities on one hand, and the over-hyped ideas and the 'enterprise reality' on the other.
Here are our takeaways from 2024, in the context of data/information businesses and investment research applications:
🔹 Everyone talks about some form of Agentic AI - where a model can 'decide on its own' what action to take and/or what tools to use (from a web search API to placing an order) for a given objective or question. In our domain, we think there are much bigger productivity gains to be earned first from a more comprehensive workflow automation that would be still deterministic in nature but with many LLM components 'talking' to each other.
🔹 The RAG Road To RAGginess - 2024 was largely a RAG year. RAG (Retrieval Augmented Generation) methods carried a promise of combining Large Language Models' general abilities to 'understand'/process unstructured text, with specific knowledge bases - be it scientific papers, SEC filings, policy documents, or medical data. The reality in practical deployment is much more challenging and limiting, yet still promising and will remain a big theme in 2025.
🔹 Focus on the domain- and use-case-specific deployments - To make LLMs work, we need to focus on a specific business problem (one at a time) and/or workflow and deploy LLMs on a highly domain-specific and use-case-specific basis.
🔹 Domain specificity needs topic models - We've been using the term 'topic model' in relation to a human-expert-defined 'map' of a particular domain. LLMs can help define such topic models (already in Kubro(TM)) but still need to be 'guided' by the human expert.
🔹 Summarization, classification, and data extraction with LLMs do work - These are the areas where LLMs can work very well and, for all practical purposes, are the most likely to deliver strong ROI.
🔹 Multimodal LLMs can serve as an alternative to OCR, data extraction, and chart documentation but not (yet) for data analysis.
🔹 The cost of human quality control remains one of the biggest issues in high-accuracy use cases. Citation / tracing back to source material in RAG or LLMs checking on LLMs, and other techniques help but that's harder to do for more complex questions.
✈️ Finally, for all the AI in the multiverse, it's hard to beat the in-person meetings around the world...