What are the key limitations of large language models (LLMs) mentioned in this chapter?

The chapter identifies two key limitations of current LLMs: they are predominantly limited to processing text, unable to natively handle image, audio, or video data, and they have limited capacity for complex, multistep reasoning. These limitations are expected to be addressed by future multimodal and reasoning models.

In this perspective chapter, the authors describe current LLM limitations as the basis for anticipated future advances. First, LLM applications to date have been mostly text-only, which means they cannot natively process the rich contextual data found in images, audio, or video. Second, current LLMs have limited ability to perform complex reasoning, such as deconstructing problems into logical steps or evaluating arguments. The chapter presents emerging multimodal models and reasoning models as responses to these shortcomings, and it cautions that additional risks and limitations of these advances are still not fully understood.