Why is the LLM-based approach for counting word frequencies considered unreliable in the chapter?

The LLM-based approach is considered unreliable because, when it was run three times, it produced word frequency results that differed from one run to another and also differed from the validated output of the standard code notebook, which served as ground truth. The chapter concludes that LLMs can generate plausible language but cannot reliably produce accurate numerical results, so direct LLM output is not dependable for counting tasks.

In the chapter, the authors tested two LLM-based approaches to obtain the 20 most frequent words from a set of posts. In the first approach, they asked the LLM directly to preprocess the posts and return the frequency table. This direct prompting approach was run three times. The results were different not only across the three runs but also compared with the output from the data_exploration_standard.ipynb notebook. The authors used the standard notebook's output as a validation reference, or ground truth, because that process could be verified step by step. Since the LLM results did not match that ground truth, the authors judged the method unreliable. They explained that the inaccuracy highlights a critical limitation: LLMs produce plausible and fluent language but lack the ability to produce accurate numerical results. For tasks requiring exact numerical counts, they therefore recommend alternative approaches, such as using an LLM as a coding assistant to generate a Python function, and emphasize that any LLM-generated code must also be reviewed and adjusted for errors.