ProcessIntermediate
In the code example for topic modeling with Gemini, what is the purpose of the `block_size` parameter in the `assign_topics` function, and how does it affect the processing of posts?
block_size determines how many social media posts are grouped into a single batch for topic assignment. The assign_topics function loops through the posts in steps of block_size, builds a sub-dataframe for each batch, and sends that batch to the Gemini API for analysis. This chunking lets the code process large datasets efficiently by handling one manageable block of posts at a time.
Key points
- block_size is the number of posts processed in each LLM batch.
- The function iterates through the DataFrame using range(0, len(df_posts), block_size), creating a slice of posts for each block.
- A separate prompt and API call are generated for each block, so results are accumulated across blocks.
Source:AI for Qualitative Research: A Hands-On Guide for Management Scholars· Clustering and Topic Modeling· p. 131–141
Related questions
AI for Qualitative Research: A Hands-On Guide for Management Scholars
Diana Garcia Quevedo
Palgrave Macmillan