AI for Qualitative Research: A Hands-On Guide for Management Scholars
Diana Garcia Quevedo
Questions & Answers from this book
Questions and answers are connected to the referenced book and its available source material.
Chapter 1: Introduction
Under what Creative Commons license is this book published, and what are its main restrictions?
The book is published under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Its main restrictions are that use must be noncommercial, you cannot share adapted or derivative material, and you must give appropriate credit to the authors and source, provide a link to the license, and indicate if you modified the licensed material.
What is the title of the book and who are its authors?
The book is titled 'AI for Qualitative Research: A Hands-On Guide for Management Scholars' and is authored by Diana Garcia Quevedo and Josue Kuri.
Chapter 2: Overview of Artificial Intelligence, Machine Learning, Natural Language Processing, and Large Language Models
What is the purpose of the synthetic dataset provided with the book, and why was it created?
The synthetic dataset was created to enable hands-on learning with LLMs for qualitative analysis while avoiding the ethical and data-protection risks of using real, private, or proprietary social media data. It consists of 1,115 fictional social media posts from ten imaginary personas and is intended exclusively for educational practice with the book's code examples.
According to the introduction, what are the practical advantages of implementing LLMs through programming applications?
Coding facilitates data handling, increases control over LLM parameters, helps manage LLM limitations, and allows combining multiple models and techniques to improve data processing and analysis.
What are the main sections covered in Chapter 2 of the book?
Chapter 2 covers the history and origins of large language models, the technology underlying them, and their limitations.
Chapter 3: Natural Language Processing in Management Research
Why have qualitative researchers historically been hesitant to adopt computational methods like NLP, and how have LLMs changed this?
Historically, qualitative researchers hesitated to adopt NLP because computational methods were seen as incompatible with interpretive analysis, and early NLP models lacked the ability to understand context and nuance. These limitations meant NLP was useful mainly for preprocessing large datasets, not for the close, context-sensitive reading qualitative scholars value. LLMs changed this because they can capture semantic meaning, subtle nuance, and context from vast unstructured texts, making them more compatible with inductive and interpretive qualitative approaches and shifting researcher acceptance.
According to the chapter, what are the main limitations of using ChatGPT for qualitative data analysis as identified in the studies by Hamilton et al. (2023) and Morgan (2023)?
According to the chapter, Hamilton et al. (2023) found that ChatGPT-generated themes are too specific to participants' immediate circumstances, disregarding nuanced context and subtleties, whereas human coders used contextual knowledge to develop more comprehensive themes. Morgan (2023) found that ChatGPT tends to emphasize generic aspects of the data, lacking contextualization and the ability to recognize subtle, interpretative themes. Both studies conclude that ChatGPT should serve as a complementary tool rather than a substitute for researcher interpretation.
According to the chapter, what are the main capabilities of large language models (LLMs) that make them useful for inductive qualitative analysis?
According to the chapter, LLMs are useful for inductive qualitative analysis because they can explore and summarize large datasets, retrieve and augment information through retrieval-augmented generation (RAG), and classify and cluster data by capturing language nuances, context, and implicit patterns. These capabilities let researchers handle vast amounts of unstructured data and reshape traditional linear analysis into dynamic, iterative inquiry.
Chapter 4: Ethical Considerations
What are the specific ethical challenges that arise when LLMs are employed as tools within research contexts, as discussed in this chapter?
The ethical challenges include algorithmic bias and discrimination in model outputs, data privacy and security risks related to retaining or leaking personal information, and the inherent lack of transparency and explainability due to LLMs' probabilistic nature. Researchers also face the challenge of rigorously validating outputs and managing the implications of relying on AI-generated insights in scholarly work.
According to the chapter, at which stages of the AI system lifecycle do ethical considerations manifest, and what are the primary concerns at each stage?
According to the chapter, ethical considerations manifest throughout the AI system lifecycle, particularly at the conception and development stages and the deployment stage. In conception and development, the primary concerns are data privacy, including informed consent, safeguarding sensitive information, and mitigating biases in training data. At deployment, the main concern is potential misuse, such as generating disinformation or creating deepfakes. Additionally, the chapter notes that in research contexts, ethical issues also arise during the analytical use of LLMs, including algorithmic bias, transparency, data protection, and reliance on AI-generated insights.
Chapter 5: Systems and Tools to Use NLP and LLMs: Getting Started
What are the main reasons for the lack of transparency in proprietary LLMs like OpenAI's GPT suite?
Proprietary LLMs like OpenAI's GPT suite lack transparency mainly because their stochastic nature makes outputs difficult to trace deterministically, and because developers withhold training data, methods, and procedures to protect competitive advantage. This lack of disclosure limits researchers' ability to audit or replicate model behavior.
Why does the stochastic nature of LLMs make explainability difficult?
LLMs are not deterministic systems; they generate responses based on a statistical representation of language rather than predictable, reproducible rules. This stochasticity makes it difficult to relate inputs to outputs in a predictable way and to explain exactly why specific content was generated, because there is no clear, deterministic pathway from prompt to response.
Chapter 6: Using LLMs in Qualitative Analysis
How do you set up and use the Groq API to query multiple LLMs, based on the example code?
To set up the Groq API, first install the groq library with `pip3 install groq`, create an account at console.groq.com, generate an API key at console.groq.com/keys, and save that key as GROQ_API_KEY in a `.env` file. To query multiple LLMs, define a list of model names, loop over that list, and for each model call `client.chat.completions.create(model=model, messages=prompts)` using a Groq client initialized with the API key. Print each response with `result.choices[0].message.content`.
What are the three NLP tasks integrated in Garcia Quevedo, Glaser, and Verzat's method for relevant data selection, and what is the purpose of each?
The method integrates sentiment analysis to classify expressions by affective tenor, topic modeling to identify underlying data patterns and clustering, and information retrieval to find similarities at the sentence and paragraph levels. These tasks together support exploration, understanding, and relevant data selection before manual inductive analysis.
What are the four main types of AI coding assistants mentioned in the chapter?
The four main types of AI coding assistants are web-based AI development tools, IDE extensions, stand-alone AI-first IDEs, and command line interface (CLI) agents.
Chapter 7: Data Evaluation and Validation
Why is the LLM-based approach for counting word frequencies considered unreliable in the chapter?
The LLM-based approach is considered unreliable because, when it was run three times, it produced word frequency results that differed from one run to another and also differed from the validated output of the standard code notebook, which served as ground truth. The chapter concludes that LLMs can generate plausible language but cannot reliably produce accurate numerical results, so direct LLM output is not dependable for counting tasks.
In the code example, what does the `process_text()` function do, and how does it clean and normalize text?
The process_text() function cleans and normalizes text in a specified dataframe column, typically 'post_text'. It converts each text to lowercase, tokenizes it using NLTK's RegexpTokenizer with pattern r"\w+" to split on word characters and remove punctuation and whitespace, removes English stop words, and filters out single-character terms. It adds a new column 'processed_text' containing lists of cleaned words and returns the modified dataframe.
What is the purpose of the `generate_metrics` function in the provided code, and what metrics does it calculate for each owner?
The generate_metrics function computes per-owner text metrics from the dataframe and returns them as a formatted dataframe. For each owner, it calculates the number of posts, total word count, average words per post, count of non-word/non-space characters treated as emojis, the owner's percentage of total words, and the owner's percentage of total posts.
Chapter 8: Classification
What are the limitations of using LLMs as classifiers in qualitative research, and how can researchers mitigate these limitations?
The main limitations are sensitivity to prompt wording and example order, the constraint on the number of examples imposed by context size, and output variability caused by the model's stochastic nature. Researchers can mitigate these by running multiple executions, validating against manually labeled ground truth, treating LLM classifications as preselectors of relevant data rather than fixed categories, and combining or testing different LLMs and classification techniques.
How do large language models (LLMs) address the limitations of traditional supervised learning for qualitative analysis?
LLMs address the limitation of relying on labeled training data by acting as general-purpose classifiers that use their statistical representation of language and simple prompts instead of supervised training. They also offer flexibility through in-context learning, allowing researchers to classify qualitative data with zero, one, or a few examples and to adapt the classification logic to specific domains.
What are the five assertive impression management behaviors proposed by Jones and Pittman (1982) as described in the chapter?
The five assertive impression management behaviors proposed by Jones and Pittman (1982) are ingratiation, self-promotion, exemplification, intimidation, and supplication, aimed respectively at being liked, looking competent, showing dedication, seeming powerful, and appearing needy.
Chapter 9: Clustering and Topic Modeling
How does topic modeling with LLMs differ from traditional methods like LDA?
Traditional topic modeling methods like LDA rely on word-level counts and statistics, so they miss the semantic structure of sentences, paragraphs, and entire documents. LLM-based topic modeling captures the nuance of human language, producing more coherent and interpretable topics, and can identify complex or specialized language use, though it also brings limitations such as topic instability and hallucinations.
What are the main limitations of using LLMs for topic modeling in qualitative research, as discussed in this chapter?
The chapter identifies several main limitations: topic instability, where each execution produces different topics and assignments; hallucinations that fabricate plausible-looking but nonexistent topics; failure to retrieve topics actually present in the data; model bias reflected in the output; and a lack of genuine interpretability and contextual understanding. These limitations mean LLM-generated topics should be treated as a starting point and require careful human oversight.
In the code example for topic modeling with Gemini, what is the purpose of the `block_size` parameter in the `assign_topics` function, and how does it affect the processing of posts?
block_size determines how many social media posts are grouped into a single batch for topic assignment. The assign_topics function loops through the posts in steps of block_size, builds a sub-dataframe for each batch, and sends that batch to the Gemini API for analysis. This chunking lets the code process large datasets efficiently by handling one manageable block of posts at a time.
Chapter 10: Information Retrieval and Retrieval-Augmented Generation
How does RAG reduce LLM limitations such as hallucinations and outdated knowledge?
RAG reduces hallucinations and outdated knowledge by retrieving current, verifiable information from a specified knowledge base and feeding it into the generative model. This grounds the LLM's response in retrieved evidence rather than relying only on pre-trained data, which may be stale or incomplete.
What is the purpose of the get_embeddings_in_batches() function and how does it handle large lists of texts?
The get_embeddings_in_batches() function generates embeddings for a list of texts by dividing the list into chunks of a specified batch_size and making one API call per chunk. It then collects all returned embeddings into a single list, allowing large lists to be processed even though the embedding model can only handle a limited number of texts per API call.
What is the purpose of the batch_size variable in the code for generating embeddings, and why is batching necessary?
The batch_size variable sets the maximum number of text items sent to the embedding model in each API call during embedding generation. Batching is necessary because the total number of items a user may want to embed can be very large, whereas the model API can only process a limited quantity of text at one time. Setting batch_size to 100 in the example allows the code to be adapted to different needs and API limits.
Chapter 11: Perspectives on LLMs in Management and Qualitative Research
According to the chapter, how can agentic AI be used in qualitative research?
Agentic AI can be used as an autonomous digital research assistant: a researcher gives it a high-level objective, and the agent breaks it down into subtasks and executes a multistep plan, such as gathering press releases, social media data, and news articles, performing sentiment and thematic analysis, and synthesizing findings into a structured report. This automates laborious parts of the workflow and lets the researcher focus on interpreting findings and crafting the final narrative. The chapter also highlights deep research agents, which synthesize hundreds of sources into a single cited report, useful for literature reviews.
What are the key limitations of large language models (LLMs) mentioned in this chapter?
The chapter identifies two key limitations of current LLMs: they are predominantly limited to processing text, unable to natively handle image, audio, or video data, and they have limited capacity for complex, multistep reasoning. These limitations are expected to be addressed by future multimodal and reasoning models.
You may also be interested in
FastAPI: Modern Python Web Development
Bill Lubanovic;
37 questions
Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play
David Foster;
33 questions
AI for the Ordinary_ A Non-technical Playbook for Citizens, Students, and Manage
Unknown
30 questions
AI and ML Techniques in IoT-based Communication
Unknown
27 questions