Schema-Guided Response Generation using Multi-Frame Dialogue State for Motivational Interviewing Systems

TL;DR

This paper introduces a schema-guided method for response generation in motivational interviewing dialogue systems, enabling LLMs to produce responses aligned with MI principles and effectively promote client deliberation.

Contribution

It presents a novel multi-frame dialogue state update and response focus strategy grounded in MI principles for improved dialogue system responses.

Findings

Generated MI-favorable responses successfully.

Encouraged user deliberation with eliciting questions.

System outperformed baseline in user study.

Abstract

The primary goal of Motivational Interviewing (MI) is to help clients build their own motivation for behavioral change. To support this in dialogue systems, it is essential to guide large language models (LLMs) to generate counselor responses aligned with MI principles. By employing a schema-guided approach, this study proposes a method for updating multi-frame dialogue states and a strategy decision mechanism that dynamically determines the response focus in a manner grounded in MI principles. The proposed method was implemented in a dialogue system and evaluated through a user study. Results showed that the proposed system successfully generated MI-favorable responses and effectively encouraged the user's (client's) deliberation by asking eliciting questions.

Tables

Table 1. Table 1: Evaluation of appropriateness of generated responses in MI principles. Competency-Good indicates the threshold for the “good” level of basic competence and proficiency for clinicians. In MITI 4.2.1, there is no threshold defined for MI Non-Adherent behavior.

	MI-FS	MI-Guide	Ours	GT	Competency- Good
% CR (cr/(sr+cr))	54.8	53.7	25.0	35.0	50.0
R:Q ((sr+cr)/q)	0.55:1	0.49:1	1.20:1	1.93:1	2:1
% MI Non Adherent	11.5	15.0	1.4	0.0	n/a

Table 2. Table 2: Proportions of question categories. We also present the results of post-hoc tests on the average differences among the three conditions: MI-FS, MI-Guide, and Ours. The labels ev-* and pl-* represent the aggregated values of question categories that elicit change talk during the evoking process and the planning process , respectively. ( ∗ : p < .05 , ∗ ∗ : p < .01 {*}:p<.05,\>{*}{*}:p<.01 )

		MI-FS	MI-Guide	Ours	GT	Post-hoc test in 3 condition
Q Rate		0.114	0.140	0.357	0.234	Ours>MI-FS**, Ours>MI-Guide**
Elicit Change Talk Q Rate	ev-*	0.009	0.024	0.039	0.050	Ours>MI-FS**, Ours>MI-Guide*
	pl-*	0.012	0.043	0.094	0.031	Ours>MI-FS**, Ours>MI-Guide*, MI-Guide>MI-FS*

Table 3. Table 3: Average number per session of the dialogues obtained in the user study. The numbers in parentheses indicate the standard deviation. We performed a Wilcoxon signed-rank test on the difference between the means of the conditions and calculated the p-value. ( † : p < .1 , ∗ : p < .05 , ∗ ∗ : p < .01 {\dagger}:p<.1,\>{*}:p<.05,\>{*}{*}:p<.01 )

	MI-FS	MI-Guide	Ours	Post-hoc test in 3 condition
Utterances (user+system)	31.8 (1.9)	31.9 (2.2)	32.6 (2.3)	Ours>MI-FS$†$
System sentences	29.5 (2.8)	34.6 (3.0)	28.0 (3.8)	MI-Guide>MI-FS**, MI-Guide>Ours**, MI-FS>Ours*

Table 4. Table 4: List of subjective evaluation questions

ID	Question
Q1	The system responded based on what you said and discussed.
Q2	The system steered the conversation in one direction and rushed to a conclusion.
Q3	The system asked questions that made you think deeply about the current issue.
Q4	You discussed your past efforts regarding the current issue with the system.
Q5	The system proposed ideas or solutions for improving the issue.
Q6	The system frequently asked for your opinions and thoughts.
Q7	The system seemed to encourage you to think for yourself rather than just giving answers.
Q8	When you asked questions, the system often responded with suggestions for improvement.
Q9	The system frequently offered proactive advice or suggestions.
Q10	You often asked the system for ideas.
Q11	The system often asked you for ideas.
Q12	You were able to create an improvement plan based on your own ideas.
Q13	The system asked about your confidence in the plan for improvement.
Q14	You felt motivated to make changes through your interaction with the system.

Table 5. Table 5: Proportion of counselor behavior categories based on MITI (Moyers et al., 2016 ) within system utterance sentences. The numbers in parentheses indicate raw counts. The behavior tags are as follows: gi (Giving Information), q (Questions), sr (Simple Reflection), cr (Complex Reflection), af (Affirm), ea (Emphasize Autonomy), c (Confront), seek (Seek Collaboration), per (Persuasion), pwp (Persuade with Permission), and n/a (not applicable)

	gi	q	sr	cr	af	ea	c	seek	per	pwp	n/a	Total sentences
GT	0.026	0.223	0.281	0.151	0	0	0	0.004	0	0.049	0.266
	(12)	(105)	(132)	(71)	(0)	(0)	(0)	(2)	(0)	(23)	(125)	(470)
MI-FS	0.059	0.112	0.028	0.034	0.032	0	0	0	0.115	0.404	0.217
	(40)	(76)	(19)	(23)	(22)	(0)	(0)	(0)	(78)	(274)	(147)	(679)
MI-Guide	0.018	0.140	0.031	0.036	0.109	0	0	0.004	0.150	0.304	0.208
	(14)	(111)	(25)	(29)	(87)	(0)	(0)	(3)	(119)	(242)	(165)	(795)
Ours	0	0.362	0.325	0.109	0.082	0	0	0.002	0.014	0.036	0.071
	(0)	(233)	(209)	(70)	(53)	(0)	(0)	(1)	(9)	(23)	(46)	(644)

Table 6. Table 6: Definition of question categories and examples of utterances

Question in evoking process:		Questions that elicit current issues and ideas in a process that involves eliciting the person’s own motivation for a particular change.
	Questions to elicit preparation change talk
	Desire (ev-d)	e.g. How would you like things to change?
	Ability (ev-a)	e.g. How confident are you that you can actually do it?
	Resons (ev-r)	e.g. Why do you want more exercise?
	Need (ev-n)	e.g. How important is it to you?
	Other (ev)	Questions that do not fit into the above categories.
Question in planning process:		Questions about specific plans during the process of eveloping a specific change plan that the client is willing to implement.
	Questions to elicit mobilizing change talk
	Activation (pl-a)	e.g. Are you willing to try it?
	Commitment (pl-c)	e.g. Do you want to do it?, Is that what you are aiming to do?
	More specific (pl-d)	e.g. How do you prepare and get ready?
	Setting a date (pl-t)	e.g. When is it likely to be ready?
	Preparing (pl-p)	e.g. What is the first step? What do you need to take with you?
	Other (pl)	Questions that do not fit into the above categories.

Table 7. Table 7: Proportion of occurrences of the question categories defined in Table 6 to the number of full sentences in the system utterances. Numbers in parentheses represent counts. ev-* is the sum of ev-d, ev-a, ev-r, ev-n. ev* is the sum of ev and ev-*. pl-* is the sum of pl-a, pl-c, pl-d, pl-t, pl-p. pl* is the sum of pl and pl-*.

	Process: Evoking							Process: Planning								Total sentences
		Elicit preparatory change talk							Elicit mobilizing change talk
	ev	ev-d	ev-a	ev-r	ev-n	ev-*	ev*	pl	pl-a	pl-c	pl-d	pl-t	pl-p	pl-*	pl*
GT	0.085	0.036	0	0.006	0.004	0.047	0.132	0.055	0.021	0.002	0.013	0	0	0.036	0.091
	(40)	(17)	(0)	(3)	(2)	(22)	(62)	(26)	(10)	(1)	(6)	(0)	(0)	(17)	(43)	(470)
MI-FS	0.032	0.003	0	0.006	0	0.009	0.041	0.059	0.001	0	0.010	0	0	0.012	0.070
	(22)	(2)	(0)	(4)	(0)	(6)	(28)	(40)	(1)	(0)	(7)	(0)	(0)	(8)	(48)	(679)
MI-Guide	0.030	0.011	0	0.013	0	0.024	0.063	0.043	0.003	0.003	0.029	0.001	0.008	0.043	0.86
	(24)	(9)	(0)	(10)	(0)	(19)	(43)	(34)	(2)	(2)	(23)	(1)	(6)	(34)	(68)	(795)
Ours	0.073	0.012	0	0.020	0.006	0.039	0.112	0.152	0.011	0	0.081	0	0.006	0.098	0.250
	(47)	(8)	(0)	(13)	(4)	(25)	(72)	(98)	(7)	(0)	(52)	(0)	(4)	(63)	(161)	(644)

Table 8. Table 8: Example of dialogue with MI Few-shot (MI-FS) dialogue system. ‘Cl’ denotes Client (user) and ‘Co’ denotes Counselor (system). Blue letters in parentheses indicate comments on the corresponding utterance. The original text is in Japanese, and its English translation is provided.

ID	Utterance
1-Cl	I am anemic and I am trying to make a nutritionally balanced meal but it is not easy.
2-Co	Maybe there is something easy or creative I can do to balance my nutritional needs. What are some things you would like to try? (The discussion is only about the plan afterwards without any discussion of the current problems.)
3-Cl	I would like to incorporate iron-rich foods into my cooking.
4-Co	What specific foods would you like to incorporate? Or are you looking for information on iron-rich foods that are easy to use on a daily basis?
5-Cl	I would like to know about iron-rich foods that are easy to use on a daily basis.
6-Co	Spinach, liver, and hijiki are rich in iron and may be easy to incorporate into your daily diet. Please let us know if there are any other ingredients you are interested in. (After providing information, MI should elicit users’ opinions about what they think about the information provided.)
7-Cl	I like all of these ingredients. I usually only use liver in stir-fries, but what other dishes could I make with it?
8-Co	Liver can also be made into patties or added to curry. Let us know if there are other cooking methods you are interested in. (Dialogue becomes a pattern of requesting and presenting information.)
9-Cl	I see. I like patties, so I would like to give it a try. How can I use hijiki in ways other than simmered dishes?
10-Co	Hijiki can be added to salads or used as an ingredient in takikomi-gohan (cooked rice). Please enjoy various variations.
11-Cl	I like takikomi gohan.
12-Co	It is fun to try different dishes, taking advantage of the flavors and textures of the ingredients. Please try it while considering nutritional balance.
13-Cl	Thank you very much. By the way, are there any other nutrients besides iron that I should take while breastfeeding?
14-Co	Calcium, vitamin D, vitamin B complex, and omega-3 fatty acids are also important during lactation. It is best to include a good balance of foods that contain these nutrients.
15-Cl	What kinds of foods contain each of these nutrients?
16-Co	Calcium is found in dairy products and small fish, while vitamin D is found in fish and mushrooms. B vitamins are found in grains and meats, and omega-3 fatty acids are found in blue fish and flaxseed oil.
17-Cl	What are some recipes you recommend using them?
18-Co	How about calcium-rich small fish in nanbanzuke or foil-baked salmon with vitamin D in mind? For omega-3, I also recommend blue fish meuniere or a salad with flaxseed oil.
19-Cl	I like the small fish in nanbanzuke. I would like to know the specific recipe for two people.
20-Co	For a specific recipe, I think it would be better to refer to the Internet or cookbooks, but if necessary, we can go over a list of ingredients together, if necessary.
21-Cl	Understood. What kind of fish do you recommend making it with this time of year?
22-Co	I recommend saury and sardines, which are easy to find. Try using seasonal fish and enjoy the flavors of the season.
23-Cl	… (Subsequent utterances were omitted.)

Table 9. Table 9: Example of dialogue with Principle-based Prompt Guide (MI-Guide) dialogue system. ‘Cl’ denotes Client (user) and ‘Co’ denotes Counselor (system). Blue letters in parentheses indicate comments on the corresponding utterance. The original text is in Japanese, and its English translation is provided.

ID	Utterance
1-Cl	I’m anemic and I’m trying to cook a nutritious and balanced diet but it’s not easy.
2-Co	That is an important goal. Can you tell us a little more about what reasons you have for wanting to cook nutritionally balanced meals?
3-Cl	I am currently breastfeeding and often feel anemic, dizzy, and sick.
4-Co	It is a difficult situation for you while breastfeeding. What changes would you like to see in your life if your anemia improves?
5-Cl	I want to be able to do my chores and go out with energy.
6-Co	That is really important. Would you consider starting with small steps to stay well?
7-Cl	Yes …
8-Co	Shall we work together to come up with a meal plan that is easy to start with? What foods and dishes do you like?
9-Cl	I like pasta.
10-Co	Coming up with nutritious recipes using pasta is a good start. You can make your meal more balanced by incorporating vegetables and iron-rich foods. What ingredients would you like to use together? (While the discussion concerns improvements to meal planning, the system takes a dominant role in determining the content, which prevents the dialogue from being grounded in the user’s own thoughts.)
11-Cl	I would like to use spinach.
12-Co	Spinach is a good choice because it is rich in iron. What other ingredients would you like to try and what flavors do you like? Based on that, let’s come up with a more specific pasta recipe.
13-Cl	I often make cream pasta with spinach and bacon.
14-Co	That sounds delicious! To add more iron and nutrients to your cream pasta with spinach and bacon, you might try adding beans or nuts, for example.
15-Cl	I would try crushed walnuts or almonds as a topping.
16-Co	That is a great idea! Walnuts and almonds are very nutritious, so I think that would be a plus. As you continue to get a little more creative, you can gradually improve your anemia.
17-Cl	Are there any other recipes you would recommend using spinach?
18-Co	For recipes using spinach, we recommend the spinach and cheese omelette. Also, soups with spinach and tofu are a good and nutritious choice. If you are interested, we can work together to come up with a more detailed recipe? (By presenting attractive solutions in response to user requests, subsequent dialogue tends to be biased toward advice.)
19-Cl	I would love to try the spinach and cheese omelet.
20-Co	It looks delicious! Spinach and cheese omelette is nutritious and easy to make. Shall I help you come up with a specific recipe?
21-Cl	Please do.
22-Co	Let’s work together to come up with a specific recipe for a spinach and cheese omelet. First, how much do you want to make and what kind of cheese do you prefer?
23-Cl	I would like to make enough for two people. I like all kinds of cheese, but I especially like mozzarella.
24-Co	That is a great choice! Let’s come up with a recipe for a spinach and cheese omelette with mozzarella. We can talk about the ingredients and steps needed, and we can make it together.
	… (Subsequent utterances were omitted.)

Table 10. Table 10: Example of dialogue with the proposed system (Ours). ‘Cl’ denotes Client (user) and ‘Co’ denotes Counselor (system). Blue letters in parentheses indicate comments on the corresponding utterance. The original text is in Japanese, and its English translation is provided.

ID	Utterance
1-Cl	I am anemic and I am trying to make a nutritionally balanced meal but it is not easy.
2-Co	What exactly do you find difficult about not being able to cook nutritionally balanced meals?
3-Cl	First of all, I don’t know what kind of ingredients to use. Also, I am tired from raising my child, so I am not able to cook elaborate meals.
4-Co	So you find it difficult to cook nutritionally balanced meals because you don’t know what ingredients to use and you are too tired to cook elaborate meals while you are raising your children.
5-Cl	Yes, I do.
6-Co	In addition to not being able to spend much time cooking, not knowing what ingredients to choose is a hurdle to creating nutritionally balanced meals, isn’t it?
7-Cl	Yes.
8-Co	While you find it difficult to cook elaborate meals and to choose the right ingredients, how much do you feel the need to improve that situation? (Ask about the importance of the issue.)
9-Cl	I would like to improve it if I could. I frequently feel anemic and sick.
10-Co	So one of your problems is that anemia occurs frequently and makes you feel sick. So you are hoping that a particularly nutritious and balanced diet will help solve that problem?
11-Cl	Yes, I do. I would like to cook to the point where it is not a burden.
12-Co	So you would ideally like to cook nutritionally balanced meals, and you would like to do so in a way that is not burdensome.
13-Cl	Yes, I do.
14-Co	Specifically, in what ways do you hope to improve your anemia and nutritional balance issues? What kind of improvement ideas do you think would not be burdensome?
15-Cl	First, I would like to know what foods can help prevent anemia. Then, I would like to know simple recipes that I can make with them on a daily basis.
16-Co	So you want to know what ingredients help prevent anemia and what simple recipes you can make on a daily basis. By doing so, you feel that you would like to make nutritionally balanced meals without being overburdened by them. (Reflect on the content of the dialogue when a client requests advice.)
17-Cl	Yes, I would like to know what ingredients are available and how to find recipes for those ingredients.
18-Co	How would you like to find recipes for these ingredients? Can you tell us specifically how or where you are trying to find them?
19-Cl	I’ve downloaded an app that offers recipes, and I’d like to use that.
20-Co	That is a great way to proceed. I think using an app to find recipes that are easy to make will help me get closer to a nutritionally balanced diet while reducing my workload.
21-Cl	Thank you.
22-Co	I think that by using the app to find simple recipes that can be made on a daily basis, it will make it easier to incorporate meals that help prevent anemia, which will in turn make it easier to achieve a good nutritional balance. (Reflection and summarization may result in repeated responses that are similar to the previous information (20-Co).)
23-Cl	What should I do when cooking feels like a chore?
24-Co	When you feel that cooking is a hassle, what kind of situation often causes you to feel that way? Can you tell us about specific episodes? (Client is asked for advice, but does not give advice but elicits more on the issue.)
25-Cl	It is often when I am busy raising my children or when I am tired. At those times, I end up taking out fast food.
26-Co	You often feel that cooking is a hassle when you are busy or tired in raising children. I understand that you sometimes end up taking out fast food in such situations.
27-Cl	Yes, I do.
28-Co	You often feel that cooking is a hassle, especially when you are busy or tired from raising your children, which is why you choose fast food. Your goal is to incorporate nutritionally balanced meals that are not too burdensome.
29-Cl	Yes, it is.
30-Co	I would like more information on easy-to-prepare recipes and ingredients that help with anemia. What specific dishes and ingredients are you interested in learning more about?
31-Cl	I would like to know about fish dishes because I don’t have a large repertoire of fish dishes.
32-Co	You want to increase your repertoire of fish dishes and aim for a nutritionally balanced diet. It would be nice if you could help me to prevent anemia.
33-Cl	I would like to do so.
34-Co	It is a great idea to work on increasing your repertoire of fish dishes while aiming to create nutritionally balanced meals.

Full text

\pdfcolInitStack

tcb@breakable

Schema-Guided Response Generation using Multi-Frame Dialogue State for Motivational Interviewing Systems

Jie Zeng

Seikei University

job.jiezeng@gmail.com

        

Yukiko I. Nakano

Seikei University

y.nakano@st.seikei.ac.jp

1 Introduction

Motivational interviewing (MI) is defined as “a collaborative, goal-oriented style of communication … designed to strengthen personal motivation for and commitment to a specific goal by eliciting and exploring the person’s own reasons for change…” Miller and Rollnick (2012). The definition clearly states that MI is a goal-oriented dialogue aiming to motivate the clients to change their behavior and this goal should be achieved by eliciting statements towards change from the client. Therefore, MI should prioritize understanding the client rather than talking from the counselor’s viewpoint (Wu et al., 2023, 2022), and high-quality MI is characterized by more Questioning and Reflecting, whereas unfavourable responses include Advising without permission and Confronting (Pérez-Rosas et al., 2019; Welivita and Pu, 2023).

Growing public interest in MI has led to the development of AI chatbots addressing smoking cessation, alcohol abstinence, and dieting, using rule-based and LLM-based approaches (Fitzpatrick et al., 2017; Inkster et al., 2018; Welch et al., 2020; Mousavi et al., 2021). To enhance the performance of MI chatbots, previous studies have proposed to generate MI-favorable responses, such as reflection (Shen et al., 2020, 2022) and Advice with permission (Welivita and Pu, 2023). Another research direction focused on generating contextually appropriate responses by considering the state of the dialogue (Park et al., 2019; Steenstra et al., 2024). Following this direction, this study aims to improve response generation in MI by enhancing dialogue state representation and dialogue strategies.

As shown in Figure 1-a to c, in professional MI, multiple interrelated topics often emerge, such as the client’s goal (e.g. “I want to lose weight”) and challenges and obstacles to achieve the goal (e.g. “I eat too much,” “I feel like I’m not getting enough exercise”). To develop a dialogue system capable of such MI interaction, we will employ schema-guided approach (Rastogi et al., 2020b; Li et al., 2023; Zeng et al., 2023): the dialogue state is represented as multiple frames, each of which consists of a set of slot-value pairs, and dialogue strategy is also represented as a frame and guides LLMs to generate MI-favourable responses. Previous dialogue generation studies that employed schema-guided approach (Rastogi et al., 2020a, b; Chen and Yang, 2020; Mehri and Eskenazi, 2021; Li et al., 2024) focused on dialogue state tracking (DST) within specific and predefined topics and domains. Although MI is goal-oriented communication Miller and Rollnick (2012), unlike DST, it requires flexible dialogue management with various alternative topics. To address this challenge, we propose a method for producing dialogue strategies, which include focused topic and content, by utilizing MI knowledge retrieved from an MI corpus.

Contributions. Employing schema-guided approach, this study will 1) propose a method for updating dialogue state using multi-frame representation, 2) introduce a mechanism for producing frame-based dialogue strategies that guide LLMs to generate responses aligned with MI principles, and 3) conduct a user study to evaluate multi-turn interaction between human users and our MI system, investigating the effectiveness of the proposed methods from the MI perspective.

2 Related Work

2.1 Response Generation in MI

Research on response generation for MI has addressed two main challenges: generating MI-favourable responses and generating contextually appropriate responses.

For generating MI-favourale responses, specifically for reflection generation, previous studies proposed fine-tuning and prompt-engineering approaches (Wu et al., 2023; Basar et al., 2024) and also attempted to use knowledge graph (KG) of medical domain (Shen et al., 2020, 2022; Deng et al., 2023). There was also a study that generated Advice with permission using rephrasing technique by LLMs (Welivita and Pu, 2023).

Another direction of previous research has addressed the generation of appropriate counselor responses by considering the state of the dialogue. Early studies employed rule-based approach, defining state transition rules which consist of client state as precondition and counselor’s response as the next action Schulman et al. (2011). Park et al. (2019) developed an MI chatbot that generates questions based on the state of the MI session, as defined by the four-stage framework of Motivational Interviewing: engaging, focusing, evoking, and planning (Miller and Rollnick, 2012). More recently, advances in prompt engineering for LLMs have enabled prompt-based approaches to MI dialogue generation. These methods incorporate dialogue history and MI principles into prompts to guide response generation (Steenstra et al., 2024). Some studies adopt a two-step prompting process consisting of dialogue act prediction and response generation (Sun et al., 2025), while others use LLMs to generate dialogue strategies, defined as pairs of desired counselor behaviors and their applicable contexts, which are then incorporated into prompt instructions (Xie et al., 2024). Although there studies considered intent categories and user state in response generation, their contextual information is still required to be improved by elaborating and structuring the representation.

2.2 Dialogue Representation and Management

Previous studies have demonstrated the usefulness of dialogue strategies and policies that contain the intention and semantic content of system responses for effective dialogue management (Moon et al., 2019; Li et al., 2023). In particular, in task-oriented dialogue (TOD), the process of updating domain-specific schemas composed of slot-value pairs is known as Dialogue State Tracking (DST). In TOD, response generation is carried out while predicting and updating the dialogue state (used as dialogue control information). Recently, methods leveraging prompt engineering with LLM have been proposed to perform zero-shot DST and response generation, showing substantial performance (Stricker and Paroubek, 2024; Li et al., 2024).

On the other hand, dialogues such as negotiation and emotional support have explicit goals, but the information that needs to be elicited from the user cannot be predefined and often varies depending on the user. Generating such dialogues is a challenging task, as dialogue systems require flexible dialog control. To address this, several models have been proposed that learn dialogue policies using annotated dialogue corpora through supervised fine-tuning (SFT) or reinforcement learning (RL) Deng et al. (2024); He et al. (2024). However, these approaches require large-scale annotated dialogue datasets and model training using a simulator. In contrast, this study proposes schema-guided dialogue control that leverages prompt engineering with LLMs, enabling effective dialogue management in low-resource settings using only a small amount of annotated dialogue data.

3 Methodology

Figure 2 illustrates the overall architecture of the proposed mechanism and the process of the integrated dialogue system. First, upon receiving user (client) input, the multi-frame dialogue state is updated through a dialogue state tracking mechanism (Section 3.1). Next, using the updated dialogue state and dialogue history as input, a dialogue strategy is generated via a strategy generation module grounded in MI knowledge (Section 3.2). Finally, based on the generated strategy, a system (counselor) response is produced (Section 3.3).

3.1 Multi-Frame Dialogue State

3.1.1 Dialogue State Representation

To represent the state of MI dialogues in a structured yet flexible manner, each topic is represented as a frame, and the dialogue state is defined as the set of all frames that have emerged during the conversation. This dialogue state provides the foundation for managing the interaction. Each frame is characterized by the types of attributes it contains. Based on an analysis of an MI corpus grounded in MI theory, we identified four schema types—Goal, Problem, Experience, and Plan—as templates for frames corresponding to the major topic categories in MI dialogues (see Figure 6). Every frame must include two core attributes: frame_type, which denotes the type of frame, and content, which provides a summary. In addition, all schema types except for Goal include a detail attribute that describes more specific information. Some frame types may also contain attributes unique to their schema. For instance, the Problem schema includes harm_effect, which describes the negative consequences of the problem, and necessity_to_improve, which indicates the perceived need for improvement regarding the problem. For example, in the dialogue segments a–c shown in Figure 1, one Goal frame and two Problem frames are generated.

3.1.2 Dialogue State Update Mechanism

The left part of Figure 2 illustrates the flow of updating the dialogue state, which is represented as a set of frames. We leverage an LLM to produce new frames and update the dialogue state.

The LLM receives the current dialogue state in addition to the dialogue history comprising the client’s latest utterance and five preceding ones (six in total). Based on this input, the LLM generates an updated dialogue state. At this point, we adopt a few-shot approach in which multiple samples of dialogue states are provided to the LLM (the prompt is shown in Figure 7). Specifically, we manually constructed and provided 10 input-output pairs as few-shot examples, where each input consists of a dialogue history and its corresponding pre-updated dialogue state, and the output is the updated dialogue state. These examples were created using an MI dialogue sample from our MI corpus (details are in Appendix B.1).

However, the LLM-generated dialogue states sometimes exhibit problems, such as omitting frames that were present in the previous dialogue state or redundantly adding new frames, resulting in duplicate values already registered in the dialogue state. To address this problem, we implemented a post-processing step that identifies the differences between the previous and generated dialogue states and merges the differing parts into the original dialogue state. (For details on the prompt and implementation, see Section A.1.)

3.2 Generating Dialogue Strategies based on MI Knowledge

The dialogue strategy decision module determines the “dialogue strategy,” which guides the generation of the next counselor response based on the dialogue history and the current dialogue state.

3.2.1 Dialogue Strategy Representation

A dialogue strategy consists of a response intent and a set of attributes along with corresponding values that represent information to be referred in the response. The attributes are defined below.

intent:

Represents the communicative intent of the response. Five types of intent are defined: four corresponding to core techniques in MI –Question, Affirmation, Reflection, and Summarization – as well as an additional Other category.

focuses:

Indicates references to the frames within the multi-frame dialogue state that are the focus of the response. Multiple frames can be referenced. To specify these references, we use a combination of the frame type, an index identifying the specific frame, and an optional frame_attributes that denote a particular attribute within the target frame.

seek_frame_type, seek_attribute:

Specify the type of information to be elicited from the client. To elicit different problems or new plans, set the seek_frame_type to the desired frame type. To request detailed information about a specific frame, assign the name of the required attribute to the seek_attribute slot.

For example, in Figure 2, a counselor strategy to explore the effect of an action taken by the client can be represented as: [intent = “Question”, focuses = [(references to the relevant dialogue state)], seek_frame_type = “experience”, seek_attribute = “effect”].

3.2.2 Dialogue Strategy Decision Mechanism

As illustrated in Figure 2, the dialogue strategy decision mechanism retrieves dialogue strategies from a pseudo-strategy pool. It selects strategies corresponding to the dialogue state similar to the current one and provides them to the LLM as few-shot examples. Based on these examples, the LLM generates a genuine dialogue strategy for the current context.

■ Creating a Dialogue Strategy Pool

The dialogue strategy pool is a collection of pseudo-strategies created using our MI corpus (details are in Appendix B.1). Each sample is required to conform to the specifications described in Section 3.2.1. However, manually creating dialogue strategy samples through human annotation is costly. To address this, we leveraged an LLM to generate pseudo dialogue strategies, which were then included as samples in the strategy pool. Figure 3 illustrates the process for generating pseudo dialogue strategy samples. First, we extracted pairs consisting of a target counselor response and the preceding five-turn dialogue history from MI dialogues in the corpus. We then obtained the dialogue state corresponding to each dialogue history using the method described in Section 3.1.2. These inputs were fed into the LLM to generate a pseudo dialogue strategy. The prompt used in this generation is shown in Figure 8. In our MI dialogue corpus, each counselor utterance is annotated with an intent label, which we directly used as the intent attribute value. The remaining attributes were generated by the LLM.

Using this method, we created 524 pseudo dialogue strategy samples from five MI dialogues. Each sample was stored in the dialogue strategy pool along with the input information used during generation—namely, the dialogue history and corresponding dialogue state. (For details on the prompts and implementation, see Section A.2.)

■ Generating a Dialogue Strategy

To determine a genuine dialogue strategy, our method first retrieves several pseudo-strategy samples from a pool, whose associated dialogue states and preceding utterances are most similar to the current dialogue state and utterances. These retrieved samples are then used as dynamic few-shot examples for prompting the LLM to generate a genuine dialogue strategy. As shown in the right side of Figure 2, the system constructs a text input by concatenating the current dialogue state with the dialogue history consisting of the five most recent utterances. This input is fed into a pre-trained encoder-based language model to obtain an embedding, which serves as the query embedding. Similarly, for each sample in the dialogue strategy sample pool, we precompute embeddings for each pair of dialogue state and history by concatenating them. Using the query embedding, we retrieve the top-N samples from the dialogue strategy pool based on cosine similarity (we set N = 5 in our experiments). The dialogue states, histories, and pseudo dialogue strategies of the retrieved samples are used as few-shot examples. These are formatted according to Figure 9 and input into the LLM to generate the genuine dialogue strategy. (See Appendix A.3 for implementation and prompt details.)

3.3 Response Generation

The response generation process takes as input a frame-based dialogue strategy, which is produced by the dialogue strategy decision mechanism (Section 3.2.2), and generates a textual response using an LLM prompt (see Figure 10).

However, LLM-generated dialogue strategies may contain slot inconsistencies. For instance, strategies that contain both Summarization and seek_frame_type = "experience", which implies information seeking, are inconsistent. To address this, we include prompt instructions to prioritize relevant attributes during generation. Specifically, when the intent of the dialogue strategy is Reflection or Summarization, the prompt instructs the model to emphasize the information focused on the current dialogue state, and to generate a response that reflects on that content. Conversely, if the intent is Question or Other, and if attributes such as seek_frame_type and seek_attribute are present, the prompt directs the LLM to generate a question that elicits information about the specified attribute from the user. (See Appendix A.4 for prompt and implementation details.)

4 Experiments

4.1 Baselines

To evaluate the performance of the proposed methods, we implemented them in a dialogue system and conducted a user study in which participants engaged in text-based conversations with the system. We prepared two baselines for comparison. As prompt-based methods for response generation have demonstrated strong performance in recent years (Wu et al., 2023), we adopt two dialogue systems that leverage prompt techniques as our baselines.

MI Few-shot (MI-FS): We include fragments of MI dialogues as examples in the prompt, along with the current dialogue history, to generate a follow-up counselor response using LLM. Specifically, following the experimental setup for reflection generation in MI by Wu et al. (2023), each dialogue sample consists of 15 turns history between the counselor and the client, followed by a counselor response. Five such samples are included in the prompt.

Principle-based Prompt Guide (MI-Guide): Following Steenstra et al. (2024), we built a system that employs a zero-shot approach, using a detailed prompt that specifies the behavior of an MI counselor. The prompt also instructs MI principles, including the four-stage framework and core skills (see Figure 12). Additionally, the system is provided with the dialogue history to generate the counselor’s next response.

Details of the prompts and implementation used in the two baseline systems mentioned above are provided in Appendix B.2.

4.2 Implementation Details

The dialogue samples used in both the MI-FS condition dialogue system and the proposed system were created from a conversation corpus between professional MI counselors and lay participants discussing concerns related to diet and eating habits. For the MI-FS condition, we constructed five dialogue samples for few-shot learning using dialogue data from four sessions conducted by different MI counselors. In the proposed method, one session from the dataset was used in the prompt for dialogue state update (Section 3.1.2), while a pool of dialogue strategy samples was created from five other sessions using the method described in Section 3.2.2.

We adopted GPT-4o (gpt-4o-2024-08-06) as the large language model (LLM) for all experimental conditions because it supports function calling, enabling the handling of information in different formats, such as structured data like dialogue states, and the generation of outputs in a specific structure.

4.3 User Study

Participants recruited through crowdsourcing engaged in text-based chat in Japanese with web-based dialogue systems under three different conditions. They were instructed that their task was to chat with a dialogue system in which an AI plays the role of a counselor and to evaluate the conversation from the perspective of MI. Consent to participate in the experiment was obtained before the experiment. Participants were instructed to act as the client and input at least 15 utterances (with one user input counted as one utterance), ensuring each dialogue contained a minimum of 30 utterances in total, including both client and counselor contributions. They could end the conversation at any point after meeting this requirement. Following the dialogue session, participants completed a 14-question survey. Each participant completed both the dialogue task and the survey for all three conditions. Participants engaged in MI sessions with AI focused on dietary habits and improvement, consistent with the MI corpus used in earlier sessions designing our methods. Prior to the experiment, they were asked to identify a concern they wished to discuss and to engage in a dialogue with each system without substantially changing that concern. (The details of the experiments are presented in Appendix B.3)

5 Evaluation Results

We analyzed impression ratings obtained from the survey and interaction logs of the dialogue systems.

5.1 Subjective Impression

After interacting with the dialogue system for each experimental condition, participants completed a 14-question survey using a 7-point Likert scale (1: Strongly disagree to 7: Strongly agree). We conducted Wilcoxon signed-rank tests for each item to examine differences in ratings between conditions. Only items with statistically significant differences are shown in Figure 4, while the complete list of questionnaire items and results is provided in Table 4 and Figure 15.

For questions related to the system’s proposing ideas or suggestions (Q5, Q8, Q9), both MI-FS and MI-Guide had significantly higher average ratings than our proposed system (Ours), suggeting that MI-FS and MI-Guide were perceived as more active in offering ideas and suggestions.

In contrast, for items related to encouraging users to think rather than providing direct answers (Q7) and prompting users to share their own thoughts or ideas (Q6, Q11), Ours received higher average ratings than MI-FS and MI-Guide. This suggests that Ours was perceived as more active in eliciting users’ own thinking, even when users requested advice.

Furthermore, Ours received lower average ratings for responding in alignment with the user’s input (Q1), asking about the user’s current issues (Q3), and motivating users to change through dialogue (Q14). It also received higher ratings for leading the conversation in a single direction or rushing to a conclusion (Q2). These results suggest that, while Ours prioritized eliciting users’ thoughts, it may have been perceived as less empathetic. Nevertheless, we believe that the MI principle that encourages the user’s deliberation has been successfully realized.

5.2 Analysis of System Response

This section analyzes system responses from two perspectives: adherence to MI principles and a detailed classification of question types.

5.2.1 Analysis for adherence to MI principles

We evaluated system responses using the Motivational Interviewing Treatment Integrity (MITI) framework, which assesses how well counselor behavior aligns with MI principles and skills. Counselor utterances were classified into 11 categories: the ten behavior codes defined in MITI 4 (Moyers et al., 2016), plus a "not applicable (n/a)" category. Annotation details are provided in Appendix C.2. Based on the MITI annotations, we computed three alignment metrics to evaluate counselor behaviors in MI. Frequencies and proportions for each MITI category are shown in Appendix Table 5.

%CR The proportion of Complex Reflections among Relections. Complex Reflection is a type of reflective listening response that adds meaning or emphasis to what the client has said, rather than slightly rephrasing it (Simple Reflection).

R:Q The ratio of Reflections to Questions.

%MI Non-Adherent The proportion of responses inappropriate as MI behavior (such as Persuasion or Confrontation) according to the MITI.

The results of these metrics are shown in Table 1. GT refers to the results of analyzing professional counselor utterances in four dialogues selected from the MI conversation corpus (Appendix B.1). Additionally, MITI defines two thresholds for assessing clinicians’ basic competence and proficiency: Fair and Good. The table displays the threshold values for the Good level.

Regarding the %CR, both MI-FS and MI-Guide exceeded 50%, approaching the threshold for the Competency–Good level. In contrast, the %CR of Ours was closer to that of GT. For the R:Q ratio, MI-FS and MI-Guide fell below 1, while Ours exceeded 1. Given that both the GT value and the threshold for a Good level are around 2, our system’s R:Q ratio aligns more closely with MI standards. Regarding the %MI-Non-Adherent metric, Ours achieved a low rate of 1.4%, similar to GT, whereas MI-FS and MI-Guide exceeded 10%. This discrepancy is likely due to the frequent unsolicited advice without user requests.

5.2.2 Analysis of Question Types

One of the primary goals of MI is to elicit change talk—client utterances that express motivation for behavioral change. Therefore, generating questions that elicit client’s change talk is crucial for supporting user behavior change (Miller and Rollnick, 2012). Based on the four-stage framework in MI Miller and Rollnick (2012), we counted the system-generated questions in evoking process (the stage focused on eliciting the client’s intention to change) and planning process (the stage aimed at creating a concrete plan for change). For example, the evoking process includes questions about the client’s current desires and reasons for change, while the planning process includes questions about client’s preparedness and commitment to the concrete action plan. If a question is identified as eliciting change talk, it is further annotated with one of the nine subcategories for detailed analysis (see Appendix C.2 for category details and annotation).

Table 2 shows the proportion of questions, while the frequencies for all question categories are provided in Table 7. In the ‘Elicit Change Talk Q Rate’, ev-* indicates the proportion of questions that elicit change talk out of the total number of sentences in the system responses during the evoking process. Likewise, pl-* indicates the proportion of such questions within the planning process.

We conducted Wilcoxon signed-rank tests to compare MI-FS, MI-Guide, and Ours. The results show that Ours achieved significantly higher question rate, including a higher proportion of questions that elicit change talk, compared to both MI-FS and MI-Guide. These findings indicate that our method not only generates more questions overall but also more effectively promotes change talk than the baseline approaches.

Figure 5 shows the question ratios over time, based on annotation results. Every three consecutive system utterances were grouped into a single bucket, resulting in seven buckets that together covered all system utterances from the start of the dialogue. The questions were classified as either evoking process questions (ev) or planning process questions (pl). We applied the same procedure to four dialogues from our MI dialogue corpus, which contains professional counselor utterances. The results, labeled as GT, were shown for comparison with the three experimental conditions (Appendix B.1).

As shown in Figure 5, the results indicate that MI-FS and MI-Guide had low proportions of questions in the middle and latter half of the dialogues. Additionally, ev appeared primarily in the early part of the diallgue, and were rarely observed after the midpoint of the dialogue. This suggests that in baseline systems, most questions focused on future planning (pl), rather than encouraging deeper exploration of the user’s current situation.

Even though Ours has a lower proportion of ev compared to GT, it still appears at an average of 0.1 throughout the dialogue, indicating that it asks about the client’s current thoughts and efforts in every situation and has a dialogue tendency similar to GT. Although Ours had a lower proportion of ev compared to GT, it maintained an average of 0.1 throughout the dialogue. This indicates that it consistently asked about the client’s current thoughts and efforts, demonstrating a dialogue style similar to GT.

6 Conclusion

In this study, we proposed a dialogue system aiming to guide the user’s deliberation and behavioral change through Motivational Interviewing. The system treats various topics discussed in the dialogue as distinct frames and manages them collectively as the dialogue state. The dialogue system also incorporates a mechanism for selecting appropriate dialogue strategies by referencing professional counselors’ strategies according to the dialogue context. Through multi-turn dialogue experiments comparing our method with existing prompt-based approaches, we demonstrated that our system reduces the frequency of MI-non-adherent behaviors and increases the proportion of questions. As a result, it effectively elicits user thoughts and promotes contemplation.

Limitations

In this study, we defined and represented a finite set of frame types and attributes based on the principles of MI and observations of actual dialogue data. We also utilized four categories to represent the intent of system responses. However, the validity of the representation of the dialogue state handled in this study is not sufficiently verified for the frames that constitute the dialogue state, and the types of attributes could be extended through automatic acquisition of attributes (Finch et al., 2024). In addition, counselor responses often convey a wider range of intentions, suggesting the need for further expansion of dialogue strategy.

Moreover, this study limited the scope of MI conversations to dietary concerns when evaluating the proposed methods. Therefore, further extensions of the dialogue system are necessary to cover wider themes addressed in MI.

Finally, the experiments conducted in this study involved interactions with participants that were completed in a single day, and the results were based on only one session of dialogue. To achieve the fundamental goal of MI, which is to promote behavioral change, it is essential to gradually enhance the client’s intrinsic motivation through sustained and repeated interactions. Consequently, future work should include empirical validation through long-term dialogues and the design of dialogue systems that account for extended interactions.

Ethics Statement

This section discusses the ethics relevant to this study.

Ethical Review

Before conducting the experiments, we consulted with the ethics board of our institution regarding the collection of MI dialogue data from general participants and obtained their approval.

Risks

We developed a dialogue system that plays the role of a counselor in MI dialogues. Experimental results showed that, although the proposed system gave participants a weaker impression of being motivated toward change compared to the baseline systems, it may still influence participants’ behavior. Moreover, there is a possibility that the generation process using the language model we employed includes inappropriate advice, unsolicited guidance, hallucinations, or responses that are not aligned with MI principles. Furthermore, the system has not been validated by MI experts. Therefore, our system, and AI more broadly, is not at a stage where it can replace human MI counselors.

Accordingly, developers of systems that perform MI must recognize the potential harms and risks involved. It is also essential to inform users who interact with such systems of these risks in advance and obtain their informed consent.

Information and Consent

The user study was conducted via crowdsourcing. Workers who applied for the task were shown a disclaimer on the instruction page before interacting with the MI systems, stating that the experiment would involve a dialogue with an AI system and that the system’s responses might contain errors or inappropriate content. Their consent was obtained at this stage. Additionally, on the questionnaire page following each system session, participants were asked to consent to the use of the collected data solely for research purposes. Furthermore, participants were provided with an opt-out option if they did not wish their dialogue data to be shared externally, even within the bounds of research use.

Appendix A Details of the Proposed Methods

We present below the prompts and generation details used in our proposed system.

A.1 Update Dialogue State

Figure 6 illustrates the structure of the dialogue states targeted in this study, represented in Pydantic format. The dialogue state comprises four frame types: GoalAndIdealFrame, ProblemAndTroubleFrame, ExperienceFrame, and ImprovementPlanFrame. These are integrated into a single class named Dialogue_State. By utilizing this structural definition in conjunction with the function calling capability of the LLM, we can obtain outputs that adhere to the intended dialogue state structure. The prompt used for extracting dialogue states is shown in Figure 7. To improve extraction accuracy, we annotate each MI session conducted by a professional with the following elements: the dialogue state immediately before the interaction (<Current_DS>), the two most recent utterances from the counselor and the client (Utterance), and the updated dialogue state after the interaction (<Updated_DS>). A total of 10 such annotated samples are prepared and presented in the Examples section. Information necessary for the current dialogue state extraction is given as the “User Message”, while the <Current_DS> field contains the immediately preceding dialogue state. The field embeds the recent dialogue history, consisting of the six most recent utterances between the counselor and the client, formatted as “Counselor:” and “Client:”. A sampling temperature of 0 was used during generation to maximize determinism in the output.

A.2 Creating Pseudo Dialogue Strategy Samples

This section details the generation process of pseudo dialogue strategy samples used as few-shot examples during dialogue strategy generation. Figure 8 presents the prompt used to generate pseudo dialogue strategies and the structure of the resulting strategies in Pydantic format. To identify the dialogue strategy employed in a counselor’s surface-level response without relying on manual annotations, we utilize utterances from professional MI dialogue datasets. Specifically, we input the counselor’s response (<Counselor_Response>), the preceding five utterances as the dialogue history (<History>), and the dialogue state derived from this history (<Dialogue_State>) into the LLM, which then generates a pseudo dialogue strategy corresponding to the counselor’s response.

Since the system at inference time selects a dialogue strategy that includes an intent attribute, it is desirable for the pseudo dialogue strategy samples to also contain this attribute. However, because the counselor responses in the MI dataset are already annotated with response categories, we leverage this category information as a proxy for the intent. Therefore, the generated pseudo dialogue strategies exclude the intent attribute from their structure. The sampling temperature during generation was set to 0.

A.3 Determine Dialogue Strategy

Figure 9 presents the prompt used to determine the dialogue strategy, as well as the schema (in Pydantic format) of the output dialogue strategy. Unlike the pseudo dialogue strategy generation shown in Figure 8, the structure generated by the LLM includes an additional intent attribute. Accordingly, the DialogueStrategy in Figure 9 contains this intent field.

In the prompt, five similar examples are retrieved from a pool of pseudo dialogue strategies and presented in the “Examples” section as references for strategies used in similar dialogue situations. Following this, the current dialogue context is embedded into the prompt: five utterances between the client and counselor are included in the <History> section, and the corresponding dialogue state is provided in <Dialogue_State>.

To retrieve dialogue situations similar to the current one from the pool of pseudo-dialogue strategies, we used the following template: “### History\n<History>\n\n### Dialogue_State\n<Dialogue_State>”. The dialogue history of five utterances is embedded in <History> and the dialogue state is embedded in <Dialogue_State>. To obtain text embeddings, we employ a RoBERTa-based XLM-R Longformer model111https://huggingface.co/markussagen/xlm-roberta-longformer-base-4096, which is capable of handling long input sequences comprising both dialogue state and history. XLM-R Longformer is released under the Apache 2.0 license and was adopted in this study to obtain multilingual embedding representations in accordance with its intended purpose. The sampling temperature was set to 0 when generating the dialogue strategy.

A.4 Response Generation

Figure 10 shows the prompt used for system response generation. In the <History> section, we embed the most recent six utterances from the dialogue history between the client and the counselor, including the client’s latest utterance. The <Dialogue_State> section contains the dialogue state, along with the pre-determined dialogue strategy elements: intent, focuses, seek_frame_type, and seek_attribute, which are inserted into their respective placeholders. The sampling temperature during generation was set to 1.0.

Appendix B Details of Experiments

B.1 Dataset

We describe the details of the professional MI dialogue dataset used in our proposed method and the few-shot baseline. In our previous work, we collected approximately 20-minute counseling dialogues in Japanese conducted by four professional MI counselors with lay individuals as clients. The conversations focused on concerns related to diet and eating habits. The dataset is not publicly available at this time. We transcribed the collected dialogues and annotated each utterance based on the Motivational Interviewing Skill Code (MISC) (Miller et al., 2003).

B.2 Baseline Details

Details of the two baselines used in the experiment are provided below.

**MI Few-shot (MI-FS):

**The prompt used for MI Few-shot (MI-FS) is shown in Figure 11. Following the setup of Wu et al. (2023), we construct five samples of professional MI dialogues, each consisting of a dialogue history of fifteen utterances between a client and a counselor, followed by the counselor’s response. These samples are inserted into the prompt without randomizing their order as few-shot examples. In the same format as the samples, the current dialogue history is appended, and the system response is generated using the LLM. When selecting the dialogue samples for few-shot prompting, we ensured that the counselor’s responses included the same type of intent (Question, Affirmation, Reflection, or Summarization) as required by the proposed system.

**Prompt used for MI Principle-based Prompt Guide (MI-Guide)):

**The prompt used for MI-Guide is shown in Figure 12. It was created based on the work of Steenstra et al. (2024), with modifications made to shift the topic from alcohol abuse to dietary issues. Additionally, since the original prompt included an MI knowledge base, we inserted a summary of the principles of Motivational Interviewing as a knowledge base in our version. In the dialogue history section (## Context section), we included up to 60 utterances exchanged between the client and counselor to provide the LLM with as much of the full conversation as possible. The default sampling temperature of 1.0 was used during generation.

During response generation testing using GPT-4o, many responses contained a large number of sentences. To simulate interactive dialogue scenarios such as voice conversations, we added a constraint to the prompts of the LLMs used in all three dialogue system conditions, limiting each response to a maximum of two sentences.

B.3 Details of the User Study

We conducted a user study using crowdsourcing, in which participants engaged in MI-based dialogues with three different dialogue systems.

Participants first access the instruction page of the experiment, where it is explained that this experiment is to evaluate a dialogue system from the MI point of view. They were given a brief overview of the goals of motivational interviewing and told that the dialogue topic would be “concerns related to eating habits.” Participants were then instructed to identify a personal issue related to this topic and to engage in conversations with each of the three dialogue systems. Before beginning, participants were required to confirm their understanding of the instructions and study content. The full description of the instructions for the experiment is shown in Figure 13.

After this orientation, participants accessed a chat interface as shown in Figure 14. The dialogue began when participants submitted an initial message describing their concern. Each participant’s utterance was reflected in the conversation history panel, followed by the system’s response. Participants continued the interaction by sending subsequent utterances. They were instructed to provide at least 15 utterances (30 in total, including the system’s responses) and were allowed to end the conversation at any point thereafter. Upon completion, they were asked to press a “Ends the dialogue and save log file” button to store the dialogue log.

Next, participants proceeded to a separate form page, where they answered 14 questions assessing their impressions of the system. They were also asked to confirm their consent for the collected data to be used for research analysis and potential public release.

We used the Japanese service CrowdWorks 222CrowdWorks: https://crowdworks.co.jp/ for crowdsourcing. We recruited 24 participants without any restrictions (age, gender). Since the experiment was conducted in Japanese, we assumed that the majority of participants were naitive Japanese speaker. We did not collect any personally identifiable information. Each participant was assigned a unique ID, and all dialogue and questionnaire data were managed accordingly.

We paid subjects 880 yen (approximately 6.15 USD) after repeated chats and impression evaluations for all three interaction systems. The cost of the OpenAI API used in the user study was 21.56 USD.

Table 3 shows the average number of utterances and the average number of sentences in the system utterances per session from the user study. We also show the results of the Wilcoxon signed-rank test for differences between conditions.

Appendix C Evaluation Details

C.1 Human Evaluation

We present the 14 questionnaire items used in the user study in Table 4, and the results for all items are shown in Figure 15.

C.2 Annotation

We describe the details of the annotation process for MITI and question categorization, which was conducted to evaluate the dialogue outcomes in Section 5.2. Each annotation unit corresponds to a single sentence, and we allowed at most one category to be assigned per sentence.

Two annotators, both of whom had received a university-level education and demonstrated a solid understanding of the principles of MI, performed the annotation of the dialogue data. Prior to the main annotation task, they aligned their annotation criteria using a separate set of dialogue data that was not included in the experimental dataset. Afterwards, they annotated three new samples to verify inter-annotator agreement. For the MITI code, which consists of eleven categories, Cohen’s $\kappa$ was 0.83, indicating excellent agreement. For the question categories, which include twelve categories, Cohen’s $\kappa$ was 0.73, indicating fair to good agreement. Then, the dialogue data obtained from the experiment was divided between the two annotators, and each performed the annotation separately.

We show the MITI annotation results in Figure 5. The definition of the question categories are shown in Table 6, and the annotation results are shown in Figure 7.

Appendix D Dialogue Examples

We present dialogue histories in which a single participant interacted with three different dialogue systems. An example dialogue with the MI Few-shot (MI-FS) system is shown in Table 8. An example dialogue with the Principle-based Prompt Guide (MI-Guide) system is shown in Table 9. An example dialogue with our proposed system (Ours) is shown in Table 10. The dialogues were conducted in Japanese, and we present the translated versions.

In the baseline systems, MI-FS and MI-Guide, there are few questions related to current problems or past efforts; the majority of the dialogue instead focuses on planning. Moreover, the system tends to provide informative advice in response to user requests, and users continue to ask for information sequentially. This results in advice-oriented dialogues that are not grounded in the user’s own ideas and do not adhere to MI (Motivational Interviewing) principles.

In contrast, the proposed system (Ours) emphasizes eliciting the user’s thoughts by asking reflective questions, even when the user explicitly requests advice. Consequently, the frequency of advice in the dialogue is reduced, and the proportion of questions is relatively high. This shift promotes a dialogue style that encourages user reflection and contemplation. However, users did not always perceive the system as actively guiding the conversation. This may be due to the system repeating responses that did not match user requests or reflect past experiences, as illustrated by utterance 22-Co in Table 10.

Appendix E Use of AI Tools

We used the AI assistant only for translation and proofreading of manuscripts.