An Experimental Test of Change Talk Exposure in Civic Contexts
Overview
Motivational Interviewing (MI) is a therapeutic technique often successfully applied in medical contexts to encourage patients to engage in pro-health behaviors, such as vaccinating children or quitting narcotics.
For this project, I wanted to see if one aspect of MI, the cultivation of change talk, could be used to affect participants’ civic engagement, adding to the growing body of interdisciplinary research seeking to drive people towards actions that make the world a better place by drawing on political theory, sociology, and psychology.
I decided to design an experiment using a Qualtrics survey to determine if viewing change talk, and its inverse, sustain talk would affect participants beliefs and intentions to engage in two forms of everyday activism, signing petitions and making small donations.
My statistical modeling in R did not provide significant results, indicating that that brief exposures to change talk may not be sufficient to shift intentions, but also that exposure to sustain talk may not be able to do the same.
To share these findings, I wrote a research paper, delivered a presentation, and devised a poster, all linked below.
Poster
Context & Constraints
This project grew from a desire to bridge related interests of mine: psychology, communication, and persuasion, through research seeking to drive civic engagement. In my reading, I stumbled upon MI and learned that there was a growing body of literature successfully applying its tactics to new contexts, including business leadership, physicians’ continued education, and limited civic engagement research. However, while MI has seen wide success as an intervention, it’s also a complex therapeutic technique.
Due to concerns about the feasibility of a project employing MI in full: lack access to paid trainings, unlikely IRB approval for such interventions, and the small sample size an interview-based intervention would garner; change talk—a core element MI associated with sustained, positive behavioral change—was assessed instead. The simple principles of change and sustain talk made it possible to develop a straightforward experiment and a fast, easy to complete survey that could host more questions and obtain more participants.
Approach
For this study, I decided to compare the impact of change talk and sustain talk on civic engagement in two distinct forms of everyday collective action to assess whether my results were domain specific. I selected petition signing and small donations as they would be familiar to the most convenient and sizable population available: college students, while lacking the social pressure of domains like recycling and voting.
From these domains and talk types, I was able to design a 4-level completely randomized design, using Qualtrics to assign each respondent to read a vignette discussing either petition signing or small donations, and a student’s decision to either change their habits of not engaging or sustain their current behavior. This random assignment helped ensure that results could be attributed to the experiment rather than any placement biases.
I then used several Likert scales to develop composite scores, averaging 3 questions each: Petition-Intentions, Petition-Beliefs, Donation-Intentions, and Donation-Beliefs. I wanted to measure Intentions and Beliefs as I felt that the distinction between intending to do something and believing in the efficacy and viability of your action are distinct outcomes that don’t always overlap (e.g. you can believe that recycling isn’t helpful and still recycle because everyone you know does). I wanted to know which of these axes might be affected by exposure to change and sustain talk.
I used AI to design the messaging and layout of a promotional poster, testing several options for tone, visibility, and ease. The flyers were hung across campus, ensuring that the sample wouldn’t simply reflect the groups I associated with. I also provided a 3-winner lottery of $10 gift cards, in line with research suggesting that smaller rewards participants were more likely to receive had the stronger impact. In total, I received 120 responses.
I used the R package ordinal to conduct several ordinal analyses of variance (ANOVA), accounting for the use of Likert scales, and analyses of covariance (ANCOVA) to control for baseline civic engagement. While mean petition intention scores were higher for the petition-change talk groups, means were often variable, and I found no significant relationships between my explanatory and response variables. I tested my assumptions and used boxplots to make it easy to interpret the means, relationships, and variability of the data.
What I Learned
As this was a class final, format was largely predetermined, a research paper and a presentation. However, I was later asked to convert the project into a poster for an event where prospective university students could learn about our department.
While the paper remains a necessity for providing the greatest wealth of information and clarity surrounding my project, I found that the poster gave me a stronger understanding of my work due to its brevity and the ability to view several aspects simultaneously. I plan to turn more projects into posters, as they give me an easy, effective way to reconsider my research projects and share them with others.
This experiment also taught me a lot about study design. In my results, I found that there were a lot of incomplete surveys, especially in later sections, as well as some surveys that, while complete, failed the attention check. There is an array of factors that could explain this and three that I think warrant discussion here.
The first has to do with study length. While completable in 2-5 minutes, the survey wasn’t short. I chose to measure both intention and belief in social action. Looking back, having two response variables introduced excess complexity and, in a future case, I would study them separately and use fewer questions. I also think that in my promotion I overemphasized ease and potential monetary reward as motivations for completion. These are useful incentives, but they can undermine more intrinsic motivations, especially for longer surveys. Going forward I’d like to provide more balanced promotion.
Finally, having the attention check end of the survey made it difficult to discern if participants had read the vignettes. In the future, I would use a reading check, that prevents participants from moving forward without the correct answer.
So What?
The insignificant results in this project offer an inspiring sign: while exposure to change talk does not appear to provide a substantial effect on participant behavior neither does its inverse—this means that more effortful attempts to change behavior may not be undermined by simple exposure to sustain talk.
This project contributes to a growing body of literature that seeks to help organizers inspire individuals to engage civically and contribute to wider movements. Motivational Interviewing has been successful in cultivating personal change in a range of environments and research that looks to expand its practice and principles to other fields—and to test its limits—offers a strong opportunity to learn more about this tactic and about non-manipulative persuasion tactics writ large.