For vaccine text-message campaigns, Brown researchers confirm success is all about timing

PROVIDENCE, R.I. [Brown University] — Text-message “nudges” sent out to unvaccinated Rhode Islanders in late spring 2021 didn’t increase COVID-19 vaccine uptake, according to a new study led by researchers at Brown University’s Policy Lab.

But the researchers posit that the messages’ ineffectiveness was mostly due to their timing: By the time unvaccinated people in the Ocean State received the nudges, all state residents ages 16 and up had been eligible to get a shot for at least four weeks. 

The study was published on Wednesday, April 6, in Nature, just seven months after researchers at the University of California Los Angeles published an influential study that found that text messages were effective at boosting vaccination rates early in 2021.

But the Policy Lab’s study — which analyzes the results of a text message campaign it carried out in partnership with the Rhode Island Department of Health — is far from a repudiation of those results, the authors said. Rather, it offers important additional information on when text-message vaccination campaigns work and when they don’t — takeaways that researchers say could be helpful to health officials across the United States as they work to keep Americans healthy and out of hospitals by promoting COVID-19 and influenza vaccine uptake.

“On the one hand, you could say, ‘What a bummer that these nudges don’t work a month after vaccinations become available to all adults,’” said Kevin Wilson, associate director of data science at the Policy Lab and a co-author of the study. “But on the other hand, we now have more information that can help people who are thinking seriously about how and when to make public health interventions. We’ve unlocked another piece of the puzzle.”

Collaborating to improve public health

Wilson said working with RIDOH on the text-message campaign was the latest in a series of collaborations with state health leaders. Since the Policy Lab’s founding in 2019, it has worked with the Executive Office of Health and Human Services, which houses RIDOH, to better understand the effectiveness of behavioral interventions targeting the opioid epidemic, and to help develop tools that could improve the state’s Medicaid and Supplemental Nutrition Assistance programs. At the start of the COVID-19 pandemic, Policy Lab staff collaborated with the state to design safe, efficient PCR testing facilities and helped leaders survey local residents to understand what behaviors they were implementing to keep their families safe, including social distancing, masking and testing. 

In May 2021, RIDOH and the Policy Lab worked together to reach out to more than 140,000 Rhode Island residents who had been eligible to receive the COVID-19 vaccine for at least four weeks but had not yet received a shot, according to state records. After developing eight different text messages with information about the vaccine, the researchers and health officials sent the messages to most of the unvaccinated residents over the course of 13 days. Some unvaccinated residents were part of a control group that didn’t receive any messages.

Wilson said they crafted the messages using findings from previous studies on Americans’ behaviors during the COVID-19 pandemic. One message touted the vaccine’s safety and another described its ability to prevent hospitalization, he said, appealing to people’s concerns about their own health. Other messages appealed to people’s instincts to protect their loved ones (“Keep your family safe”) and emphasized access (“​​You don’t need an appointment, insurance or other documents”).

“At this juncture, Rhode Islanders could just walk into any CVS or Walgreens without an appointment, an ID or an insurance policy and get the vaccine for free,” Wilson said. “We thought, what if not everyone knows that’s possible? Someone who wants the vaccine but doesn’t know how fast and easy it is is exactly the kind of person you’d want to reach.”

But in the end, the text messages had no meaningful effect on vaccine uptake. No single message seemed to compel a significant number of unvaccinated Rhode Islanders to visit a clinic in the following days and weeks. Even the text message that had the largest positive effect on uptake, which described how the vaccine could prevent “bad COVID-19 outcomes,” showed an effect so small as to be statistically indistinguishable from the control group: It drove 2.2% of recipients to get a vaccine, compared to 2% of those who didn’t receive any message.

Nathaniel Rabb, a project manager at the Policy Lab and a Class of 2014 Brown graduate, said all the messages contained what researchers call “ownership language,” or words and phrases that speak directly to the message’s recipient (“a vaccine is available for you”). He said the use of ownership language had previously been shown to help drive flu vaccine turnout.

“Ownership language could be a way of cutting through the perception that getting a vaccine is complicated,” Rabb said. “It conveys this message of, ‘We’re going to make this easy for you. There’s no red tape. We’ve got a vaccine ready, set aside, just for you.’”

Wilson said the use of ownership language worked well in the study conducted by UCLA researchers in January 2021. Those researchers worked with a Los Angeles health system to send COVID-19 vaccine reminders to about 93,000 people who were among the very first patients to become eligible for the shot. The authors reported that a text-message reminder containing ownership language boosted appointment and vaccine rates within the health system by 6.83 percentage points within six days.

That campaign’s success as compared to the Rhode Island campaign, Wilson said, points to the importance of timing when sending out vaccine reminders.

“In the beginning of 2021, the vaccines were still a novelty,” Wilson said. “Everyone who wanted a vaccine was strategizing how to get one as early as possible. I remember friends frantically calling me, saying, ‘You need to log in to the appointment system at exactly 12:00 and fill out the form as quickly as possible!’ I would have been ecstatic to get a text message with a link that showed me exactly where to go and what to do. But three months later, it’s going to be more difficult to get good results from a text campaign. There’s an abundance of vaccines, there’s less urgency, and the number of people who could get vaccinated has shrunk.”

Rabb said the study confirms that it’s crucial for public health officials to kick off text-message campaigns at the right “inflection point,” when there’s an increased demand for vaccinations. There’s a reason why text-message campaigns for the flu shot work better in October than in February, he said: It’s a key juncture at which children return to school, physicians emphasize the importance of immunization, and the flu season begins. 

Rabb said finding that inflection point could be trickier for COVID-19, which hasn’t yet become endemic or reliably seasonal. But he predicts that a good time to start a text campaign might come when children younger than 5 become eligible for a shot, or when vulnerable adults are advised to receive another booster.

But even if the “right” timing doesn’t become clear, Wilson said, there’s little harm in sending “nudges” to the unvaccinated. Previous studies have shown that text-message campaigns have no negative effect on vaccination rates, which suggests that the worst they can do is inspire no new vaccine uptake. Plus, the campaigns are eminently affordable: In May 2021, RIDOH spent less than a penny per message sent.

“Many states have the data that are necessary to make these kinds of interventions,” Wilson said. “I hope public health leaders who see this study consider trying these sorts of low-cost messaging campaigns, as long as the timing is right.”