Peer Review #2 of "An exploratory pilot study of mechanisms of action within normative feedback for adult drinkers (v0.1)"
Normative feedback (NF), or receiving information about one's drinking compared to peer drinking norms, is one of the most widely used brief interventions for prevention and intervention for hazardous alcohol use. NF has demonstrated predominantly small but significant effect sizes for intention to change and other drinking related outcomes. Identifying mechanisms of action may improve the effectiveness of NF; however, few studies have examined NF's mechanisms of action, particularly among
... icularly among adults. Objective: This study is an exploratory analysis of two theorized mechanisms of NF: discrepancy (specifically personal dissonance-the affective response to feedback) and belief in the accuracy of feedback. Method: Using Amazon's Mechanical Turk, 87 men (n=56) and women (n=31) completed an online survey during which they were asked about their perceptions about their drinking and actual drinking behaviors. Then participants were provided tailored NF and evaluated for their reactions. Severity of discrepancy was measured by the difference between one's estimated percentile ranking of drinking compared to peers and actual percentile ranking. Surprise and worry reported due to the discrepancy were proxies for personal dissonance. Participants were also asked if they believed the feedback and if they had any plans to change their drinking. Mediation analyses were implemented, exploring whether surprise, worry, or belief in the accuracy of feedback mediated severity of discrepancy's impact on plan for change. Results: Among this sample of adult drinkers, severity of discrepancy did not predict plan for change, and personal dissonance did not mediate severity of discrepancy. Severity of discrepancy was mediated by belief in the accuracy of feedback. In addition, viewing one's drinking as a problem prior to feedback and post-NF worry both predicted plan for change independently. Conclusions: Results revealed that NF may not work to create personal dissonance through discrepancy, but belief in the accuracy of feedback may be important. It appears the more one believes the feedback, the more one makes a plan for change, suggesting practitioners should be mindful of how information within feedback is presented. Findings also indicate NF may work by validating a preexisting perception that drinking is a problem instead of creating concern related to discrepancy where none existed. Limitations regarding generalizability are discussed. PeerJ reviewing PDF | Manuscript to be reviewed 1 Running header: EXPLORING MECHANISMS OF FEEDBACK 2 Abstract word count: 360 3 Word count: 4,569 4 Tables: 4 5 Figures: 0 6 Number of references: 7 An exploratory pilot study of mechanisms of action within normative feedback for adult drinkers Abstract 31 Background: Normative feedback (NF), or receiving information about one's drinking 32 compared to peer drinking norms, is one of the most widely used brief interventions for 33 prevention and intervention for hazardous alcohol use. NF has demonstrated predominantly 34 small but significant effect sizes for intention to change and other drinking related outcomes. 35 Identifying mechanisms of action may improve the effectiveness of NF; however, few studies 36 have examined NF's mechanisms of action, particularly among adults. 37 Objective: This study is an exploratory analysis of two theorized mechanisms of NF: 38 discrepancy (specifically personal dissonance-the affective response to feedback) and belief in 39 the accuracy of feedback. 40 Method: Using Amazon's Mechanical Turk, 87 men (n=56) and women (n=31) completed an 41 online survey during which they were asked about their perceptions about their drinking and 42 actual drinking behaviors. Then participants were provided tailored NF and evaluated for their 43 reactions. Severity of discrepancy was measured by the difference between one's estimated 44 percentile ranking of drinking compared to peers and actual percentile ranking. Surprise and 45 worry reported due to the discrepancy were proxies for personal dissonance. Participants were 46 also asked if they believed the feedback and if they had any plans to change their drinking. 47 Mediation analyses were implemented, exploring whether surprise, worry, or belief in the 48 accuracy of feedback mediated severity of discrepancy's impact on plan for change. 49 Results: Among this sample of adult drinkers, severity of discrepancy did not predict plan for 50 change, and personal dissonance did not mediate severity of discrepancy. Severity of discrepancy 51 was mediated by belief in the accuracy of feedback. In addition, viewing one's drinking as a 52 problem prior to feedback and post-NF worry both predicted plan for change independently. PeerJ reviewing PDF | Manuscript to be reviewed 53 Conclusions: Results revealed that NF may not work to create personal dissonance through 54 discrepancy, but belief in the accuracy of feedback may be important. It appears the more one 55 believes the feedback, the more one makes a plan for change, suggesting practitioners should be 56 mindful of how information within feedback is presented. Findings also indicate NF may work 57 by validating a preexisting perception that drinking is a problem instead of creating concern 58 related to discrepancy where none existed. Limitations regarding generalizability are discussed. 59 PeerJ reviewing PDF | Manuscript to be reviewed 76 a primary care physician, or, in some cases, on a computer. One such brief intervention is 77 normative feedback (NF), a widely used intervention for prevention of and early intervention for 78 hazardous alcohol use 10 . During NF, individuals are assessed and then provided with information 79 about how their drinking patterns compare to drinking norms of their peers of the same gender 80 and age. The purpose of NF is to provide a drinker with a tool for understanding their own, 81 potentially excessive, drinking and ultimately motivate them to reduce their drinking to safe 82 levels. PeerJ reviewing PDF | Manuscript to be reviewed 83 There are a variety of forms of NF, and it is implemented in a myriad of ways. For 84 example, NF can be implemented as a standalone intervention; as part of more detailed feedback 85 that includes information on health consequences, referred to as personalized feedback 11 ; or 86 alongside or within another therapeutic intervention, such as Motivational Enhancement Therapy 87 (MET) 11,12 . NF is often provided in community-based settings, such as primary care 13,14 , via 88 multiple modalities (e.g., web-based, mailed, in-person) 11,15-18 . Given its relative simplicity, it is 89 somewhat surprising that NF demonstrates consistent effectiveness in helping individuals reduce 90 drinking at statistically significant levels across all modalities, contexts (e.g., within therapy or 91 alone), and structures (e.g., a single item or detailed feedback about multiple health 92 indicators) 4,9,11,12,15,17,19-27 . 93 It is important to note that the preponderance of the above described research on NF has 94 primarily focused on college students and/or their non-college same-age counterparts 11,28,29 . This 95 is in part because of the well documented prevalence of high risk drinking on college campuses 96 and among individuals at this stage of late adolescence 3 . Only a few studies have examined the 97 impact of NF among adult drinkers beyond the college age years (e.g., 4,10,13,17,18,23,27 ). Within 98 those few studies, NF again demonstrates preliminary effectiveness among adults at a spectrum 99 of ages; however, this effectiveness is less well established, especially long term 27 , than for 100 college-aged individuals. 101 While NF is considered an effective intervention, particularly for individuals in late 102 adolescence, it can potentially be improved to better facilitate both individual and population 103 health 8 . While some studies demonstrate impressive effect sizes of NF, most studies demonstrate 104 small effect sizes 9,17,25 . There is also evidence that NF does not impact drinking outcomes as PeerJ reviewing PDF | Manuscript to be reviewed 105 much as has been previously suggested 30,31 , particularly outside the context of primary care or 106 for more than six months 27 . 107 One way to improve an intervention is to understand its mechanisms of action. If one 108 identifies and understands how an intervention works, one can hone the intervention--improving 109 its efficacy and cost-effectiveness. Although some studies have examined the mechanisms of 110 action of personalized feedback 8 and NF 32 , mechanisms remain relatively unexplored 31 . 111 Proposed mechanisms of action of NF. Two potential mechanisms of action of NF 112 explored in the literature are 1) discrepancy, including both external discrepancy 33 , the difference 113 between one's perceived norm and the actual norm, and internal discrepancy 33 , the dissonance 114 between an individual's drinking and an internal standard of comparison, such as one's "ideal" 115 drinking behavior; and 2) perceived accuracy of feedback. 116 Discrepancy has been explored in a number of ways due to its different forms and 117 definitions. Social Norms Theory 34 is used to explain the effect of NF-such that once the 118 perception of how much others drink is reduced, then the individual's drinking will be reduced in 119 kind. Some studies support this theory, demonstrating change in perceived norms as a significant 120 mediator of NF in reducing drinking among college students 22,32, . Other studies show that 121 while change in perceived norms relates to intentions to change alcohol use among college 122 students, it does not necessarily lead to actual reduced drinking 33,38,39 . Notably, young adults may 123 react to discrepancy differently from older adults due to their particular life stage 40 . For example, 124 a heightened need for peer approval during late adolescence may cause a more potent effect of 125 NF 23 . As such, mechanisms of action of NF may differ across age groups. 126 Within the context of MET 41 , NF is thought to develop internal, self-ideal 127 discrepancy 39,42 . In MET, a heightened awareness of discrepancy between one's image of oneself PeerJ reviewing PDF | Manuscript to be reviewed 128 and one's actual behavior is thought to create conflict for the individual, resulting in discomfort 129 with the status quo 43 and thus enhancing motivation to change. This conflict is described many 130 ways-including affectively 43 -such as fear, worry, or a neutral uneasiness with the difference 131 between perception and behavior. This conflict is conceptualized here as an affective response to 132 discrepancy and is termed personal dissonance. Only two studies thus far directly examined the 133 affective response related to NF and drinking with any population 23,43 . Results of one study 134 demonstrated that among adult problem drinking men-who-have-sex-with-men (MSM), NF 135 within MET elicited concern and worry, but these responses were not related to reduction in 136 post-NF drinking 23 . The second study 43 also found no relationship between developed affective 137 dissonance and drinking outcomes among heavy drinking college students. Given that only one 138 study has examined this process among an adult population beyond college age and only in the 139 context of MET, further research is needed to explore affective responses to NF and their role in 140 behavior change among adults, at a range of ages. 141 Perceived accuracy of NF (e.g., belief that norms used for drinking comparisons and the 142 results of the evaluation are true or valid) may also mediate NF's efficacy. Decision dilemma 143 theory is a behavioral economic theory that posits that poor decision making (e.g., continued 144 hazardous drinking) results from receiving "equivocal" feedback 44,45 or "feedback for which 145 multiple (positive or negative) interpretations can be constructed" 45 . An individual who perceives 146 NF as highly equivocal might view the information as inaccurate or not applicable to their 147 personal situation. As a result, the individual would dismiss the information, nullifying the 148 effects of NF. In their study of adult drug users who received NF, Amrhein and colleagues 46 149 noted a subgroup of participants who disagreed with the feedback they received (thus viewed it 150 as "highly equivocal"). This subgroup demonstrated more stable or increased drug use post-NF. PeerJ reviewing PDF | Manuscript to be reviewed Self-reported quantity and frequency of drinking. Participants were asked to report on 215 their actual drinking through a series of questions based on the Form 90 QFV-30 49 . Responses 216 were used to provide NF, described below. 217 Personal dissonance. Immediately after feedback was provided, participants were asked 218 to rate, again on a scale of 1 (Not at all) to 8 (Extremely), 1) how surprised are you by the PeerJ reviewing PDF | Manuscript to be reviewed 219 feedback? (surprise) and 2) how much does the feedback worry you? (worry). These were used 220 as proxies for personal dissonance resulting from NF. 221 Expectation. Participants were asked "Is this feedback, ..?" and then provided with three 222 options: better than expected, as expected and worse than expected. 223 Newness of information. Participants were asked if the information provided in the NF 224 was new. They could respond yes or no. 225 Plan to change. In the last section of the survey, participants were asked about their plan 226 related to drinking for the next 30 days. This question was categorical and responses ranged from 227 (0) no change in drinking to (3) quitting drinking. Participants were also asked to rate on a scale 228 of 1 (Not at all) to 8 (Extremely) their commitment to and confidence in their ability to achieve 229 their plan. 230 Belief in the accuracy of NF. To avoid biasing item responses earlier in the survey, the 231 last question was "how accurate does the information you received about your drinking seem to 232 you?" A 5-point Likert response scale included anchors -2 Not at all accurate to 2 Definitely 233 accurate. Zero was "Not sure". 234 Severity of discrepancy. Participants were asked to rank themselves as to how much they 235 drink compared to their same gendered peers in the form of a percentile rank. Severity of 236 discrepancy was calculated as the difference between the participant's initial reported percentile 237 rank and actual percentile rank (calculated based on reported standard drinks per week)--a 238 measure similar to those used in other studies 22,50 . Higher numbers indicated greater discrepancy. 239 Positive severity of discrepancy indicated that participants drank more compared to their peers 240 than they estimated. 241 Normative Feedback PeerJ reviewing PDF | Manuscript to be reviewed 242 Normative comparisons were based on the 2009 National Survey on Drug Use and 243 Health 51 . Based on participants' reported average number of standard drinks per week, they were 244 directed to one of 18 different possible statements of feedback, each with a unique percentile 245 rank based on gender and quantity of standard drinks per week. Feedback contained two 246 sentences; for example: "You reported drinking 1 standard drink per week. You drink more than 247 27% of men." While we asked about binge drinking, binge drinking was not a factor in 248 determining feedback, nor were participants educated about binge drinking. 249 Analytic Plan 250 Descriptive analyses were performed to evaluate overall participant perceptions of health 251 behaviors, their reactions to NF, and their plans for change. Correlations between the perceptions 252 of drinking pre-NF and post-NF reactions were calculated. Mediation analyses were performed 253 using the PROCESS procedure 52 in SPSS 22.0 (IBM, 2013). Mediation was formally tested 254 using the product of coefficients and bias-corrected bootstrap confidence intervals 52 , with 10,000 255 samples specified. Mediators were first tested independently, and effect sizes of the indirect 256 effects were calculated using Preacher and Kelley's kappa-squared 53 , where appropriate. In this 257 context, Kappa-squared provides information about the size of the indirect effect relative to its 258 maximum possible value, given the natural constraints imposed by variances and correlations 259 between variables 52 . If more than one independent mediator was significant, then multiple 260 mediation was tested. Procedures were reviewed and granted approval by the Institutional 261 Review Board at the New York State Psychiatric Institute. 262 Results Pre-NF drinking is a problem significantly predicted plan to change (b = .24, SE = .05, p 327 < .001). When post-NF worry was entered as a second step in the model, both remained 328 significant (drinking is a problem: b = .24, SE = .05, p < .001; worry: b = .24, SE = .05, p < .001; 329 adjusted R 2 = .38; F(2, 84) = 26.8, p < .001), suggesting that both baseline perception that 330 drinking is a problem and post-NF worry predict plan for change. Tolerance statistics 331 demonstrated no multicollinear effect. 332 Discussion PeerJ reviewing PDF | Manuscript to be reviewed 333 This is the first study to test both personal dissonance and perceived accuracy of feedback 334 as mechanisms of action within NF among adult drinkers outside the context of MET. 335 Hypotheses that severity of discrepancy would be mediated by personal dissonance and belief in 336 the accuracy of feedback were only partially supported. While the participants in this study were 337 primarily light to moderate drinkers, a number of them greatly under-estimated their level of 338 drinking compared to their same gendered peers, demonstrating a wide range of discrepancy. 339 Still, severity of discrepancy was not associated with plan to change, which is inconsistent with 340 previous studies on college students 22,50 . Interestingly, findings were consistent with previous 341 research on a sample of hazardous drinkers from the general population 10 . This recent study 342 found that norm comparisons, provided in the context of more detailed personalized feedback, 343 did not appear to be the active mechanism for change among a sample of adult hazardous 344 drinkers 10 . It is possible that peer comparisons work especially well for individuals in their late 345 adolescence and emerging adulthood because it leverages their particular need for conforming 346 with peers' behaviors. Given the lack of association between severity of discrepancy and plan for 347 change, it also suggests that among a sample of mostly light to moderate drinkers, factual 348 normative discrepancy does not act alone to instigate thoughts about change. 349 While there were few heavy drinkers in this sample, lack of findings regarding personal 350 dissonance was consistent with previous studies on affective responses 23,43 of heavy drinkers. 351 According to evidence thus far, it appears that affective responses are not associated with 352 outcomes of feedback among both college-age and an age-inclusive sample of adults with a 353 spectrum of drinking patterns. 354 Perceived accuracy of feedback was demonstrated to be a potential mediator of severity 355 of discrepancy, providing initial support for decision dilemma theory. Results suggest that the PeerJ reviewing PDF | Manuscript to be reviewed 356 more an individual is aware of how much he or she drinks compared to others, the more likely he 357 or she will perceive the feedback to be accurate and thus more likely to set a plan for change. 358 Among this sample, this was true regardless of intensity of drinking. It is important to note that 359 the mediation effect of belief in the accuracy of NF was quite small-only 11.6% of its possible 360 maximum value. More research is needed to further understand this potential mediator of 361 change. 362 Post hoc analyses support the possibility that feedback for drinkers may not operate to 363 create concern about or awareness of excessive drinking where none existed. Instead, it seems to 364 potentially capitalize on a pre-existing perception that drinking is a problem, particularly among 365 those who may drink more heavily. Some studies demonstrate NF works best for those with the 366 most severe drinking patterns 54,55 . It is therefore possible that feedback works within a 367 framework of consonance rather than dissonance. This may explain why a "boomerang effect", 368 an effect in which light drinkers who receive feedback about their drinking increase instead of 369 decrease their drinking 56 , has yet to be found within the empirical literature. Furthermore, given 370 the correlational relationships, perceiving alcohol is a problem may also indicate an awareness of 371 one's drinking and already how it compares to existing norms. 372 Given that NF is one of the most widely used interventions to prevent and/or reduce 373 hazardous drinking, the above findings have potentially important implications for application to 374 clinical practice. When implementing NF to an adult sample, providers should be aware of both 375 the existing concerns and perceptions a drinker may have, as well as how believable the 376 information imparted is for the recipient. Inquiring about a patient's drinking and their 377 perception of its role in his or her life a priori may help providers navigate providing feedback in 378 an optimal way. For example, there may be little need to offer much additional feedback when PeerJ reviewing PDF | Manuscript to be reviewed 379 one already perceives their drinking as problematic. Instead, offering an opportunity to explore 380 goal planning and/or treatment may be the best use of a clinician's time. For those who do not 381 perceive their drinking as problematic, feedback interventions may be best improved by 382 increasing the credibility of the feedback for the individual. By being selective in how one 383 provides feedback, providers may enhance its effectiveness. Limitations 385 There are several limitations to this study, and findings should be interpreted in the 386 context of an exploratory pilot study. While this study demonstrated the feasibility of measuring 387 affective responses and belief in the accuracy of feedback post-NF, generalizability is extremely 388 limited. These participants were recruited from an online labor market, not the general 389 population. The measurement of belief in the accuracy of feedback was simple in that it was a 390 single item, self-report measure; however, it had face validity in that it captured the participant's 391 perception rather than behavior, which was the specific construct of interest. A major limitation 392 of this study is that there were few individuals who were heavy or hazardous drinkers, in terms 393 of quantity of standard drinks per week, and thus results cannot be generalized to hazardous 394 drinkers. More research on understanding this relationship with hazardous drinkers is needed; 395 however, the greater variability in the sample allowed us to understand feedback in a 396 heterogeneous drinking sample, such as those found in primary care or other opportunistic 397 settings, in which NF is often used preventively. 398 Drinking was entirely self-report and could not be corroborated by other sources. 399 Outcomes were based on intention to change health behaviors, not actual behavior change. While 400 other studies have used a similar outcome, it is not equivalent to behavior. Finally, it is not 401 known whether personal dissonance is experienced only in a proximal fashion. For example, a PeerJ reviewing PDF | Manuscript to be reviewed 402 person who receives feedback may not initially experience worry but may find his or her concern 403 grows over a period of time. This potential sleeper effect of NF could in turn potentially affect 404 the timing of making a plan for change. Longitudinal research is needed to better understand the 405 timing of potential mediators and subsequent behavioral outcomes. Future Research 407 Future research could include experimental manipulation with better operationalization of 408 normative feedback, self-ideal feedback and emotional responses in order to unpack the active 409 ingredients of NF. Expanding the population upon which NF is tested to heavy drinkers and 410 other undesirable health behaviors will be crucial to confirming these preliminary findings. 411 Additionally, both actual behavioral outcomes and longer term, prospective perceptions of 412 feedback must be explored to understand the mechanisms of maintenance of behavior change. Conclusions 414 Despite the limitations, findings have the potential for immediate clinical application, and 415 thus implications for individual and population level health. For individuals with a skewed 416 perception of how much they drink compared to their peers, helping them to understand NF in 417 context and making it believable (e.g., supported by empirical data) may be critical for maximum 418 effectiveness of NF among adults. In addition, exploring with drinkers how much they are 419 already perceive their drinking to be a problem may help to potentiate the effects of the simplest 420 form of NF. These are relatively minor and feasible changes to NF that may in fact help to 421 increase its efficacy rate, potentially helping to reduce the prevalence rate of hazardous alcohol 422 use or prevent the development of AUD across a wide group of drinkers. 424 PeerJ reviewing PDF |