An Empirical Investigation of Non-GAAP Exclusion Quality Indicators
We examine commonly used indicators of non-GAAP exclusion quality and find they perform poorly at capturing low-quality (i.e., more persistent) exclusions. Further, low-quality non-GAAP earnings, as identified by any of the indicators used in prior research, are more value relevant than GAAP earnings. We propose a new indicator that performs better at identifying low-quality non-GAAP exclusions. Specifically, we consider cases where GAAP earnings are highly persistent (e.g., low-magnitude
... low-magnitude unexpected earnings; Freeman and Tse 1992), but managers report non-GAAP earnings anyway. In these cases, exclusions have strong negative persistence for future operating earnings/cash flows, suggesting exclusions are of low quality. Further, the GAAP ERC is significantly greater than the non-GAAP ERC. In total, our results suggest existing indicators of low-quality non-GAAP earnings are of little use and that researchers and regulators should focus on firms that disclose non-GAAP earnings despite high quality GAAP earnings. 1 INTRODUCTION Much of prior research on non-GAAP reporting relies on one or more of the following indicators to identify low-quality non-GAAP exclusions: (1) meeting or beating the analyst consensus with non-GAAP earnings when GAAP earnings fall short (e.g., Baik, Billings, and Morton 2008; Doyle, Jennings, and Soliman 2013) , (2) exceeding GAAP earnings through income-increasing exclusions (e.g., Bentley, Christensen, Gee, and Whipple 2018), and (3) turning GAAP earnings losses into non-GAAP profits, or "avoiding losses" (e.g., Bhattacharya, Black, Christensen, and Larson 2003; Brown, Christensen, Elliott, and Mergenthaler 2012). 1 The maintained assumption underlying all of these commonly used indicators is that exclusions that are both necessary and sufficient to achieve a strategic reporting benchmark are more likely driven by managers' opportunism than exclusions that are either unnecessary or insufficient to achieve that benchmark. However, there is little evidence regarding how well these indicators segregate exclusions into low-versus high-quality. 2 In this paper, we assess the construct validity of these non-GAAP exclusion quality indicators. We also propose a new indicator to better capture non-GAAP exclusion quality. Non-GAAP reporting quality is a topic of interest to regulators and market participants. Scrutiny of non-GAAP measures has grown recently as their use has become more widespread. Regulators, practitioners, and news outlets frequently express concern about managers using non-GAAP measures to report opportunistic assessments of firm performance (Golden 2017; Rapoport 1 We use the term "exclusions" to refer to the earnings components managers exclude from GAAP earnings when disclosing non-GAAP earnings. 2 One exception is recent evidence by Leung and Veenman (2019) , who examine the incremental information in loss firms' non-GAAP earnings. While the focus of their study is not to evaluate indicators of non-GAAP reporting quality, Leung and Veenman find evidence suggesting that non-GAAP earnings are highly predictive of future performance for GAAP loss firms. Their result is consistent with the notion that, on average, firms that report non-GAAP profits in the presence of GAAP losses do so for informative reasons. 2 2016). The Sarbanes-Oxley Act of 2002 and subsequent SEC regulatory actions such as Regulation G were intended to constrain managers from opportunistic use of non-GAAP earnings. However, the SEC has recently expressed increased concerns that current requirements are 'not working', and that some non-GAAP disclosures may mislead investors. 3 This concern led to the percentage of SEC comment letters referencing non-GAAP measures increasing from roughly 9% in 2010 to 35% in 2017 (Audit Analytics). We test the construct validity of the three indicators used in prior research by comparing the persistence of exclusions the indicators classify as low-quality to exclusions the indicators classify as high-quality. We measure persistence by the mapping of exclusions into future operating earnings and future operating cash flows (e.g., Kolev et al. 2008; Landsman, Miller, and Yeh 2007; Frankel et al. 2011; Bentley et al. 2018; Kyung et al. 2019). Exclusions the indicators classify as low-quality should have higher associations with future operating earnings or cash flows than exclusions the indicators classify as high-quality. 4 For example, if meeting or beating the analyst consensus with non-GAAP earnings when GAAP earnings fall short is a good indicator of low exclusion quality, then exclusions should have more persistence for future operating earnings and cash flows when the firm's non-GAAP earnings meet or beat the consensus forecast (but GAAP does not) than when both non-GAAP and GAAP earnings either miss or beat the forecast. Our primary results suggest that the three indicators perform poorly at identifying lowquality exclusions. For meeting or beating the analyst consensus (MOB) and avoiding losses by 3 WSJ March 16, 2016 https://www.wsj.com/articles/sec-scrutinizing-use-of-non-gaap-measures-by-publiccompanies-1458139473. Last accessed May 2019. 4 The basic notion behind exclusion persistence is as follows: If excluded earnings components are transitory in nature, do not reflect core operations, or are otherwise not useful when assessing firm performance, then the excluded items should have little to no relation with future operating earnings and/or cash flows (Doyle et al. 2003; Kolev et al. 2008; Whipple 2016; Bentley et al. 2018). 3 turning GAAP losses into non-GAAP earnings profits (AVOID), we find that exclusions that help meet/beat the analyst consensus (turn GAAP losses into non-GAAP profits) are not of lower quality than exclusions that do not help meet/beat the analyst consensus (turn GAAP losses into non-GAAP profits). In fact, our evidence suggests that these exclusions are of higher quality (i.e., are of lesser persistence). The result for the MOB indicator is surprising given this indicator is the most commonly used in prior research to identify low-quality exclusions. The result for the AVOID indicator complements the evidence in Leung and Veenman (2018) and suggests that, on average, non-GAAP earnings are more informative than GAAP earnings in the presence of GAAP losses. On the other hand, we find exclusions are of lower quality (i.e., have greater persistence) when non-GAAP earnings exceed GAAP earnings (EXCEED), suggesting this indicator appropriately identifies low-quality exclusions. However, one limitation of the EXCEED indicator is that it identifies nearly all exclusions as low quality. More specific to our sample, non-GAAP earnings is greater than GAAP earnings for 83% of the observations. We next seek to improve the identification of low-quality non-GAAP earnings. Managers generally justify the disclosure of non-GAAP earnings by arguing that GAAP earnings contain items that are transitory, non-cash, and less relevant for assessing firm fundamentals (Black et al. 2018). In other words, managers claim they disclose non-GAAP earnings to compensate for lowquality GAAP earnings. The implication is that when GAAP earnings are of low quality, non-GAAP exclusions should be of higher quality. However, managers' motivation to provide non-GAAP earnings in the presence of high-quality GAAP earnings is unclear. We conjecture that non-GAAP exclusions are of lower quality in this scenario. While prior research considers several earnings characteristics that indicate quality, one common measure of quality in both the GAAP and non-GAAP literatures is persistence (Doyle et al. 2003 , Kolev et al. 2008 Barth et al. 2012; 4 Bentley et al. 2018; Leung and Veenman 2019). Generally, earnings persistence cannot be measured for each firm-quarter because estimating persistence requires a time-series of earnings. However, evidence in Freeman and Tse (1992) suggests that the magnitude of GAAP earnings surprise is negatively correlated with GAAP earnings persistence. 5 Therefore, we use the magnitude of GAAP earnings surprise as a firm-quarter specific measure of GAAP earnings quality. Specifically, we construct and validate a new indicator of low quality exclusions as equal to one for firms disclosing non-GAAP earnings when their absolute GAAP surprise (i.e., actual GAAP earnings minus the GAAP earnings analyst consensus forecast) is small, i.e., five, three, or one cent(s) per share or less. This new indicator identifies low-quality exclusions as occurring when GAAP earnings are highly persistent, but managers report non-GAAP earnings anyway. We assess our new indicator of exclusion quality through persistence tests similar to our tests of the extant low-quality indicators. We find that our new indicator identifies exclusions that are of lower quality, i.e., highly persistent for both future operating earnings and future operating cash flows. Next, we compare the value relevance of low-quality non-GAAP earnings to the same firms' GAAP earnings. Prior research generally concludes that non-GAAP earnings are, on average, more informative or value relevant than GAAP earnings (e.g., Bhattacharya et al. 2003; Bradshaw and Sloan 2002) . However, it is unclear from prior research whether low-quality non-GAAP earnings are more, less, or equally informative relative to GAAP earnings. We identify low-quality non-GAAP earnings using both the indicators used widely in prior research and our 5 The rationale behind this notion is that analysts and investors place greater emphasis on forecasting high-persistence components of earnings than low-persistence components because high-persistence components have greater valuation weight. Therefore, forecasts of high-persistence components of earnings will be more accurate than those of low-persistence components. This leads to a negative association between the persistence of earnings and the absolute magnitude of unexpected earnings.