827 Hits in 3.5 sec

Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation [article]

Yuexiang Xie, Fei Sun, Yang Deng, Yaliang Li, Bolin Ding
2021 arXiv   pre-print
In light of these challenges, we propose a novel metric to evaluate the factual consistency in text summarization via counterfactual estimation, which formulates the causal relationship among the source  ...  Among the key factors to ensure factual consistency, a reliable automatic evaluation metric is the first and the most crucial one.  ...  CoCo), to measure the factual consistency via counterfactual estimation.  ... 
arXiv:2108.13134v2 fatcat:rtwaagkbjnbabcpdjsyj2pwlyi

A Game Theoretic Approach to Class-wise Selective Rationalization [article]

Shiyu Chang, Yang Zhang, Mo Yu, Tommi S. Jaakkola
2019 arXiv   pre-print
Each class involves three players set up competitively to find evidence for factual and counterfactual scenarios.  ...  We evaluate the method in single- and multi-aspect sentiment classification tasks and demonstrate that the proposed method is able to identify both factual (justifying the ground truth label) and counterfactual  ...  Regina Barzilay for her support and help.  ... 
arXiv:1910.12853v1 fatcat:sj2krzjm5vgxnmvtcvgnacnskq

XPROAX-Local explanations for text classification with progressive neighborhood approximation [article]

Yi Cai, Arthur Zimek, Eirini Ntoutsi
2021 arXiv   pre-print
Our work focuses on textual data and our explanations consist of both word-level explanations from the original instance (intrinsic) and the neighborhood (extrinsic) and factual- and counterfactual-instances  ...  Several attempts have been made to construct a better neighborhood for high dimensional data, like texts, by using generative autoencoders.  ...  We also thank Philip Naumann for the insightful discussions.  ... 
arXiv:2109.15004v1 fatcat:ea7wl2pk2zegbkdqrbac3htu3e

Cultural Capital and Educational Inequality: A Counterfactual Analysis

Mads Jæger, Kristian Karlson
2018 Sociological Science  
These findings are consistent with cultural mobility theory (which argues that low-SES children have a higher return to cultural capital than high-SES children) but not with cultural reproduction theory  ...  We use National Longitudinal Survey of Youth 1979 (NLSY79) data and a counterfactual approach to test the macro-level implications of cultural reproduction and cultural mobility theory.  ...  We thank the participants at these events for their excellent comments.  ... 
doi:10.15195/v5.a33 fatcat:a6hvewyduvggfmctbllbpkb76i

Enhancing Counterfactual Classification via Self-Training [article]

Ruijiang Gao, Max Biggs, Wei Sun, Ligong Han
2021 arXiv   pre-print
We may only observe outcomes for the chosen actions, but not the counterfactual outcomes associated with other alternatives.  ...  In addition, we show input consistency loss can further improve CST performance which is shown in recent theoretical analysis of pseudolabeling.  ...  By doing so, we have the full knowledge of counterfactual outcomes for evaluation. Data statistics are summarized in the Appendix.  ... 
arXiv:2112.04461v1 fatcat:sws5bjzvh5awziegf3crrod5am

New Methods Metrics for LFQA tasks [article]

Suchismit Mahapatra, Vladimir Blagojevic, Pablo Bertorello, Prasanna Kumar
2021 arXiv   pre-print
The authors evaluate the factual consistency in the generated summaries via counterfactual estimation and remove the impact of language prior (refer to Figure 1 for details 1 ), which is a potential  ...  Automatic metrics for LFQA tasks For automatic metrics for LFQA tasks, adapting the CoCo (Xie et al., 2021) metric was evaluated. It was originally intended for text summarization tasks.  ...  It can be clearly observed by the similarity score distribution, which has shifted to the left for the new alpha dataset, in each of the results.  ... 
arXiv:2112.13432v1 fatcat:neirmp7a2rcsvcslme6wujtqeu

Poincare: Recommending Publication Venues via Treatment Effect Estimation [article]

Ryoma Sato, Makoto Yamada, Hisashi Kashima
2020 arXiv   pre-print
Choosing a publication venue for an academic paper is a crucial step in the research process.  ...  We estimate the number of citations a paper will receive if the paper is published in each venue and recommend the venue where the paper has the most potential impact.  ...  This was because we evaluated the methods using only factual observations.  ... 
arXiv:2010.09157v1 fatcat:t76qt2oecfca5nsbisfdhdiwvu

VACA: Design of Variational Graph Autoencoders for Interventional and Counterfactual Queries [article]

Pablo Sanchez-Martin, Miriam Rateike, Isabel Valera
2021 arXiv   pre-print
Finally, we apply VACA to evaluate counterfactual fairness in fair classification problems, as well as to learn fair classifiers without compromising performance.  ...  As a result, and as shown by our empirical results, VACA accurately approximates the interventional and counterfactual distributions on diverse SCMs.  ...  The authors thank for the generous funding support. The authors also thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Pablo Sańchez Martín.  ... 
arXiv:2110.14690v1 fatcat:fazpeu5srvd3hmu4ojkdzjy2ki

Targeted Maximum Likelihood Estimation for Dynamic and Static Longitudinal Marginal Structural Working Models

Maya Petersen, Joshua Schwab, Susan Gruber, Nello Blaser, Michael Schomaker, Mark van der Laan
2014 Journal of Causal Inference  
Data from the International Epidemiological Databases to Evaluate AIDS, Southern Africa are analyzed to investigate this question using both TML and IPW estimators.  ...  We compare this pooled estimator to a stratified TMLE (Schnitzer et al. 2014 ) that is based on estimating the intervention-specific mean separately for each intervention of interest.  ...  As a consequence, our TMLE will be a consistent estimator of ψ 0 if either is consistent for Q 0 or g n is consistent for g 0 [38] .  ... 
doi:10.1515/jci-2013-0007 pmid:25909047 pmcid:PMC4405134 fatcat:bch3rzm7lbc6nmhi2xin7dfcyu

The Cumulative Cost of Regulations

Bentley Coffey, Patrick A. McLaughlin, Pietro F. Peretto
2016 Social Science Research Network  
We estimate the effects of federal regulation on value added to GDP for a panel of 22 industries in the United States over a period of 35 years .  ...  on a text analysis of federal regulatory code.  ...  Acknowledgments The authors thank Oliver Sherouse, Scott Eastman, and Laura Stanley for their research assistance.  ... 
doi:10.2139/ssrn.2869145 fatcat:rzu7hoz4h5do5ha7bpsklqjdhy

Hindsight Bias

Neal J. Roese, Kathleen D. Vohs
2012 Perspectives on Psychological Science  
explanations for a given outcome can reduce hindsight bias.  ...  Hindsight bias stems from (a) cognitive inputs (people selectively recall information consistent with what they now know to be true and engage in sensemaking to impose meaning on their own knowledge),  ...  Acknowledgments We thank Angie Chung, Subarna Ranjit, and Elizabeth Wilson for clerical and library assistance.  ... 
doi:10.1177/1745691612454303 pmid:26168501 fatcat:iqnvnitssbdpxdk7mybphewfqi

A Survey on Causal Inference [article]

Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, Aidong Zhang
2020 arXiv   pre-print
Moreover, the commonly used benchmark datasets as well as the open-source codes are also summarized, which facilitate researchers and practitioners to explore, evaluate and apply the causal inference methods  ...  Embraced with the rapidly developed machine learning area, various causal effect estimation methods for observational data have sprung up.  ...  This is a natural application of treatment effect estimation as it is consistent with the objective of treatment effect estimation. (2) Counterfactual estimation.  ... 
arXiv:2002.02770v1 fatcat:kcedeyseevb3doht4pop26rqvy

Counterfactual Learning of Stochastic Policies with Continuous Actions: from Models to Offline Evaluation [article]

Houssam Zenati, Alberto Bietti, Matthieu Martin, Eustache Diemert, Julien Mairal
2021 arXiv   pre-print
Counterfactual reasoning from logged data has become increasingly important for many applications such as web advertising or healthcare.  ...  Second, we empirically show that the optimization aspect of counterfactual learning is important, and we demonstrate the benefits of proximal point algorithms and differentiable estimators.  ...  Acknowledgments The authors thank Christophe Renaudin (Criteo AI Lab) for his help in the data collection and engineering which was necessary for this project.  ... 
arXiv:2004.11722v5 fatcat:wjalqh2eujc6fhk4j6354wcyqq

Benchmarking and Survey of Explanation Methods for Black Box Models [article]

Francesco Bodria, Fosca Giannotti, Riccardo Guidotti, Francesca Naretto, Dino Pedreschi, Salvatore Rinzivillo
2021 arXiv   pre-print
The widespread adoption of black-box models in Artificial Intelligence has enhanced the need for explanation methods to reveal how these obscure models reach specific decisions.  ...  In [25] is presented a study that evaluates the understandability of factual and counterfactual explanations.  ...  In Figure 6 we report the factual and counterfactual rules of lore for the explanation of the same records showed for anchor.  ... 
arXiv:2102.13076v1 fatcat:6iw6nhwltzb7fhaawnvy7oxndu

Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning [article]

Vivek Gupta, Riyaz A. Bhat, Atreya Ghosal, Manish Srivastava, Maneesh Singh, Vivek Srikumar
2021 arXiv   pre-print
its underlying pre-trained language model in a nuanced, context-sensitive fashion (e.g. handling counterfactuals).  ...  The tabular nature of the input is particularly suited for the study as it admits systematic probes targeting the properties listed above.  ...  We summarize the combined prohibited transitions for each label in Table 8 .  ... 
arXiv:2108.00578v1 fatcat:75a7njpznnbmjbpohiu77ldxbu
« Previous Showing results 1 — 15 out of 827 results