If you were signifcantly impacted by an algorithmic decision, how would you want the decision to be reviewed? In this study, we explore perceptions of review processes for algorithmic decisions that difer across three dimensions: the reviewer, how the review is conducted, and how long the review takes. Using a choice-based conjoint analysis we fnd that people prefer review processes that provide for human review, the ability to participate in the review process, and a timely outcome. Using a

more »

... vey, we fnd that people also see human review that provides for participation to be the fairest review process. Our qualitative analysis indicates that the fairest review process provides the greatest likelihood of a favourable outcome, an opportunity for the decision subject and their situation to be fully and accurately understood, human involvement, and dignity. These fndings have implications for the design of contestation procedures and also the design of algorithmic decision-making processes. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI.