Interactive Q-Learning for Quantiles

Kristin A. Linn, Eric B. Laber, Leonard A. Stefanski
2017 Journal of the American Statistical Association  
A dynamic treatment regime is a sequence of decision rules, each of which recommends treatment based on features of patient medical history such as past treatments and outcomes. Existing methods for estimating optimal dynamic treatment regimes from data optimize the mean of a response variable. However, the mean may not always be the most appropriate summary of performance. We derive estimators of decision rules for optimizing probabilities and quantiles computed with respect to the response
more » ... tribution for two-stage, binary treatment settings. This enables estimation of dynamic treatment regimes that optimize the cumulative distribution function of the response at a prespecified point or a prespecified quantile of the response distribution such as the median. The proposed methods perform favorably in simulation experiments. We illustrate our approach with data from a sequentially randomized trial where the primary outcome is remission of depression symptoms. We first characterize the optimal regime for a probability and quantile using potential outcomes (Rubin, 1974) and two treatment time-points. We assume that the observed data, , comprise n independent, identically distributed, timeordered trajectories; one per patient. Let (X 1 , A 1 , X 2 , A 2 , Y) denote a generic observation where: X 1 ∈ ℝ p 1 is baseline covariate information collected prior to the first treatment; A 1 ∈ {−1, 1} is the first treatment; X 2 ∈ ℝ p 2 is interim covariate information collected during the Linn et al.
doi:10.1080/01621459.2016.1155993 pmid:28890584 pmcid:PMC5586239 fatcat:o2huynvmzrhjjjqntikmlw4znm