As mentioned, our approach builds on the explanations proposed by Martens & Provost (2014), who developed and applied counterfactual explanations for document classifications, defining an explanation as an irreducible set of words such that removing them from a document changes its classification. 4 We generalize counterfactual explanations in three important ways. First, we generalize to broader system decisions, which may incorporate predictions from multiple predictive models. Second, their explanations consist of removing features by setting them to zero, whereas we generalize to arbitrary methods for removing features (and note the important relationship to methods for dealing with missing data). Third, while their approach has been applied in other contexts beyond document classification (Chen et al., 2016; Moeyersoms et al., 2016), these applications all have the same data structure: high-dimensional, sparse, binary features. Our generalization applies to data with arbitrary data types.