Loss aversion—the idea that losses loom larger than equivalent gains—is one of the most important ideas in Behavioral Economics. In an influential article published in the Journal of Experimental Psychology: General, Walasek and Stewart (2015) test an implication of decision by sampling theory: Loss aversion can disappear, and even reverse, depending on the distribution of gains and losses people have encountered. In this manuscript, we show that the pattern of results reported in Walasek and Stewart (2015) should not be taken as evidence that loss aversion can disappear and reverse, or that decision by sampling is the origin of loss aversion. It emerges because the estimates of loss aversion are computed on different lotteries in different conditions. In other words, the experimental paradigm violates measurement invariance, and is thus invalid. We show that analyzing only the subset of lotteries that are common across conditions eliminates the pattern of results. We note that other recently published articles use similar experimental designs, and we discuss general implications for empirical examinations of utility functions.