Human perception is increasingly described as optimal. This view reflects recent successes of Bayesian approaches to perception but ignores an extensive literature documenting suboptimal performance in perceptual tasks. Here we review several classes of suboptimal perceptual decisions, including improper placement, maintenance, and adjustment of perceptual criteria, inadequate tradeoff between speed and accuracy, and inappropriate confidence ratings. We examine suboptimalities in the cue combination literature, which has often been taken as evidence for optimality in perception. We further discuss how findings regarding visual illusions, adaptation, and appearance relate to the concept of optimality. We extract a number of principles that account for suboptimal behavior across all of these studies. Finally, we outline an "Objectives-Constraints-Mechanisms" approach for guiding future investigations and discussions of optimal and suboptimal perception. In this approach, findings of optimality or suboptimality are not ends in themselves but serve to characterize the perceptual system's objectives (i.e., what it aims to achieve), constraints (i.e., what costs it incurs), and mechanisms (i.e., what processes it uses to trade off the objectives and constraints). We suggest that this conceptual framework, more than a focus on optimality per se, can advance our understanding of perception.