Gene expression is imperfect, sometimes leading to toxic products. Solutions take two forms: globally reducing error rates, or ensuring that the consequences of erroneous expression are relatively harmless. The latter is optimal, but because it must evolve independently at so many loci, it is subject to a stringent "drift barrier" - a limit to how weak the effects of a deleterious mutation s can be, while still being effectively purged by selection, expressed in terms of the population size N of an idealized population such that purging requires s < -1/N. In previous work, only large populations evolved the optimal local solution, small populations instead evolved globally low error rates, and intermediate populations were bistable, with either solution possible. Here we take into consideration the fact that the effectiveness of purging varies among loci, because of variation in gene expression level and variation in the intrinsic vulnerabilities of different gene products to error. The previously found dichotomy between the two kinds of solution breaks down, replaced by a gradual transition as a function of population size. In the extreme case of a small enough population, selection fails to maintain even the global solution against deleterious mutations, explaining the non-monotonic relationship between effective population size and transcriptional error rate that was recently observed in experiments on E. coli, C. elegans and Buchnera aphidicola.