Gamma band oscillatory neural activity, considered critical in synchronizing brain areas, appears abnormal in patients with schizophrenia. Stimulus evoked gamma band power is lower, compared with healthy subjects, but baseline gamma power is higher. Based on the observation that cholinergic modulation regulates interactions between primary auditory cortex (A1) and an association cortex in the gamma frequency bands, we hypothesize that disrupted cholinergic modulation may underlie abnormal gamma rhythms in schizophrenia. In this study we use a computational model composed with pyramidal cells, fast-spiking (FS) and non-FS inhibitory interneurons. In accordance with experimental data, we simulate the pathological condition by lowering the excitability of non-FS cells. The spectral power densities of local field potentials (LFPs) are compared between control and pathological conditions in pre-stimulus and stimulus periods, respectively. In the simulations, disrupted cholinergic modulation enhances gamma band power in the pre-stimulus period but reduces it in the stimulus period, which is consistent with the patterns of experimentally observed abnormal gamma rhythms. We further note that top-down gamma rhythms suppress A1 responses and that A1 Pyr cells respond to top-down gamma rhythms in the pathological condition. Thus, we propose that cholinergic modulation can underlie the functional dysconnectivity and that the erroneous activation of A1 induced by top-down gamma rhythms in the pathological condition can account for auditory hallucination.