Despite much effort to probe the properties of dimethyl sulfoxide (DMSO) solution, effects of DMSO on water, especially near plasma membrane surfaces still remain elusive. By performing molecular dynamics (MD) simulations at varying DMSO concentrations (XDMSO), we study how DMSO affects structural and dynamical properties of water in the vicinity of phospholipid bilayers. As proposed by a number of experiments, our simulations confirm that DMSO induces dehydration from bilayer surfaces and disrupts the H-bond structure of water. However, DMSO enhanced water diffusivity at solvent-bilayer interfaces, an intriguing discovery reported by a spin-label measurement, is not confirmed in our simulations. In order to resolve this discrepancy, we examine the location of the spin-label (Tempo), relative to the solvent-bilayer interface. In accord with the evidence in the literature, our simulations, which explicitly model Tempo-PC, find that the Tempo moiety is equilibrated at ~ 8-10 Å below the bilayer surface. Furthermore, the DMSO-enhanced surface water diffusion is confirmed only when water diffusion is analyzed around the Tempo moiety that is immersed below the bilayer surface, which implies that the experimentally detected signal of water using Tempo stems from the interior of bilayers, not from the interface. Our analysis finds that the increase of water diffusion below the bilayer surface is coupled to the increase of area per lipid with an increasing XDMSO (< 10 mol%). Underscoring the hydrophobic nature of Tempo moiety, our study calls for careful re-evaluation of the use of Tempo in the measurement on lipid bilayer surfaces.