BACKGROUND: Poor sleep quality is comparable to traditional risk factors for diabetes. This study evaluated the association between sleep quality index (SQI) and the risk of diabetes incidence in China. METHODS: We used data from the China Kadoorie Biobank, comprising 462,743 participants aged 30 to 79 years without diabetes at baseline, who were recruited between 2004 and 2008 from ten diverse regions across China. We developed a SQI to assess sleep quality, based on seven self-reported sleep behaviors. The association of SQI (a lower value reflects a better sleep quality) on diabetes risk was assessed using Cox proportional hazards models, adjusting for a range of covariates including mainly potential risks factors of diabetes, i.e., waist circumference, body mass index, frailty, and poor health. RESULTS: During an average follow-up of 11.7 years, a total of 18,397 incident cases of diabetes were identified among study participants. Compared with individuals with a SQI of 0, those with SQI of 1, 2, 3, or ≥ 4 had adjusted hazard ratio of 1.07 [95% confidence interval: 1.05–1.10], 1.17 (1.13–1.21), 1.12 (1.06–1.19), and 1.28 (1.17–1.39), respectively. Additionally, each one-unit higher SQI was associated with 6% higher risk. The association was very consistent between sexes and across various subgroups. Our study also found that sleep quality is closely linked to various socioeconomic and lifestyle factors, including education, occupation, levels of physical activity, sedentary habits, and adiposity. CONCLUSIONS: Poor sleep quality was a significant risk factor of developing diabetes in Chinese adults from both rural and urban areas. Promoting a healthy sleeping habit may be a promising public health strategy for the prevention of diabetes. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12889-025-25993-y.