The detection of polycyclic aromatic hydrocarbons (PAHs) on fruit and vegetable surfaces is important for protecting human health and ensuring food safety. In this study, a method for the in situ detection and identification of PAH residues on fruit and vegetable surfaces was developed using surface-enhanced Raman spectroscopy (SERS) based on a flexible substrate and lightweight deep learning network. The flexible SERS substrate was fabricated by assembling β-cyclodextrin-modified gold nanoparticles (β-CD@AuNPs) on polytetrafluoroethylene (PTFE) film coated with perfluorinated liquid (β-CD@AuNP/PTFE). The concentrations of benzo(a)pyrene (BaP), naphthalene (Nap), and pyrene (Pyr) residues on fruit and vegetable surfaces could be detected at 0.25, 0.5, and 0.25 ug/cm2, respectively, and all the relative standard deviations (RSD) were less than 10%, indicating that the -CD@AuNP/PTFE exhibited high sensitivity and stability. The lightweight network was then used to construct a classification model for identifying various PAH residues. ShuffleNet obtained the best results with accuracies of 100%, 96.61%, and 97.63% for the training, validation, and prediction datasets, respectively. The proposed method realised the in situ detection and identification of various PAH residues on fruit and vegetables with simplicity, celerity, and sensitivity, demonstrating great potential for the rapid, nondestructive analysis of surface contaminant residues in the food-safety field.