Lavrio is a Greek town with several abandoned Ag/Pb mines. In this study, 19 potentially toxic elements (PTEs) were measured in soil, weeds, and olives. Levels of seven of the studied PTEs in soil were highly elevated: Zn (56.2–58,726 mg kg^-1), Pb (36.2–31,332), As (7.3–10,886), Cu (8.3–1273), Sb (0.99–297.8), Cd (0.17–287.7), and Ag (0.09–38.7). Synchrotron-based X-ray absorption near edge structure analysis of the soils revealed that As was predominantly associated with scorodite, Pb with humic substances, Zn with illite, Zn(OH)₂ and humic substances, and Fe with goethite-like minerals. The transfer of the PTEs to weeds was relatively low, with the transfer coefficient being less than 1.0 for all PTEs. Cadmium in table olives surpassed 0.05 mg kg^-1 fresh weight (the limit in EU), while Pb surpassed its limit in approximately half of the samples. Health risk assessment confirmed soil contamination in the study area where As and Pb hazard quotients were well above 1.0 and the average hazard index equaled 11.40. Additionally, the cancer risk values exceeding the 1 × 10^-4 threshold. The results obtained in the study indicate that Lavrio urgently requires an adequate ecofriendly remediation plan, including revegetation with tolerant species and targeted efforts to chemically stabilize harmful PTEs. The presented approach may serve as a pivotal study for industrial areas with similar contamination levels.