Eutrophication triggered by internal phosphorus (P) poses a substantial threat to the biodiversity of organisms in freshwater ecosystems. However, little is known about the linkages between P resource partitioning and microbial succession, especially in karst sediments. Here, we studied the diversity patterns and assembly processes of bacterial and archaeal communities in sediment cores from two historically hyper-eutrophicated karst lakes, Hongfeng Lake and Aha Lake, and investigated the relative contribution of P fractions to them. Our null and neutral models consistently indicated that bacterial and archaeal community assembly was judged to be deterministic rather than stochastic. We found a monotonically decreasing pattern for bacterial Shannon diversity toward deep sediments in Aha Lake, but U- or hump-shaped patterns for archaea in Hongfeng and Aha Lakes. Intriguingly, the community dissimilarity Bray–Curtis of bacteria and archaea consistently increased with increasing depth distance, with slopes of 0.0080 and 0.0069 in Hongfeng Lake and 0.0078 and 0.0087 in Aha Lake, respectively. Such cross-taxon congruence was well-supported by equivalent ecological processes (i.e., environmental selection). For bacteria and archaea, Shannon diversity was primarily affected by the total P (TP) fractions such as the loosely adsorbed TP or calcium-bound TP and sediment TP. Their community composition was significantly (P < 0.05) affected by calcium-bound inorganic P (Pi), loosely adsorbed Pi and reductant-soluble Pi. Although sediment properties were important, bacterial and archaeal diversity or community composition were well-explained by the Pi fractions, with high direct or indirect effects. In particular, Pi fractions exhibited stronger effects on bacterial and archaeal characteristics than organic P fractions. Taken together, our study provides novel insights into the ecological importance of P resource partitioning to microbial succession, which has crucial implications for disentangling the biogeochemical processes of P cycling in aquatic ecosystems.