The Late Paleozoic Taiyuan Formation in North China is mainly composed of a mixed shallow-marine carbonate and terrigenous clastic deposits. Its basal limestones have been constrained in the late Pennsylvanian to Early Permian. To further constrain the age of the lowest Taiyuan Formation, we obtained two genera and 16 species of conodonts from the bottom limestones of the Taiyuan Formation in two sections in Henan Province, southern North China. The fauna includes Idiognathodus hebeiensis, Streptognathodus isolatus, S. elongatus, S. cf. longus, S. acuminatus, S. cf. recreatus, S. cf. cristellaris, S. bellus, S. invaginatus, S. wabaunsensis, S. glenisteri, S. conjunctus, S. binodosus, S. fuchengensis, S. nodulinearis, and S. sp. A genus of Streptognathodus dominated conodont assemblage with some Idiognathodus elements. It indicates a late Gzhelian (latest Pennsylvanian) age for the lowest Taiyuan Formation in southern North China. This is consistent with recently published high-precision zircon U-Pb ages from ash layers. Based on conodont biostratigraphy, the basal limestones of the Taiyuan Formation in Henan Province (southern North China) can be correlated with the upper part of Miaogou limestones of the Taiyuan Formation in Shanxi Province (northern North China). This correlation might reflect a significant sea-level rise in North China, possibly corresponding to a deglaciation event at the Permo-Carboniferous transition.