Sulfate aerosols exert profound impacts on human and ecosystem health, weather, and climate, but their formation mechanism remains uncertain. Atmospheric models consistently underpredict sulfate levels under diverse environmental conditions. From atmospheric measurements in two Chinese megacities and complementary laboratory experiments, we show that the aqueous oxidation of SO2 by NO2 is key to efficient sulfate formation but is only feasible under two atmospheric conditions: on fine aerosols with high relative humidity and NH3 neutralization or under cloud conditions. Under polluted environments, this SO2 oxidation process leads to large sulfate production rates and promotes formation of nitrate and organic matter on aqueous particles, exacerbating severe haze development. Effective haze mitigation is achievable by intervening in the sulfate formation process with enforced NH3 and NO2 control measures. In addition to explaining the polluted episodes currently occurring in China and during the 1952 London Fog, this sulfate production mechanism is widespread, and our results suggest a way to tackle this growing problem in China and much of the developing world.