The polarity of ionic liquids (ILs), usually denoted as E_T(30) by the solvatochromic probe Reichardt’s dye, is one of the most fundamental properties that remarkably affect the solvation and chemical reaction in ILs. It was generally accepted that the E_T(30) of ILs was dominated by the nature of the cation. However, in this work, it was found that the common ammonium-based ILs showed strongly anion-dependent E_T(30). For example, the E_T(30) value for [N₁₁₂₄][DCA] and [N₁₁₂₄][NTf₂] is 49.0 and 59.0 kcal mol⁻¹, respectively, while the corresponding imidazolium ILs bearing the same anions possess nearly identical E_T(30), the E_T(30) value for [BMIm][DCA] and [BMIm][NTf₂] is 51.4 and 51.6 kcal mol⁻¹, respectively. Moreover, introduction of an ether group was found to increase the E_T(30) of imidazolium ILs while having no obvious effect on that of ammonium-based ILs. The Kamlet–Taft parameters and density functional theory (DFT) calculations indicated that the distinct result is related to different stabilization of the ground state of Reichardt's dye 30. In imidazolium ILs, the main interactions between ILs and zwitterionic dye involve both coulombic interaction (between the cation and the phenolate oxygen atom) and H-bonding interaction (between the acidic hydrogen on imidazolium ring and the phenolate oxygen atom). However, with the ammonium ILs lack of active hydrogen, the dye is only stabilized by the coulombic interaction between the cation and the phenolate oxygen atom. Interestingly, in both imidazolium and ammonium-based ILs, the spiropyran–merocyanine equilibrium exhibit obvious anion-dependent photochromism, solvatochromism, and thermal relaxation.