The primary goal of this review is to present a clear chemical perspective of borates in order to stimulate and facilitate the discovery of new borate-based optical materials. These materials, which exhibit structures as varied as they are complex, are needed to meet the urgent technological milestones. In the current period of rapid socio-technological breakthroughs, the need for the rational design and discovery of novel borates with superior performance is greater than ever before. Through the sustained efforts of chemists and material scientists, more than 3900 boron-containing compounds, including borate minerals and synthetic borates, have been documented in the scientific literature. This review provides a survey of all the reported anhydrous borates and an analysis of their complex structural chemistry. State-of-the-art progress related to technological advances in borate-based nonlinear optical, birefringent, and self-frequency-doubling materials is surveyed, with special emphasis on the relationships between structural architectures and optical properties. More importantly, this review serves both as a scientific introduction for graduates and post-doctoral researchers to the chemical richness of solid-state borates and as a comprehensive reference for researchers interested in borate-based optical materials.