The scattering of fast radio bursts (FRBs) by the intergalactic medium (IGM) is explored using cosmological hydrodynamical simulations. We confirm that the scattering by the clumpy IGM has significant line-of-sight variations. We demonstrate that the scattering by the IGM in the voids and walls of the cosmic web is weak, but it can be significantly enhanced by the gas in clusters and filaments. The observed nonmonotonic dependence of the FRB widths on the dispersion measures (DM) cannot determine whether the IGM is an important scattering matter or not. The IGM may dominate the scattering of some FRBs, and the host galaxy dominates others. For the former case, the scattering should be primarily caused by the medium in clusters. A mock sample of 500 sources shows that tau(IGM) alpha DMIGM (1.6-2.1) at z < 1.5. Assuming that the turbulence follows Kolmogorov scaling, we find that an outer scale of L0 similar to 5 pc is required to make tau(IGM) 1-10 ms at nu = 1 GHz. The required L-0 similar to 5 pc can alleviate the tension in the timescales of turbulent heating and cooling but is still similar to 4 orders of magnitude lower than the presumed injection scale of turbulence in the IGM. The gap is expected to be effectively shortened if the simulation resolution is further increased. The mechanisms that may further reduce the gap are shortly discussed. If future observations can justify the role of the IGM in the broadening of FRBs, it can help to probe the gas in clusters and filaments.