We present near-IR (NIR) and optical observations of the Type Ic supernova (SN Ic) SN 2020oi in the galaxy M100 and the broad-lined SN Ic SN 2020bvc in UGC 9379, using Gemini, Las Cumbres Observatory, Southern Astrophysical Telescope, and other ground-based telescopes. The NIR spectrum of SN 2020oi at day 63 since the explosion shows strong CO emissions and a rising K-band continuum, which is the first unambiguous dust detection from an SN Ic. Non-LTE CO modeling shows that CO is still optically thick and that the lower limit to the CO mass is 10?3 M⊙. The dust temperature is 810 K, and the dust mass is ～10?5 M⊙. We explore the possibilities that the dust is freshly formed in the ejecta, heated dust in the preexisting circumstellar medium, and an infrared echo. The light curves of SN 2020oi are consistent with a STELLA model with canonical explosion energy, 0.07 M⊙ Ni mass, and 0.7 M⊙ ejecta mass. A model of high explosion energy of 1052 erg, 0.4 M⊙ Ni mass, and 6.5 M⊙ ejecta mass with the circumstellar matter reproduces the double-peaked light curves of SN 2020bvc. We observe temporal changes of absorption features of the IR Ca ii triplet, S i at 1.043 μm, and Fe ii at 5169 ?. The blueshifted lines indicate high velocities, up to 60,000 km s?1 for SN 2020bvc and 20,000 km s?1 for SN 2020oi, and the expansion velocity rapidly declines before the optical maximum. We present modeled spectral signatures and diagnostics of CO and SiO molecular bands between 1.4 and 10 μm.