In this paper, a novel data-driven control solution to cooperative output regulation problems is proposed for a class of discrete-time multi-agent systems. Different from existing solutions to cooperative output regulation problems, the dynamics of all the followers are presumed unknown. Based on the combination of the internal model principle and the value iteration technique, a distributed suboptimal controller is learned by means of online input-state data collected from system trajectories. Notably, the developed learning algorithm does not rely on a priori knowledge of a stabilizing control policy. Rigorous theoretical analysis guarantees the convergence of the algorithm and the stability of the closed-loop system. Numerical results validate the effectiveness of the proposed control methodology.