Discovering the underneath causal relations is the fundamen tal ability for reasoning about the surrounding environment and predicting the future states in the physical world. Coun terfactual prediction from visual input, which requires simu lating future states based on unrealized situations in the past, is a vital component in causal relation tasks. In this paper, we work on the confounders that have effect on the physi cal dynamics, including masses, friction coefficients, etc., to bridge relations between the intervened variable and the af fected variable whose future state may be altered. We propose a neural network framework combining Global Causal Rela tion Attention (GCRA) and Confounder Transmission Struc ture (CTS). The GCRA looks for the latent causal relation s between different variables and estimates the confounders by capturing both spatial and temporal information. The CTS integrates and transmits the learnt confounders in a residu al way, so that the estimated confounders can be encoded into the network as a constraint for object positions when perform ing counterfactual prediction. Without any access to ground truth information about confounders, our model outperforms the state-of-the-art method on various benchmarks by fully u tilizing the constraints of confounders. Extensive experiments demonstrate that our model can generalize to unseen environ ments and maintain good performance.