With the development of technology, product ecologies have emerged, and cross-device interactions have become increasingly common. Previous research found that operation latency has a significant impact on user experience for single devices, but no study has investigated this relationship in cross-device interaction. This study explored the effect of operation latency on user experience and operation performance for four cross-device tasks: transfer content (click a mobile button to connect with a TV and transfer a mobile page), turn pages (click a button on a mobile device to turn the page on a TV), zoom (enlarge a circle on a TV by zooming a phone screen with two fingers), and drag (slide on a phone to drag a circle on a TV). The results showed that across four cross-device interaction tasks, as the operation latency increased, user satisfaction dropped significantly, and the stalling experience became increasingly noticeable. Specifically, the percentage of users with positive ratings dropped to 50% with a latency of 1300 ms in the transferring content task and 200–350 ms in the page-turning, zooming, and dragging tasks. The percentage of users with negative ratings was larger than the percentage of users with positive ratings and moderate ratings when the latency reached 3500 ms and 500–700 ms in the transferring content task and the other three tasks, respectively. This study provides a reference for companies, engineers and designers for time delay setting and evaluation.