Taxi is one of the most common public transport, predicting taxi demand precisely within an area is of great signifcance for improving efciency of trafc. Taxi demand is spatialtemporal data, and highly influenced by many external factors, such as time, weather. So there are two main problems on taxi demand prediction: the one is that modeling both spatial and temporal non-linear correlations is not easy, the other is that real scenarios exist temporal but non-spatial side information, which is hard to be fused with spatial-temporal taxi demand. To handle the two problems, in this paper,we propose a novel Side information fused Spatial-Temporal
Network(SideInfo-STNet) framework to model correlations of time, space and side information. The framework has three main components: Spatial-temporal Taxi Demand (transforming raw taxi demand to taxi demand image sequences); Side Information (transforming side information to time series vectors); SideInfo-STLSTM (extending LSTM to has convolution structures and fusing side information into LSTM gate units). By using SideInfo-STNet to conduct extensive experiments on large-scale TLC trips of New York City, we validate that our model outperforms traditional and deep learning based models on taxi demand prediction.