Accurate pre-operative overall survival (OS) prediction of gastric patients is of great significance for personalized treatment. However, the accuracy of OS prediction has been limited by existing methods. To facilitate improvement of survival prediction, we propose a novel joint multi-task network equipped with multi-level features simultaneously predicting clinical tumor and node stages. Two independent datasets including a training set (377 patients) and a test set (122 patients) are used to evaluate our proposed network. The results indicated that the multi-task network exploits its recipe by capturing multi-level features, and sharing prognostic information from correlated tasks of clinical stages prediction, which enable our network to predict OS accurately. Our method outperforms the existing methods with the highest c-index (training: 0.73; test: 0.72). Meanwhile, our method shows better prognostic value with the highest hazard ratio (training: 3.77; test: 4.28) for dividing patients into high- and low-risk groups.