Data-driven prognostic methods centered on deep learning are attracting ever-increasing attention. However, most existing methods mainly provide point estimates about RUL without quantifying predictive uncertainty, making it difficult to determine the confidence of the prediction results. Additionally, the uncertainty of the degradation process can cause distribution discrepancies between different machines, even under the same working conditions. While most probabilistic deep learning methods can provide reliable predictive uncertainty over in-distribution data, these methods tend to output overconfident predictions under the covariate shift situation where data distribution changes across domains while the conditional distribution stays the same. To solve the above two issues, this paper proposes a Bayesian framework-based method that is easy to implement and can provide high-quality predictive uncertainty under covariate shift. The proposed method provides a framework to unify uncertainty quantification and distribution alignment by probabilistic modeling. First, this method models both model and data uncertainty to make more reliable maintenance decisions. The model uncertainty is handled by variational inference that approximates the posterior distribution over the model parameters. The data uncertainty is captured by Gaussian distribution parameterized. Then we adopt domain adaptation to reduce the distribution discrepancy of different domains, which are extracted by the Bayesian framework-based model. The calibration of uncertainty estimates can provide a more appropriate quantification of uncertainty under covariate shift. The proposed method is validated using vibration signals obtained from the accelerated degradation of rolling element bearings. The results of the experiments show the effectiveness of RUL prediction of machinery.