Recommender system (RS) has been evaluated in many but incomparable ways beyond accuracy and thus proposing an evaluation framework to synthesize the existing strategies seems a solution. However, few scholars did it so far. Through literature review, user interview and expert assessment, this study proposed a theoretical evaluation model of RS and then formed the assessment tool, RS Evaluation Questionnaire (RSE). The results showed that RSE was an effective tool to evaluate a recommender system, with its reliability (Cronbach’s α=0.803) and validity meeting the requirements of psychometrics. Seven factors such as Perceived Quality and Perceived Ease of Use were generated by factor analysis, accounting for 63.126% of the variance. Furthermore, regression analysis indicated that different combinations of RSE factors could significantly predict User Satisfaction, Reuse Intention and positive Word-Of-Mouth (WOM) spreading willingness. Enlightenments for future research and practice were discussed as well in the end.