In this study, relevant work on autonomy evaluation (AE) in recent years was comprehensively reviewed and classified from the perspective of task models, and a closed-loop task models based theoretical framework for AE was developed. The main contributions of this study are as follows. 1) A taxonomy for AE based on task models was introduced to classify current theories, methods and standards. 2) The limitations of the current autonomous evaluation methods were addressed to provide a theoretical framework for quantitative evaluation based on task models, and evaluation metrics for each stage were proposed based on the AE theoretical framework. 3) Qualitative analyses of the superiority of the proposed AE framework based on the closed-loop task models were conducted. This study attempts to provide a reference for researchers and engineers in the autonomous unmanned systems field and inspire future development of AE.