Mining users’ opinions towards the rising and falling

trends of the stocks may help the management department

estimate the risk and make timely decision. Existing methods

ignore the effective fusion of domain information and pre

trained language models, hindering mining implicit semantic

information. This paper proposes a hybrid method that adopts

masked language modeling to obtain a domain-information

enhanced language model. Firstly, it generates an attention

mechanism-oriented masking based on words’ importance,

word-level polarity and terminology. Then, the masked words

and their corresponding knowledge are predicted to acquire

domain-aware language representation. Experimental results

on two public financial sentiment analysis datasets show the

efficacy of the proposed model.