The innovation behavior (IB) of followers is vital for individuals and organizations. It is not only an important part of individual performance but also an essential foundation of organizational innovation. In general, understanding the formation mechanism of followers’ IB could enhance organizational innovation performance and sustainable competitiveness. The innovation of this study includes the following points. First, in the previous research on the antecedent variables of the IB of followers, the key factor of implicit follow cognitive difference (IFCD) was not examined. We add this key factor to the independent variables of our formation mechanism. Second, in the relationship between IFCD and followers’ IB, we investigate the mediating role of psychological empowerment (PE) and the moderating role of person-organization Fit (P-O Fit), further shown as a mediation model with moderating variables. Third, this study adopts a longitudinal research design, and the data were obtained from 1:1 matched leaders and their followers of large and medium-sized enterprises in China. To avoid deviation in the homologous method, this study gathers data at three- month intervals to ensure that leadership’s influence on followers is effective. In this study, Amos 24.0 and SPSS 24.0 are used for empirical research. The results show IFCD has a negative effect on the IB of followers; PE has a partial mediating effect on the relationship between IFCD and IB of followers; P-O Fit plays a positive regulatory role in the relationship between IFCD and PE, and P-O Fit moderates the mediating effect of PE on the relationship between IFCD and IB. Based on our empirical research, we put forward some feasible suggestions for company managers to increase sustainability in market competition by promoting the formation of the IB of followers.