Pre-training details with a little proportion of multi-activity instruction information improves the general model effectiveness On this training aim, tokens or spans (a sequence of tokens) are masked randomly along with the model is asked to forecast masked tokens presented the past and long