# MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer. (arXiv:2401.04821v1 [cs.CL])


Transformer-based pre-trained language models (PLMs) have achieved remarkable
performance in various natural language processing (NLP) tasks. However, pre-
training such models can take considerable resources that are almost only
available to high-resource languages. On the contrary, static word embeddings
are easier to train in terms of computing resources and the amount of data
required. In this paper, we introduce MoSECroT Model Stitching with Static Word
Embeddings for Crosslingual Zero-shot Transfer), a novel and challenging task
that is especially relevant to low-resource languages for which static word
embeddings are available. To tackle the task, we present the first framework
that leverages relative

[Read Full Article](http://arxiv.org/abs/2401.04821)