Welcome to the BUET CSE NLP Group. We are a group of researchers focusing on tackling problems on natural language processing and machine learning, specifically machine translation, multi-lingual NLP, adapting NLP techniques for programming language and natural language understanding.
The target language of a multilingual model for cross-lingual summarization is limited to only the language it is fine-tuned on, and we have observed that fine-tuning with multiple languages without cross-lingual supervision can not help control the language of the generated summaries.
In this work, we are aiming to generate summaries in any target language for a given article by fine-tuning multilingual models with explicit (albeit limited) cross-lingual signals. By aligning identical articles across languages via cross-lingual retrieval on the XL-Sum dataset, coupled with a multi-stage sampling technique, we are aiming to perform large-scale cross-lingual summarization for 45 languages.
Synthetic paraphrase datasets are typically generated with round-trip machine translation. Since these back-translation-based data generation approaches have been shown to generate appropriate paraphrases.
In this work, we are trying to directly distill the knowledge of translation models into a paraphrase generation model. We are aiming to use two teachers, namely a forward translation model and a backward translation model, to distill two types of knowledge into the paraphrase model: the cross-attention distribution and the output distribution. In constrast to traditional knowledge distillation, here we have two teacher models instead of one and the task of the student model is different from the teacher models.
