WP5: Integrative AI for Natural Language Understanding

Task Lead: FBK, PI: Bernardo Magnini, co-PI: Luisa Bentivogli

We will tackle fundamental challenges related to Integrative AI for Natural Language Understanding. The state-of-the-art neural models will be able to integrate knowledge from heterogeneous sources, will be well integrated with societal needs, adapted into realistic and rich communicative contexts, and combined to obtain comprehensive models. 

Connected to TP Vision, Languages in Multimodal Challenges (TP2).

Effort 36p/m + cascading funds.

Description of work 

This WP addresses new challenges in Integrative AI for Natural Language Understanding (NLU), through the combination and  integration of state-of-the-art neural models.

Task 2.5.1 – Integrating heterogeneous knowledge in NLU Task leaders: Bernardo Magnini, Luisa Bentivogli

Recent approaches to NLU are based on large-scale pre-trained neural models, which are then fine-tuned to downstream tasks, domains, or languages.  While such models achieve good performance when common sense knowledge is required, they are still unable to take advantage of the variety of (structured/unstructured) knowledge available for downstream tasks and domains, as well as to cope with specific constraints that might be imposed. Integrating heterogeneous knowledge sources into pre-trained neural models may produce high performing NLU, including models able to cope with implicit knowledge, not expressed in text (e.g. for explainable AI). Several challenges will be addressed in this task, including: (i) the creation of models that can be easily adapted and better controlled to meet specific application requirements (e.g. for machine translation, subtitling, simultaneous translation); (ii) the design of novel approaches to integrate external knowledge into models, such as databases, ontologies, knowledge graphs, glossaries, and terminologies; (iii) approaches for instruction-based learning (e.g. chain of thoughts); (iv) the generation and integration of realistic synthetic data; and (v) methods to leverage user feedback in model training.

Task 2.5.2 – Align NLU models with societal requirements Task leaders: Bernardo Magnini, Luisa Bentivogli

AI systems at large, and therefore also NLU components, are increasingly integrated into our daily life and in today's society. In light of their widespread use (e.g. autonomous driving,  assisted diagnostic and clinical decision-making), their reliability and accountability are of utmost importance. Also, their output should not be offensive or discriminatory toward specific socio-demographic groups. Last but not least, their massive scale deployment should safeguard the environment. This task aims to develop NLU models that tackle real societal needs, account for inherent technology risks, and can therefore be reliably integrated guaranteeing fair behavior. Among other aspects, the task will focus on i) identifying and mitigating the biases affecting current NLU technology (e.g. towards gender identities and minority groups), ii) increasing NLU reliability in critical sectors (e.g. medical diagnosis), and iii) reducing environmental impact (e.g. through compact, easily trainable and efficient models).

Task 2.5.3 – Adapt NLU models to complex and dynamic communicative contexts Task leaders: Bernardo Magnini, Luisa Bentivogli

Current NLU models are trained to operate in rather simplified situations, which limits their capabilities to cope with more realistic and complex communicative contexts.  As an example, task-oriented dialogue systems are often trained on data that capture only a fraction of the collaborative behaviors of human-human conversations. This task aims at integrating NLU models into realistic communicative contexts, so that models can adapt to contextual situations and evolve as the context changes.  There are several challenging aspects that will be addressed, including (i) copying with domain changes during time, in order to avoid models that soon become obsolete; (ii) increasing model robustness, particularly in situations where the input quality varies significantly, or the input was not seen during training; (iii) face with rich communication contexts, addressing pragmatic aspects of language recognition and generation based on emotions, persuasive communication, and rhetoric phenomena; (iv) design context-informed methodologies, able to recognize context situations that require model adaptation (e.g. in human-computer dialogues); (v) allow multimodal text generation and understanding (e.g. multimodal associations between visual information and text); (vi) design NLU models that can be grounded on the physical world (e.g. for the interaction with a robot). 

Task 2.5.4 – Design comprehensive NLU models Task leaders: Bernardo Magnini, Luisa Bentivogli

NLU tasks have typically been approached in isolation, with the sole objective of reaching high performance. With a few exceptions, often based on multitasking and transfer learning, this has led to a fragmentation of technological solutions. The consequent redundancy of components dedicated to different tasks raises the need to orchestrate them for generating, manipulating, and extracting information from text and other input sources (e.g., speech). In order to generate increasingly rich outputs with systems featuring reduced computational cost and multilingual capabilities, this task aims to create integrated models capable of (i) performing different activities by means of a single technology, (ii) making the most of heterogeneous data, (iii) transferring knowledge to cope with under-resourced multilingual settings (e.g., cross-lingual transfer through adapters), and (iv) jointly optimizing different objective functions and evaluation metrics. Particular emphasis will be given to the development of multifaceted evaluation approaches able to measure performance accounting for several dimensions in an integrated framework.