Skip to Main content Skip to Navigation

Intégration de Connaissances aux Modèles Neuronaux pour la Détection de Relations Visuelles Rares

Abstract : Data shared throughout the world has a major impact on the lives of billions of people. It is critical to be able to analyse this data automatically in order to measure and alter its impact. This analysis is tackled by training deep neural networks, which have reached competitive results in many domains. In this work, we focus on the understanding of daily life images, in particular on the interactions between objects and people that are visible in images, which we call visual relations.To complete this task, neural networks are trained in a supervised manner. This involves minimizing an objective function that quantifies how detected relations differ from annotated ones. Performance of these models thus depends on how widely and accurately annotations cover the space of visual relations.However, existing annotations are not sufficient to train neural networks to detect uncommon relations. Thus we integrate knowledge into neural networks during the training phase. To do this, we model semantic relationships between visual relations. This provides a fuzzy set of relations that more accurately represents visible relations. Using the semantic similarities between relations, the model is able to learn to detect uncommon relations from similar and more common ones. However, the improved training does not always translate to improved detections, because the objective function does not capture the whole relation detection process. Thus during the inference phase, we combine knowledge to model predictions in order to predict more relevant relations, aiming to imitate the behaviour of human observers
Document type :
Complete list of metadata

Cited literature [174 references]  Display  Hide  Download
Contributor : ABES STAR :  Contact
Submitted on : Wednesday, August 19, 2020 - 9:32:13 AM
Last modification on : Friday, January 15, 2021 - 5:24:10 PM
Long-term archiving on: : Monday, November 30, 2020 - 9:30:25 PM


Version validated by the jury (STAR)


  • HAL Id : tel-02917340, version 1



François Plesse. Intégration de Connaissances aux Modèles Neuronaux pour la Détection de Relations Visuelles Rares. Apprentissage [cs.LG]. Université Paris-Est, 2020. Français. ⟨NNT : 2020PESC1003⟩. ⟨tel-02917340⟩



Record views


Files downloads