Publications

Large Language Models (LLMs) have shown impressive capabilities in understanding and generating coherent natural language, but they still suffer from hallucinations, i.e. answers that seem coherent but are incorrect. Retrieval-Augmented Generation (RAG) aims at reducing hallucinations by grounding the LLMs with external relevant data sources. The Meta KDD Cup 2024 introduced the Comprehensive RAG (CRAG) challenge, which evaluates Question-Answering (QA) systems across five domains and eight question types. The challenge consists of three tasks: Web-Based Retrieval Summarization, Knowledge Graph and Web Augmentation, and End-to-End RAG. This paper summarizes the UM6P Team’s participation in the first task. We describe our experimental framework including hyperparameter tuning, sampling strategy selection (tobest align the offline and online results), and an extensive evaluation of various RAG pipelines. We also share key insights about the contribution of each RAG component to the overall performance, covering chunking, retrieval, and enhancement techniques. The pipelines were assessed offline using Llama 3 and online using GPT-4 based on the number of correct answers, missing answers, and hallucinations. Our experiments indicate that the best-performing pipeline consists of Facebook AI Similarity Search (FAISS), sentence chunking, re-ranking, and Hypothetical Document Embedding for input enhancement (HyDE), achieving a competitive accuracy score of 0.339 compared to the top score of 0.393, despite a lower overall CRAG score (0.05 vs. 0.204) due to hallucinations (0.288 vs. 0.189). We conclude with a discussion of the main technical and performance challenges encountered during the competition, and some pointers on future research directions.

Nuclei semantic segmentation is a key component for advancing machine learning and deep learning applications in digital pathology. However, most existing segmentation models are trained and tested on high-quality data acquired with expensive equipment, such as whole slide scanners, which are not accessible to most pathologists in developing countries. These pathologists rely on low-resource data acquired with low-precision microscopes, smartphones, or digital cameras, which have different characteristics and challenges than high-resource data. Therefore, there is a gap between the state-of-the-art segmentation models and the real-world needs of low-resource settings. This work aims to bridge this gap by presenting the first fully annotated African multi-organ dataset for histopathology nuclei semantic segmentation acquired with a low-precision microscope. We also evaluate state-of-the-art segmentation models, including spectral feature extraction encoder and vision transformer-based models, and stain normalization techniques for color normalization of Hematoxylin and Eosin-stained histopathology slides.Our results provide important insights for future research on nuclei histopathology segmentation with low-resource data. Code and dataset:https://github.com/zerouaoui/AMONUSEG