Question Answering through Retrieval-Augmented Large Language Models: an Experimental Evaluation

Large Language Models (LLMs) show impressive performance in many natural language processing (NLP) tasks, including code generation and question answering. However, they suffer from various limitations, such as the generation of hallucinated content and reliance on outdated internal knowledge. To address these challenges, Retrieval-Augmented Large Language Models (RA-LLMs) have emerged as a promising solution, enhancing response accuracy by dynamically incorporating external knowledge through retrieval-augmented generation (RAG). Despite the advantages of RA-LLMs, the implementation of effective retrieval-augmented pipelines remains a complex task, since various techniques exist for document chunking, text similarity evaluation, and model selection, each influencing the overall system performance. In this work, we propose a general methodology for designing a RA-LLM pipeline, outlining key approaches for each phase and evaluating the effectiveness of different configurations. We also perform an experimental evaluation of the pipeline’s performance using real-world data, analyzing the impact of different techniques at each stage. The findings of this study offer insights into optimizing RA-LLM architectures for enhanced information retrieval and response generation.