Supervision
Students I have supervised or currently supervising
Student name: Kevin NanheKhan
Topic: Efficient retrieval for Open-Domain Fact Checking
Description:Efficient retrieval approach for fact-checking. Imporves efficiency on CPU by 10x and GPU by 20X.
Status: Graduated
Student name: Alexandru Dumitru
Topic: TimeLine based List Question Answering (TLQA)
Description: Timeline-based List Question Answering or TLQA benchmark that requires structured answers in list format with corresponding time periods. Our TLQA benchmark, can be considered a specialized subset of ListQA that requires both list comprehension and temporal understanding. We investigate the temporal reasoning and list comprehension capabilities of state-of-the-art generative models on TLQA in closed-book and open-domain settings. Our findings reveal significant shortcomings in current models, particularly their inability to provide complete answers and temporally align facts in a closed-book setup. To address these issues, we further explore how retrieval augmentation for the LLM in an open-domain setup can enhance performance in this task. Our results indicate that while augmenting with evidence from retrieval improves performance, there remains considerable room for further improvement in retrieval and downstream reasoning, providing clear future directions for research on TLQA. The benchmark and the evaluation of the models can be accessed at repo.
Project Link
Status: Graduated
Student Name: Deepali Prabhu
Topic: Towards Improving Retrieval for the Verification of Natural Numerical Claims
Description: Verification of numerical claims is critical as they tend to be more believable despite being fake and have previously demonstrated the potential to cause catastrophic impacts on society. While there currently exist several automatic fact verification pipelines, only a handful focus on natural numerical claims. A typical human fact-checker first retrieves relevant evidence addressing the different numerical aspects of the claim and then reasons about them to predict the veracity of the claim. Hence, the retrieval thought process of a human fact-checker is a crucial skill that forms the foundation of the verification process. Emulating a real-world setting is essential to aid in the development of automated methods that encompass such skills. Hence, we introduce \quantempplus{}: a dataset consisting of natural numerical claims, an open domain corpus, and the corresponding evidence relevance and veracity labels. Given this dataset, we also aim to characterize the retrieval performance of key query planning paradigms, especially those of decomposition as they have shown promising results in other tasks. Finally, we observe their effect on the outcome of the verification pipeline and draw insights.*
Status: Graduated