RAG Information Extractor
RAG Information Extraction System uses RAG (Retrieval Augemented Generation) to streamline the process of retrieving, and answering questions on vast amounts of online an latest information.
Created on 28th February 2024
•
RAG Information Extractor
RAG Information Extraction System uses RAG (Retrieval Augemented Generation) to streamline the process of retrieving, and answering questions on vast amounts of online an latest information.
The problem RAG Information Extractor solves
Problem:
Searching information online is very tedious task, searching and filtering every site consumes a lot of time and effort.
alternatively, using pre-trained models like GPT to search information may result in failure in the integrity of the data. Additionally, the data generated is not exactly correct and accurate because it relies on training of the model, due to which the model is able to provide a very limited data only, on which it is trainied. (Example: ChatGPT fails in the query "Indian railway accidents in 2023")
Solution:
This RAG based information extractor scraps latest and true form of any data from online sources and retrives very relevant information from the corpus of the scrapped data, based on the queries given by the user.
The information provided by the App is very ordered, highly accurate, and relevant to the query given.
This helps in searching accurate information on any given topic, efficiently. The Application is very simple to use and user friendly. Plus the searched information can be exported into PDF for storing records also.
Challenges we ran into
The major challenge we faced was converting the scrapped data into Vector embeddings to insert the embeddings into the vector database for similarity search.
However, we ultimately made it by using advanced language models like BERT from google.