The problem My Research Pal solves

Research and experimental development have long been recognized as fundamental pillars of the advancement of the human race. According to the Western Sydney University’s Department of Education, research can be defined as ‘the creation of new knowledge and/or the use of existing knowledge in a new and creative way so as to generate new concepts, methodologies and understandings’. A UNESCO Science Report from 2021 stated that by 2018, the world was home to 8.854 million full-time equivalent researchers. Every year, millions of research publications are published in thousands of peer-reviewed journals, contributing to an ever-evolving corpus of academic literature.
In this age where the scholarly publications inventory is only augmenting with each passing day, researchers are faced with the strenuous task of sifting through the large number of resources available, in order to find the most relevant and valuable, high-quality papers. This task becomes even more daunting due to the increased pressure of being up-to-date with the latest research and development in their area of study, leading to most of their time being spent in trying to filter and sort apropos content from the sheer volume of research work available.
The academic search engines that are currently available contain substantial data and are often thoroughly profound. However, they lack one essential aspect that can make the lives of researchers drastically uncomplicated and that is- personalized recommendations. In order for research to actually yield the best possible results and truly become successful, it is vital to ensure that no opportunity to discover pertinent work is missed, especially due to the abundance of information, which by all means should be a boon and not a bane.

Challenges we ran into

An initial web application was developed utilizing Next.js, Node.js, Express.js, and Clerk. However, challenges arose in constructing an API to integrate a Google Colab file with the frontend. To circumvent this, the project transitioned to Streamlit for streamlined integration. Despite this change, the desired level of accuracy in results was not attained. To enhance model performance, an abstract text box was introduced, enabling the extraction of keywords directly from the abstract. This modification significantly improved the relevance and precision of recommendations and predictions, surpassing the limitations encountered during the initial integration phase. Additionally, to mitigate overfitting, an early stopping mechanism was implemented to terminate the training process upon reaching a plateau in model accuracy.