The problem Fresh Eye: Eat Everything You Buy solves

Food waste is a global crisis, with roughly one-third of all food produced for human consuption being wasted or lost throughout the supply chain, from farms to consumer. To combat the inefficiencies in food storage, Fresh Eye aims to implement computer vision using our trained machine learning models to estimate food spoilage solely using a camera. By assigning classificaitons to food based on their color, texture, and blemishes, by continuously monitoring changes over time, it accurately predicts when food is likely to spoil. This empowers users to make informed decisions regarding their food consumption, reducing unnecessary discards and saving money.. Unlike current technology which uses expensive and inacccessible sensor technology to monitor freshness, Fresh Eye solely uses simple cameras that can easily be implemented in consumer fridges to continously monitor the state of their produce for widespread adoption. It is also accounts for every individual food in its fieldview, something sensor technology cannot. It also implements real time tracking of cost loss from food waste, allowing users to see the financial effects of letting food go to waste, which users can see on a simple dashboard on their smart fridge, a growing industry that already includes 13% of households. The impact of our project extends beyond individual households. It can be seamlessly integrated into commercial settings, such as grocery stores and restaurants, to prevent spoilage on a larger scale, including production farms and transport. By implementing our system at various points in the supply chain, we can significantly reduce food waste and promote sustainability. By empowering individuals and businesses to proactively manage their food inventory, we can make significant strides in reducing global food waste and building a more sustainable future.

Challenges we ran into

When we decided to create our own model instead of using a pre-trained one, we ran into two issues: time and data. Finding data sets for fruit was easy, but specific ones for stages of decomposition was more difficult and we were only able to find one on some fruits within this time period, even though these images are widely available. The second is time, in order to have more accurate models we wanted to use a baseline of 50 epochs when training them, however because of computer and financial constraints, training one model one time took 10 hours on average. We ended up fine-tuning a pretrained model at the start so that we would have something to build the rest of an app off of and we also used Google Colab's extra GPU's to speed up the process for both the created and fine-tuned model. Even so, we had to use both models in order to implement accurate reading of fruit freshness. Next was implementing this model using OpenCV's library, which prior to this hackathon only one team member had experience with. In order to accurately label and get data from frames to real time object detection and tracking, several of our group members had to use online googling resourceful and chatGPT4 prompt engineering in order to parse for the information we wanted as well as label our objects in real time and convert that into usable data on the front-end. We also had to worry about non max suppression, as the food items were logging various classes segmented, we had to resolve this with a non max suppression function. We created a test frontend for this purpose, as the main frontend was still in development until the end, however we were able to create a beautiful frontend UI that users will be able to log in to, add food to already tracked items, and to find recipes based on inputted ingredients.