The problem IoT Relayer solves

IoT is expected to impact all aspects of our lives in the coming years (if it hasn't already). There were 26.66 billion of active IoT devices as of August 2019. There are 127 new IoT devices connecting to the web every second. With so many uses of IoT and the sheer number of devices using IoT, there is an immense amount of data to be processed and stored. Hence it is important for the IoT systems to manage this data efficiently.
This project simulates 50 bots firing messages on an MQTT topic simultaneously at 500ms intervals. A central computer subsribes to this topic and receives all this data. It then segregates the data according to the type of data, such as temperature, humidity, pH and air pressure and places them into different files. At 5 minute intervals, the data of the past 5 minutes is compressed (upto 85%) using LZMA algorithm and then this compressed data is pushed to the AWS cloud (Dynamodb). If internet connectivity is not available, it it is stored locally and will be pushed chronologically once the system reconnects to the internet.
On the cloud, the data is decompressed and retrieved without any loss, using AWS Lambda functions. This data is then visualized on a Flask server using matplotlib.

Challenges we ran into

1. Finding the optimum algorithm to compress data. We tried LZW, Huffman, Shannon, ZPAQ, ZLIB and BZ2. We finally chose LZMA.
2. Visualization of data on the AWS cloud was difficult since we had no prior experience of AWS Lambda, Athena, S3, Redash or Quicksight.
3. Compression and lossless retrieval of images and videos.