98.57% Accuracy Using Image Recognition To Identify Food Dishes & Menu Items | Case Study

Matt Payne
January 27, 2021

Best Place For was looking for an image recognition based software solution that could be used to detect and identify different food dishes, drinks, and menu items in images sourced from blogs and Instagram. The images would be pulled from restaurant locations on Instagram and different menu items would be identified in the images. This software solution has to be able to handle high and low quality images and still perform at the highest production level, while accounting for runtime as well as accuracy.

Solution: built an image recognition software solution to recognize different food and drink menu items in images sourced from Instagram and blogs. The model was built using a specific set of popular menu items from restaurants and used a custom built high variance dataset to improve generalization and scalability as we add more restaurants to the model. This allows us to build a noise distortion function that shows the model very difficult images during training and forces the model to learn the exact features of menu items that distinguish each of them from each other. The architecture of the model perfectly takes into account both accuracy and run time requirements, especially for a domain like food recognition which outside of academia has very little open source information.

This heatmap shows what parts of the image the model uses

Results: On the full menu item classification dataset, the model is 98.57% accurate at finding the correct menu item in different images from all the above sources. The model runs at a production speed to be used in real time to grab and label images straight from the different menu sources.

Looking to add image recognition to your companies software stack? Want to see results like 98.57% on different datasets? Lets talk