Grant Abstract: Nutri-DeepDish: Enhancing AI-Powered Dietary Assessment for Accurate, Real-World Food Intake Measurement

Grant Number: 1R41DK146772-01
PI Name: Hennessy
Project Title: Nutri-DeepDish: Enhancing AI-Powered Dietary Assessment for Accurate, Real-World Food Intake Measurement

Abstract: This research seeks to produce a rapid and reliable tool for portion size estimation and nutrition logging that uses computer vision and deep learning to predict the nutritional content of a meal based on an image. This system will aid nutritionists and patients in supervised treatment programs as well as individual users in understanding dietary intake as an indicator of overall health. The project will expand a dietary assessment system based on semantic segmentation and monocular depth estimation, improving the system’s ability to recognize a wide variety of foods and adapt to various real-world serving scenarios. This differs from existing methodologies in that it does not require a calibration marker to be included in the image and uses state-of-the-art monocular depth estimation techniques instead of spherical or other geometric estimation that is constrained to a specific shapes and patterns. The goal of this research is to produce a tool that performs better than human estimation on real-world images of meals. This STTR will expand the recognition capabilities of the existing system, strengthen its capabilities with non-homogeneous foods and leftovers, and statistically validate the system against weighed plate waste and human estimation capabilities in a real-world eating environment. The proposed research will focus on three main aims: • Aim 1: Expand the recognition capabilities of Nutri-DeepDish to include over 250 classes of foods prepared in a free-living environment. • Aim 2: Strengthen the system’s capabilities to accurately recognize non-homogenous foods and leftovers with over 85% accuracy for calorie estimation. • Aim 3: Statistically validate the system against weighed plate waste and determine its added value over human portion size estimation capabilities. This research will enhance the effectiveness of nutritional interventions and improve the rigor of nutrition research by providing easily recorded, highly accurate portion consumption data. Ultimately, our goal is to produce a tool that performs better than human estimation on real-world images of meals, significantly reducing user estimation error and cognitive burden in food logging.

Back to Grants Page