Recognizing Multiple Ingredients in Food Images Using a Single-Ingredient Classification Model

Recognizing food images presents unique challenges due to the variable spatial layout and shape changes of ingredients with different cooking and cutting methods. This study introduces a new approach for recognizing multiple ingredients segmented from food images. The method localizes the candidate regions of the ingredients, and then, these regions are assigned into ingredient classes using a CNN-based single-ingredient classification model trained on a dataset of single-ingredient images covering 110 kinds of foundational ingredient categories. Subsequently, the multi-ingredient identification is achieved through a decision-making scheme, incorporating a novel top n algorithm with integrating the classification results from various candidate regions to improve the ingredient recognition accuracy. Experimental results validate the effectiveness and efficiency of our method, particularly highlighting its competitive performance in recognizing multiple ingredients compared to state-of-the-art (SOTA) methods, while offering high interpretability.