Glasshouse Images / Getty Images

You can put MIT's Pic2Recipe to the test.

July 21, 2017

When you post your latest food porn pic to your Instagram feed, you have one thing on your mind: being the envy of all your hungry friends. But imagine just how envious you would be if your followers could take that pic and instantly translate it into a recipe to make themselves. Computer scientists believe that future may be more fact than fiction.

According to MIT News, researchers from the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL) were able to train an artificial intelligence system called Pic2Recipe (okay, so they’re scientists, not marketing gurus) to analyze a photo of a dish, then predict the ingredients and recommend similar recipes. “In computer vision, food is mostly neglected because we don’t have the large-scale datasets needed to make predictions,” said Yusuf Aytar, who co-authored a paper on the research. “But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences.”

One of the most difficult parts of the project was creating datasets large enough for the AI to “learn” from so it could then go on to make accurate predictions. The CSAIL team eventually built “Recipe1M,” an annotated database of over 1 million recipes built by combing websites like All Recipes and Food.com. The resulting Pic2Recipe AI is currently available as an online demo where anyone can upload food photos and test it out.

The researchers admit Pic2Recipe has its strengths and limitations. The AI works well with baked goods like cookies or muffins, however, the MIT team says “more ambiguous foods, like sushi rolls and smoothies,” have proven more difficult. It also struggles when one dish can be made with a wide variety of recipes. (Lasagna was given as an example.)

Courtesy of Jason Dorfman / MIT CSAIL

At the same time, however, artificial intelligence is still a new and rapidly developing technology, and the researchers believe the system has lots of avenues for improvement in the future. The eventual applications could also go beyond simply getting cooking instruction—perhaps helping with dietary information. “This could potentially help people figure out what’s in their food when they don’t have explicit nutritional information,” said Nick Hynes, another co-author. “For example, if you know what ingredients went into a dish but not the amount, you can take a photo, enter the ingredients, and run the model to find a similar recipe with known quantities, and then use that information to approximate your own meal.” The implication is that even if you’re not intelligent enough to eat healthily, artificial intelligence hopefully is.