Nucona

Our client, Nucona - The American Foundation for Nutrition Consumption Advancement, approached us with two challenges - extending their Chrome extension and creating a user-friendly mobile app - both dedicated to promoting healthier eating habits and providing nutritional guidance. They sought to expand their existing Chrome plug-in, which at that time worked only for Walmart, also to include Amazon, Costco, etc., allowing them to reach a broader audience and expand their user base. Additionally, Nucona wanted to create a mobile app for both iOS and Android to offer users a similar experience when shopping in physical stores, including scanning products and receiving health guidance. The challenge for Interactivated was to obtain product information for online products (including but not limited to category, volume, nutrition information, and ingredients) since this data is not always readily available on online product pages. With that, Nucona aimed to validate the accuracy, performance, and cost of storing product data and using AI for advanced data matching when implementing these products.

For the Chrome extension, we developed a comprehensive Proof of Concept (POC) to address the key requirements. This included validating the cost of storing product data, the cost and speed of using AI to classify products, and the accuracy of product categorization based on names, ingredients, and categories. We expanded the Chrome extension to include Amazon by implementing a semantic search algorithm to intelligently search Amazon Fresh product names in the USDA database, retrieving necessary information to calculate diet scores. The backend stack for this solution included TypeScript for the fully functional backend. The POC then turned to a fully functional Python FastAPI API, using under-the-hood: open-source embedding models, Qdrant vector database, and custom code for advanced semantic search.

For the mobile app, we designed it to allow users to scan barcodes, recognize products, scan receipts, validate products, see the HEI breakdown, calculate rewards, configure payout options, and receive push notifications. The apps front-end was developed using React Native for cross-platform compatibility, with Zustand for state management, and React for user navigation. The backend utilized Node.js and Next.js for API requests and Nhost with GraphQL queries to PostgreSQL DB. We also integrated the Vision Camera library, Passio Nutrition AI SDK and API for product recognition, OpenAI vision 4.0 for receipt scanning, and geolocation features using React Native Geolocation and Google Geofencing API.

The outcome of these efforts was the development of an optimized Chrome extension which will eventually be compatible with any product marketplace and a user-friendly mobile app that effectively promotes nutrition education and encourages healthier food choices. By expanding the Chrome extension to include Amazon (with semantic search incorporated to get product information for any food item) and creating a mobile app that supports in-store shopping, Nucona now offers a comprehensive solution that incentivizes individuals to make healthier dietary decisions. This technological advancement enhances the user experience and significantly contributes to Nuconas mission of fostering healthier eating habits on a wider scale.

You can check the Chrome Extension for yourself by following this link. The mobile app will soon be available for use.