Nima Peanut Sensor’s Early Performance Results
The Nima Peanut Sensor is currently available for pre-order through March 8, 2018. During this time, the Nima R&D team is putting the sensor through rigorous testing and validation to ensure that our proprietary antibody as well as full system (chemistry, capsule, sensor) are all performing according to our quality standards.
In previous blog posts, we’ve mentioned that our goal for the Peanut Sensor was to detect below 20 parts per million (ppm) of peanut protein, but research and studies were still being done to determine our exact level of detection. Below, we’re sharing the results from our latest round of testing on Nima’s chemistry and sensor.
In this study, Nima’s R&D team found the Nima Peanut Sensor sensitivity to be 98.8% when detecting 10 ppm of peanut in a food sample.
Why 10 ppm?
As peanut does not have an FDA determined limit for “safe” levels in foods as they do with gluten, our test is set to detect at the lowest part of the range for LOAEL, or lowest-observed-adverse-effect level (the lowest amount that has been shown to cause an adverse reaction). LOAEL for peanut ranges from 0.25 to 10 mg of protein (FDA/Threshold Working Group: Journal of Food Protection Vol. 71, No. 5, 2008). The lowest part of this range corresponds to 10 parts per million for a standard serving size of 100 grams. LOAEL is a safety assessment-based approach, based on clinical data linked to biological effects.
Based on this testing, we recommend diluting non-peanut nut butter samples with water before testing with Nima, as the sticky/thick nature can cause longer test times.
We are extremely encouraged by this round of testing, and our continued efforts will focus on improving system accuracy and making the test more broadly applicable for additional food types.
Continue below to read more about the types of foods tested, methodology and other recommendations on usage of Nima Peanut Sensor.
Obtaining accurate, reliable and usable results has been at the core of our development process. While our goal was to be highly sensitive and specific to peanut proteins at 10 ppm, we also wanted to see if our antibody was even more sensitive. To that end, we conducted a study to assess Nima’s ability to detect both 5 ppm and 10 ppm of peanut in multiple food matrices. Our food samples were varied in texture, from meats to liquids, and in cooking, from raw, to fried, to baked. We tested more than 50 foods, including chicken, nougat, ice cream, muffin, cookie and almond butter. (We have also tested peanut butter!) In this latest round of testing, we collected more than 1,000 data points on foods tested at either 0 ppm, 5 ppm, or 10 ppm.
This study demonstrated Nima’s current sensitivity to be 98.8% when detecting 10 ppm of peanut in the foods tested, excluding chocolate from the analysis. Sensitivity at 5 ppm was 85.9%.
Prior to any food testing, all foods used were tested in order to determine that they were peanut-free prior to use. This pre-testing was done at Nima Labs, and in some cases, samples were sent to an external lab for verification. As this study was intended to determine our accuracy at a specific ppm level, we needed to prepare foods and spike each food with a known amount of peanut prior to testing in the Nima sensor. For spiking, we use the same ‘standard’ as is recommended by the FDA (Trucksess et al.: Journal of AOAC International Vol. 87, No. 2, 2004). This standard is spiked into the food sample so that the final concentration is 5 ppm or 10 ppm. For 0 ppm, foods were tested without spiking. Foods that were spiked prior to cooking included most of the baked foods we tested (the industry term for this is “incurred samples”); we also spiked foods after their initial preparation, like ice cream.
Once foods were spiked to the target ppm levels, they were added to Nima peanut test capsules, inserted and tested in the Nima sensor. For each food type and at each ppm level, 10 tests were performed.
Table 1 is a summary of data collected using the Nima device for foods containing 10 ppm of peanut. Of the 638 tests performed, 626 yielded the expected result, resulting in an overall accuracy of 98.1%. The true positive rate (correct “peanut found” result or “sensitivity”) was 98.8%.
The three instances of false negatives (false peanut-free “smile” result) in the system came from soy sauce (1/10) and chocolate chip cookie dough ice cream (2/10). These issues in the system are being investigated and addressed. The false “peanut found” results in the system were driven by one food type – an energy drink – accounting for six of these instances; we are currently investigating causes. The remaining three false “peanut found” results were from bread, cookie, and vodka, which are likely due to spiking issues.
Table 2 summarizes the testing done at 5 ppm. The sensitivity at 5 ppm (true “peanut found” rate) is 85.9% – lower than what is observed at 10 ppm but expected by our team. Although we can detect a significant amount of samples at the lower limit of 5 ppm, ongoing research is being conducted to continually improve our accuracy at lower ppm levels. However, we are confident that 10 ppm is an appropriate detection level for the majority of peanut avoiders.
During our testing, we noticed a decrease in performance for almond butter, a fairly viscous – or thick and sticky – sample. Undiluted, this sample has a longer test time than we target to have for Nima. Therefore, we recommend testing non-peanut nut butters diluted with a bit of water in equal parts in the test capsule: 1 part nut butter to 1 part water.
An additional food type tested that is not included in this analysis was chocolate. Chocolate is abundant in molecules such as tannins and polyphenols. The peanut protein adheres strongly to these molecules, preventing it from being freely released in the sample and limiting its ability to be detected. We are still testing to see if there are ways to help loosen the molecules for better detection. This is an area of active study for the team, and data on testing chocolate with Nima will be published in the near future.
Nima detected 10 ppm of peanut protein in 42 non-chocolate containing foods, with a sensitivity of 98.8%. Up to our launch of the peanut sensor, our efforts will focus on improving system accuracy and making the test more broadly applicable for additional food types. In the future, testing will also be done in partnership with a third party to validate Nima’s accuracy.