Nima Peanut Sensor’s Performance Assessment
Nima is the world’s first portable allergen sensor. The Peanut Sensor is Nima’s second product – the first product was the Gluten Sensor, that came out on the market a little over a year ago. (You can read more about the Nima Gluten Sensor sensitivity study here.)
In a blog post this past February we presented some of our early data regarding the efficacy of our peanut sensor. Since then we have continued our development, improving on the variety of foods Nima can test and have verified our results with internal testing using an AOAC validated ELISA.
We tested and reported on a wide variety of 62 foods with the Nima Peanut Sensor, with 53 foods spiked with peanut to our detection level of 10 ppm. We were able to achieve a Sensitivity of 99.0% and Accuracy of 97.6%. With all results being verified by ELISA.
Sensitivity describes Nima’s ability to detect peanut in samples that are known to contain the allergen.
Accuracy describes Nima’s ability to correctly detect peanut (or the lack thereof) in a population of samples that both contain and do not contain peanut (more about this later).
In this latest round of testing, we collected thousands of data points on foods at either 0 ppm or 10 ppm. Foods tested included bread, chips, cereals, cakes, chocolate, coated meat (baked), coated meat (fried), nut butters, ice cream, muffins, pasta, marinades, meat substitutes, candy, chili, soups, and sauces. You can see the full list of tested foods in the validation results on our science page as well as third party validation reports. We tested a total of 2,242 samples. The tested foods list is a combination of requests from our community, AOAC’s guideline for recommended foods, as well as a collection from the Food Allergy Research & Education program’s list of possible peanut sources. We tested these foods using our peanut test capsules and peanut sensor and also tested them using an AOAC approved ELISA kit. Collectively, our scientific staff has over 20 years of experience developing and running ELISA tests.
Prior to any food testing, all foods used were tested with ELISA in order to determine that they were peanut-free prior to use. As this study was intended to determine the performance of the Nima device at a specific peanut concentration level, we needed to prepare foods at that level. This involved spiking each food with a known amount of peanut prior to testing with Nima. For spiking we use the same standard as is recommended by the FDA (Trucksess et al.: Journal of AOAC International Vol. 87, No. 2, 2004). This standard is spiked into the food matrix such that the final concentration is 10 ppm (10 parts per million, by weight). You can read more about why we selected this level here.
Foods that were spiked prior to cooking included most of the baked foods we tested; we also spiked foods (e.g. ice cream) after their initial preparation. Once foods were prepared to target ppm levels they were added to peanut capsules and run in the Nima Peanut Sensor. For 0 ppm, foods were tested without spiking. For each food tested and each ppm level, 10 replicates were performed. Results were recorded and analyzed.
In addition to Nima device testing, we prepared the same foods for ELISA testing. Data was analyzed and compared to our device results.
Presented below are device results for the foods tested. As a reminder:
– The foods were prepared as described above
– The spiked samples were verified as containing peanut by ELISA testing
– The negative samples were verified to be below the quantification limit of the ELISA.
During this round of testing Nima was able to achieve an accuracy of 97.6%. The sensitivity, or true positive rate, was 99.0%.
Table 1: Food Testing Results at 10 ppm peanut.
During our testing, viscous samples such as almond butter were diluted to help ensure a successful test. We recommend that users do the same. If not diluted, testing these foods has the potential for a false smile due to the thickness and slow flow of the sample. The sample will take too long to develop properly.
As our chemistry cross-reacts with certain foods, we recommend not testing the below foods due to the increased chance of obtaining a false result:
- Sesame seed (false peanut found risk)
- Cayenne (false peanut found risk)
- Paprika (false peanut found risk)
- Tamarind (false peanut found risk)
- Tomato paste / Sauce (false peanut found risk)
- Eggplant (false peanut found risk)
- Solid chocolate (false smile risk)
- Solid chocolate refers to bars, unmelted chips. Chocolate in other forms (melted, powdered, softened, etc) are acceptable to test.
- Alcohol (false smile risk)
Nima detected 10 ppm peanut in foods such as cereals, cakes, chocolate, meat and meat substitutes, nut butters, ice cream, marinades, candy, soups, and sauces! Our Sensitivity and Accuracy are 99.0% and 97.6%, respectively. We are currently in partnership with a third party testing lab to further validate Nima’s performance. Stay tuned!