Glucomen Day CGM – A two week review

Over the past two weeks, I’ve worn the Glucomen Day CGM in its recommended place on my abdomen. During that time I’ve done 91 blood tests to provide a “MARDf”, which is a measure of variation accuracy compared to the meter I’m using for calibration, which is the Ascensia Contour Next One. Alongside this, I’ve run a Dexcom G6 on my arm, which is my preferred sensor and location.

With all this in mind, how did the Glucomen do versus the Dexcom? Was it accurate and is it worth paying what amounts to a £20 discount per month from the Dexcom G6 system? As always, bear in mind that this is an n=1 experiment and your use and diabetes may vary…

I’ll look at this in two ways. The first, what’s it like to live with, and the second, data accuracy.

If you’re of the “Too long, didn’t read” brigade, then it wasn’t very accurate and I felt it was overpriced at the subscription model Menarini are using. If you want to know the quantifiable details behind that, read on.

What’s it like to live with?

Once you’ve inserted it, as per the last post on this topic, it works pretty well. It tells you when to calibrate, and if you don’t, it drops back to a display with grey dots giving estimated glucose values until the calibration is given, as can be seen in the below image middle section.

One of the biggest frustrations of day-to-day living with it is that the only way to get the glucose data (on the iPhone at least) is to go into the app. There’s no “today” screen widget and no way of getting it onto a watch. There is also no mechanism for remotely monitoring the data, which won’t appeal to parents. 

In this sense, it feels like a step backwards. 

I use it with as many of the alarms turned off as possible, and looking at the “History” page in the alerts view, I can see all of the alerts it would have triggered.

I had a couple of weird times where the transmitter dropped out and couldn’t be found by the app. This might normally happen if there is a major bluetooth interference, or multiple systems using bluetooth on the phone, but given the only devices in the vicinity at the times this occurred were talking to my other phone, either they produce very strong signals, or the Glucomen transmitter is very sensitive. Overall, the experience of using it was okay. Nothing special and a few irritations, as per normal with a CGM system. 

The other point I’d note is that the system comes with overplasters that really need to be applied and probably held this sensor on for it’s full duration (and a bit more). For the final four days, I’ve been on holiday, and undertaking the associated swimming that usually entails. The sensor hasn’t fallen off yet, but it’s looking fairly grubby!

If you remember in the original article, this system comes with two transmitters, the idea being that you change sensor and put in the new one, which is fully charged. Instead of applying a new sensor, I swapped the transmitter ( a very easy undertaking), and it was able to restart the sensor and carry on with the 55 minute warm up. Even in the above state. So you do have the option of restarting these things, if you can figure out how to keep them attached for long enough.

One of the other points I made when putting it on for the first time was that I was sure the shape would result in me pulling it out on clothing. Thankfully, that didn’t materialise, although it did pull a little on shirts.

All in all, it’s not too difficult to live with, as long as you don’t mind getting your phone out and opening the app to check glucose levels, and don’t want remote monitoring.

System accuracy

This is always the interesting question in respect to new systems, the claims they make based on clinical data and how that translates to the real world. As usual, I’ve tested versus blood glucose measured against the Ascensia Contour Next One, and present the data as MARDf (MARD compared to fingersticks) rather than as anything else.

So how does this stack up?

First up, we have the Surveillance Error Grid assessment as provided by the Diabetes Technology Society.

 

The outcome here isn’t all that pretty. There is a lot of variation from the blood tests and at the lower end there were frequently missed lows. 

When compared to the Dexcom, we see a slightly different picture.

For the Dexcom G6, the values are a lot closer to the 45 degree line indicating good measurements compared to the fingersticks. 

For reference, during this testing period, neither of them was particularly good, with the descriptive statistics showing that even the Dexcom was getting nowhere near its supposed accuracy. The other “highlight” of the two datasets is the co-efficient of variation. Again, this is skewed by the first day, but the Glucomen shows almost twice the level of the Dexcom. This means that it has a much greater dispersion of data points around its mean, and this can be seen later on in the Modified Bland Altman plots.

It’s worth highlighting that the Glucomen’s data was badly skewed on day one, which was truly awful, so if we exclude the day one data, we see the following:

MARD excluding day 1 Dexcom	12.4%
MARD excluding day 1 Glucomen	17.2%

The key here is that this is nowhere near the supposed 11.4% in the literature, and 9.9% that we’ve been told verbally. The modified-Bland-Altman plots just highlight this further:

In terms of risk associated with this variance, I looked at the average glucose level shown by the CGM system when glucose was below 3.9mmol/l and also when below 3.5mmol/l.

Average reading when blood <3.9
Dexcom	3.9
Glucomen	4.3

 

Average reading when blood less than 3.5
Dexcom	3.6
Glucomen	4.3

As we can see here, these numbers don’t look too great for the Glucomen system. Taking this to its logical conclusion, how many times did the Glucomen miss lows compared to the Dexcom?

Overall I measured 14 levels below 3.9mmol/l using fingerpricks. The Glucomen didn’t show these as low in 11 of the 14 times (79%) whereas the Dexcom didn’t show them as low in 8 cases (57%).

There were a few where the level was below 3.5mmol/l, and in those cases, for the Glucomen 83% of the numbers that showed below 3.5mmol/l on the fingerprick showed above 3.9mmol/l on the Glucomen. Conversely, the Dexcom showed 83% of the same numbers as below 3.9mmol/l. Given the reported accuracy of the Contour Next One by the Diabetes Technology Society, these are definitely missed lows.

Overall then, the Glucomen missed more lows than it captured, and considerably more in the more dangerous area. Not a very good result.

So what does this show us in terms of variance over the lives of each sensor?

Each of these three views reflect what we saw in the Bland Altman plot, in that the variance of the Glucomen is pretty high, while the Dexcom got worse as it got older. The Glucomen on the other hand, maintained a high variability throughout its life. As an adjunct here, in this case the Dexcom was mostly positively biased, which isn’t great for a system running an automated insulin delivery device.

The box and whisker chart really highlights the difference between the two systems, as does the co-efficient of variation in the descriptive statistics.

To put this level of variation in perspective, only the Medtrum A6 that was tested in the Royal Rumble has come out worse than the Glucomen Day CGM, and even the Dexcom G4 published MARD was 13% before they updated the software and 9% afterwards.

The question for me relates to the number of tests over which the MARD in the documentation was evaluated. The only documentation I can find relates to Agamatrixx testing and was from 15 people, which is a very small sample to be providing data for wide consumption, as is n=1 in this test.

Some thoughts

While MARDf is not a clinical measure, it gives an indication of how far from the “calibration” data the systems stray. In this particular study, the Dexcom G6 has underperformed other experiments that I’ve done, and I wonder whether this is linked to their changes to lock out DIY users. 12.1% MARDf for the G6 is considerably inferior to previous tests that I’ve done with it.

Having said that, the results for the Glucomen have been very disappointing. For a system that charges £20 per month less than Dexcom, you get considerably worse accuracy while still having to calibrate once daily. There’s little useful data sharing or display, so it’s much less usable than the Dexcom system, and fundamentally, the sensors are priced at the same cost per day as the Dexcom ones. It was also worse at picking up hypoglycaemic events than the Dexcom, especially those where levels dropped below 3.5mmol/l, that might be considered critical.

On the plus side, you get a lot less waste with it.

Is it worth £139 per month

For me it’s a resounding “No”.

Personally, I think this is overpriced for what you’re getting. Even starting from the base measure of MARD in the documentation, for something that’s 15% less accurate than the Dexcom, I’d expect to pay 15% less, and to top that off, as it still requires daily calibration, although Menarini do include strips so that this doesn’t cost you any more.  

Based on the testing that I’ve done, where there’s near enough 62% difference in accuracy, and if we’re being generous and ignoring day 1, then it’s 39%, and it’s worse at detecting low glucose events, one of the primary reasons for having CGM, then £139 per month looks to be a lot overpriced. 

When you take into account the other features, ie. worse usability of the application and no sharing, £139 per month starts to look really expensive. While it produces values every minute, these don’t make a huge amount of difference to how you use the system, and don’t really drive up the value.

Overall, I am disappointed with the Glucomen Day CGM. The accuracy was not good enough, the variance wasn’t predictable and the usability of the software is lacking.

On the plus side, you can restart sensors, it produces data on a minute by minute basis, and I’m sure it will be fairly easy for WeAreNotWaiting to crack the bluetooth. I also had no issues with the adhesive, and the insertion was very low pain. 

I will give it another chance and try running the sensor off label on my arm to see if that improves outcomes, and I seriously hope that Menarini re-negotiate their agreement with Agamatrixx, as I feel that they are being ripped off if they have to price it at this level right now. I hope that there is a major program in place to improve the accuracy, because this just wasn’t good enough.

So would I recommend it? No. For Menarini to charge almost Dexcom G6 prices for something that for me is nowhere near G6 accuracy and usability, even with the reduction in waste, then there is a problem. And with the best will in the world, in my opinion a CGM system needs to:

1. Be accurate;
2. Be usable;
3. Reduce its cost;
4. Reduce its waste footprint.

In that order, to be competitive and try and charge Dexcom prices (which while still overpriced, shows why people are willing to pay it).

I couldn’t justify the expenditure of something this inaccurate for £139 per month, even with the reduction in waste. If it cost £100 per month, then maybe my tune would be rather different.