Following the release of the FDA statement about DIY systems, and the implication that it is a DIY CGM that caused the issue (essentially Libre being used with one of the third party adaptors and one of xDrip, Spike or Glimp), I thought it timely to find out how variable the dataset can be.
To put this in context, when I tested Libre against the G6 previously, I wasn’t impressed with the outcomes, and yet, anecdotally I regularly hear that it’s great, and that people get very accurate results.
As a result, I’ve decided that it’s time to run a comparison of the G6 versus the Libre, and split it into four weeks of testing. The set-up will use the 21st May Nightly build of xDrip software running on Android for collecting Libre data.
For each week, I’ll use a different calibration method on the Libre. The four approaches will be:
- Allow automatic calibration in xDrip
- Calibrate once daily
- Calibrate twice daily
- Adjust the slope of the line to 1 and calibrate “as required”
Hopefully using these four different approaches will give some insight into how the data varies under the circumstances of each type of calibration.
These will be run against the G6 operating in “Code-Mode” and not requiring calibrations.
The data will be stored in a test Nightscout instance which will mean that it can then be extracted and quantitatively analysed to try and determine variance, and of course, there’s also a bunch of finger prick tests required, so time to say thanks to my fingers ahead of time!
What do people predict as the outcomes? Which of the calibration strategies will result in the best outcomes? At this stage, I’m not sure, but it should be interesting finding out.
Hi,
First of all thanks for running this test. I think it is important to have proper data rather than anecdotal evidence. I personally use a Libre-MiaoMiao-xDrip+ setup and overall I am very happy with it.
I think that there are a lot of people who are not getting the best out of their MiaoMiao data because they have not taken the time to really appreciate how xDrip works.
When I first started with mine, I hadn’t setup xDrip properly and the readings were not as good as I would have liked.
By regularly tinkering with the settings I have got it to the point where it works well for me.
There IS always a day or so after starting a new sensor where I have to perform a number of finger prick tests, especially at meal times.
I also try to add a few “HIGH” & “LOW” calibrations just to settle in the new sensor.
I look forward to the result of your tests.
Oh, I ought to say that if I could get the G6 on prescription then I definitely would. I am hoping to be given a T-slim X2 pump soon. If I get this pump, then I will SERIOUSLY consider self funding the G6 (£159 a month, ouch!)
“Hopefully using these four different approaches will give some insight into how the data varies under the circumstances of each type of calibration.” – the real challenge would be to do some stress testing with changing temperatures. For example: calibrate at ambient temperature X and see what happens at different temperature Y. Calibrate at cool house/outside and then sleep(sensor) under duvet/blanket. Maybe you will be able to simulate false hyper overdose condition. Not saying that this actually happened. Calibrate inside and go running outside in cool weather…
I’d say that’s initially less of an issue. Simply testing it under normal user based calibration conditions and then seeing how the results vary as glucose levels vary when using different calibration strategies has to be the biggest item to confirm.
Stress testing these outcomes is a nice to have, and if standard calibration doesn’t produce safe results, is almost a side thought.
I’m curious to see how this will work with one test versus two a day. I can’t really afford the G6 and have been generally pleased with the accuracy of my Libre/Miaomiao with one daily calibration.
I predict it will be significantly off compared to G6 regardless of calibration method. I would have liked to see a comparison to a second Libre using standard software and reader. Until the recent issues with Spike, I was using Spike with G6. I had a lot of fluctuations, and often found Spike value was off by 40 points or more. Since Spike issues I have returned to using Dexcom software. It is very accurate, rarely being off by as much as 10 points. That’s including “with code” restarts. My experience indicates there’s a significant margin of error with non-factory algorithms. Thanks for taking the time and effort to make the Libre comparison, we appreciate your contributions.
I’ve compared G6 to a standard Libre set-up in the past, and to the Libre Algorithm being run as the OOP setup. You can find the outcomes of those at:
Libre vs G6 (and others): https://www.diabettech.com/cgm/the-cgm-royal-rumble-4-systems-face-off-one-will-win/
OOP: https://www.diabettech.com/abbott-freestyle-libre/freestylelibre-miaomiao-two-weeks-what-did-i-think/
Will you be testing against the OOP algorithm? This has be n my preferred calibration method for some time now, and I would love to see how it works for you.
I’ve done that in the past and the results for me were not very good. You can see the write up here:
https://www.diabettech.com/abbott-freestyle-libre/freestylelibre-miaomiao-two-weeks-what-did-i-think/
When I tested the Libre (both earlier 10 day version and later 14 day version) using an iPhone as the reader, the Libre was consistently 10 mg/dL or more lower than the G6 as read from an iPhone.
I was told the Libre is factor calibrated so you are not testing the libre but the app you are using with Miao-Miao. I went with the newest verison the 14 day Libre and it is not able to be used with the Miao-Miao or anyother device.
Yes, that’s exactly what I’m testing. May 21st nightly version of xDrip to be precise. But it should be taken in context. I can use the same app with the Dexcom G6 and use the onboard transmitter algorithm, so what we’re really looking at is how calibration of the app affects the data it produces.
Have you considered using Spike in your n=1 experiment?
A lot of Loopers are using Spike and Libre/MM. Your experiment has a bias for Android, so will be even less conclusive if Spike isn’t considered.
Hi Cheese, thanks for your comment. Yes, it has crossed my mind, but given that (using UK data), three times as many people use xDrip with Libre to loop as use Spike with Libre, it seemed like the more sensible place to start.
On the question of whether outcomes will be conclusive, the issues that have occurred and led to the FDA warning seem to stem from Android users, so that’s where this is focused. The outcomes will be conclusive for this particular n=1 experiment, controlling for Android and calibration patterns.
I agree that it won’t necessarily be conclusive for Spike, but as Miguel points out in this issue closure (relating to the latest release), https://github.com/SpikeApp/Spike/issues/166 , Spike and xDrip use the same calibration algorithm as long as you’re using the “Non Fixed Libre Slopes” option in the latest release, so in reality, it shouldn’t make any difference to the outcomes.
If someone wants to supply more Libre sensors, I am happy to run a similar experiment with Spike.
Thank you for your efforts. I always wondered what the best way for calibrating a libre would be and apreciate your systematic approach a lot. Regarding the FDA statement, I think that this incident was special. It seems an extreme offset was calibrated. This usually means for me that some data is wrong (BG, xDrip, Scanner) or that the sensor will fail in minutes or hours with plainly wrong values meanwhile.
I think it’s less uncommon than people think, having seen two similar incidents happen at the same time at the weekend. In one case there was a 9mmol difference and the other 4. One sensor recovered and the other didn’t.
The issue being that there’s no type of error code being determined from the Libre, and this can happen even with good calibration.
This sounds intense. What I wasn’t aware of until now is that I use some kind of error code replacement. I started this in the beginning just for refund reasons. The offset from BG to OOP/Scanner is an indicator for me of reliability and soon happening failures. But I must say that with me sensors with an offset above 60mg/dl usually never really recover and I even hope for the sensor to fail or run-off quickly to keep the period of unreliability as short as possible.
So there you identify the real issue here, which is that you’re acting as an expert user, as others have been with tweaking slopes, etc. As people use the system who have no idea about this kind of stuff, they rely on the tools as is, which is why it’s important to understand how it works, what works best and how to identify variation and potential issues.
I totally agree. I’m really looking forward to your testing results.
I was very fortunate to be covered by my insurance for the MiniMed 670G & the Dexcom G6. They just arrived in the mail this week. I am new to all of the technology, systems, apps & terminology. I have an appointment with a rep to guide & train me how to use. I must say after reading the posts here I am ready to pack all up & ship back. Sounds like these systems are not as reliable as one would anticipate. Please give me encouragement to understand that this equipment will not malfunction, especially when I am sleeping.
Hi Denise, I’m interested that you’ve got G6 and 670G, as the two are incompatible, and effectively you’ll only have a pump from Medtronic. It seems a little odd to me.
Very interesting comments.
I have been using Libre with xdrip+ and Bluecon and now MiaoMiao for 18 months and definitely fall in to the non expert category. I just use connect the Libre/Transmitter/phonexdrip and SW3 and let it roll. No altering slope or settings etc.
I check twice a day using the Contour Next BGM and calibrate only when necessary.
The variable is the Libre sensor. Mainly they read high especially around 10mmoll/l and above but below 5 they start to read low e.g at 4 on the Contour they are 3 to 3.6.
However this is not consistent starting 5 weeks ago I had 2 sensors that had xdrip spookily accurate at lower levels and almost no calibrations needed but with the one I put on last week while Libre has been very close to the Contour at all levels the xdrip has jumped around and needs calibrating at least once a day (I generally calibrate if it is out by more than .5 when under 8 mmoll/l).
So what I suppose I am saying is that any experiment over just a few weeks is very dependent on the quality of sensors used.
Overall I find the system very effective and I am normally in range over 90% and had my best HbA1c ever in January of 5.7
Thanks Anthony. I’m trying to mitigate for that a little by switching calibration technique at 7 days, so that we get two different calibration approaches from the same sensor. That should show, for one sensor, what the effects of the two different approaches are.
Ideally I’d run the same process comparing all the different calibration approaches across multiple sensors, however, that starts to get a bit pricey….