If you have type 1 and use social media, you can’t have missed the adverts for the newly CE marked SiBionics GS1 CGM system.
They are everywhere. Facebook. TikTok. Instagram. You can’t avoid them.
In general they tell you that the GS1 is great. It doesn’t need fingerpricks and it has an “amazingly accurate” MARD of 8.83%.
But if you dig a little deeper, it’s not all roses, and if you’re using insulin, you might want to consider whether these CGMs are really safe…
Some things to consider
1. The Advertised MARD
As I’ve already mentioned, the MARD value being discussed is 8.83%. This is used based on all users in their clinical trial (documented here.
However, the trial had three subgroups. Type 1, Type 2 and “pre-diabetic”.
In the Primary Outcomes table, we see that stating 8.83% encompasses all three of these groups, whereas for those with T1 using insulin, we have a value of 9.15% (or 9.2% if you round it up).
WHat’s also very clear from this table is that the performance in the type 1 group when levels are below 4.4mmol/l is pretty poor, with a MARD of 15.03% and a 20/20 consistency of a little under 83%. This suggests that in the “low” zone, it doesn’t perform very well.
So their measure of accuracy is skewed by type of diabetes and even more so by glucose level at which the measures are taken, which is not evident from any of their social media advertising.
2. Social media claims about the system
There have been multiple statements from “SiBionics” on their Facebook pages in response to potential customer queries.
Unfortunately, they don’t seem to align with either their approval or their user guide. The two main areas of concern relate to the following items:
- The age for which the device is approved
- The device is not approved for determining treatments, so the advice that you don’t need fingerpricks is a little misleading
In both the below examples, the SiBionics user can be seen to be suggesting inappropriate use.
It appears that the SiBionics user is deliberately using allegory to suggest that users use the CGM against direction.
It’s almost as though they don’t want to accept that while you don’t need fingerpricks for calibration, because you simply can’t, they don’t want to admit that to make dosing decisions, you do need fingerpricks. As they makes the system wholly uncompetitive in the type 1 world. It also goes directly against their user guide, which clearly states that high and low readings can’t be relied upon, and need fingerpricks. The user guide is attached at the end of this article.
3. The price
It’s one of the more expensive options, at face price, coming in at £56 and appearing to not offer VAT removal for those with medical conditions. Given how much lower cost other sensors are, this seems a little silly. There are plenty of discounts available when you go o to the website, but for what you’re getting, it seems overpriced at face value.
An n=1 test of GS1 and some thoughts
Given the much-vaunted performance, I felt it was worthwhile testing this against fingerpricks, and the other sensor that comes as a two piece, isn’t calibratable and has a sensor component that looks very similar, the Libre2. Unfortunately the only Libre2 I had was 6 months out of date, but hey- ho. So it went on.
The measurements in use here were versus a reference device of the Contour Next One, which has been calibrated using solution. I put my hand up and agree that it is not perfect, but it is about as good an option as possible without a YSI.
Firstly, how well does a six month beyond use-by date Libre2 work?
Much better than expected, it turns out. Aside from occasional loss of signal, it stayed on for 12 days before the adhesive gave up. It’s MARDf (MARD Vs Fingerpricks) over that period was 7.4%.
The GS1 managed 13.5% over it’s 14 days. This was a long way from the somewhere near 9.2% I was expecting, and on a day by day basis, was very variable.
Metrics
If we look at the consensus error grid, we get a better impression of this data. The three grids shown are from the clinical trial used for approval, my n=1 vs fitngerpricks and the Libre2 n=1 vs fingerpricks.
It’s possible to see in the n=1 charts above that the GS1 tended to produce results that were generally lower than the reference glucose level, compared to the libre2 which is much more evenly distributed around the diagonal blue line.
This can further be seen when we compare the relative variation between the Libre2 and GS1.
The GS1 was, more often than not, lower, or a lot lower, than the Libre2. Which was 6 months out of date.
Thoughts
I’ve done a lot of these sensor comparisons to fingerpricks, and one thing that’s been very clear is that where there has been a high quality clinical trial, the data that they produce compares pretty well to fingerprick comparisons. Dexcom G5 and G6, Libre2 and Libre3 have all stuck to this pattern.
Other newer sensors from a multitude of companies with some less reliable studies have proven to be much less close between fingerpricking and clinical study data.
This appears to be one of the latter. The marketed MARD at 8.83% clearly isn’t true for those with type 1, and this is present in their own data. Even the user guides acknowledge that this sensor isn’t accurate enough to determine insulin doses.
And for me, it was miles off the clinical trial, when comparing with fingerpricks.
It’s always possible that Abbott’s Libre2 is calibrated to show up well against the Contour Next fingerpricks, but if that’s the case, then the same must be true for the Dexcom sensors as well. If you’re a manufacturer that’s including a significant proportion of type 2s in the clinical trial and then publicising the combined value, why wouldn’t you calibrate the same sensor a certain way?
But the main question is, would I use it?
The simple answer is no. Given all that I’ve experienced and seen above, I don’t really trust the sensor and I don’t like the behaviour of company representives on social media. There’s a need for trust in a CGM system and its provider. I’m struggling with that a little.
Hopefully over time the accuracy will improve. To be fair, when we started our 3yo on the Dexcom G4, it was off-label use as well. I don’t put much stock by that frankly unless there are physical issues that prevent successful insertion and wearing in a child. It just comes down to the clinical trials, and what they get approval to market. We wanted our son’s BGL and didn’t feel like waiting 15 years.
Also, the libre is well-known to be a lot more inaccurate in hypo region.
A difference in MARD between 8.83% and 9.15% (0.32%) is not particularly exciting or significant either. It could easily be noise in their data from sample size. Even a small number of outliers could skew results like that depending on the number of participants in the trial.
Comparing a CGM vs finger-sticks is always dubious. The MARD on your Countour is not zero either. It also measures capillary blood vs the CGM measuring ISF, so unless your BGL is level, there will be an error simply due to the delay between blood and ISF. This is why the CGM companies put predictive components into their algorithms. Testing ISF readings vs capillary blood samples is therefore comparing apples to oranges a bit. It would be interesting to see the accuracy vs an ISF reference (I don’t think there even is one? Do YSI do one?). Personally I would rather know that it’s giving an accurate representation of ISF glucose levels, because ISF levels are the actual reality when it comes to low and high symptoms and problems.
Hopefully these guys will improve whatever is needed. It’s always good to get more competition in the market, but it seems they can’t really justify a price premium so that’s weird. Seems also they need to educate their social people on what claims they are allowed to make. That can even vary by territory, and so it can be a minefield commenting on social media.
As mentioned in the article, I know that comparisons to fingerpricks aren’t perfect, and there are potentially many excuses, that can be given. Experience suggests that the CGMs produced by the market leaders are generally better in terms of readings proximity to good finger stick measures compared to some of the more recent entries. I wouldn’t expect to see the wide variation I saw here from products provided by Dexcom, Abbott or Medtronic (indeed, a G6 sensor was run alongside the two mentioned in the article and performed similarly to the Libre, and a Medtrum Nano, over the first seven days at least, was also better).
In terms of approvals, UK licensing is based on EU CE marks, so if their UK manual says don’t use for treatment decisions or children under 18, then you can be pretty sure that’s true across Europe. I’m fairly certain they don’t have approval in the US…
The price premium is very hard to understand.
I think one of the major issues has become that it’s so easy to get products to market in Europe that we become guinea pigs for companies, who aren’t necessarily being completely truthful about the capabilities of their products. That will, at some point, end in an unpleasant way.
Thanks for the interesting article and considerations. I’m a T1 for more than 35 years, and a libre 1 user for 7 years (unfortunately libre 2 and 3 aren’t available in my country yet). In the past 12 months, I experienced an absurd number of faulty libre sensors. I had to replace probably 20 faulty sensors in the last year, some of them didn’t work from start, some others showed very low readings compared to fingerstick and stopped working after 1-10 days. I tried 2 sibionics sensors already and my experience was good, especially compared to libre 1. It is true that sometimes the sensors seems to go out of track and show lower readings. It happened 3 times, but after a few hours it came back on track by itself. Obviously, it is clear that no one should blindly trust it, but the same applies to libre, at last in my experience. The fact is: more competition will push the industry to improve their products, and this is good for us.
That’s a fair comment. I wasn’t aware that Abbott were still producing Libre1 and distrusting it, as that’s all but ceased in many places, in favour of Libre2.
It may explain why, in their clinical trial paper they refer to the Libre1 and Dexcom G5.
Also, about the price, it is cheaper to buy a GS1 from AliExpress and have it shipped to my country than buy a Libre 1 in a drugstore here 😦
Does it work with the local app?
They’re distribution model certainly seems a little strange.
Sorry, what do you mean by “does it work with the local app?”
Does the Ali Express version work for your country or do you have to download an app from a different country app store?
Since Sibionics is not officialy available in my country, I cannot download the app from the Google Play Store (unless I change my Google account to another country where GS1 is officialy available). But Sibionics provides an URL to download and install the app manually on Android phones, and that’s what I do.
You can also use Juggcluco as alternative app:
https://www.juggluco.nl/Juggluco/changes.html#end
(If you use AndroidAPS and Libre 2/3 this sensor would be a drop-in-replacement).
At 12 euro per sensor, and it lasts up to 22 or more days? It might still be a great proposition for people without sensor access.
Depends where you’re buying it from on the price, and I’d be very wary about using it with AID. Many people report them conaistently reading well below real glucose levels, which isn’t great for an AID.
Disappointing. Like the other commenters I was hoping to use it as an alternative to the Freestyle Libre 1 (combined with a Miaomiao and Xdrip+), as it’s cheaper here (by almost a third) and the Freestyle Libre 2 (let alone 3) is not available locally. I have already ordered a couple so will see how they go.
One thing that occured to me was the conversations had with ‘SiBionics user’ might be AI generated.
AI simply trawls through data to produce commentary to try and respond to questions.
Quite a dangerous thing to do when recommending use of medical devices.
Could be wrong but it would be interesting to know if it was a real person responding rather than a database query.
That’s an interesting thought. You could well be right.
Hi Rob, Team Sibionics here. No, we don’t use AI generated responses for our interactions. But we do require our teams to use standardized responses for as many questions as possible (to avoid potential misunderstandings as outlined in this blog post!), which may make it feel a little “robotic”. We’ll see what we can do to make it a better experience.
You understand why this is worse? You are creating standardised responses that are either:
a. Misleading in respect of the use of your product, or;
b. Encourage use of your product outside of the approval.
And these have been approved by your internal control teams for anyone to issue.
There are two types of responses:
1) Standardized responses for common questions, which as you note are pre-approved
2) Tailored responses to specific questions, which rely on individual judgement
All of the examples highlighted in this post are the second type. We always want these answers to be helpful and informative, but there is always the risk of misinterpretation with tailored answers.
Don’t you think you should provide guidance to anyone giving tailored answers that says “Don’t suggest something that could cause us to breach the terms in our approval?”
You do realise that every time a representative does that you should be issuing a report to the local regulator, as it’s your representative doing it? MHRA Yellow Card in the UK case.
Hi, Team Sibionics here, many thanks for this thoughtful and honest assessment and feedback on our product. Very helpful!
We do not have a pediatric indication in the EU / UK yet, and had an internal review with our team on how to avoid misunderstandings when asked about pediatric usage.
Accuracy is always a hot topic, and we are always interested in hearing real life usage cases. Most accuracy trials are done using venous blood glucose as the comparator (as was ours), which is more accurate than BGM, hence the lower MARD values with venous blood testing vs BGM. You can check out this paper if you really want to nerd out on the details:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5375072/
A higher MARD in the low glucose ranges is pretty common across devices; it is both a technical issue as well as a mathematical issue (similar absolute level of variation as with higher BG levels but with a much lower base value). Other devices show 15-16% MARD with T1D in hypo ranges, so the Sibionics should not have any particular safety issues in this regard.
Regarding your personal experience, users have a different protein binding rate to the sensor electrode, and higher rates of protein binding can result in artificially low readings. Perhaps this was the case for you?
We love connecting with users, so please feel free to reach out to us if you want to learn or discuss more.
teamsibionics@sibionics.com
Team Sibionics
Thanks for your feedback Team SiBionics. This site is well aware of accuracy and MARD, and what manufacturers are able to do to improve study outcomes.
On your hypoglycemia detection comment, your 20/20 value below 4.4mmol/l was 82.9%.
Dexcom’s G6 between 3 & 3.9mmol/l is published as 94.4% and Freestyle Libre <3.9mmol/l is 98.4% amongst an adult population. These equate to single digit MARD values. Which sensors are you comparing yourselves to?
It's interesting that you think it's my personal biology that might be causing issues with GS1 sensors. Even an out of date Libre2 managed better, and I don't see these issues with Dexcom either (which is well documented throughout this site). All this suggests that rather than it being me that's the problem, it's more likely to be your sensor or interpretation algorithm. I'm also not the only one that has expressed concerns around excess low readings. Your Facebook group has plenty.
Happy for you to get in touch directly to discuss further.
Thanks for the quick response! To answer your question, I was referring to the G6 FDA iCGM filing, which includes all of the trial data to reach the iCGM standard. I use this as our trial was operated in the same way, so it makes for the best comparison. See pg 14 here if you want to research the original data:
https://www.accessdata.fda.gov/cdrh_docs/reviews/DEN170088.pdf
As you note, accuracy can be influenced by many factors: the design of the electrode sensor, the material used to coat the electrode, the way it is calibrated in the factory, and the algorithm that converts the current data into BG data… It’s a fascinating topic, please feel free to write us direct at teamsibionics@sibionics.com and we can dig into more detail.
Even in that paper, the comparison to Dexcom still isn’t the best for GS1. On page 18 you have the 20/20 outcomes (separated by Paediatric and Adult) and MARD.
For adults (to compare to your study) it shows 20/20 in the range 54-69 of 96.1%. That’s significantly better than GS1. MARD is shown as 12.4%, which makes GS1 more than 20% worse than G6.
Page 14 specifically isn’t relative for comparison, because it shows 15/15 and 40/40, neither of which are presented in the SiBionics study. And perhaps interestingly, the 15/15 value at <70mg/dl for G6 is still significantly better than the 20/20 at <80mg/dl for GS1.
The data starts on pg 14, the chart at the bottom of pg 18 was indeed what I was referring to. They have good performance, it is clear.
GS1 is our first product version, and we are continually working to improve both the hardware and algorithm performance — we’ll keep you updated as we progress!
Well, Team Sibionics, thats what you need to advertise, and not 8.83%.
Just stareted using one today.
In general (I used fingerprick more times today then ever, just to compare):
1. When it shows Low blood sugar (3.6 to 4.0) difference is about +0.5 (finger test would be 4.1-4.5).
2. Between 5.0 and 6.4, +0.8-1.0
3. Above, about +1.1 – 1.2.
If at least it would stay that way, I could at least estimate. If it changes day by day… I will spend more money on comparing with finger device, than what CGM cost me.
Wouldnt buy it again. Sorry, but you have a looooot to improve.
Anyone knows about iCan3? Also chinese provider.
I’ve been using Sibionics since the beginning of 2024. I checked my blood sugar by fingerpricks a few times and the readings were identical to the CGM or sometimes a bit lower than the CGM. I haven’t noticed CGM readings to be higher than fingerpricks. We also tested my wife and her readings were 100% the same as from the finger. So it makes me to trust this device. I’m prediabetes btw, not T1.
I’m pleased it works for you. Whereabouts are you based?
I’m in my 4th sensor now and the experience has been really good. After dozens of faulty libre sensors, the fact that the four gs1 sensors worked fine for the promised 14 days brought a much needed relief to my spirit. Accuracy is better than libre v1 and usually it is pretty close to fingerprick (0-20%).
Hi, I am using GS1, already my 4th sensor. It works very well. I have noticed there is a small delay in time if to compare to fingerpicks. I would say 10 to 15 minutes. However, sensors work fine and help me to control my glucose. I am pregnant and was not diagnosed with diabetes.