A long time ago in a galaxy far, far away….
The CE mark was created. But what is it?
The definition according the the EU website is:
The letters ‘CE’ appear on many products traded on the extended Single Market in the European Economic Area (EEA). They signify that products sold in the EEA have been assessed to meet high safety, health, and environmental protection requirements.
Please note that a CE marking does not indicate that a product have been approved as safe by the EU or by another authority. It does not indicate the origin of a product either.
If we dig into what’s known as the “Blue Guide” we can see that for continuous glucose devices it means, well, not a lot. There’s nothing specific in the guide about them. Instead we have to refer to the Medical Devices standards which say:
Devices shall achieve the performance intended by their manufacturer
and shall be designed and manufactured in such a way that, during
normal conditions of use, they are suitable for their intended purpose.
They shall be safe and effective and shall not compromise the clinical
condition or the safety of patients, or the safety and health of users or,
where applicable, other persons, provided that any risks which may be
associated with their use constitute acceptable risks when weighed
against the benefits to the patient and are compatible with a high level
of protection of health and safety, taking into account the generally
acknowledged state of the art.
Diagnostic devices and devices with a measuring function, shall be
designed and manufactured in such a way as to provide sufficient
accuracy, precision and stability for their intended purpose, based on
appropriate scientific and technical methods. The limits of accuracy
shall be indicated by the manufacturer.
It’s clear that “appropriate methods” are not defined.
Why have I opened up with this definition?
Thanks to the plethora of CGMs at ATTD that are either not yet CE marked or are CE marked but not yet available.
New CGMs on the European Market
As shown in the table below, there were a lot of challenger CGMs showing off their wares. And almost to a tee, they presented with MARD values that were as good as, or better than, those of the market leaders (ignoring that some present with a value to 2 or more decimal places, as if it makes them more accurate).
| Manufacturer | Device | MARD (%) | Study Status |
| Menarini | i-Can | 8.7 | No study available at stand. Publishing soon. |
| Yuwell | Anytime CT3 | 9.07 | Chinese only. UK study to be done in April/May |
| SiBionics | GS3 | Better than 8.83 | No study information |
| Bionime | Rightest iFree2 CGM | N/A | No study details available because it’s not published (also no CE mark yet) |
| Syai | Tag | 8.106 (!) | Not handed out, but presentation shown privately at booth |
| iSens | Caresens | 8.7 | No study available to back up data |
| Microtech | Linx | 8.3 | Can’t share study because it’s not published |
But note the final column. This is the availability of the study used to provide that MARD value.
There’s been a lot of talk about standardisation of the process by which CGM accuracy is measured to allow reliable comparisons across CGMs. It’s even now been discussed in symposia led by Abbott (who have themselves been called out for changing the process they used to create MARD between Libre2 and Libre3). But that still feels like a hope rather than likelihood.
With this in mind, the table above presents a bit of a problem.
As end users, we’ve been trained by the industry to use MARD as our way of evaluating whether a CGM is good. It features prominently in the marketing materials of most of those listed. But as end users, it’s just a number. And the contextual details are not provided, usually because “The study is private and hasn’t been published yet”.
We don’t know how they got it:
- How many people did they have in their study?
- How many sensors did they use?
- What was the population distribution?
- What were the numbers of datapoints in different glucose ranges?
- Were there any glucose manipulation challenges in there?
This is the crux of the issue.
We have a number with no context. It is meaningless, but it’s okay to publish it, because apparently, study data was, or will be, included in the CE mark submission. A submission to a body that knows nothing about CGM.
Fixing what’s broken
Once again, the call for standardisation of the process is loud and clear.
But in the meantime, let’s go a step further.
If a company submits data for a CE mark, then that study data needs to be available for them to market with a MARD value. If they won’t or can’t, then any marketing must not include the MARD value. And if that study information isn’t available to the end user, the device shouldn’t be allowed on to a national tariff for easy access via the country healthcare provider.
No “waiting to publish the data”.
No “we’ll only supply it to clinicians”.
I wonder how long those studies would remain “unpublished” in this scenario.
If you’re able to market and sell direct to consumers, you, as a manufacturer have an obligation to give us information to make the best and safest choices. Understanding how you arrived at a number (that you’ve published to 3 decimal places in some cases) is critical.
It’s that simple. Make all the data available to the end user or be forbidden from sharing the headline or accessing the healthcare system.
What do you think?
I agree we need more standards but newbies to CGM are making a mistake if they want to compete on MARD – the battle is not how accurate the sensor is – it must be accurate reliable and provide users with actionable information they understand
I was employed in the aerospace industry for over 40 years. We manufactured components for both military and commercial aircraft and the space shuttle. A 20% error allowance would in my industry put you out of business. We are required to be certified to ISO 9000, 9001, and AS 9000. In addition to these standards, our customers, Boeing, Lockheed-Martin and others would not accept these the so called MARD Standards.
When you check a CGM with a glucometer, the glucometer also has a 20% allowance as well. So the 20% is checking the 20%. I feel that there are two measures we need to get straight here. The first is accuracy and then repeatability of the accuracy. If this can be standardized, now we would have a higher level of confidence that what we are using is accurate and repeatable. Furthermore, any testing of equipment for accuracy and repeatability should be done by a CERTIFIED and INDEPENDENT company where there is absolutely no bias. The company that is chosen should have no ties and or people in their company who represent the products that are being tested.
The only way we are going to know for certain which of these measuring devices that diabetics use on a daily basis are what they say they are, is to take the testing out of the hands of the device makers and put this into a impartial and qualified company that has no ties to the companies that are submitting their products.
How can we make this a Call To Action campaign?
Dennis’ recommendation summed it all up perfectly and I completely agree:
“The only way we are going to know for certain which of these measuring devices that diabetics use on a daily basis are what they say they are, is to take the testing out of the hands of the device makers and put this into a impartial and qualified company that has no ties to the companies that are submitting their products.”
In day to day life, people with T1 are not infrequently completely frustrated by their devices alleging they’re ‘in range’ – eg BG of 6.5, whereas a blood meter check says 3.2, and if they’re using for example, a Libre and a Dexcom CGM, the 3 readings can be very different.
Really helpful article Tim, thank you.
NB: MARD = mean absolute relative difference (MARD) – I only knew D-MARD – disease modifying anti-rheumatic drugs.
Thank you for your comments. We as advocates have to push the envelope to make changes are made to make these systems are more honest in their measurements and all devices are measured in the same manner! This way we will have objective outcomes to make better choices.
News from Sunny Poland – We have the same situation here
Up until recent the following CGMs were available with funding from the Polish Health Service:
FreeStyle Libre 2
Dexcom G6, G7, One+
Medtronic Guardian, Simplera Sync
Eversense
In the last few month the following CGM products have become available, all with funding from the Polish Health Service:
TouchCare
Anytime CT3
CareSens Air
Sibionics GS1
iCan i3 CGM Sinocare
I read this Blog frequently, so thankfully I understand a fair amount.
But I wonder how much the average user know or cares about the quality of these products, or indeed how much the attention the average Doctor gives to the specification of these products. Also, I have no idea how much due diligence the Polish Health Service does before it agrees to fund anyone product.
Competition is normally a good thing, but I cannot help but to think that there are probably lots of company reps running around trying to “persuade” Doctors to prescribe their products …………
Everyone claims to have the best mouse trap. Nevertheless, you have prove that this is verified. We will se where this leads to! There might need to be more players in the CGM marketplace to put the needed presures on all of these manufacturers to be more clear. Correct and Concise!
Totally agree to need of standards to measure mard values. CGMs are the core point of any aid system and they need to be accurate and reliable, but also comparable. Only then we can make good choices and treat our diabetes safely.
Just a few words on CE marking.
1) To not dig too much into detail, in-vitro diagnostics are subject to third-party assessment prior to CE marking. One can see a 4-digit number next to the CE symbol – that is the ID number of the third-party involved, also known as “notified body”. Devices do get assessed.
2) Annex I, General Safety and Performance Requirements, of the IVD Regulation says “Devices … shall be safe and effective and shall not compromise the clinical condition or the safety of patients, or the safety and health of users … taking into account the generally acknowledged state of the art.”
The key term is “state of the art”. In practice, this means that
– the manufacturer either uses so-called “harmonised standards”
– or suggests a better solution.
For the new Regulation, the harmonisation is yet under ways, and the listed standards [https://single-market-economy.ec.europa.eu/single-market/european-standards/harmonised-standards/iv-diagnostic-medical-devices_en] are too few. The list under the old IVD Directive is more comprehensive [https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A02020D0439-20210415]. Some of the listed standards may be outdated.
It is expected that manufacturer will take the best available standard or, when using other methods, justify these are better or at least as good. If the standard used is not harmonised, the manufacturer must justify why it is chosen.
Hope this helps to understand “appropriate methods”.
To answer both your questions in a single point.
The assessment is whether the device meets the requirements of the CE mark, not whether the data is of high quality. There is no specific requirement for CGM, along the lines of the ISO standard for SMBG devices, so the standard is that there is data to support the manufacturer claims.
As long as there is a study to do this, then there is no investigation of whether that study meets standards required for marketing a CGM.
Which is why we have a lot of CGMs with a lot of claimed MARDs using a lot of studies with different methods.
Just because a notified body signs off on it, doesn’t mean that there are any standards defining the accuracy of the device!