With the release yesterday of the Medtronic Hybrid Closed Loop pivotal study data, I thought it would be interesting to take a look at the only other widely, and publicly trialled closed loop system available, and see how the two compare.
The two studies
OpenAPS provided a poster to the American Diabetes Association Scientific Sessions with self reported data in June 2016, which, while not presenting full user details, contains some of the same indicators that Medtronic published from their pivotal study.
The Medtronic study encompasses 124 participants and ran for 12 weeks following a 2 week run in period, amounting to, just under 34 years of data. Medtronic have not allowed the EASD presentation to be republished as a webcast.
The self reported data for OpenAPS is from 18 participants, of 40 that were using the system at the time of data collection. OpenAPS has over 150,000 hours of use in the wild, equating to a little over 17 years. At the current time there are at least 150 people using the OpenAPS implementation, with a fair number of others using similar open source APS systems.
The key figures that OpenAPS provided in their poster were Hba1C reductions, and change in Time in Range. We therefore don’t have the ability to compare the breakdown of Sensor Glucose Values (SGV) percentage of that time and the Standard Deviation of the SGVs in total. We also can’t state the period of time over which these changes were achieved, so whilst I’ll make a comparison, it’s worth being aware of these limitations.
Hba1C
The headline from the Medtronic Study was that over the three month period the Hba1C level of participants decreased from a median of 7.3% to a median of 6.8%. This was an 8% reduction in Hba1C level. A tightening of the range of results across the population was also demonstrated.
With OpenAPS, the reported changes in Hba1C were from a median of 7.1% (SD 0.8%) to 6.2% (SD 0.5%). This was a 13% reduction in Hba1C level.
Whilst we see a bigger drop in the Hba1C reports from OpenAPS participants, as we don’t have a reporting time frame for comparison, the best we can say is that both Hybrid Closed Loop solutions effectively reduced Hba1C by a statistically significant amount, and that in both cases, they reduced the spread across the user base.
Time-in-range (Glycaemic Variation)
For the participants of the Medtronic Study, the reported time-in-range, where the range was reported as 71-180 mg/dL, was 72.2%, (SD 8.8%). During the run-in (calibration) phase, this was 66.7% (SD 12.2%).
The participants in the OpenAPS study reported a time-in-range, where the range was reported as 80-180 mg/dL, of 81% (SD 8%). We know from the OpenAPS reports that this was an increase from 58% (SD 14%).
What stands out for me from this set of data is that in both the cases here, the APS system was theoretically calibrated to operate at full efficiency. Within this, the OpenAPS algorithm, oref0, appears to do a substantially better job of managing within a tighter boundary. Users of OpenAPS spent 12% more time within range than the Medtronic HCL users. The variation in the two cases is similar.
Discussion
Whilst the Medtronic publication appears to provide a lot of data compared to the OpenAPS poster, the reality is that the pertinent data is there, limited and mostly comparable. As mentioned earlier, due to no information on the duration of use of the OpenAPS systems, the comparison of drop in Hba1C of the two systems is not possible. What is clear is that both achieved a better result and that both also resulted in a tightening of the range of Hba1Cs achieved.
It’s the time in range that is the more interesting comparison, as this is less reliant on time spent using the system (if both are properly calibrated), although this may be a factor. What we don’t know from the Medtronic study was the time-in-range prior to the intervention with the HCL system. What’s also interesting is the reporting of the considered range. Medtronic’s range was broader than that of OpenAPS.
Whilst this information is interesting, it’s also the one component of the study that is most open to variation as a result of outside factors. We can’t know, for example, how well the participants of either study carb counted, or their bolus strategy (before, at or after a meal). We know from comments made in the presentation by Medtronic that the HCL will suggest adjustments to users if it calculated that the meal bolus wasn’t working effectively.
There are two observations I’d make in relation to the OpenAPS participants:
- They are self reporting, which potentially could lead to positive bias as a result of poorer results self-selecting out and a smaller sample will be more open to skew from this.
- The users of OpenAPS are clearly very engaged in their diabetes management as they have built the systems themselves. As a result, they are more likely to have determined strategies for better managing their glucose levels in relation to mealtime bolusing.
The other aspect of the two systems that we have not looked at, and to a great extent, at this point can’t, is the predictive and dosing algorithm. oref0 is easily available to go through and understand. The Medtronic HCL isn’t.
From a purely subjective perspective, based on the model I’ve seen with the 640G SmartGuard algorithm, I would expect that oref0 will make, what might be considered by a Med Tech company to be, more aggressive decisions about when to resume insulin post a low and how much to give when dealing with an elevated SGV. Whilst a commercial company may have one eye on liability, I’ve yet to see anyone reporting issues with the oref0 from OpenAPS being too aggressive. Without running them side by side, this is simply a supposition.
Conclusions
Having a commercial APS product undertake a pivotal study for submission to the FDA is a massive step forward in the introduction of artificial pancreas, or closed loop systems. The news from Medtronic in this respect can only be taken as positive.
The results of their study should also be considered positive. As a commercial product, the company producing it will be keeping a close eye on their liabilities and trying to make sure that they are not in a position where legal action may take place. They therefore have to make certain decisions about algorithm action.
The biggest take away is that the outcome of the Medtronic trial paints the work done by Scott, Dana and Ben in a new light. They brought to market, for those willing to do it themselves, a platform that seems to perform as well as the only commercial product publishing data, over 18 months ago. And they made it available to everyone to use for free. The data from OpenAPS (even though a small sample) is corroborated by that of Medtronic.
Let’s hope the powers that be understand the glycaemic variability range benefits of these systems. A broad roll-out of the commercial technologies would benefit the majority of T1s!
From here, for me, the only possible way is up, or rather, down….
Leave a Reply