DENVER, CO – Once again the buzz of conversation filled the auditorium at Industry, a 152,000 square foot co-working space in Denver’s River North Art District, as the hundred-plus members of Prime Health who had arrived for the organization’s July 16^th Meetup milled about, greeting one another and catching up. They packed the hallway that ran beside the auditorium, with some assembled around an informal bar, and others standing along a pizza-lined table. A few were even seated in the auditorium already, having wanted to sit down before the best seats were taken.

But whether seated or standing, everybody at July’s Prime Health meeting was talking, either sharing the details of their own digital health endeavors, or commenting on what their companions were doing. And as the busy noise of their chatter echoed throughout the vast building, it was once again Jeffrey Nathanson’s job to call them to order.

A tall, lean man with gray hair and a serious demeanor, Nathanson took the microphone from an assistant and stepped onto the auditorium floor. CEO of Prime Health and a businessman himself, he understood the excitement that animated the conversations going on around him. The digital health boom was in full swing, and everywhere they looked, the members of Prime Health were seeing opportunity.

When he made his first call to begin the meeting, a handful of attendees trickled in from the hallway and took their seats. He asked again that people come into the auditorium so the presentation could start, and a few more Prime Health members walked in and sat down. Yet it was only after his fourth request for everyone to be seated that enough people had gathered in the auditorium for the presentation to commence.

A small number of Prime Health attendees remained by the bar as Nathanson started to speak, talking with one another in low voices like high school students who were lingering in the halls to avoid going to homeroom. Little did they know, for the members of Colorado’s digital health community, class was already in session.

Fitbit and the Need for Demonstrating Clinical Efficacy

The recent Fitbit IPO revealed an interesting dilemma that many digital health companies are facing these days.

“How many of you have heard of Fitbit,” Nathanson asked the crowded auditorium, his strong voice capturing the attention of his talkative audience.

A majority of those in attendance raised their hands.

“How many of you know that Fitbit just had an IPO?”

He looked around at the hands that remained in the air.

“What was the valuation of Fitbit? Anyone know?”

“A billion,” someone in the crowd offered.

“Three billion dollars,” Nathanson answered, causing the members of Prime Health to murmur in surprise. “Does anyone know how many weeks or months or years a user will use the Fitbit Product?

“Six months,” one person guessed.

“Three months,” another called out.

“Six weeks,” Nathanson replied. His audience laughed nervously. “So what the IPO actually said is that the revenues for Fitbit were born on advertising, on getting people to try something.” He paused for dramatic effect. “And once they tried it, they stopped using it.”

Those in the auditorium grew quiet as they considered the implications of this statement.

“It’s my personal hypothesis,” Nathanson continued, his tone growing more casual as he saw he had their full attention, “that had Fitbit been embedded in a condition-management program – for example, for Type II Diabetes – and demonstrated clinical efficacy, they probably would have had a better engagement rate.”

He went on to describe the demonstration of clinical efficacy as “a critical component” in integrating digital health products into the healthcare delivery system. He also tied it to the “triple aim” of Prime Health: to foster the development of products that generated increases in patient engagement, improvements in population health, and reductions in healthcare costs.

“Because if it doesn’t increase engagement, improve health, and reduce costs,” Nathanson finished nonchalantly, “why bother?”

Dr. Susan Moore on Using Research to Stand Out

The Innovation Adoption Curve divides tech users according to the speed with which they adopt new technology.

In contrast to the sobriety that had characterized Jeffrey Nathanson’s introduction, Dr. Susan Moore graced her talk with welcome charm and wit. “I’m the Assistant Director of Health Services Research at Denver Health,” she told her audience while waiting for her presentation to load, “and the Lead of the Mobile Health Research Methodological Core at the ACCORDS Center on the University of Colorado’s Anschutz Campus.”

“ACCORDS stands for Adult and Child Outcomes Research Delivery Science,” she explained, “and you can see right there why we have an acronym for it.” Her humor caused the audience to laugh, breaking the tension that the revelation of Fitbit’s shortcomings had generated, without diminishing the interest in research and evaluation that Jeffrey Nathanson’s comments had inspired.

“In the digital health industry,” Dr. Moore said as she began her presentation “Building Value for Digital Health,” “everybody starts in the same place: with an idea. You have this product, and you’re going to do something wonderful with it. You’re going to improve healthcare in Colorado and beyond. And maybe if you’re lucky and everything goes perfectly right, you might end up making some money from it.”

“It’s that middle part that’s hard to figure out,” she noted. “Research and evaluation can help you get from an idea to a product that’s widely used.” Dr. Moore went on to explain that while the FDA had recently concluded that all mobile medical apps would have to follow rigorous steps to demonstrate clinical efficacy, research and evaluation procedures were still a valuable asset for those digital health products that did not fall under the FDA’s new jurisdiction.

“Research and evaluation will help you do three things,” she told the crowd. “It will help you deliver products that are of higher quality. It will help you engage with your market by demonstrating your value. And it will help you distinguish yourself from your many, many competitors.”

She then showed her audience a slide entitled, “There’s an app for that.” It stated that over three million apps were currently available for download, and that 250 million annual app downloads were expected by 2017. Then it cited a study by Flurry Analytics, which had found that in 2014 the number of digital health apps on the market had increased 87% more than any other app category.

“It’s growing really fast,” she added. “You have to be able to stand out.” A few heads nodded in agreement, as the audience quietly considered the new competition that the booming digital health market had created.

To emphasize the challenges the members of Prime Health were facing, Dr. Moore then introduced the Innovation Adoption Curve. It divided tech users into five types: innovators, early adopters, early majority, late majority, and laggards.

“These are your healthcare systems,” she said upon arriving at the final category. “These are the people who are not going to trust anything unless it’s proven to them that it’s not going to kill their patients.”

She concluded her presentation by framing research and evaluation as a means of encouraging laggards to adopt new technologies. “By using rigorous methods and presenting strong evidence,” she explained, “you can get clinicians comfortable with the idea of trying new things and incorporating them into practice.”

Provided with a means of avoiding the pitfall into which Fitbit had stumbled, the crowd regained its characteristic animation. And as Dr. Moore fielded questions from her newly invigorated audience, it became apparent that the members of Prime Health had shaken off their sense of foreboding, and were ready to learn more.

Research 101 with Dr. Mandy Dempsey

Adapted from one of Dr. Dempsey’s slides entitled “Levels of Evidence,” this diagram demonstrates where each type of study falls according to the strength of the evidence it produces.

“I am not a business person,” Dr. Mandy Dempsey, a pediatrician at the University of Colorado, informed her audience at the beginning of her presentation, “so I don’t know very much about your world. But I do know a lot about clinical care delivery, and about how to use research to better implement products in the clinical care environment.”

Focused mainly on study design, with a few minutes near the end spent discussing outcomes and predictors, Dr. Dempsey’s “Research 101: The Basics” was a crash course in the types of studies that digital health companies can use to demonstrate clinical efficacy.

“There are many factors that you have to take into consideration when undertaking a study,” Dr. Dempsey explained. “The first is cost. Research studies are not free. It’s not going to work if you show up on our campus and say, ‘Here’s my invention. Go for it.”

Dr. Dempsey then outlined the various costs associated with implementing a study, including extracting and analyzing data, and persuading patients and providers to take part. She also pointed out that the time needed to complete a study was often a source of friction between entrepreneurs and researchers. “My world moves very slowly compared to your world.”

Having forewarned the members of Prime Health about the challenges of performing research on their products, Dr. Dempsey then introduced them to the types of studies at their disposal.

“In the world of scientific research and medicine,” she told her audience, “we have standards by which we can validate evidence.”

She covered the concepts of strong and weak evidence first, and then listed the types of research studies from weakest to strongest. Beginning with usability testing, Dr. Dempsey then moved into pilot studies for user experience, pre/post comparison studies, and comparison to control studies, before ending with the “gold standard of research,” the randomized clinical trial.

“As you increase in your levels of evidence, generally there’s an increase in the time, money, and human resources your study needs,” she cautioned the crowd.

Dr. Dempsey concluded her presentation by noting the differences between outcomes from a business perspective and those from a clinical one. After reminding the members of Prime Health that doctors were on the far-end of the adoption curve, she warned them that their innovations had to be both clinically meaningful and implementable.

“There are many examples of innovative interventions that have been shown to be highly effective,” she explained, “but which weren’t implemented because they weren’t clinically feasible.”

The moment her presentation ended, those seated in the auditorium started discussing ways to put the knowledge they just had gained to good use. And as Dr. Dempsey answered their questions, it grew clear to everyone present that many of the Prime Health attendees were eager to begin pursuing studies of their own.

Making the Grade

The July 16^th Prime Health Meetup was an excellent example of the Colorado-based digital health organization’s Integrator model in action. By promoting awareness of how research and evaluation can help digital health companies overcome whatever obstacles they may be facing, Prime Health had once again positioned its members to make the grade when it came to bringing their products to market.

Like the coverage that CyberMed News provides? Follow us on Twitter, LinkedIn, and Facebook to make sure you keep up to date on the most recent developments in Colorado’s digital health community.