Regulators regulating digital scanners
In the words of Harry Caray – "Holy Cow!" Karen Titus does an excellent job putting together this piece. Who else could use "Gentlemen, start your turtles", "Alan Greenspan" and also work in "From that perspective, a Class III, or even a Class II, classification, is overkill—like dropping a V8 engine into an Amish buggy" in the same article.
So much blog fodder here I have copied the entire article available for free from CAP Today with my comments below on some of my thoughts on this matter.
Courtesy of CAP Today - Regulators scanning the digital scanners by Karen Titus
A recent panel on whole-slide imaging launched a clear message from the Food and Drug Administration: The agency views WSI systems as Class III medical devices and plans to regulate them as such. Gentlemen, start your turtles.
– The FDA has about 1 million pages that are surprisingly easy to navigate on their website including a "How to Classify Your Device Page". If I am reading this correctly, microscopes are Class I devices, as are colposcopes to diagnose cervical dysplasia and cancer, ditto for stethoscopes, holders for artificial heart valves and some defibrillators are Class 2 (roman numerals should only be used for really important things like Super Bowls). Defibrillators [CITE: 21CFR870.5300] are Class 2! 360 joules of energy that could save your life in a moment or cause death if you do not respond to the TV "CLEAR!". And a slide scanner is Class 3 because? Oh, image quality, right. Apparently the FDA didn't look through the microscope I used today. It was like rice crispies were stuck to the lenses. I am sure the article will provide clear detail on why and how these are Class 3 devices. Let's read on.
While the FDA’s decision was clear, the next steps are anything but. Vendors, pathologists, the FDA, and the Centers for Medicare and Medicaid Services could head in any number of directions next, but they won’t be moving swiftly. In fact, those who were at the meeting are still dissecting the information presented at the panel, as if Alan Greenspan had delivered one of his famously tortured pronouncements from the Federal Reserve.
– Yeah, but unemployment was lower, at a nadir (don't get to use that word often) of 4% in the 1990s. Double digit unemployment, financial collapse, Greece, Spain, housing crisis, fall of Lehman, etc… he only predicted once he started doing stand up to not be forgotten when the oft jovial and always comical Bernake took the chairmanship. Those tortured pronouncements in retrospect weren't as bad as this.
Depending on one’s view, the news will slow efforts to bring WSI for primary diagnosis into U.S. laboratories, with some vendors looking to Europe for regulatory relief; have virtually no impact on large vendors, who, while not necessarily enamored of the FDA’s decision, concede it’s one they can live with; kill the market completely; choke innovation among vendors, especially component makers; possibly put laboratories in jeopardy if they try to validate these systems as laboratory-developed tests under CLIA; or encourage laboratories to use WSI for other, already approved purposes, readying themselves for the inevitable day when whole-slide imaging transforms surgical pathology.
- Sprechen Sie Deutsch? - Come è il tuo italiano?
– I predict no impact, no choking, killing, or jeopardy or pocket translators needed to replace US sales; pathologists are conservative folks with supportive industry innovators and inventors; we will test, test and re-test, then test again and we will transform safely and accurately.
What most agree on is that for the first time, the FDA, which regulates the vendor portion of the vendor-laboratory equation, has “put a stake in the sand regarding digital pathology,” says David Wilbur, MD, professor of pathology, Harvard Medical School, and chair, CAP Technology Assessment Committee.
Note that the stake is in sand. “I suspect there’s going to be a whole lot of give-and-take that comes about in the future,” says Dr. Wilbur, who was in the audience at the panel discussion, held at the annual Digital Pathology Association meeting last fall in San Diego.
In a follow-up interview with CAP TODAY, the FDA’s presenter on the panel, Tremel Faison, noted that her remarks reflect the agency’s current thoughts on digital pathology as it works through the issues, rather than an official announcement. “We anticipate eventually having another public meeting, and/or publishing the guidance,” possibly in the next year, she says.
– I think a formal document would minimize confusion on this matter and time is of the essence, particularly since this issue was first addressed in public forum in October of 2009 and mention of pre-market requirements was stated at that time which are similar in many ways to slides and comments from a few months ago.
Download Faison_DPDevicesPanelMeeting2009
Nonetheless, this was not the usual runic message coming out of a federal bureaucracy. Faison, a former cytotechnologist who is now a regulatory scientist in the FDA’s Office of In Vitro Diagnostics, drew praise from those at the meeting. “There was some clarity from the FDA,” says Walter H. Henricks, MD, who represented the CAP on the panel. Until now, he says, industry and labs have largely been in the dark about how the FDA planned to regulate WSI systems for primary diagnosis. “This was the biggest piece of news coming out of the panel — and it was a big piece of news, even if not entirely unexpected,” says Dr. Henricks, medical director, Center for Pathology Informatics, and staff pathologist, Pathology and Laboratory Medicine Institute, Cleveland Clinic.
– A few editor's notes at this point: Tremel taught me everything I know about cytology as a pathology resident at The National Naval Medical Center (now The Walter Reed National Naval Medical Center) and I know she is doing what she can on this and we will all come out the other end better for doing so. I made some comments in November regarding what the CAP did, should have done and could do to help facilitate what is mentioned below as a several year process. See:
The Class III label is used for devices the FDA deems as highest risk; to be approved, such devices require general controls (such as quality system regulation and good manufacturing procedures) and premarket approval. A lower level of clearance, Class II, refers to moderate risk devices that already have a predicate device on the market. The lowest-risk device, Class I, requires no pre-market notification.
Dr. Henricks sees no gain in dwelling on the FDA’s reasoning in classifying WSI systems as Class III. “The facts are what were presented,” says Dr. Henricks, who is also a member of the CAP’s Council on Accreditation and of the Diagnostic Intelligence and Health Information Technology Committee.
- In seeking absolute truth we aim at the unattainable and must be content with broken portions.
– One of the first duties of the physician is to educate the masses not to take medicine.
Sir William Osler
A couple of the larger vendors also show an unwillingness to engage in debate; they prefer to keep plugging away, like infantrymen, to bring their systems to market. The Class III announcement barely made them look up. And while it may have opened a door, no one expects to pass through it anytime soon. The last time the FDA participated in a public forum to discuss WSI regulation was 2009, says Dirk Soenksen, president of Aperio. At that time, observers say, the agency appeared to be gathering information. “Now, two years later, we’re finally able to hear some of the learnings they’ve digested. That shows you the pace at which FDA is working,” says Soenksen, who was at the recent DPA panel.
– A snail's pace? Already used "turtle" twice in this post. Besides, he beats the hare so not sure we are good using turtle (OK, 4 times in this post).
Soenksen says the Class III label didn’t surprise him. “But the fact that it surprised some shows you the confusion that exists in the marketplace,” he says.
The confusion exists even at the most basic level, particularly among those who think the FDA’s regulatory hand smacks a little too hard. Faison says she’s routinely asked about the agency’s reach by those who say the microscope isn’t regulated—and since it’s not, they argue, why should devices performing similar functions be tightly regulated? From that perspective, a Class III, or even a Class II, classification, is overkill—like dropping a V8 engine into an Amish buggy.
In fact, Faison explains, microscopes are regulated as Class I devices. That astounds some pathologists, who think, “Nobody regulates my microscope . Why would they regulate my scanner? It’s doing the same thing,” says Anil Parwani, MD, PhD, who spoke at the DPA meeting about the CAP’s recommendations for validating WSI. Digital pathology may be a familiar topic, having been around for a decade or so, but until now regulatory oversight hasn’t been a big part of the conversation.
That’s especially true at trade shows, says Dr. Parwani, where the mushrooming presence of digital devices over the last five years is devoid of anything as mundane as regulatory information. “Not many people know that FDA is even looking at regulating whole-slide imaging,” says Dr. Parwani, division director of pathology informatics, University of Pittsburgh Medical Center.
Those who expected WSI systems to be Class II devices can debate all they want, Soenksen says, but, “That ship has sailed. They’ve made up their mind that this is a Class III, which is why most people are going to Europe with this technology, not the U.S.”
For vendors committed to the U.S. market, the pace to market will be somewhat stately. “You’re talking five years at the earliest when someone’s going to get an approval,” Soenksen says. “People don’t like to face up to that truth, but that’s the timeline.” The FDA will need to clear a vendor to do a clinical study; the vendor will need to do the study; and the FDA will have to approve the PMA.
– Propose 5 pathologists, 5,000 cases, 5 days to achieve "ground truth/panel/consensus disgnosis", then 5 different pathologists each looking at 1,000 cases on both screen and (exempt) microscopes with 5 week washout. Get cheap monitors from BestBuy as to establish minimal technical equipment needed and microscopes with rice krispies dessicating on objectives, typical of many clinical laboratories to replicate "real life". 5 years too long. Eli and Tom will be in the Superbowl again.
Faison declined to comment on when the FDA anticipated receiving vendor submissions.
– After football season is over and before baseball begins. Also known as "February".
Aperio had been talking with the FDA about clinical studies even before the Class III announcement, and it hopes to have an acceptable study design soon. “We’re going to be the first company to get FDA approval,” Soenksen predicts.
Another large vendor, Omnyx, has been in talks with the FDA as well, says Michael Montalto, PhD, one of the company’s founders and vice president of clinical and regulatory affairs. The Class III billing didn’t surprise him, either. “We have a pretty good sense of what we need to do,” says Dr. Montalto, who also attended the panel. “But that’s not as a result of the announcement—it’s because of our continued back and forth with the FDA.”
Dr. Montalto puts a positive spin on the news. “The device will be safe when it comes out. You have to be happy about that.”
– The best interest of the patient is the only interest to be considered
William J. Mayo
Safety, after all, is at the heart of the Class III label. Listing the potential risks of WSI systems, Faison says, “We’re very concerned that the image quality is as good or better than when using the light microscope. Is it like that for all surgical pathology specimens or only for a segment of surgical pathology specimens? What are the differences in human interaction between viewing under the microscope and navigating on the computer screen?”
To answer such questions, the FDA will require clinical studies to validate performance. Here’s where confusion re-enters the room, forcing players to engage in, if not quite brinkmanship, at least a little blinkmanship.
It’s not clear, for example, what types of clinical studies vendors will need to conduct as part of their PMA submissions. Faison gave some general guidelines at the panel, but until the agency receives its first vendor submission, the FDA’s specific desires are likely to be a mystery. “We don’t have all the answers,” Faison says. The more specific vendors can be with their proposed clinical studies, observers say, the easier it will be for the agency to decide whether to grant a green light.
Another unanswered question: How broad or narrow can an intended use be? Will approval be given for diagnosing, say, breast cancer, but not colon cancer? Prostate biopsies but not endometrial biopsies? Or cancer, but not inflammatory skin conditions? “A huge question,” Dr. Henricks says. “I wish I could give you more clarity. I wish I could give me more clarity.”
“This is a tough question,” Faison says. “We don’t want to see a submission for just one organ system—say, breast.” That’s not a realistic intended use, she says, “and we realize that a laboratory would not buy for just that indication.
“On the other hand,” she continues, “performing a study for all of surgical pathology, including frozens, special stains, etc., would be one huge and hardly manageable submission. We are encouraging sponsors to take a hard look at how these devices will most likely be used in the laboratory, employ a ‘fit for purpose’ mentality, and frame their intended use (and therefore clinical studies) around that.” She adds that vendors will need to define the physical and technical characteristics, such as focus, resolution, and color, prior to beginning their clinical studies; in addition, they’ll need to look at what she calls a clinically balanced population.
– Paul Valenstein, MD I think gives the best talk on the issues raised in the last 6 paragraphs. I heard him speak on these issues at a talk several of us gave at USCAP last year.
Download Valenstein_companion06handout
I recall something about needing 65,000 cases but not hemepath, cytology or pigmented lesions
Vendors are dropping few clues themselves. Regarding Aperio’s submission, “It will be as broad as FDA allows it to be,” Soenksen says, punctuating that sentence with laughter.
Vendors are struggling with this issue, Dr. Montalto says, and some are irked that they’ll need to make the first move. But he reminds his industry colleagues that this is a relatively new field. Previous summary statements and clearances aren’t useful guides; every device will have its own nuances, and it’s up to vendors to discuss them with the FDA. “I think they learn a lot from their discussion with vendors. They’re getting educated on this process, too,” he says.
While vendors and the agencies continue their parry, Soenksen sees an opportunity for pathologists to step up. “My personal view is the College needs to lead this,” he says. He suggests that the FDA is looking for cover from the pathology community—if pathologists, and the CAP, made it clear they support WSI and are ready to use it, he says, the FDA would feel more comfortable bringing the systems to market.
The FDA has also irked some pathologists, Dr. Montalto observes, though he speaks diplomatically, gently pointing out that the AP community, in comparison to CP and other clinical specialties, may be less familiar with the demands of bringing new technologies to the marketplace, including the regulatory environment and its requirements for technical validation and understanding the risk profiles of every device.
The FDA will look at the accuracy of the whole-slide imaging approach and the accuracy of the traditional light microscopy approach, comparing both to an adjudicated standard. This reference standard will likely be determined by a panel of three pathologists; agreement by two of the three creates the reference diagnosis. Dr. Wilbur’s preference would have been to consider the glass slide interpretation the de facto gold standard, and then compare digital to that. This approach is more in line with submitting a 510(k) rather than a PMA, showing essential equivalence to a similar, standard technology. “The glass slide is the current gold standard—this type of PMA approach tests not only WSI interpretation, but also the glass slide standard. It will be interesting to see how this sorts out. WSI could turn out to be better with this approach—who knows?” says Dr. Wilbur.
– A growing number of studies have shown superiority of virtual microscopy versus light microscopy (See: https://tissuepathology.com/weblog/2011/10/superiority-of-virtual-microscopy-versus-light-microscopy-in-transplantation-pathology.html)
– This could be bad for microscope manufacturers and what about all the diagnoses made on these barbarian, exempt devices?
The FDA’s approach also requires a so-called washout period, during which the pathologist theoretically forgets the initial diagnosis before making a diagnosis on the second technology. “How long do you need to make the study not biased?” Faison asks. “I think randomizing the read order may help with that.”
– I may not remember your name, but I never forget a face. Excuse me, have we met somewhere before? But if you change the read order you already know that the first case is not the first case, or the last the last, unless of course it actually is which sounds like something Dr. House would say but most of us know if you are playing Monopoly and you are the thimble and on Connecticut Avenue and roll a nine then you go to Tennesse Avenue and a subsequent 7 puts you on B&O railroad and 8 more gets you Community Chest. With enough cases (see reference to 65,000 cases above), this might work.
If all this sounds familiar, that’s because it’s similar to the FDA’s approach to regulating automated cytology, says Dr. Wilbur. But it may be more problematic for WSI systems, he says, especially the washout period. “Cytology slides are more difficult to remember, but I would suspect that memory of surgical pathology specimens will be more difficult to wash out,” he says. The FDA’s proposed washout period is a week minimum, Faison says, though she adds that two to three weeks would be optimal. (A CAP workgroup on WSI validation said it has found no widely accepted washout length and has recommended a three-week period.)
– Propose minimum of 2 weeks. Absolute minimum. More than 3 weeks ideal. Increases the chances the slides could be lost, broken, misfiled, destroyed or reused. Usually still in the pathologists office for 2 weeks and cannot be uncovered or identifed as broken or destroyed.
“In addition,” queries Dr. Wilbur, “what about other important aspects of a surgical pathology case?” Compared to cervical cytology, he says, where each case has only one reference diagnosis, surgical pathology specimens may have many aspects to test. In addition to a diagnosis of, say, colon cancer, the pathologist is also expected to grade the cancer, assess the margins, the depth of invasion, and so on. If these parts of the case do not match, how will the FDA handle that? Such patient care issues will make design of the studies potentially complicated, he says.
Beyond this, Dr. Wilbur fears that the FDA’s proposed studies will be too expensive and too difficult for smaller companies to conduct. With the advantage falling to larger companies, it could curb innovation.
He’s particularly concerned about how component makers will fit into the picture. Right now, they don’t. The FDA regards WSI as a system, and that’s the regulatory pathway it’s providing. Dr. Montalto suggests the FDA will eventually take another look at this. But near term, it will likely have a chilling effect on component providers, Soenksen says.
Some fear the decision could be stifling. Pathologists won’t be able to mix and match components as they see fit, and large vendors will have little if any incentive to design flexible systems. “What the FDA presented is the easiest solution,” says Dr. Wilbur, who wants more thought devoted to this issue. How will makers of scanners, image-management systems, or viewing stations break into the primary interpretation market? “They’ll be left out in the cold. This has to be addressed.”
Dr. Wilbur’s concerns point to another mudslide in the making. By recognizing WSI as a system, rather than individual components, the FDA also stated it did not see whole-slide imaging as a laboratory-developed test, which originates in the lab and is put together from initial components. Where does that leave labs that want to validate a non-approved WSI system?
“I’m doing my best to piece this together,” says Dr. Henricks, who adds that the matter has now been tipped into CMS’ lap. “What is CMS going to do about this if they find a laboratory using it? What if the laboratory has done a good validation for its intended use in the lab? What happens?”
– Take home message: We are not actually talking about regulators regulating whole slide scanners (without a predicate device), we are actually talking about regulating whole slide systems. Entire ecosystems – stainers?, scanners, monitors, servers, viewers, pathologists?
It’s not hard to extrapolate further and ask about the implications for CAP inspectors enforcing CLIA. The answers could be scary.
“It’s a panic issue right now,” says Dr. Parwani.
– A perfection of means, and confusion of aims, seems to be our main problem.
Albert Einstein
Attempts to clarify matters further at the panel failed, attendees and panel members say. It wasn’t clear, for example, whether WSI systems that have already received FDA clearance for select use (for example, automated image analysis of breast markers) or for research use only can be validated as LDTs, Dr. Henricks says. FDA regulates manufacturers of medical devices, whereas CMS/ CLIA regulates testing that occurs in clinical laboratories. “I think sometimes it’s a misperception that the FDA directly regulates clinical laboratories, outside of blood bank,” Dr. Henricks says.
Dr. Montalto says that in his conversations with the FDA, the agency appears understandably uncomfortable with the idea of labs employing WSI systems for off-label use. He says the potential for this is a major reason the FDA wants vendors to move quickly on their submissions, so the devices can be proven safe for their intended uses.
– I hope not too quickly here we still need another public meeting and possibly a guidance document possibly in the next year.
Dr. Henricks makes it clear that the CAP accreditation program is not taking a public position on this and will harmonize with the FDA and CLIA and their requirements. “We look to them for some guidance on how to approach this topic,” he says. At the same time, he says, it appears that the CMS would welcome input from the CAP on how to address WSI for clinical purposes.
– I recognize CAP is in a tough spot here and everyone is looking to everyone else for guidance. Please give these folks enough guidance to make the decisions we need them to make. A blocked path also offers guidance. (Last 2 lines with apologies to Mason Cooley and Jimmy Johnson. Who else can use these 2 names in the same sentence, huh?). See if CMS would welcome input from the CAP on additional billing codes for some of these services.
In the meantime, the CAP has already begun addressing WSI via the aforementioned workgroup, which was convened by the College’s Pathology and Laboratory Quality Center. The group (Dr. Parwani and Dr. Henricks are members) put together 13 draft statements for laboratories that want to validate WSI systems. The CAP currently has no accreditation program checklists on WSI validation, but the recommendations might be part of a future such checklist.
– The only question then is who drives this may be, could be, future such checklist, The College’s Pathology and Laboratory Quality Center, CAP’s Council on Accreditation, Diagnostic Intelligence and Health Information Technology Committee or The CAP Technology Assessment Committee. I think a committee should be formed to organize these committees.
The CMS representative on the panel, Debra Sydnor, CT(ASCP), says CLIA is interested in the workgroup’s recommendations. “That is very helpful to us,” she says. But it’s hard to know how that interest will translate into practical action and, ultimately, regulatory compliance.
– One should avoid using the terms "practical" and "regulatory" in the same sentence. Kind of like saying "Notre Dame" and "football" for the past decade and a half. It doesn't sound right. And are we talking about regulations or compliance with said regulations.
Ideally, labs should consider holding off on using WSI for clinical purposes until a system has FDA approval for the appropriate intended use, says Sydnor, cytologist, CMS Division of Laboratory Services. She realizes this is a quixotic notion. Sydnor says she’s been fielding calls from laboratories that intend to use—or are already using—WSI for testing that involves H&E. Most of the questions concern the holder of the CLIA certificate—i.e. where is the final testing done?—rather than validation. For CLIA purposes, the pathology test is the specimen grossing and the microscopic slide interpretation; therefore, the location where they are performed must have the appropriate CLIA certificate and meet the applicable requirements.
– Increasingly grossing/histology services are becoming consolidated and where the tissue is grossed and slide read are different facilities. And a third location could be where the image being used to render the diagnosis is reviewed.
She advises laboratories to look to CLIA regulation 42 CFR 493.1253 for guidance regarding off-label use of the device under CLIA, but notes that additional formal guidance, specific to WSI, will be forthcoming from the agency.
– Until then go to http://edocket.access.gpo.gov/cfr_2010/octqtr/pdf/ 42cfr493.1254.pdf for the aforementioned reference above.
What will happen if a CLIA inspector encounters a laboratory using WSI for clinical purposes? The lab will have to demonstrate appropriate validation, policies and procedures, and other CLIA-related quality assurance practices, as it would for any test, she says, but that’s not the end of the story. “This will involve training and instruction within CLIA,” Sydnor says. “This area of automation is all new to them [CLIA inspectors] as well.”
– What? Level of automation? What level of automation? Validate the slide scanning, disk spinning, pixel transfer? What is being manufactured that will reduce the need for hard physical labor and/or monotonous work. We are actually adding additional steps and work and effort in this process. What humans are being replaced by what instrumentation that would justify the sheer mention of "automation".
She makes clear that CLIA is neither granting permission nor encouraging laboratories to use WSI imaging for clinical purposes right now. At the same time, “CLIA is not out to witch hunt anyone,” she says. “We basically want to know what you’re doing, how you’re ensuring quality testing, and what it is you’d like to do.” Like everyone else at the table, she says, CLIA is seeking data about how well, and how safely, these systems perform.
– Translation: We work for the government and we are here to help. We are not saying that you can't, but we are not saying that you can either.
Meanwhile, what’s a less-adventurous lab to do?
A surprising amount, as it turns out. As Dr. Henricks notes, digital pathology remains viable for uses other than primary diagnosis, including quality assurance, secondary consultations, education and research, and automated image analysis.
Labs should continue using WSI in approved ways, Dr. Parwani says, which will let them move quickly once the systems earn approval for primary diagnostic use. Here the CAP working group guidelines will be valuable, he says, since they’re extensively annotated and draw on available data as well as user experience. Labs can use the guidelines to ensure they have all the components in place and the right workflow as they prepare for the eventual shift to WSI.
– In 5 years we can jump on this right away.
Dr. Wilbur and his colleagues mostly use WSI for continuing education, but in mid-December they inked a contract with an image-management system company, setting them up to do what he calls “intramural” consultation. This will allow pathologists to share cases in the system across multiple desktops, including those at regional affiliates, and enable second opinion consultations to flow into the institution from outside sources.
At UPMC, Dr. Parwani and his colleagues continue to use digital pathology, as they have for the past couple of years, for education, research, QA, and getting opinions from colleagues. They use it for image analysis of breast markers, and they are starting to accept consults from other countries and institutions for second opinions. “We’re trying to use it for all the intended uses that are approved,” he says. They’re participating with a vendor in clinical trials to prepare its system for premarket approval, and their interest in primary diagnostic use looms large. “Most of our pathologists are very comfortable with looking at digital images and looking at digital slides,” he says.
– Who mentioned anything related to pathologists actually being able to read these images and help providers take care of people. When was pathologists abilities to do their jobs to the best of their abilities with the right training, experience and equipment discussed in this process? You mean pathologists can actually do this today? Read images? Like through a microscope? Or a gross photo? Or an electron micrograph?
“There are so many things you can do,” he adds. “This should not stop your march toward digital pathology.” The DPA panel, in his view, was merely one step in the process. He, like Dr. Montalto, even sees it as a positive one. “FDA is looking at it, and we’re going to have a good product in the end.”
- “Everything will be all right in the end. If it’s not all right, then it’s not the end.”
Karen Titus is CAP TODAY contributing editor and co-managing editor.
Of the FDA’s decision to regulate whole-slide imaging systems as Class III devices, Aperio president Dirk Soenksen says, “They’ve made up their mind. . . . You’re talking five years at the earliest when someone’s going to get approval.” How broad will Aperio’s submission be? “As broad as FDA allows,” he says.
Faison
Dr. Henricks
Dr. Montalto
Dr. Wilbur
