Presentation

Presenter

Transcript

Introduction

Hello. I'm Dr. Bobbi Pritt, the chair of the Division of Clinical Microbiology and the medical director for the Clinical Parasitology Laboratory at Mayo Clinic in Rochester, Minnesota. And I'm delighted to be here with you today to talk about the use of artificial intelligence and digital slide scanning for the detection of intestinal protozoa in trichrome stain stool specimens. 

Objectives

Now, the objectives for this presentation are that, following the presentation, participants will be able to describe the challenges to traditional microscopy for detection of protozoa in stool specimens, essentially, what are the challenges that we would like to overcome; list potential uses of artificial intelligence in parasite detection; and then discuss workflow modifications that may be needed when implementing digital slide scanning and AI-assisted interpretation. 

Disclosures

I'll note that I have no disclosures for this presentation, but I do want to comment that Mayo Clinic has formed a partnership with Techcyte® Inc. to co-develop artificial intelligence or AI algorithms, and we have implemented the Techcyte algorithm in the Parasitology Laboratory, and I'll be telling you about that today.

Background

So, let's talk a little bit about the background of parasite detection in the traditional clinical parasitology laboratory. Now, morphologic examination of clinical specimens by light microscopy remains the gold standard method for detecting many parasites today. And you can see on the right I have an image of two of our technologists looking in the microscope for parasites, screening blood and stool, and on a busy day we will have seven or eight individuals all sitting at their microscope for the entire shift. Common specimen types, I mentioned blood and stool. There's also urine, skin scrapings, tape preparations for pinworm, and then we also get a good number of macroscopic worms and arthropods for identification. 

Challenges of microscopy

There are challenges to microscopy, however. It's a manual, subjective examination and it requires highly trained and skilled technologists. Also, in high-income countries such as the United States, most specimens submitted for parasitic examination do not contain parasites.

So, this leads to low staff satisfaction from screening negative slides, challenges in maintaining competency for detecting rare parasites, and then with high volumes, such as seen in my laboratory, we have a potential for ergonomic issues and also a potential to miss positive specimens amidst all of the negative specimens. 

Digital imaging and artificial intelligence

Now that's where digital imaging and artificial intelligence can play an important role. Pathology is currently undergoing a transformation through digitalization, and this does apply to all microscopy-based testing. Glass slides can be digitalized for subsequent analysis, reviewed on a high-resolution monitor, they can be analyzed by artificial intelligence algorithms that have been trained to find objects of interest, and those objects could be anatomic pathology specimen objects, or it could be something like malaria, seen here on this peripheral blood smear examination, or in this case, Giardia parasites on a trichrome stain stool specimen. 

Now, with digital imaging and artificial intelligence, there are many potential advantages. We could use this technology to screen out negatives that can save a lot of worker fatigue and really allow our highly trained personnel to focus on more time-consuming positive specimens or suspicious findings. It could potentially decrease turnaround times, especially for those negative specimens, and it can also potentially increase sensitivity of finding parasites. Artificial intelligence algorithms don't suffer from fatigue and will continue to maintain sensitivity throughout the day, whereas traditionally, workers may suffer from fatigue and ergonomic injuries from continuous reviewal of slides. It could also facilitate telepathology, settings where expertise is not available, and it could be used for teaching. And last but not least, there may be an opportunity in the future for our technologists or other professionals to work from home and view digital images in that location. 

Now, there are potential disadvantages. Digital imaging and artificial intelligence require expensive systems. You need a slide scanner, and then you need the software that's been trained. The commercially available systems at this point are not FDA-cleared or approved, so they have to be validated as a lab-developed test. And it requires a high degree of specialized knowledge and expertise if a laboratory is going to create their new AI algorithm themselves. And the last thing that's very important to mention for individuals interested in bringing in this type of technology to their laboratory is there may need to be process modifications to prepare their lab for implementing AI. 

Stool parasitology as an example

Now, let's use stool parasitology as an example. The stool parasitology exam, often called the ova and parasite examination, is comprised of at least two important parts: identification of parasites and examination of concentrated wet mount, looking for helminths and protozoa, and then also examination of a permanently stained preparation, looking for protozoa primarily. This is usually made from unconcentrated stool specimens and then stained with trichrome stain or another similar stain. I'm really going to focus on this because this is what we have newly implemented artificial intelligence for in my laboratory, and I'll be telling you about this process. 

Techcyte intestinal protozoa algorithm

We were initially inspired by this particular article in the Journal of Clinical Microbiology where Mathison and colleagues used this deep, convolutional neural network to detect intestinal protozoa in trichrome stain stool specimens, and the particular company is Techcyte. So, this is a commercially developed system, and we decided to bring it in and validate it as a laboratory-developed test in our laboratory. 

Workflow

The workflow for this algorithm requires slides to be prepared and stained, and they have to be coverslipped with a permanent mounting medium. They are then scanned by the slide scanner and images are required and analyzed by the software system, which assists but does not replace the technologist in making the final interpretation.

Hamamatsu NanoZoomer 360 Digital slide scanner

The system that it has been validated with and that we are using in my laboratory is the Hamamatsu NanoZoomer 360 Digital slide scanner. It's a freestanding scanner. You can see it's rather large, but that's good for our laboratory because we're a high-volume laboratory. It uses a 40x dry and digital zoom to capture images at a thousand times magnification, which is what we would use in the traditional microscopy. It reads barcodes and communicates bi-directionally with our laboratory information system (LIS). It self-adjusts to focus points so someone doesn't have to manually adjust the slide and the focus each time, and it can load up to 360 slides at a time for scanning. So perfect for a high-volume laboratory like ours. 

Techcyte Inc. (Orem, UT) artificial intelligence software

This is an example of a screen that is then captured by the software system, which again is Techcyte, and uses artificial intelligence software. The software searches through the matrix of material — in this case it’s stool — and looks at the slide's digital image for objects of interest that it has been trained to find. In this case, it's trained to find protozoan parasites. The objects of interest are then grouped into suggested categories and presented for the laboratory technologist's analysis and review. The technologist can choose to use the generated result or they can perform a manual evaluation, or both. Now, in this instance you can see that Dientamoeba fragilis has been checked off, and there are multiple instances where we can see pictures of what does appear to be Dientamoeba fragilis. Our technologist agreed with this review and checked on this; this is the result that's going to be transferred over into the LIS. The other parasites that were potentially identified when reviewed by the technologists were not thought to be true parasites. There were only a few of them. For example, you could see that Blastocystis species, there was one potential object identified as this. The technologist was able to very quickly review that one instance and then just determine it wasn't a true parasite. 

Detectable organisms/objects

These are the organisms that are detectable by the Techcyte algorithm, and you can see it's the normal culprits. It's almost all of the protozoan parasites that you could find on a trichrome stain stool specimen. 

Preparing for our digitalization journey

Now I'd like to tell you a little bit about what we needed to do to prepare for our digitalization journey. We initially evaluated two different commercially available systems and we chose the Techcyte system. We then had to undertake the not-insignificant changes to our workflow to accommodate digitalization. So let me tell you a little bit about what that entailed.

Step #1 – Slide preparation

So, step number one, we had to change the way we prepared slides. First of all, very importantly, we had to create a thin monolayer of stool on the slide for scanning, and we did that while maintaining sensitivity by using the concentrated stool specimen. And you may recall me mentioning that normally the stained slide preparation uses unconcentrated stool. But by using this concentrated stool specimen, we were able to make thinner layers of stool, and you can see the examples here of how thick that monolayer was, while still having similar or even superior sensitivity for detecting parasites. Also, I want to point out that we had to change our barcodes. Note that the barcode labels cannot overhang the slide, and so we had to change from our previous barcodes to these new barcodes, which are actually superior to what we were using before. When you scan the slides, you can't have interference with barcodes that could interfere with the automated processes of slide scanning.

Step #2 – Coverslipping

We also had to change the method we used for coverslipping. We had to permanently mount the coverslips, and before this, we just used immersion oil to mount the slides and then we discarded them afterwards, but now we need the coverslip to be on very firmly so it doesn't move around during the scanning process. And we needed a fast-drying mounting medium. The previous medium, when we did permanently mount our slides, took up to 24 hours to dry. So, we decided at this point to evaluate two automatic coverslippers given that we have a high volume and we'd be coverslipping all of our slides, and we ended up going with a coverslipping automated system that articulated directly onto our stainer, and you could see it here. So, after our slides were stained, this instrument automatically took them out of the stainer and then coverslipped them one by one.

Step #3 – Simplifying processes

We also used this as a very important time to simplify our processes, and we reevaluated our entire ova and parasite process and were able to identify a few areas we definitely could become more efficient and have a simplified workflow. We consolidated to a single version of a trichrome stain, which happens to be the Ecostain®, a commercially available product. We also limited the types of acceptable preservatives that worked best with the system, which are Ecofix®, which we provide to all of our Mayo Clinic Laboratories customers, and then also PVA without mercury or copper. We used this as an opportunity to get rid of those toxic heavy metals that were in some of the fixatives we used to accept. 

Validation of the AI algorithm

So, all of this required validation. All of those previous changes we implemented before we validated the AI algorithm, the coverslipping and mounting medium, the use of a single stain, the comparison of the unconcentrated, thick specimen to the concentrated, thin specimen. These all required thorough validations to show that it did not impact our results at all, and we actually found that they gave superior results. We were quite pleased with this when we then undertook the validation of the AI algorithm itself. And we specifically looked at accuracy, precision, and limit of detection, and that included analytical sensitivity and specificity. 

Results

We used 142 stool specimens that included positives and negatives, and the positives represented all classes that were identifiable with the algorithm. And our findings are shown here, and I want to call your attention to this right here, where you could see that overall, the manual result detected 82 positive cases and the AI-assisted result showed 83 positive cases. And this additional positive was confirmed by manual blinded review of the slides. This is our most important finding, because any positive slide that is identified by the AI system will then be pulled for manual review. You'll see that the manual review detected more organisms, slightly more, 127 compared to 123, but those would all be detected by the manual review, so the slide level positive or negative result is most important. With this, we felt comfortable that the AI result was equivalent, if not better than the manual result. And then you'll also notice that the AI algorithm detected more white blood cells and more red blood cells, so we were very pleased with this analysis. 

And this is just a list to break down by each parasite, whether it was a pathogen or a commensal protozoan, and also by white blood cell and red blood cell, how the manual compared to artificial intelligence. And for all of them, we did not detect a single category that was significantly skewed towards one or the other. One difference I will point out is you can see here, Dientamoeba fragilis, there were 15 detected by manual and only eight by AI. But then you'll see that AI often grouped parasites together. For example, Dientamoeba fragilis and Endolimax nana, because they look so similar, and there were 15 detected here by AI and only one in manual. So all in all, each parasite ended up being equally detected by both manual and AI, and so we felt very confident in the AI's ability to detect all parasite classes. Again though, what's most important is AI's ability to detect parasites in general so that positive slides can then be pulled for manual review while negative slides could be very quickly excluded. 

Precision/Reproducibility

We also looked at precision and reproducibility. We had three different specimens, and we ran them on multiple days and also multiple runs on a single day, and positive and negative results did not change. We actually ran 10 specimens in this manner for both intra-assay and inter-assay precision, and they all maintained the same results. So excellent precision.

Analytical sensitivity

And then, last but not least, we looked at analytical sensitivity, that is, using artificial intelligence and comparing it to manual microscopy. What was the lowest dilution of a specimen we could still detect? So we identified three positive specimens with a range of commonly seen parasites. We performed 2-fold dilutions going from a 1:1 dilution down to 1:128, and we made six slides from each dilution, so a total of 144 total slides. And then all slides were reviewed in a blinded manner by both conventional and AI-assisted microscopy, and we were very pleased to see that the AI algorithm provided equal or improved sensitivity for each specimen.

And this shows the actual results. You can see here, manual and AI, and we have where six out of six were detectable. That was our limit of detection for that dilution. So, for this particular specimen, manual was able to detect parasites in six out of six dilutions of 1:32, where AI was able to go to another dilution, 1:64. And they were overall for the three specimens, nearly equivalent. AI was slightly better. And I'd like to point out that a dilution even of 1:128, the AI still detected four or five slides out of the six slides, whereas manual dropped off very quickly. So we feel confident that the AI algorithm is as sensitive, if not slightly more sensitive, than manual microscopy. And remember, this is AI-assisted technologist review, so the AI algorithm is detecting the parasites that the technologist can then identify and make a call of positive or negative. So very promising. 

Take-home points

So, I'll leave you all with these take-home points. AI algorithms can be applied to digital slides. That's what we did in this instance, and we will be using this clinically very shortly. And they can be used specifically to increase the detection of parasites in trichrome stain stool specimens. Improved turnaround time in screening out negative specimens. I didn't provide this data, but we were able to show that we could go from about five minutes to roll out a negative specimen down to 15 to 30 seconds, so significant improvement in our turnaround time of calling out negatives. And we could in this manner decrease fatigue and ergonomic injuries associated with the manual microscopy of reviewing all those negative slides manually. And this does improve technologist job satisfaction. So we are very excited that we're going to be implementing this. We feel it's going to improve our processes, and we wanted to provide this information for other laboratories that may also be interested in providing this in their setting for their patients. 

Thank you

I'd like to thank you for joining us today and for staying with me as we talked about this very exciting topic.

Questions?

Contact us: mcleducation@mayo.edu

Image credit: Getty Images