The Region 4 Stork (R4S) Collaborative Project Part 4b: High-Throughput Data Entry Portals – All Conditions Tool
Accurate identification of newborns with metabolic disease can significantly improve patient outcomes. Conversely, a missed diagnosis can result in significant morbidity and may even result in death. While a false-positive diagnosis does not carry the burden of increased morbidity or mortality, there are social and psychological costs that may generate significant harm. The Region 4 Stork Collaborative was developed to improve detection of true positive cases of metabolic disease and improve accurate diagnosis. The R4S project uses Mayo-developed software that provides postanalytical interpretation of complex metabolic profiles. The R4S project offers physicians worldwide the opportunity to utilize this software to analyze their patients’ test results, and compare them with other locations’ results.
R4S Collaborative Project Part 4 discusses high-throughput data entry portals. To improve download time, Part 4 has been separated into 2 subjects: 4A focuses on the Tool Runner tool, while 4B focuses on the All Conditions tool.
Presenter: Piero Rinaldo, M.D., Ph.D.
- Co-director of the Biochemical Genetics Laboratory
- Professor of Laboratory Medicine and Pathology
- T. Denny Sanford Professor of Pediatrics at Mayo Clinic
The final tool to be presented here is the all conditions tool.This presentation is the second portion (part B) of the fourth segment of the series describing the products and clinical tools of a newborn screening quality improvement project called Region 4 Stork, or R4S. The title of both presentations is “High-throughput data entry portals”. During editing this topic has been divided in two portions to remain within the required time limits of this type of educational presentation.
Like the tool runner, this tool also covers all conditions with an active tool, but it does it for just one patient at a time.
The second major difference is the type of report. The all conditions tool generates a report in the form of a composite graphic, not a table. It does calculate every possible score, and it reports on any missing data required to calculate a score with a particular tool. Finally, in this format all tools can be readily accessed, even if the score was zero.
A single case file can be extracted from the batch file formatted for the tool runner. From that point on, the process is exactly the same: uploading to the data entry portal, calculation of scores for all one condition tools, and creation of a report. This is, of course, not the only way to access the all conditions tool.
Just like all other post-analytical tools, once a user has logged in the R4S website the all condition tool can be accessed from the link on the home page and then selecting the link to the all conditions tool.
Moreover, direct access is provided from the report of the tool runner for every case with at least one informative score. This tool can also be used retrospectively, for review of true positive cases already posted on R4S. From the main menu bar below the website blue banner a user can select the Tool and Reports menu, site tools (in this case Minnesota), and the True Positives menu. A selection window (only the top section of this window is being shown here) allows to select a condition and then a specific case. At the top of the next window a link to the all conditions tool is displayed. This option is also provided in the True Positives data submission menu once a new case has been added to the R4S database.
This slide summarizes the elements and optional choices available on the selection window of the all conditions tool.
The first dialog allows the selection of condition types. The default setting is to show all types, but a user interested to see, for example, only fatty acid oxidation disorders can easily unclick the two other groups and generate a report showing only the scores of tools for this type of conditions.
The distinction between primary and secondary targets is based on the recommended uniform screening panel, or RUSP. It was established in 2006 by an expert panel assembled by the American College of Medical Genetics with funding from HRSA, the same agency that funded the R4S project. In the following years, this panel was first reviewed and approved by the Health and Human Services Secretary Advisory Committee on Heritable Disorders in Newborns and Children. The committee then made a recommendation to the HHS Secretary, Dr. Kathleen Sebelius. In May 2010 the Secretary agreed to make RUSP the national standards for newborn screening programs in the United States. Like in the case of condition types, a user can choose to display only the conditions included in one group, for example only primary targets.
The selection of score type drives the visual appearance of the lower section of the report. The default is the MinMax type, the one that was discussed extensively in the analysis of the dual scatter plot.
Briefly, the display of the same data using the three types is shown here: MinMax, Regular, and Z-score. The Z-score is based on the calculation patient value minus the average of the population divided by the standard deviation of the population. The z-score is multiplied by 100 and a value of 500 is added. This transformation expands the range of scores so that 95% of them fall between 300 and 700. The addition of 500 shifts any score reduced by differentiator rules to a positive number. The application of z-scores will become increasingly more important once the next version of the software, for convenience referred to as “version 2.0”, is completed, and will be further discussed in part VI, the last segment of this series.
If the all conditions tool is accessed without previous processing of a batch file, it might be necessary to upload a single case .csv file. A macro script to create such file is available to all users and could be customized easily to match the format of raw data files generated by virtually any type of instrument.
Once a file had been uploaded, or transferred from another tool, it is always possible to review the data by clicking the banner “Show analytes”. This is also the process to manually upload data, which is an available option but could be very time consuming and therefore it is not recommended. Once the review and/or data entry is completed, the window is compressed by clicking “Hide Analytes”.
After any desired modification of the settings and inclusion of a data set, “Run Tools” generates a visual report like the one shown here.
The elements included in the All Conditions Tool Report are the following: file name, percentile rank of calculated case scores, the range of condition scores according to the selected type, a color legend for the diamond symbols representing the scores, and, if applicable, a list of missing analytes, if any (none is listed in this case).
This slide shows a few examples of the information provided by the all conditions tool: it could be a straight answer, but for a rare condition like Malonic acidemia that most programs have little or no previous experience with, and therefore could be at risk of being missed by an improperly set cutoff value. Alternatively, the report could reveal a situation of moderate complexity, like this case where there are informative scores for maternal vitamin B12 deficiency and different types of inherited Methylmalonic acidemias. The complexity is considered moderate because all these conditions require the same 2nd tier test and confirmatory testing in plasma and urine. The tool may provide a completely unexpected answer, like this case who is affected with a remethylation disorder, a group of at least three conditions not yet included in the recommended panel but likely to be added sometime soon. One of the two main features of the biochemical phenotype of these conditions is a low concentration of the amino acid methionine, and related ratios. It is likely that many cases like this will continue to be overlooked because many laboratories do not even have a low cutoff for this amino acid, so a result near zero would still be considered to be “normal”. In the R4S database, 124 laboratories have posted a high cutoff for methionine, but only 66 of them also have indicated their use of a low cutoff value. This is unfortunate because the availability of a highly dependable 2nd tier test for the other characteristic finding of this disorder, hyperhomocysteinemia, can drive the specificity of screening for these disorders to virtual perfection, with no false positive outcomes. The next example is the same case that was analyzed in a previous segment of this series to demonstrate the clinical utility of the dual scatter plot, one of the so-called first generation tools: the comparative evidence provided by the all conditions tool is compelling and should prevent the referral of a case with moderate elevations of branched chain amino acids when the underlying explanation is total parental nutrition (TPN) instead of MSUD. Finally, the most important example is the one shown here: a completely negative profile that exemplifies the extremely common situation where random elevations of one or more analytes, something we can describe as random analytical noise, could trigger an unnecessary referral even when the overall profile is not consistent with any known condition.
The clinical utility of the All Conditions tool can be best described as the ability to serve as an effective gateway to a comprehensive and unbiased differential diagnosis of all possible conditions. In the report, all scores (shown as diamonds) and ranges are active links to the respective one condition tool. This design allows a user to explore in rapid sequence multiple one condition tools, even those with uninformative scores, which are shown as gray diamonds. The All Conditions Tool has been utilized 3,389 times in 2012, by more than 100 users worldwide. Similar to the increased awareness experienced with the tool runner, in 2013 the average utilization per month has increased by 67%.
This is the conclusion of the second portion of part IV of the R4S series of Mayo Medical Laboratories Hot Topics. In part V we will describe how a user with access to the tool builder can accomplish a site-specific customization of the post-analytical tools, particularly of the one condition tools and of the dual scatter plot, and their seamless incorporation in the high throughput tools we just presented in this segment.
Please do not hesitate to contact us if you have any questions or requests related to the content of this presentation. Thank you very much for your attention.