This curation tool must enable the gathering, weighting and interpretation of genomic evidence, results and data, as well as offer clear data visualisation and data handling. Most importantly, the tool must provide a streamlined, collaborative and easy-to-use process to interpret, share and store genomic information for clinical and research purposes.
“We’re pursuing a tool that improves the current systems we have; that makes the process of genetic variant analysis and curation easier, more streamlined and more robust,” said curation tool evaluator, Dr Sebastian Lunke, Head of the Translational Genomics at Victorian Clinical Genetics Services.
“Ultimately, we’re seeking a tool to help future medical scientists doing this work; to make their job easier and, therefore, more effective.”
Two tools to test
After an extensive product selection process earlier this year, two curation tool vendors were chosen. This selection was based on the tools' ability to meet the Alliance’s specific requirements, and their vendors' capacity to collaborate with us as an innovation partner into the future. A rigorous evaluation of each tool was then planned and conducted in mid-2017.
“Our detailed hands-on evaluation brought together 16 expert evaluators to perform a highly structured and well-controlled assessment of both curation tools,” said Dr Natalie Thorne, the Alliance’s Clinical Bioinformatics and Genomics Project Manager. “The comprehensive design and planning of this evaluation pilot involved selecting genomic test data and crafting relevant, yet intricate, case-scenarios to accurately test and address the real-world requirements of the Alliance laboratories.”
Two days to trial
Armed with a diverse set of more than 30 detailed ‘test scenarios’ – addressing basic and advanced variant curation, assay and curation system set-up, and numerous quality measures – the evaluators spent two full days trialling each of the tools, following training with the tool’s vendor.
“Our meticulous and systematic approach was essential to give each tool a fair and equal assessment, and to ensure that no relevant aspects were overlooked,” Natalie commented.
A curation tool to lead the way
The extreme detail of the evaluation, the highly skilled evaluators and the diverse mix of professional expertise in genomics software have all contributed to a well-informed selection of the most appropriate curation tool for our clinical system for genomics.
“The evaluation pilot was labour-intensive for everyone involved but, importantly, we’ll end up knowing the exact capabilities and pitfalls of the curation tools,” said Sebastian.
“It’s fair to say there’s no perfect curation tool out there – we’re very much on the cutting-edge of existing technology. I haven’t heard of anyone doing such a detailed evaluation pilot of these types of tools. We’re all learning as we go, because the whole field is so new. As a world-first evaluation of this type, it’s an exciting process to be part of.”
Following the evaluation pilot, recommendations were made on a selected tool. Negotiations with the selected vendor are now nearing completion.