April 3rd, 2019

Validation and Verification: Trusting Clinical Products

Verification and validation of vendor products... the Radformation way.

When that shiny new installation CD arrives (or your amazingly hip vendor lets you download the setup files), you are always left with that lingering question: what’s next?

Verification and validation are important steps for any clinically-used product. But how much testing is enough and when do we hit the point of “diminishing returns?” The short answer is that testing should be directly proportional to the complexity of the product. The long answer is that there is no single solution, as the magnitude and depth of validation and verification testing are very specific to the product being implemented: the more variables introduced by the software, the more extensive testing should be. Radformation was started by clinical medical physicists with first-hand knowledge of the requirements and complexities of the clinic and our goal was to make our products easy to integrate into your clinical workflows while minimizing the need for complex testing.

With validation testing, you are ensuring that a product is working in a manner that is beneficial to the end-user. This involves testing items such as the software launching properly, being able to access the product easily, and configuring product access for an entire department. Since Radformation software is typically installed in an accessible location on the Eclipse Image server, this step is one that is easy to configure and test for all users. After ensuring that the software is operating properly and correctly configured for all users, the next step is to confirm that the software is working as expected with verification testing.

The most apparent verification test is verifying that the product is supplying the correct information, i.e. if you expect answer “x,” does the product output answer “x” or something unexpected? This can lead to further tests that were not originally outlined, working to perhaps identify if the unexpected output was caused by user-error or the product malfunctioning. There is quite a bit of critical investigation that is often times needed when testing new products. Radformation provides a verification test as a part of our installation process that we also advise to repeat with each upgrade. When asked for supplemental verification testing, we typically suggest finding a few patients with different treatment sites that are tested when the product is first installed, cross-verifying the values obtained in Eclipse. Additionally, those same patients may be repeated with each upgrade the software to ensure that results are reproducible from version to version. While specific to vendor-provided data, TG332 also provides guidance and resources relevant to vendor product/data verification and may provide beneficial information.

At the end of the day, product validation and verification is just one of the many reasons that medical physicists are so critical to the department and patients: we are the first line of defense to ensure products are working as intended. Radformation has products for clinical medical physicists created by clinical medical physicists. As such, we recognize that the need for validation and verification is critical to the safety of clinics and we strive to make this process as safe, but as efficient for our users as possible.

John Theurer Cancer Center at Hackensack Meridian Health performed a succesful internal validation on Radformation’s ClearCheck software, shown below.

via AAPM