Every few months I am asked whether randomization.com is validated. The answer has always been a puzzle to me.

I know how to validate deterministic software. When I wrote my software for determining sample sizes, I took all of the examples from a comprehensive textbook and replicated them. In the very few cases where my program calculated a value 1 greater than that in the text, I was able to show that the text had rounded calculated power values like 0.799 to 0.80 whereas I demanded that the power be no less than 0.80.

However, randomization.com does not do deterministic calculations^*. It generates random assignments. There is no standard set of published examples one can appeal to.

While validation is not as simple as for deterministic programs, there are some things that can be done. On January 11, 2002, The FDA issued General Principles of Software Validation; Final Guidance for Industry and FDA Staff OC, which says that it considers validation to be

"confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled."

It also says, with regard to software controlling medical devices,

The level of confidence, and therefore the level of software validation, verification, and testing effort needed, will vary depending upon the safety risk (hazard) posed by the automated functions of the device.

The function of randomization.com is to produce randomization plans to guard against conscious or unconscious bias in assignement of subjects to treatments that might affect the results of a study. There are two ways in which it might fail to achieve its goals.

1. It could be using a faulty algorithm to generate its random assignments. The methods used to generate random assignments are documented fully in the help file. I am unaware of any work questioning the adequacy of the Wichmann-Hill random number generator.
2. The algorithm, even if adequate, could be implemented in a faulty manner. I have received no reports suggesting the the generators are not behaving properly, but other than that and releasing the source code, I have no way to demonstrate the the algorithm was implemented properly and that randomizaion.com is behaving as it should.

If anyone has any suggestions for ways to validate the site, I would be delighted to hear them!

---------------

^* Actually, it does (do deterministic calculations, that is)! The algorithm that produces the random numbers and assignments is a deterministic algorithm designed to generate values that appear random. However, there is no standard set of examples to reproduce, regardless.