Speeding up TestU01 with the use of HTCondor
Joshua Beizer
TL;DR
This paper demonstrates how integrating HTCondor with TestU01 significantly reduces testing time for random number generators, from hours to minutes, while minimizing user disruption and comparing performance with existing parallel implementations.
Contribution
The paper introduces a method to accelerate TestU01 testing by leveraging HTCondor, providing a practical setup guide and performance analysis against existing parallel versions.
Findings
TestU01 run time reduced from 5.5 hours to 5.5 minutes.
User downtime minimized with HTCondor integration.
Performance surpasses standard and existing parallel implementations.
Abstract
Testing random number generators is a very important task that, in the resent past, has taken upwards of twelve hours when testing with the current agship testing suite TestU01. Through this paper we will discuss the possible performance increases to the existing random number generator testing suite TestU01 that are available by offering the executable to an HTCondor pool to execute on. We will see that with a few modifications we are able to decrease the running time of a sample run of Big Crush from about five and a half hours to only five and a half minutes. We will also see that not only the time taken for the testing to complete is shortened, but also the amount of time the user is unable to use their testing computer is reduced to almost none. Additionally, during this paper we will look at the standard implementation of TestU01 and how it compares with a preexisting Parallel…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Distributed and Parallel Computing Systems · Advanced Data Storage Technologies
\college
College of Arts and Sciences \departmentDepartment of Computer Science \manuscripttypeProject \degreeMaster of Science \degreeyear2016 \defensedateAugust 2nd 2016
\committeepersonMichael MascagniProfessor Directing Project \committeepersonXin YuanCommittee Member \committeepersonJeff BauerCommittee Member \committeepersonYu WangCommittee Member
Speeding Up TestU01
With the Use of HTCondor
Joshua Beizer
Abstract
Testing random number generators is a very important task that, in the resent past, has taken upwards of twelve hours when testing with the current flagship testing suite TestU01. Through this paper we will discuss the possible performance increases to the existing random number generator testing suite TestU01 that are available by offering the executable to an HTCondor pool to execute on. We will see that with a few modifications we are able to decrease the running time of a sample run of Big Crush from about five and a half hours to only five and a half minutes. We will also see that not only the time taken for the testing to complete is shortened, but also the amount of time the user is unable to use their testing computer is reduced to almost none. Additionally, during this paper we will look at the standard implementation of TestU01 and how it compares with a preexisting Parallel version of TestU01. We will be comparing the performance of the standard version of TestU01 with the parallel version so that we have a performance baseline to test our own modifications against. Finally, this paper will also discuss the use of the distributed computing system HTCondor, and cover some basic instructions related to setting up and using HTCondor. HTCondor is already known to increase the performance of a networked group of computers by allowing super users to utilize additional resources from other lesser user’s idle workstation. We will relate the model recommended for HTCondor to a standard computer lab found at a University and explain why they are the perfect candidates for the system.
keywords:
random number generator, HTCondor, Condor, TestU01
\makecommitteepage