HTCondor has a built-in mechanism to transfer binaries and files to and from compute nodes. If users have relatively small amounts of data and binaries to transfer (<1 GB per job) or need to do ad-hoc job submissions, then this mechanism can be effective.
Before getting started, users should login to login01.osgconnect.org and get a copy of the tutorial files:
Word Distribution Example
This example will use the HTCondor transfer mechanisms to transfer a binary (distribution) and a file with a list of words (random_words) to compute nodes that are running the jobs. The condor file that will be used is shown below:
The key parts of the submit file are the
transfer_input_files parameter that gives a comma separated list of paths to the files that will be transferred. In addition,
ShouldTransferFiles needs to be set to YES and
when_to_transfer_output needs to be set to
ON_EXIT in order to make sure that the HTCondor will return the output.
Finally, change submit file to by replacing PROJECT_NAME with the appropriate value before submitting the file:
When the jobs are completed, verify the output: