This module will illustrate how software can be transferred and accessed in OSG Connect. Different methods to access software without transferring it are described in Distributed software access using the OASIS service and Software access using OASIS and Parrot.
In the section OSG Connect Data you have seen the same mechanisms used to transfer data. The techniques and the recommendation are the same, e.g. use HTCondor 's built-in transfer mechanisms only for relatively small amounts of data and binaries to transfer (<100MB) or if you need to do ad-hoc job submissions.
Transferring software is very similar. When using HTCondor to transfer software keep in mind that the
Executable specified in the submit file is transferred by default. Then remember that you must have execute permission on the software files, that the PATH (folders where executables can be found) may be set differently on the remote host (pay attention on where the software is).
First login to login.osgconnect.net. Then, to create a working directory, you can either run 'tutorial software1' or type the following:
Then make sure that you have a public directory exported via Web. This directory should have been created for you and be accessible at an URL like https://stash.osgconnect.net/+user/ where "user" is your user name on OSG Connect :
Distributing Applications Using Stash and HTTP
This example is very similar to the one in Access Stash remotely from your job using HTTP. You will be preparing the input files for the job, including an executable, and transferring them via HTTP. When you are transferring an executable instead of data you have to pay attention because Web servers or commands like
wget frequently change the file permissions and this can cause the job to fail. To make sure that your program is executable either set manually the permission on the transferred file (
chmod +x filename) or transfer a tar bundle and uncompress it.
Prepare the file bundle in your public html space (~/data/public):
words.sh is the auxiliary script to run the job:
Here is the submit file. Note that HTCondor allows to specify an URL as input file and it will download it for you. Substitute your user name for marco in the transfer_input_files line:
Submit the job:
Once the jobs are completed, you can look at the output in the logs directory and verify that the job ran correctly: