Contents
Skip to end of metadata
Go to start of metadata

Overview 

This module will illustrate how software is accessed in OSG Connect via OASIS. Different methods to access software are described in Software access using HTCondor or HTTP and Software access using OASIS and Parrot

OASIS is the OSG Application Software Installation Service. OASIS is a value-added service the OSG offers to Virtual Organizations (VOs) to assist with the distribution of VO application software. OSG offers a central login to a software manager for each VO. The manager will be able to login and install their application software in a central location. Once installed in the central location, the OSG is responsible for distributing the VO software to as many sites as possible. Some Campus Grids, like HCC and UC3, have their own VO. Other Campuses can take advantage of the software installed by OSG or other VOs that let them access their software stack.

By doing this tutorial you will se how to check via PALMS the software installed in OASIS, how to use PALMS and OASIS on login01.osgconnect.com to test your jobs and how to run your jobs on resources that provide the software installed in OASIS.

Starting Up

Before going through the examples, login to login01.osgconnect.net and setup a workarea. , you can either run 'tutorial oasis' or type the following:

$ ssh login01.osgconnect.net
$ mkdir -p osg-oasis/log
$ cd osg-oasis

 

Use the OSG Application Software Installation Service (OASIS) 

When one requires software that is frequently used (e.g. community software such as R, NumPy, MatLab, Mathematica, etc.), OASIS may be the way to go.

OASIS  is an OSG service that presents itself to the users as a network file system based on HTTP and optimized to deliver community or group software in a fast, scalable, and reliable way. Files are installed on a central server by librarians (aka software managers) and are available on all the nodes where they are downloaded on demand and cashed. The OSG VO maintains a software repository specifically for the OSG Production and Campus Grid communities. Using OASIS has the advantage that only the portions of your application accessed will be transferred. In addition, the OASIS FUSE software will cache accesses to the OASIS repository using a web proxy as well as a local disk cache so network utilization is minimized particularly if multiple jobs on a single machine use the same applications.

To use OASIS, the target compute cluster must either have OASIS installed or one must use SkeletonKey/Parrot (see Using Parrot when OASIS is not mounted). Clusters in the OSG environment that have OASIS installed will have the HAS_CVMFS_oasis_opensciencegrid_org ClassAd set to TRUE. The previous tutorial An example: an R (statistical analysis) run on OSG Connect is making use of OASIS to access the R software.

To access software on OASIS you can use PALMS

After sourcing the multi-shell setup you can list the available software and setup the environment for the one you'd like to use. The palmsdosetup command takes care of preparing the PATH, LD_LIBRARY_PATH and the environment that you need to run that application. E.g. anaconda is a bundle including NumPy and other scientific libraries for Python:

[user@login01 osg-oasis]$ source /cvmfs/oasis.opensciencegrid.org/osg/palms/setup
[user@login01 osg-oasis]$ palmsdosetup anaconda

And to view the available software (a table is available also at the bottom of this document):

[user@login01 osg-oasis]$ palms list
SOFTWARE       SYSTEM  ARCH    VERSION
snoplus-osg    el6     x86_64  20130806.1
snoplus-osg    el5     x86_64  20130806.1
root           el6     x86_64  5.34.09-gcc4.4
root           el5     x86_64  5.34.09-gcc4.3
R              el6     x86_64  3.0.1-2
R              el5     x86_64  3.0.1-2
osg-client     el6     x86_64  3.1.21-1
osg-client     el6     i386    3.1.21-1
osg-client     el5     x86_64  3.1.21-1
osg-client     el5     i386    3.1.21-1
octave         el5     x86_64  3.6.4-1
anaconda       Linux   x86_64  1.6.1-1
anaconda       Linux   i386    1.6.1-1
osg-wn-client  el6     x86_64  3.1.21-1
osg-wn-client  el6     i386    3.1.21-1
osg-wn-client  el5     x86_64  3.1.21-1
osg-wn-client  el5     i386    3.1.21-1
lmod           el6     x86_64  5.1.1-1
lmod           el5     x86_64  5.1.1-1
Lmod           el6     x86_64  5.1.1-1
Lmod           el5     x86_64  5.1.1-1

Software available in the OASIS repository

The following software is available on the OASIS server via PALMS. To list the available packages you can use palms list and to setup them for your machine you can type:

palmsdosetup APPLICATION

To setup a software use the name in the first column as argument for palmsdosetup. The location in the last column is important only if you want to inspect the code or do a manual setup.

ApplicationDescriptionVersionPlatformsLocation
anaconda

Python 2.7 distribution including NumPy/SciPy/...

see http://docs.continuum.io/anaconda/pkgs.html for a list of packages

1.6.1Linux, 32/64bit/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/anaconda
RR software environment for statistical computing and graphics.3.0.1

el5, 64bit

el6, 64bit

/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/R
rootROOT (note the GCC requirements. The default gcc in SL5 is 4.1.2)

5.34.09-gcc4.3

5.34.09-gcc4.4

el5, 64bit

el6, 64bit

/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/root
snoplus-osgSoftware bundle for the SNO+ experiment. Built with Snoing. It includes ROOT20130806.1

el5, 64bit

el6, 64bit

/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/snoplus-osg
osg-client

OSG Client software, it includes basic Grid clients (proxy mgmt, file transfer,

job submission.

3.1.21-1

el5, 32/64bit

el6, 32/64bit

/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/osg-client
osg-wn-client

OSG Worker Node Client software, it includes basic Grid clients

(proxy mgmt, file transfer, job submission.

3.1.21-1

el5, 32/64bit

el6, 32/64bit

/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/osg-wn-client
octave

GNU Octave is a high-level interpreted language, primarily intended for

numerical computations. 

3.6.4el5, 64bit/cvmfs/oasis.opensciencegrid.org/osg/palms/sw/octave
Icon

In order to make sure that OASIS is installed on the compute node used for your job you have to add the following requirement in your HTCondor submit file. The first requirement assures that OASIS is available, the second one that it is not a stale copy:

Requirements = (HAS_CVMFS_oasis_opensciencegrid_org =?= TRUE) && (CVMFS_oasis_opensciencegrid_org_REVISION >= 729)

 

  • No labels