The data transfer between login.osgconnet.net and desktop (or laptop or some backup machine) may be performed to backup data or move data base for calculations. We use scp, sftp or rsync for transferring data of less than 10 GB. Globus transfer is highly recommended for transferring data of more than ten gigabytes.
We can transfer files using
scp which is a counterpart to ssh that allows for secure, encrypted file transfers between systems using your ssh credentials.
To transfer a file using
scp, you'll need to run
scp with the source and destination. Files on remote systems are indicated using user@machine:/path/to/file .
First, log in to OSG Connect:
ssh firstname.lastname@example.org #Connect to the login node with your username passwd: # your password
Once done, you can change to the data directory in your home area:
$ cd ~/data
This directory is an area on Stash that you can use to store files and directories. It functions just like any other UNIX directory although it has additional functions that we'll go over shortly. Let's create a file :
$ echo "Hello world" > my_hello_world
Let's copy the file we just created from Stash to our local system (Desktop or laptop):
$ scp email@example.com:~/data/my_hello_world .
As you can see,
scp uses similar syntax to the
cp command. To copy directories using
scp, you'll just pass the
-roption to it. E.g:
$ scp -r firstname.lastname@example.org:~/data/my-directory .
Transfer between existing servers
From the portal.osgconnect.net web site, click the Transfer -> Start Transfer menu.
On one side, enter "osgconnect#stash" into the endpoint field, and press "Return" or the "Go" button.
On the other side, enter the other endpoint from which you want to transfer data.
Browse the files and folders on the two endpoints to configure you transfer.
Optionally modify the advanced options below the two file browser panes.
Click on one of the arrows at the top to submit the transfer request:
Go to the View Transfer window to see your transfer progress. Click on the progress bar to drill in on transfer details.
Transferring between your laptop and OSG Connect Stash
To add your own laptop as an endpoint, so that you can transfer data to/from stash, you need to install Globus Connect, by clicking on the "Get Globus Connect" in the upper right corner of the Start Transfer page. More detailed instructions for installing Globus Connect can be found here: https://www.globusonline.org/globus_connect/.
Using the Globus Online Command Line Interface
There are a few other quick start guides here: https://www.globusonline.org/quickstart/.
There's lots of good Globus Online material, including documentation, faqs, etc here: https://support.globusonline.org/forums
Making data accessible over HTTP
All user accounts on OSG-Connect have a directory that is automatically web accessible. This directory is located at ~/data/public. To make a file or directory accessible, copy it to this directory or a subdirectory of this directory and give files permissions of 644 and directories permissions of 755. E.g. :
Manually Accessing Stash using HTTP
All the contents of the public directory are made available over HTTP. Go to http://stash.osgconnect.net/+username to view the files and directory that you just made available in the previous section. You can also use wget to retrieve the files, e.g: