LoCI logo
Home
About Us
Projects
Publications
Software
Documents
Screencasts
Related Projects
Related Pubs

People
FAQs
News Archive
Contact Us


Links

IBPvo
Record your favorite show on TV!
Linux ISOs
Download the latest version of Linux!
 
 



 
Overview Publications Software

LoRS Command Line Tool Tutorial


The LoRS command line tools let you do three primary things:

  • Store a file into the wide-area network (Upload),
  • Change where the file is located in the network (Augment or Trim), and
  • Retrieve the file (Download).
They also give you the ability to see where the pieces of the file are stored (List) and to extend the allocations' expiration times (Refresh).


The tools have built-in defaults which can be over-ridden by settings in the .xndrc preferences file.


UPLOAD

To start, let's upload a file into the network. The multi-threaded tool is lors_upload. The simple usage is:

    lors_upload <input_file> -f

This tells the tool to store the input file into the network using the preset defaults and then create an exNode that uses the original file name with .xnd appended to it. So if we called:

    lors_upload foo -f

It would create foo.xnd. The foo.xnd file is an XML file that contains information about the stored file (name and size), information about where the data is stored (IBP allocations), and the mapping between those allocations and the logical file (offset and length per allocation).

Let's do a little more sophisticated upload where you have more control. You want to call this exNode bar.xnd and you want to store 2 copies of the file. You want each copy to be broken into 4 pieces. You would like the data to be available for at least 3 days without renewing your leases and you would like the data near Los Angeles. Here is what you would do:

lors_upload foo -o bar.xnd -c 2 -F 4 -d 3d -l 'state= CA city= Los Angeles'

The arguments follow my description. The -o parameter sets the output file name. If you do not specify -f or -o, the exNode will be printed on stdout. The duration parameter (-d) can be specified in seconds (e.g. 600), in minutes (e.g. 30m), in hours (e.g. 12h) or in days (e.g. 2d). The location hints are outlined on the L-Bone Client API page.

You have other choices in addition to the above:
  • You can override the default LBone server hostname and port number,
  • You can specify the type of IBP storage (HARD or SOFT),
  • Instead of number of fragments per copy, you can set a constant blocksize (before End-to-End services are applied),
  • You can limit the number of threads the process uses,
  • You can limit the number of unique depots where the file is stored,
  • You can use a static list of depots instead of contacting the LBone.
The standard policy for the number of threads is Copies times Fragments (or Blocks). In the first example, it would use 1 thread because we used the preset defaults of 1 copy and 1 fragment. In the second example, it would use 8 threads (2 copies x 4 fragments). You can reduce this number using the -t parameter, but you can't specify more than that.

Also, the LoRS tools support End-to-End (E2E) services including checksums, encryption and compression. By default, DES encryption and checksums are used. If you are concerned with maximizing performance, you can choose not to use E2E services with the -n parameter. Encryption, in particular, slows performance by 50%.

UTK home        DOE NSF
 
corner
Home   About Us   Projects   Publications   Software   Docs   Screencasts   Related Projects   Related Pubs   People   FAQs   News  Contact Us
corner