LoCI logo
Home
About Us
Projects
Publications
Software
Documents
Screencasts
Related Projects
Related Pubs

People
FAQs
News Archive
Contact Us


Links

IBPvo
Record your favorite show on TV!
Linux ISOs
Download the latest version of Linux!
 
 



 
Overview Publications Software

LoRS Command Line Tool Tutorial


The LoRS command line tools let you do three primary things:

  • Store a file into the wide-area network (Upload),
  • Change where the file is located in the network (Augment or Trim), and
  • Retrieve the file (Download).
They also give you the ability to see where the pieces of the file are stored (List) and to extend the allocations' expiration times (Refresh).


The tools have built-in defaults which can be over-ridden by settings in the .xndrc preferences file.


THE .xndrc PREFERENCES FILE

You may use the .xndrc in your home area to store your preferred settings for almost all the options available in the LorS tools. These will override the default settings as described on the defaults page.


You may also have .xndrc files in other directories in addition to your home area. The .xndrc in the current working directory will override the .xndrc in your home area. For example, if you are in ~/data and you have a .xndrc in your home area and another one in ~/data, the tools will read ~/.xndrc first and then the tools will read ~/data/.xndrc will override settings from ~/.xndrc.


You may temporarily override both the defaults and the .xndrc settings by using the command line options.



Keyword Value Comment
LBONE_SERVER
LBONE_SERVER
...
hostname1 port
hostname2 port
...
e.g. vertex.cs.utk.edu 6767

There may be several L-Bone servers listed in the .xndrc. They must be on separate lines. The hostname and port must have a space between them. The tools will start with the first and continue down the list until a server replies.
LOCATION zip= 37001 This can be in any form as outlined on the L-Bone Client API page. It may be only listed once and all keywords and values must be on the same line.
DURATION_DAYS 2.4 This is the number of days (whole or decimal) to request allocations for.
MAXDEPOTS 5 Sets the maximum number of depots to get from the L-Bone for Upload and Augment.
STORAGE_TYPE SOFT The options are HARD (the default) or SOFT.

HARD allocations last until the expiration time, but there is less HARD space available.

There is more SOFT space available, but the IBP depot may delete it at any time if the filesystem starts to become full.

VERBOSE 0 This is not being used currently.
DEMO 0 Set to 1 to output the messages used by the visualization tool. This is the same as setting -D.
DATA_BLOCKSIZE 512K Sets the size of the Logical Data Blocksize.

The format is n[k|K|m|M|g|G] where n is a number (whole or decimal). K and k are kilobytes. M and m are megabytes. G and g are gigabytes.

When using this setting, comment out FRAGMENTS_PER_FILE option by inserting a # in front of FRAGMENTS_PER_FILE.

FRAGMENTS_PER_FILE -1 Use as many as necessary. Leave as -1 when specifying DATA_BLOCKSIZE.

Set to a whole number if you want the file stored in a specific number of pieces. If so, also comment out DATA_BLOCKSIZE option by inserting a # in front of DATA_BLOCKSIZE.
COPIES 1 Sets the number of copies for Upload and Augment.
THREADS 8 Set the maximum number of threads to use.
MAX_INTERNAL_BUFFER 48M Sets the maximum amount of memory to use during Upload adn Download.

Currently,when using the E2E services, the tools can use twice this amount.
TIMEOUT 600 Number of seconds before a tool will give up.
E2E_BLOCKSIZE 1 MB This value must be less than or equal to the Logical Data Blocksize. This unit must also be consistent within the exNode.

An entire E2E block must be retrieved in order to get any or all of the data represented by it. This is the smallest amount of data that may be Augmented or Downloaded. If the value is too large, then downloading a small number of bytes has a higher latency. However, using to small a value requires more CPU processing to encode and decode the data.
E2E_ORDER checksum aes_encrypt
The available options are:
  • none
  • checksum
  • aes_encrypt
  • des_encrypt (decremented)
  • zlib_compress
Except for the "none" option, the other options may be combined in any order and they may be repeated. For example,

   checksum  aes_encrypt  checksum

is a valid E2E ordering.

CONNECTIONS_PER_DEPOT -1 Used in lors_download only. By default, lors_download will open as many connections to any particular depot as the number of threads allow. By changing this value to a positive number, it will place a ceiling on the number of connections to any one depot. It is equivalent to using -a.
JOB_PROGRESS 1 Used by lors_download only. This sets the number of successfully downloaded blocks after a slow block to retrieve before trying to get the slow block from an alternate depot. This requires setting JOB_REDUNDANCY above 1 to try getting theblock from an alternate source. A good rule of thumb is to set JOB_PROGRESS to two times the number of THREADS. This is the same as setting -p.
JOB_REDUNDANCY 1 Used by lors_download only. This sets the number of alternate depots to try to get a slow block from. The best value is usually 2. This is the same as setting -r.
DEPOT
DEPOT
...
hostname1 port
hostname2 port
...
If the user want to specify IBP depots by name, use the --depot-list option instead of the -l option on the command line.
ROUTE_DEPOT
ROUTE_DEPOT
...
hostname1 port
hostname2 port
...
When using lors_augment with the default IBP_copy(), destination depots are specified by the ROUTE_DEPOT keyword rather than DEPOT keyword. We introduce a new keyword to allow the same .xndrc file to be used for a lors_upload and lors_augment to different depots. As with DEPOT, the --depot-list parameter should be specified on the command line.
TARGET_DEPOT
TARGET_DEPOT
...
hostname1 port
hostname2 port
...
When using lors_augment with IBP_mcopy(), intermediate depots are specified by ROUTE_DEPOT and destination depots are specified by TARGET_DEPOT. In this way, a simple tree can be created using IBP_mcopy(). The --depot-list parameter should be specified on the command line, as well as --mcopy.
UTK home        DOE NSF
 
corner
Home   About Us   Projects   Publications   Software   Docs   Screencasts   Related Projects   Related Pubs   People   FAQs   News  Contact Us
corner