"VL-e Proof-of-Concept Quickstart", owner=>"Dennis van Dok", email=>"dennisvd@nikhef.nl", footer=>"Comments to Dennis van Dok or Jan Just Keijser.", bodyappend=>"onload='collapse_all();'", style=>"quickstart.css")); ?>
Last Update: $Date: 2009-01-30 10:00:56 $
This HOWTO document outlines the necessary steps to get access to the grid resources offered by the VL-e Proof-Of-Concept environment. Although it is somewhat tailored for vlemed, it is generally applicable for all wannabee grid users. Please help us keep this guide useful by sending your comments, corrections, updates, etc., to the P4 mailing list ( vle-pfour-team@lists.vl-e.nl ).
If you need more information on which resources are actually available within VL-e, consult the VL-e Resource Guide.
This is a step-by-step guide to get pole position on the grid. You can follow this guide blindly, which will be the fastest way to get started. If you get confused and/or need more details, you can always select "Show more info" to get in-depth background information, explanations, and references to relevant websites.
See? This text just became visible because you selected the above link. If you didn't see the link and didn't do anything to see this text, that's ok too - it just means that javascript is disabled in your browser and all the extra information is visible by default.
If javascript works, the link functions as a toggle. The purpose of all this is merely to keep the view from becoming cluttered by a lot of complex and potentially confusing information. So maybe you'd be better off without reading all the extras, in case you become even more befuddled!
See also The PoC website.NOTE: some of the steps can take a few days, since they require manual processing.
The steps outlined below should be followed in linear order, except steps 2a, 2b, and 2c, which can be done in parallel.
A grid certificate is a "passport" for using grid resources.
Are you sure you want to be reading this paragraph? I explained in the previous "extra info" block that it may do more harm than good. So unless you absolutely, definitely, really need to know everything about grid certificates you should probably skip this section.
A grid certificate is a personal electronic document that testifies your true identity, much like a passport. Let's see how they compare.
Before the CA can sign your certificate, your true identity has to be checked. That means that you will have to
Identity checking and signing is done with mathematical sorcery called public key cryptography. The gist of it is that someone holding the public key of your certificate can challenge you to prove your identity by asking a question that only the holder of the private key is able to answer. To prevent identity theft, you should never hand over your private key to anyone, not even to the CA.
Digital signing is the other way around: your certificate is signed vith the CA's private key. The CA's public key can be used to 'decrypt' the signature to verify that
Since the CA's certificate is publicly available, anyone can check the validity of your certificate.
Even more information:
Get a grid certificate from the DutchGrid certificate authority, by going to http://www.dutchgrid.nl/ca/request/ and filling out the web form:
You'll will be asked to follow a sequence of steps including
downloading and running a script on your machine. You should create a new directory for that and call it .globus.
filling in a paper form which you then need to have signed by the indicated person.
choosing a passphrase. Remember this well, since this is the password that you'll have to type every single time that you'll access grid resources (for example, to access data or run jobs).
IMPORTANT: choose a strong passphrase!
!Doct0r Jone$$ likes 2 cut ### patients X-raying 34% of the 20+ PEOPLE? (...)but I'm sure you can think of something better. Just mind that you should be able to memorize it, because writing it down on a sticky note makes the whole thing pointless.
IMPORTANT: your private key will be stored by the script in the file "userkey.pem". There are three golden rules:
After your request has been processed, you'll receive a message from the CA. Simply follow the instructions in the message, and save it into a file usercert.pem in the same directory as userkey.pem. You can also see your Distinguished Name (DN) and the validity of your grid certificate in this message, which will look like this:
Subject: O=dutchgrid, O=users, O=uva, OU=wins, CN=Silvia Delgado Olabarriaga Valid till: Feb 14 16:37:19 2007 GMT
Finally, you have to "install" your certificate in all computers used to access the grid. This means that the directory where the files mentioned above were stored must be copied into your HOME directory in all the computers you'll be using to access grid resources. Configuraton of your environment becomes easier if this directory is named ".globus".
$ ls -la ~/.globus/user* -rw-r--r-- 1 silvia silvia 6659 Feb 15 11:53 usercert.pem -r-------- 1 silvia silvia 963 Feb 13 14:37 userkey.pem(which means that no one can read the private key file other than the owner.) The best is to use zip or tar to copy the complete ".globus" directory into the computers you'll use.
See also
To get an SRB account, send an e-mail to grid.support@sara.nl.
The request will be processed manually and confirmed via e-mail. You'll get an e-mail containing your user name and a password.
NOTE: this is a research platform, so make sure to keep your own backups.
You can later change the password with a command-line utility (Spasswd). See also below Configuring your account at the UI.
O=dutchgrid,O=users,O=nikhef,CN=Dennis van Dok
Instructions about how to access the files (configuration, upload, download) are given in http://poc.vl-e.nl/srb/
This is necessary to associate your certificate to one "virtual organization" (= group of people that have access to shared grid resources).
To do this, you need to load your certificate into your browser. Here is a page explaining how.
To register, follow the instructions on https://voms.grid.sara.nl:8443/vomses.
Choose vlemed and follow New User Registration.
You'll get back an e-mail confirming your registration to the grid and VO.
You may think that this step is superfluous after having gone through all the trouble getting a grid certificate. But you should realise that while a certificate helps to establish your identity, it does not give you the right to use any resources. Those rights are usually handed down through virtual organisations, and that is why you need to register your affiliation.
Note that loading your certificate in your browser adds to the risk of having your identity stolen from you. You should be aware of all the places where your private key is stored and used, and never ever load your certificate in a browser that is not under your control, such as on a public terminal, in an internet cafe, or on a friend's laptop.
on Windows, use your favourite SSH client (e.g. PuTTY) but remember to turn on X-forwarding.
on linux/UNIX, use ssh -Y or ssh -X
The distinction is that since ssh 3.8, X forwarding has become more secure; however, some applications can not deal with this and crash. If you suffer from crashing X11 applications (possible with BadWindow error codes), use -Y.
export DISPLAY=:0
Get and extract the srb-userenv.tar.gz file in your home directory. This will create a directory .srb.
Typically with commands like these:
cd tar xvfz srb-userenv.tar.gz
run voms-proxy-init --voms vlemed to create a grid proxy. You'll be asked to type your certificate passprase selected in step 1.
A proxy is like a certificate, only shorter-lived. Also, it is not signed by a CA, but by your own private key. This is called delegation of your credentials. This way grid jobs can act on your behalf, without having access to your private key: there is a verifiable chain of signatures leading back to a trusted party: the CA.
Grid proxies do carry temporary private keys, and anyone getting a hold of your proxy can act in your name. Although this is a security risk, it is limited by the fact that the signature on the proxy gives it a limited validity of only 12 hours.
You can run voms-proxy-info -all to see the status of your proxy, and voms-proxy-destroy to remove it from the system.
A proxy is just a file in the /tmp directory. So it will remain on the system even when you logout.
This machine has all the environment necessary to access grid resources.
NOTE: <tab><tab> means that you have to type the character "tab" twice, and the names of all available utilities will be shown.
For more info on the utilities, type man <utility>
Goal: run a job that writes Hello vlemed user! into a file.
Get hello.jdl into your home directory.
Delegate your VOMS proxy to the WMS:
glite-wms-job-delegate-proxy -d dennis1234
This will return a delegation identifier, which will look this this:
Connecting to the service https://graszode.nikhef.nl:7443/glite_wms_wmproxy_server ================== glite-wms-job-delegate-proxy Success ================== Your proxy has been successfully delegated to the WMProxy: https://graszode.nikhef.nl:7443/glite_wms_wmproxy_server with the delegation identifier: dennis1234 ==========================================================================
Submit job to queue:
glite-wms-job-submit -d dennis1234 hello.jdl
This will display the job identifier (jobid), which will look like this:
https://grasveld.nikhef.nl:9000/eD7ha_9J7iuU7jnmyqVM_Q
This jobid should be used to check the job status and retrieve the generated files.
Get job status:
glite-wms-job-status https://grasveld.nikhef.nl:9000/eD7ha_9J7iuU7jnmyqVM_Q
Once the job is in state "Done", it is possible to obtain the generated files:
glite-wms-job-get-output -dir . \ https://grasveld.nikhef.nl:9000/eD7ha_9J7iuU7jnmyqVM_Q
The output files (std.err, environmentOnNode.txt) will be stored in the given directory.
To see the output:
cat silvia_tWoc6ZfjIwU-c0ifAWowAg/environmentOnNode.txt
Another example: getenv_job.tar.gz. Retrieve and unpack the tar file.
glite-wms-job-submit -d dennis1234 getEnvironment.jdl
This will dump the environment on the computing node into the "environmentOnNode.txt" file. To be used as an illustration only.
If you followed all the steps in this 'quick start' tutorial, congratulations! You have taken the first difficult hurdles on the grid and you are now ready for the next challenge: to make optimal use of the available resources and to embed them seamlessly in your workflow.
Unfortunately, there is no easy, step-by-step guide to help you there, nor would it be possible to write such a guide. Every user has specific needs which cannot be addressed by generic solutions. So you have to gather bits and pieces of information from colleagues, websites, presentations and other sources to build up your knowledge and learn your way around.
Another unfortunate fact is that the available information – however plentyful – is scattered and sometimes hard to find. And when you do find it, it is often outdated because the grid is rapipdly maturing.
The VL-e Resource Guide could be a useful starting point when you are looking for information.