You use the command condor_submit to submit batch processing manually to the Condor system.
If you are new to batch processing please see the previous section using condor_submit_util
In the RCE, you must include the attribute Universe = vanilla in every submit file. If you do not include this statement, Condor attempts to enable job-check pointing, which consumes the central manager resource.
Perform the following to submit batch processing manually:
Before you submit your program for batch processing, create a directory in which to run your submission, and then change to that directory. Make sure that you set permissions to enable the Condor software to read from and write to the directory and its contents.
Also make sure that your program is batch ready.
Create a submit file for your program.
For information about how to create a submit file, see Submit file basics.
Note: You can use the HMDC Automated Condor Submission script and include the -k option to create a submit file, and then edit and reuse that submit file for other submissions.
Submit your program for batch processing.
Type the following at the command prompt:
condor_submit <submit file>
Condor then checks the submit file for errors, creates the ClassAd object, and places that object in the queue for processing. New jobs are added to the end of the queue. For example:
condor_submit myjob.submit
Submitting job(s)..........
Logging submit event(s)..........
10 job(s) submitted to cluster 24.
View your job queue to ensure that execution begins.
condor_q <username>For example:
condor_q wharrell
You send input to the Condor system using a submit file, which is a text file of <attribute> = <value> pairs. The naming convention for a submit file is <file name>.submit. Before you submit any batch processing, you first set up a directory in which to work, and create the executable script or program that you choose to submit for processing.
Basic attributes used in the submit file include the following:
Universe - At HMDC you specify the vanilla universe, which supports serial job processing. HMDC does not support use of other Condor environments.
Executable - Type the name of your program. In the job ClassAd, this becomes the Cmd value. The default value in the RCE for this attribute is the R program.
Arguments - Include any arguments required as parameters for your program. When your program is executed, the Condor software issues the string assigned to this attribute as a command-line argument. In the RCE, the default arguments for the R program are --no-save and --vanilla.
Input - Type the name of the file or the base name of multiple files that contain inputs for your executable program.
Output - Type the name of the file or the base name of multiple files in which Condor can place the output from your batch job.
Log - Type the name of the file or the base name of multiple files in which Condor can record information about your job's execution.
Error - Type the name of the file or the base name of multiple files in which Condor can record errors from your job.
Queue - The command queue instructs the Condor system to submit one set of program, attributes, and input file for processing. You use this command one time for each input file that you choose to submit. (Note: this keyword should be specified without an equals sign, e.g. "Queue 10".)
Request_Cpus - The number of physical CPU cores required to run each instance of the job, as an integer.
Request_Memory - The amount of physical memory (RAM) required to run each instance of the job, in MB. Your job may also have access to additional system swap memory if available, but this value guarantees a minimum amount of available system main memory for your job.
The full documentation of all available submit file options can be found at http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html
An example submit file with the minimum required arguments is as follows:
cat myjob.submit Universe = vanilla Executable = /usr/bin/R Arguments = --no-save --vanilla input = <program>.R output = out.$(Process) error = error.$(Process) Log = log.$(Process) Request_Cpus = 2 Request_Memory = 4096 Queue 10
This file instructs Condor to execute ten R jobs using one input program (<program>.R) and to write unique output files to the current directory. Each job process will run with access to 2 CPU cores and 4GB of RAM.
When you specify file-related attributes (executable, input, output, log, and error), either place those files within the directory from which you execute the Condor submission or include the relative path name of the files.