We created the condor_submit_util script to automate the process of writing a submit file and submitting a cluster of jobs to the Condor queue. When you execute this script, you can include all arguments on the command line. Or, you can execute the script in interactive mode and be prompted for your submit file attributes.
The default settings for the Automated Condor Submission script support creation of submit files for programs that are written in the R language. To submit another type of program to the Condor queue, such as an Octave program, specify the full path and program for the executable (in this example, Octave). You then define your program file as the input to the executable.
Note: To use the condor_submit_util script, you must have an RCE account. See for more information.
The following are example uses of the condor_submit_util script and options to submit batch processing in the RCE. A complete description of options is provided in .
Example Using Multiple Input Files
Start with an executable program (named foo) that uses a set of input data files (named data0 - data4) and does some analysis.
To save the submit file and receive notification when processing is done, type the following command:
> condor_submit_util -x foo -i "data" -k -N
The submit file for this batch looks like this:
Universe = vanilla
Executable = /usr/bin/foo
Arguments = --no-save --vanilla
when_to_transfer_output = ON_EXIT_OR_EVICT
transfer_output_files = out.$(Process)
Notification = Complete
input = data.$(Process)
output = out.$(Process)
error = err.$(Process)
Log = log.$(Process)
Queue 5
Example Using Multiple Iterations of One Executable Program
An R program (named random.R) produces random output.
To execute this program eight times and place the output of each execution in separate files in your default working directory, type the following command:
> condor_submit_util -i random.R -n 8 -o "outrun"
Following is the submit file for this batch:
Universe = vanilla
Executable = /usr/bin/R
Arguments = --no-save --vanilla
when_to_transfer_output = ON_EXIT_OR_EVICT
transfer_output_files = outrun.$(Process)
input = random.R
output = outrun.$(Process)
error = error.$(Process)
Log = log.$(Process)
Queue 8
Example Checking Process Status
To check the status of the Condor queue after submitting your program for processing, type:
> condor_q
-- Submitter: x1.hmdc.harvard.edu : <10.0.0.47:60603> : x1.hmdc.harvard.edu
ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD
24.4 mcox8/18 16:35 0+00:00:01 R 0 0.0 R --no-save--vani
24.5 mcox8/18 16:35 0+00:00:00 R 0 0.0 R --no-save--vani
24.6 mcox8/18 16:35 0+00:00:00 R 0 0.0 R --no-save--vani
24.7 mcox8/18 16:35 0+00:00:00 I 0 0.0 R --no-save--vani
24.8 mcox8/18 16:35 0+00:00:00 I 0 0.0 R --no-save--vani
24.9 mcox8/18 16:35 0+00:00:00 I 0 0.0 R --no-save--vani
6 jobs; 3 idle, 3 running, 0 held
The column ID lists the process IDs for your jobs. The column ST lists the status of each job in the Condor queue. A value of R indicates that the job is running. Valid status values are listed in .