How can I make my batch submission track iteration number?

Condor refers to the iteration number of your batch job as the "Process" number, and the descriptor for your job in aggregate as its "Cluster" number. A Job ID consists of both, separated by a dot: $(Cluster).$(Process)

To track iteration number for batch submissions, use one of the following methods:

  • Pass the iteration number as an argument to the executable. For instance, for an R job:
    • Add --args '$(Process)' to the Arguments line of your Condor submit file. This passes to the R interpreter the process number of the R run, which progresses from 0 to one less than the number of runs.
    • Capture the argument in a variable in your R code by entering the following line: run <- commandArgs(TRUE). The R object run contains the run number. You then can use this object to construct appropriate output file names for your job.
  • Inspect the value of the environment variable CONDOR_JOB_JobId. The iteration number of the job will be the portion after the dot.
  • Set an environment variable with the value of the Condor-internal variable $(Process), using the environment directive in your submit file as follows:
    • environment = ITERATION_NUMBER=$(Process)