Introduction

Pegasus takes in a DAX which is composed of tasks. Pegasus plans it into a Condor DAG / Executable workflow that consists of Jobs. In case of Clustering, multiple tasks in the DAX can be captured into a single job in the Executable workflow. When DAGMan executes a job, a job instance is populated . Job instances capture information as seen by DAGMan. In case DAGMan retires a job on detecting a failure , a new job instance is populated. When DAGMan finds a job instance has finished , an invocation is associated with job instance. In case of clustered job , multiple invocations will be associated with a single job instance. If a Pre script or Post Script is associated with a job instance, then invocations are populated in the database for the corresponding job instance.

Index

A

Abstract Workflow
Input graph of tasks and dependencies, independent of a given run on specific resources.

C

CondorQTime
The time between submission by DAGMan and the remote Grid submission. It is an estimate of the time spent in the condor q on the submit node .The value is calculated as [GRID_SUBMIT/GLOBUS_SUBMIT/EXECUTE -SUBMIT]. 

D

DAX
Input graph of tasks and dependencies, independent of a given run on specific resources.

DAGMan
The workflow execution engine used by Pegasus.

DAG
Result of mapping a DAX to a specific set of resources. The cardinality of the DAX task to DAG job mapping is many-to-many. In Pegasus, this step is called planning.

E

Executable Workflow
Result of mapping a DAX to a specific set of resources. The cardinality of the DAX task to DAG job mapping is many-to-many. In Pegasus, this step is called planning.

I

Invocation
One or more executables associated with a job instance. Invocations are the instantiation of tasks, whereas jobs are an intermediate abstraction for use by the planning and scheduling sub-systems.

J

Job
Node in the DAG. May represent part of a task (e.g., a stage-in/out), one task, or many tasks.

Job Instance
Job scheduled or running by underlying system (e.g., DAGman). Due to retries, there may be multiple job instances per job.

Job Name
The name of the job.

Job Site
The site where the job instance ran.

K

Kickstart time
The actual duration of the job instance in seconds on the remote compute node. 

P

Post time
The postscript time as reported by DAGMan . The value is calculated as [POST_SCRIPT_TERMINATED - POST_SCRIPT_STARTED/JOB_TERMINATED].

R

Resource delay
The time between the remote Grid submission and start of remote execution . It is an estimate of the time job instance spent in the remote queue .The value is calculated as [EXECUTE -GRID_SUBMIT/GLOBUS_SUBMIT]. 

Runtime  
The time spent on the resource as seen by Condor DAGMan . Is always >=kickstart . The value is calculated as [JOB_TERMINATED - EXECUTE].

S

Seqexec time
The time taken for the completion of a clustered job instance .

Seqexec-delay
The time difference between the time for the completion of a clustered job instance and sum of all the individual tasks kickstart time .

Sub Workflow
A workflow that is contained in another workflow

T

Tasks
Representation of a computation in the DAX.

Total Failed Jobs
The total number of failed jobs in the workflow execution.For hierarchical workflow the job count will include the count of failed jobs of the sub workflows.The calculation considers the state of last job instance retry corresponding to a planned job.

Total Failed Tasks
The total number of failed tasks.The count doesn't includes retires of a given task . The calculation considers only the state of the last invocation retry corresponding to  the task.

Total Failed Sub Workflows
The total number of failed sub workflows in the workflow execution.The calculation considers the state of last sub workflow retry.

Total Incomplete Jobs
The total count of jobs that are not in succeeded or failed state. This includes all the jobs that are not submitted, submitted but not completed etc.
This is calculated as difference between 'total' count and sum of 'succeeded' and 'failed' count.

Total Incomplete Tasks
The total count of tasks that are not in succeeded or failed state. This includes all the jobs that are not submitted, submitted but not completed etc.
This is calculated as difference between 'total' count and sum of 'succeeded' and 'failed' count.

Total Incomplete Sub Workflow
The total count of sub workflows that are not in succeeded or failed state. This includes all the jobs that are not submitted, submitted but not completed etc.
This is calculated as difference between 'total' count and sum of 'succeeded' and 'failed' count.

Total Jobs
The total number of jobs  in the workflow planned by the pegasus system. For hierarchical workflow the job count will include the jobs of the sub workflows . 

Total Jobs Retries
The total number of times job instances corresponding to jobs were retried during the workflow run.

Total Jobs Runs
The total number of job instances that where  run during the execution of the workflow. The count includes all the job retries.

Total Succeeded Jobs
The total number of succeeded jobs in the workflow execution. For hierarchical workflow the job count will include the count of succeeded jobs of the sub workflows . The calculation considers the state of last job instance retry corresponding to a planned job.

Total Succeeded Tasks
The total number of succeeded tasks.The count doesn't includes retires of a given task . The calculation considers only the state of the last invocation retry corresponding to  the task.

Total Succeeded Sub Workflow
The total number of succeeded sub workflows in the workflow run..The calculation considers the state of last sub workflow retry.

Total Tasks
The total number of tasks defined in the DAX file. 

Total Task Run
The total number of invocations corresponding to tasks that where actually run during the workflow execution. The count includes the retries.

Transformation
The logical name for a given executable or code run by the pegasus system.

Transformation Count
The number of times the invocations with a  given logical transformation name was executed.

Transformation Max
The maximum runtime value  of invocations with a given logical transformation name.

Transformation Min
The minimum runtime value  of invocations with a given logical transformation name.

Transformation Mean
The mean of the invocation runtimes with a given logical transformation name.

Transformation name
The name of the transformation.

Transformation Total
The cumulative of runtime value of invocations with a given logical transformation name.

W

Workflow wall time 
The walltime from the start of the workflow execution to the end as reported by the DAGMAN.In case of rescue dag the value is the cumulative of all retries.

Workflow cumulative job wall time
The sum of the wall time of all jobs as reported by kickstart. In case of job retries the value is the cumulative of all retries. For workflows having sub workflow jobs (i.e SUBDAG and SUBDAX jobs), the walltime value includes jobs from the sub workflows as well. 

Cumulative job walltime as seen from submit side
The sum of the walltime of all jobs as reported by DAGMan. This is similar to the regular cumulative job walltime, but includes job management overhead and delays. In case of job retries the value is the cumulative of all retries. For workflows having sub workflow jobs (i.e SUBDAG and SUBDAX jobs), the walltime value includes jobs from the sub workflows as well.

  • No labels