You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Table Of Contents:

Last Updated Feb 28th , 2011

Main Focus

  • Improving how we get data to worker nodes. Being able to retrieve data on the worker nodes from various sources.
  • Add notification support to Pegasus
  • Moving auxillary tools to stampede db
  • Other Stampede Related Changes
  • User Guide Reorganization ? Dependant on Bill.

Data to Worker Nodes

The main change for 3.1. The developers document has complete details

  • Refactor how clustered jobs are handled
  • Addition of staging-sites option to pegasus-plan
  • New Shell Job Wrapper for jobs when running on worker nodes
  • Changes to pegasus-cleanup
  • Changes to pegasus-createdir
  • Special S3 support into pegasus-transfer, pegasus-cleanup and pegasus create dir
  • Bypassing of staging-site while staging in input data
  • Bypassing of staging-site while staging out output data
  • Transfer of braindump file to remote workflow execution directories

Notification Support in Pegasus

  • Monitord needs to support notifications
  • Requires changes to Pegasus to generate input file for monitord
  • Come up with default notify scripts in the toolkit that notify the user and generate some status reports.

Auxillary Tools to Stampede DB

  • pegasus-statistics
  • pegasus-plots
  • pegasus-analyzer

Stampede Related Changes

  • Improve Rescue DAG semantics
  • Additional DB schema changes to be able to connect jobs/tasks in the DAX with corresponding kickstart records
  • Addition of workflow metrics file distribution of jobs into the DB

User Guide Reorganization ? Dependant on Bill.

  • No labels