Pegasus
Child pages
  • Development Roadmap
Skip to end of metadata
Go to start of metadata

Short Term

  • Reorganize website with respect to non-Pegasus software products
    • Maybe create "Products" or "Other Software" section?

Intermediate Term

  • Develop tutorial screencasts
  • Add support for Globus Online
  • Add support for rsync

Long Term

  • Resource prediction service
  • Pegasus Appliance
  • Galaxy Integration

Development Releases

  • 4.4

    • plan for June 20th
    • automatic data dependencies
    • wrap up existing stuff
    • documentation
    • JDBCRC change
    • documentation of FAQ's
  • 4.5
    • pegasus-service
      • some form of multi tenancy and authentication mechanism
      • python dependencies especially for external stuff is tricky
      • rename of dashboard database tables so that everything can be put in a single database for the service.
    • ensemble manager enhancements
    • pegasus-dashboard enhancements
      • ability to delete a workflow from the dashboard database
      • support caching of gantt charts etc
    • support for job checkpointing file ( critical for LIGO Virgo needs)
    • support for bypassing staging of output files to staging site ( OSG/Mats)
    • enhancements to data reuse algorithm ( RSEQFlow)
    • support leaf cleanups in hierarchal workflows
    • sqlite jdbcrc
    • software cleanup
    • transfers with hierarchies
    • leverage condor asynch transfers in pegasus lite
    • separate planning job from prescript
    • 5 minute youtube video
  • 4.6
    • metadata
    • dax annotation
    • enhanced notifications
      • monitord
    • PMC data locality
    • globus online support ??
      • get credentials . at least do more research.
    • skipping symbolic links
  • 4.8
    • Containers 
    • Jupiter Support
  • 4.9
    • Hierarchal 
      • mix of compute jobs and sub workflows in the same hierarchy
      • revisit how to do cleanup for the sub workflows always
        • and also handle outputs correclty between compute jobs and DAX jobs.
    • SWIP features
      • DAX schema changes
      • having integrity checking turned on by default
      • planner to add extra jobs for doing integrity checks
      • come up with dial
      • integrating checks into 4.9
    • AWS Batch integration as a clustering executable
    • Burst Buffers
  • 4.10
    • Move the API's from XML to JSON / YAML
      • Site Catalog infromation 
        • Information in there is sufficient
        • the representation
      • DAX API to JSON
        • transfer=true to final output
      • Transformation Catalog Format
      • Replica Catalog
    • proper pegasus-dagman
    • Classification of errors in monitord and show in statistics
    • track more things from kickstart into monitoring database
      • tagging error types
      • pegasus-analyzer will be updated to give suggestions
      • pegasus-monitord to populate to elastic search
  • No labels