Before the Student Arrives
Default group: pegasus.
Default shell: /bin/bash
Group Number: ???
Employee Number: ???
Ask Lorna to find an office for the student. Try to avoid offices without a window.
Lorna will also get an ID badge for the student. Make sure they get a key that works for their office.
Ask Lorna to create a basic computer account for the student. He will need a project account to charge it to.
We have several computers the students can use, including butterfly.isi.edu and juve.isi.edu.
The computer should have the latest CentOS, with a recent Condor and the latest Pegasus release.
Install iptables and open port 22, install fail2ban, and disable passwords via ssh.
Make sure the computer is mounting the ISI NFS home directories properly, and configure it to use the ISI NIS and ISI NTP servers.
Give the student sudo privileges on the machine.
Subscribe the student to the group email list, firstname.lastname@example.org. The files are in: /nfs/asd2/pegasus/mailing-lists
After the Student Arrives
NSF has a new requirement that all students (undergraduate, graduate, and postdoctoral) must have had training in Responsible Conduct in Research (RCR) before being supported (i.e., paid) by NSF.
An online RCR certification system has been set up for this purpose. It is found here:
Please note the following requirements as they apply to all VSoE students:
- all currently NSF-supported students (undergraduate, graduate, and postdoctoral) must complete the on-line course this spring, as per the Provost's memo;
- all new incoming PhD students and postdocs must complete the on-line course before being set up on payroll;
- all undergraduates being put on an NSF support must complete the on-line course before being set up on payroll;
- PIs who have or hope to have NSF funding are urged to have all their PhD students and postdocs complete the on-line course in any case, to avoid possible future delays in student payroll if students/postdocs are moved from one grant to another.
Wired Access for Student's Laptop
Register student's wired NIC address with CIS from https://action.isi.edu/services/ip_address
Sync LDAP with NIS
After password change ask CIS to sync NIS and LDAP. Include username in ticket.
Action should have sent an email about the student's account along with a username and temporary password.
Open a terminal for the student and have them telnet to darkstar.isi.edu. Change their password with:
Add to Crowd
Add student to Crowd with ccg-users, confluence-users, jira-users
Sync user directories in JIRA, Confluence, etc.
Add to Salt
Add student to salt stack.
Give the student write access to Confluence via Crowd.
Action always creates the accounts with ntcsh, but that doesn't work properly on any of our machines. Have the student change their shell to bash with:
Have the student set up their email client to access their ISI account, or create a .forward file.
Set up SSH Keys
Windows 10: Has OpenSSH support. To enable see https://youtu.be/iyLUTgyDkGI?t=40
Papers to Read
List of Pegasus papers:
- "Pegasus, a Workflow Management System for Science Automation"Ewa Deelman; Karan Vahi; Gideon Juve; Mats Rynge; Scott Callaghan; Phil Maechling; Rajiv Mayani; Weiwei Chen; Rafael Ferreira da Silva; Miron Livny; Kent Wenger
- "Rethinking Data Management for Big Data Scientific Workflows." Karan Vahi, Mats Rynge, Gideon Juve, Rajiv Mayani, and Ewa Deelman.
- "Enabling Large-scale Scientific Workflows on Petascale Resources Using MPI Master/Worker" Mats Rynge, Gideon Juve, Karan Vahi, Scott Callaghan, Gaurang Mehta, Philip J. Maechling, Ewa Deelman.
- "Scheduling Data-Intensive Workflows onto Storage-Constrained Distributed Resources" Arun Ramakrishnan, Gurmeet Singh, Henan Zhao, Ewa Deelman, Rizos Sakellariou, Karan Vahi, Kent Blackburn , David Meyers and Michael Samidi. Seventh IEEE International Symposium on Cluster Computing and the Grid - CCGrid 2007
- WorkflowSim: A Toolkit for Simulating Scientific Workflows in Distributed Environments. Weiwei Chen, Ewa Deelman. The 8th IEEE International Conference on eScience 2012 (eScience 2012), Chicago, Oct 8-12, 2012. (Funding Acknowledgement: NFS IIS-0905032)
- Workflow Overhead Analysis and Optimizations. Weiwei Chen, Ewa Deelman. 6th Workshop on Workflows in Support of Large-Scale Science (WORKS 11), Seattle, Washington, November 14th, 2011.
- Cost- and Deadline-Constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds Maciej Malawski, Gideon Juve, Ewa Deelman, Jarek Nabrzyski in 24th IEEE/ACM Conference on Supercomputing (SC12), 2012.
Older Pegasus Papers:
- "Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems" Ewa Deelman, Gurmeet Singh, Mei-Hui Su, James Blythe, Yolanda Gil, Carl Kesselman, Gaurang Mehta, Karan Vahi, G. Bruce Berriman, John Good, Anastasia Laity, Joseph C. Jacob, Daniel S. Katz. Scientific Programming Journal, Vol 13(3), 2005, Pages 219-237
- "Pegasus : Mapping Scientific Workflows onto the Grid" Ewa Deelman, James Blythe, Yolanda Gil, Carl Kesselman, Gaurang Mehta, Sonal Patil, Mei-Hui Su, Karan Vahi, Miron Livny, Across Grids Conference 2004, Nicosia, Cyprus
- "Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges" Ewa Deelman, Yolanda Gil. Workflows in e-Science, e-Science 2006, Amsterdam, December 4-6, 2006
- "Condor: A hunter of idle workstations"
- "Condor-G: A Computation Management Agent for Multi-Institutional Grids" James Frey, Todd Tannenbaum, Ian Foster, Miron Livny, and Steven Tuecke Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10) San Francisco, California, August 7-9, 2001
Optional grid papers:
- "Anatomy of the Grid" Ian Foster, Carl Kesselman, et al