...
Before the Student Arrives
CIS Account
Link: https://action.isi.edu/services/new_cis_account
Default group: pegasus.
Group Number: 23
Office, Badge
...
Ask Larry Lorna to find an office for the student. Try to avoid offices without a window.
Larry will Lorna will also get an ID badge for the student. Make sure they get a key that works for their office.
Ask Larry Lorna to create a basic computer account for the student. He will need a project account to charge it to.
Laptop
Loaner or a new laptop
New Mac Setup: https://gist.github.com/mayani/7f55a3a6453fa6fbb9b540418ce5de49
Create a CIS ticket to add the student to the ccg@isi.edu mailing list.
Computer
We have several computers the students can use, including butterfly.isi.edu and juve.isi.edu.
The computer should have the latest CentOS, with a recent Condor and the latest Pegasus release.
Install iptables and open port 22, install fail2ban, and disable passwords via ssh.
Make sure the computer is mounting the ISI NFS home directories properly, and configure it to use the ISI NIS and ISI NTP servers.
Give the student sudo privileges on the machine.
...
Welcome Bag
After the Student Arrives
Wired Access for Student's Laptop
Register student's wired NIC address with CIS from https://action.isi.edu/services/ip_address
AD Setup
https://action.isi.edu/help/ad_self_service
Action should have sent an email about the student's account along with a username and temporary password.
Change password
Set Duo
Set Okta
Add to Crowd
Add student to Crowd's isi directory with ccg-users, confluence-users, jira-users, bamboo-users groups
Sync user directories in JIRA, Confluence, Bamboo, etc.
Add to Salt
Add student to salt stack.
Add to Slack
Add to USC ISI Slack
Add to Pegasus-WMS Slack
Add to Pegasus-Users Slack
Add to other project specific Slack workspaces as applicable.
Add to CCG mailing list
Confluence
Give the student write access to Confluence via Crowd.
...
Have the student set up their email client to access their ISI account.
Set up SSH Keys
Windows 10: Has OpenSSH support. To enable see https://youtu.be/iyLUTgyDkGI?t=40
Code Block theme Emacs mkdir ~/.ssh cd ~/.ssh ssh-keygen -t ed25519 -C "$USER" # Enter a password when prompted # Execute on Student's Linux Desktop # cat id_ed25519.pub > authorized_keys
NSF RCR
NSF has a new requirement that all students (undergraduate, graduate, and postdoctoral) must have had training in Responsible Conduct in Research (RCR) before being supported (i.e., paid) by NSF.
...
- all currently NSF-supported students (undergraduate, graduate, and postdoctoral) must complete the on-line course this spring, as per the Provost's memo;
- all new incoming PhD students and postdocs must complete the on-line course before being set up on payroll;
- all undergraduates being put on an NSF support must complete the on-line course before being set up on payroll;
- PIs who have or hope to have NSF funding are urged to have all their PhD students and postdocs complete the on-line course in any case, to avoid possible future delays in student payroll if students/postdocs are moved from one grant to another.
Papers to Read
List of Pegasus papers:
- "Pegasus, a Workflow Management System for Science Automation"Ewa Deelman; Karan Vahi; Gideon Juve; Mats Rynge; Scott Callaghan; Phil Phil Maechling; Rajiv Mayani; Weiwei Chen; Rafael Ferreira da Silva; Miron Livny; Kent Wenger
- "Rethinking Data Management for Big Data Scientific Workflows."Karan Karan Vahi, Mats Rynge, Gideon Juve, Rajiv Mayani, and Ewa Deelman.
- "Enabling Large-scale Scientific Workflows on Petascale Resources Using MPI Master/Worker" Mats Mats Rynge, Gideon Juve, Karan Vahi, Scott Callaghan, Gaurang Mehta, Philip J. Maechling, Ewa Deelman.
- "Scheduling Data-Intensive Workflows onto Storage-Constrained Distributed Resources" Arun Ramakrishnan, Gurmeet Singh, Henan Zhao, Ewa Deelman, Rizos Sakellariou, Karan Vahi, Kent Blackburn , David Meyers and Michael Samidi. Seventh IEEE International Symposium on Cluster Computing and the Grid - CCGrid 2007
- WorkflowSim: A Toolkit for Simulating Scientific Workflows in Distributed Environments. Weiwei Chen, Ewa Deelman. The The 8th IEEE International Conference on eScience 2012 (eScience 2012), Chicago, Oct 8-12, 2012. (Funding Acknowledgement: NFS IIS-0905032)
- Workflow Overhead Analysis and Optimizations. Weiwei Chen, Ewa Deelman. 6th Workshop on Workflows in Support of Large-Scale Science (WORKS 11), Seattle, Washington, November 14th, 2011.
- Cost- and Deadline-Constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds Maciej Malawski, Gideon Juve, Ewa Deelman, Jarek Nabrzyski in 24th IEEE/ACM Conference on Supercomputing (SC12), 2012.
Older Pegasus Papers ( for reference. not reqd to read on first go):
- "Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems" Ewa Ewa Deelman, Gurmeet Singh, Mei-Hui Su, James Blythe, Yolanda Gil, Carl Kesselman, Gaurang Mehta, Karan Vahi, G. Bruce Berriman, John Good, Anastasia Laity, Joseph C. Jacob, Daniel S. Katz. Scientific Programming Journal, Vol 13(3), 2005, Pages 219-237
- "Pegasus : Mapping Scientific Workflows onto the Grid" Ewa Ewa Deelman, James Blythe, Yolanda Gil, Carl Kesselman, Gaurang Mehta, Sonal Patil, Mei-Hui Su, Karan Vahi, Miron Livny, Across Grids Conference 2004, Nicosia, Cyprus
- "Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges" Ewa Ewa Deelman, Yolanda Gil. Workflows in e-Science, e-Science 2006, Amsterdam, December 4-6, 2006
...
HTCondor Papers
- "Condor: A hunter of idle workstations"
- "Condor-G: A Computation Management Agent for Multi-Institutional Grids" James Frey, Todd Tannenbaum, Ian Foster, Miron Livny, and Steven Tuecke Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10) San Francisco, California, August 7-9, 2001
Optional grid papers:
- "Anatomy of the Grid" Ian Ian Foster, Carl Kesselman, et al
Computer Stuff
Change password
Action should have sent an email about the student's account along with a username and temporary password.
Open a terminal for the student and have them telnet to darkstar.isi.edu. Change their password with:
$ yppasswd
Change shell
Action always creates the accounts with ntcsh, but that doesn't work properly on any of our machines. Have the student change their shell to bash with:
$ ypchsh
Have the student set up their email client to access their ISI account, or create a .forward file.
Set up SSH Keys
Help the students set up their ssh keys so that they can log on to ISI machines from their laptops / home computers
- log on to your linux box using action provided username and password
- mkdir ~/.ssh
ssh-keygen -t dsa
Code Block Enter passphrase (empty for no passphrase): ******** Enter same passphrase again: ******** Your identification has been saved in /nfs/asd2/username/.ssh/id_dsa. Your public key has been saved in /nfs/asd2/username/.ssh/id_dsa.pub. The key fingerprint is: 14:f9:31:1a:2d:e0:11:86:cd:3a:3d:9f:35:73:24:8c user isi key 93.340u 0.070s 4:55.37 31.6% 0+0k 0+0io 363pf+0w
- Make sure you enter a passphrase
- Now you have a private key/public key pair generated.
- cat id_dsa > authorized_keys