This is a rough outline of all the connections that are made to run a Condor-G job in the grid universe using the GT2/GT5 GAHP.
- The gridmanager launches the GT2/5 GAHP.
- The GAHP binds to a port in the ephemeral port range of the client (GAHP_PORT).
- The GAHP launches a GASS server for file transfers (stdin, stdout, stderr, others).
- The GASS server binds to a port in the ephemeral port range of the client (GASS_PORT).
- The GAHP contacts the Gatekeeper on well-known port 2119.
- The gatekeeper launches the jobmanager.
- The jobmanager binds to a port in the ephemeral port range of the server (JM_PORT).
- The gatekeeper sends jobmanager address to the GAHP.
- The GAHP contacts the jobmanager on JM_PORT.
- The jobmanager sends status updates to the GAHP on GAHP_PORT.
- When the job is done, the jobmanager sends data to GASS server on GASS_PORT.
Ephemeral Port Range
The client will need to specify a port range for services to listen on. We typically use 50000–51000.
The client needs to:
- Accept incoming connections from the server for the designated ephemeral port range of the client.
- Allow outgoing connections to the server for port 2119 and anything in the server's ephemeral port range. Note: If you don't know what the server's ephemeral port range is, or if it is not set to a specific range, then you probably need to open up all ports.
The server needs to:
- Accept incoming connections from the client to port 2119.
- Accept incoming connections from the client to the local ephemeral port range of the server.
- Allow outgoing connections to the client's ephemeral port range.
Globus command line tools such as globus-job-run look for an environment variable called GLOBUS_TCP_PORT_RANGE. It should be set like this:
for example, if the lowport is 50000 and the highport is 51000, it should be:
You can also set GLOBUS_TCP_SOURCE_RANGE if you want to control the local port of outgoing connections.
Condor-G is going to use the Condor configuration variables LOWPORT and HIGHPORT to determine the local ephemeral port range. Set them in your condor_config like this:
If you want to configure Condor to use a different range for outgoing ports than incoming ports, then you need to specify OUT_LOWPORT / OUT_HIGHPORT and IN_LOWPORT / IN_HIGHPORT. In this case, OUT means outgoing (client side) connections, and IN means incoming (server side) connections. If those aren't defined, then LOWPORT and HIGHPORT are used for both incoming and outgoing connections.
The gridmanager will use these to set GLOBUS_TCP_PORT_RANGE and GLOBUS_TCP_SOURCE_RANGE in the environment for the GAHP and GASS servers.