schrodinger.test.jobserver module

Convenience functions for dealing with job server

(starting, killing, writing config file).

class schrodinger.test.jobserver.ServerInfo(hostname, schrodinger, job_server_directory, username, pid, job_server_port)

Bases: tuple

__contains__(key, /)

Return key in self.

__len__()

Return len(self).

count(value, /)

Return number of occurrences of value.

hostname

Alias for field number 0

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

job_server_directory

Alias for field number 2

job_server_port

Alias for field number 5

pid

Alias for field number 4

schrodinger

Alias for field number 1

username

Alias for field number 3

schrodinger.test.jobserver.get_user(hostname)

schrodinger.test.jobserver.get_supervisord_dir(ssh: paramiko.client.SSHClient, hostname: str) → str

Gets the remote directory to start or interact with the supervisord.

Parameters

• ssh – ssh client connection established with the hostname.

• hostname – name of the host where supervisord is being setup.

Returns

the remote supervisord directory.

schrodinger.test.jobserver.supv(ssh: paramiko.client.SSHClient, hostname: str)

Gets the command to work with remote supervisord.

Parameters

• ssh – ssh client connection established with the hostname.

• hostname – name of the host where supervisord is setup.

Returns

the supervisorctl command to interact with remote supervisord.

schrodinger.test.jobserver.get_ini_file(ssh: paramiko.client.SSHClient, hostname: str, proc_id: str)

Returns the complete path of the configuration file to manage the process under supervisord in the remote host.

Param

ssh: ssh client connection established with the hostname

Param

hostname: name of the host where supervisord is setup.

Param

proc_id: program name to use as the basename of the configuration file.

Returns

full path of the configuration file to manage the process under supervisord in the remote host.

schrodinger.test.jobserver.get_log_dir(ssh: paramiko.client.SSHClient, hostname: str)

Returns the complete path of the directory to access log files of processes being managed by supervisord in the remote host.

Param

ssh: ssh client connection established with the hostname

Param

hostname: name of the host where supervisord is setup.

Returns

full path of the log directory.

schrodinger.test.jobserver.get_log_location(ssh, hostname, proc_id)

Returns the complete path of the log file corresponding to the managed process under supervisord.

Param

ssh: ssh client connection established with the hostname

Param

hostname: name of the host where supervisord is setup.

Param

proc_id: program name to use as the basename of the log file.

Returns

full path of the log file corresponding to the process.

schrodinger.test.jobserver.get_tls_config(cert_dir)

Parameters

cert_dir (str) – path to wildcard certificates

Return type

dict

Returns

dict for webserver tls config

schrodinger.test.jobserver.get_job_server_directory(basedir: str, username: str, hostname: str) → str

Returns a unique path to a directory to store job server data in.

Parameters

• basedir – prefix of path, usually tmpdir on remote

• hostname – hostname on which to create the job server directory

Rparam

Path to a directory

schrodinger.test.jobserver.job_server_exe(schrodinger)

schrodinger.test.jobserver.job_server_setup_exe(schrodinger)

schrodinger.test.jobserver.run(schrodinger)

schrodinger.test.jobserver.get_job_server_config(job_server_directory)

Return a path for a server-specific job server config in the job server directory.

schrodinger.test.jobserver.get_queue_type(hostname: str) → str

Return a queue type in the format for jsc_admin for a given hostname.

schrodinger.test.jobserver.setup_host(hostname, schrodinger, job_server_directory, username, serve_queue_jobs)

Set up authentication in a new directory.

schrodinger.test.jobserver.setup_supervisord(hostname, username)

Setup supervisord to start jobserver in given hostname

schrodinger.test.jobserver.setup_server(hostname, schrodinger, job_server_directory, username, append=True, licensing=False, use_certs=False, certs_dir=None, use_shared_supervisors=False, use_ldap=True, use_socket_auth=True, hosts_yml_path: Optional[str] = None)

Setup new server on arbitrary ports.

Parameters

• hostname (str) – name of host to set up jobserver

• schrodinger (str) – path to SCHRODINGER

• job_server_directory (str) – base directory for job server

• username (str) – username to use on hostname

• append (bool) – If True, add server config to jobserver.config, If False, overwrite jobserver.config

• licensing (bool) – If True, pass licensing check params to queued jobs

• use_certs – If True, use wildcard certificates in standard internal locations

• use_shared_supervisors (bool) – Use shared supervisor executables

• use_ldap (bool) – Enable LDAP authentication

• use_socket_auth (bool) – Enable unix socket authentication

• hosts_yml_path – transfer a local hosts.yml file to the configured jobserver

schrodinger.test.jobserver.run_command(ssh, command, login=False)

Runs a command.

Parameters

• ssh (paramiko.SSHClient) – a paramiko.SSHClient with an established connection to the remote machine. If ssh is None, the command will be invoked by subprocess.run

• command (list[str]) – The command to run as a list of string arguments

• login (bool) – If True, command requries login shell for ssh

Returns

The output of the executed command over ssh; None if local using subprocess

Return type

str, or None

This function is duplicated in server_management.py for backward compatibility.

schrodinger.test.jobserver.setup_log_dir(log_dir, ssh=None)

schrodinger.test.jobserver.write_ini_file(ini_file, ini_contents, ssh=None)

Write the configuration file to manage jobserver

schrodinger.test.jobserver.run_supervisord(ssh: paramiko.client.SSHClient, hostname: str)

Run the daemon with the configuration in the supervisord directory.

Parameters

• ssh – a paramiko.SSHClient with an established connection to the remote machine.

• hostname – name of the host where supervisord is being setup.

Raises

RuntimeError – skip raising the exception if another program is already listening on the port.

schrodinger.test.jobserver.monitor_job_server_with_supervisord(cmd, hostname, job_server_directory, username)

monitor the given job server command under supervisord

schrodinger.test.jobserver.get_ports_from_file(job_server_directory, ssh=None)

Parse the output of jobserver to get the pid and ports.

Parameters

• log_filename (str) – path of file to open

• ssh (paramiko.SSHClient) – open ssh client, or None to read locally

Return type

tuple(int,int,int,int)

schrodinger.test.jobserver.start_server(hostname, schrodinger, job_server_directory, username, serve_queue_jobs=False)

Start a job_server in the schrodinger directory on the hostname provided, using the job_server_directory as its local storage.

schrodinger.test.jobserver.create_job_server_config(hostname, username, job_server_directory)

Copy the user_authentication that is created automatically at server setup from the remote server machine to the local launch host and dump it into a job server config file.

Parameters

hostname (str) – name of hostname where job server config is located

schrodinger.test.jobserver.modify_jobserver_yml(hostname, job_server_directory, username, licensing, use_shared_supervisors, certs_dir=None, use_ldap=True, use_socket_auth=True)

Modify the jobserver config to respect licensing and web server certificates for a multi-user queue server.

Parameters

• hostname (str) – name of hostname where job server config is located

• job_server_directory – path to job server on hostname

• licensing (bool) – If True, enable license checking on job_server

• certs_dir (str) – If provided, use wildcard certificates from that directory.

• use_shared_supervisors (bool) – Use shared supervisor executables

• use_ldap (bool) – Enable LDAP authentication

• use_socket_auth (bool) – Enable unix socket authentication

schrodinger.test.jobserver.transfer_hosts_yml(hostname: str, username: str, job_server_directory: str, local_hosts_yml_path: str)

Use the provided hosts.yml file to configure jobserver host entries, instead of reading from schrodinger.hosts installations.

schrodinger.test.jobserver.copy_server_log(server, destdir)

Copy jobserverd log file

schrodinger.test.jobserver.clean_localhost_server(server)

Clean localhost jobserver

schrodinger.test.jobserver.clean_remotehost_server(ssh, server)

Clean remotehost jobserver

schrodinger.test.jobserver.cancel_active_jobs(server: schrodinger.test.jobserver.ServerInfo)

Cancels all active jobs managed by the given localhost server. And waits for the job supervisor processes to exit before the function returns.

schrodinger.test.jobserver.kill_server(server: schrodinger.test.jobserver.ServerInfo, cancel_jobs: Optional[bool] = False)

Kill the job_server on server.hostname that is using server.job_server_directory as its local storage.

Parameters

• server – The metadata maintained about the managed jobserver.

• cancel_jobs – cancel the active jobs before killing the jobserver.

schrodinger.test.jobserver.clean_server_dir(server)

Remove job_server_directory on hostname.

schrodinger.test.jobserver.log_job_processes()

Log information about running jobserverd/job_supervisor/job_supervisord processes, which may be preventing removal of the job server directory.

schrodinger.test.jobserver.update_supervisord(ssh: paramiko.client.SSHClient, hostname: str, proc_id: str)

Re-read the configuration file corresponding to the given proces group name and restarts the supervised program if the configuration has changed on disk.

Parameters

• ssh – ssh client connection established with the hostname.

• hostname – name of the host where supervisord is setup to monitor the given supervised program.

• proc_id – group name of the supervised program.

schrodinger.test.jobserver.start_localhost_server(hostname, schrodinger, job_server_directory)

schrodinger.test.jobserver.stop_localhost_server(server)

Stop the jobserver running in localhost

schrodinger.test.jobserver.get_ssh_client(hostname, username)

Return ssh client for hostname. Closes ssh connection automatically.

Parameters

hostname (str) – name of remote host

schrodinger.test.jobserver.get_username_from_host_entry(host_entry_name)

Return username from host_entry_name to find the correct remote user. This user name is useful for directory creation, and remote authentication.

Parameters

host_entry_name (str) – name of host entry (localhost, bolt-gpu)

Return type

str

schrodinger.test.jobserver.schrodinger_jobserver_config()

Set job server configuration to a temporary location. Cleans up the file after use.

This function is duplicated in server_management.py for backwards compatibility.