pysqa.base.config.QueueAdapterWithConfig

pysqa.base.config.QueueAdapterWithConfig#

class pysqa.base.config.QueueAdapterWithConfig(config: dict, directory: str = '~/.queues', execute_command: ~typing.Callable = <function execute_command>)[source]#

Bases: QueueAdapterCore

The goal of the QueueAdapter class is to make submitting to a queue system as easy as starting another sub process locally.

Parameters:

config (dict) – Configuration for the QueueAdapter.
directory (str) – Directory containing the queue.yaml files as well as corresponding jinja2 templates for the individual queues.
execute_command (funct) – Function to execute commands.

__init__(config: dict, directory: str = '~/.queues', execute_command: ~typing.Callable = <function execute_command>)[source]#

Methods

`__init__`(config[, directory, execute_command])
`check_queue_parameters`(queue[, cores, ...])	Check the parameters of a queue.
`convert_path_to_remote`(path)	Converts a local file path to a remote file path.
`delete_job`(process_id)	Delete a job.
`enable_reservation`(process_id)	Enable reservation for a process.
`get_job_from_remote`(working_directory)	Get the results of the calculation - this is necessary when the calculation was executed on a remote host.
`get_queue_status`([user])	Get the status of the queue.
`get_status_of_job`(process_id)	Get the status of a job.
`get_status_of_jobs`(process_id_lst)	Get the status of multiple jobs.
`get_status_of_my_jobs`()	Get the status of the user's jobs.
`submit_job`([queue, job_name, ...])	Submit a job to the queue.
`transfer_file`(file[, transfer_back, ...])	Transfer a file to a remote location.

Attributes

`config`	Get the QueueAdapter configuration.
`queue_list`	Get the list of available queues.
`queue_view`	Get the Pandas DataFrame representation of the available queues.
`queues`	Get the available queues.
`remote_flag`	Get the value of remote_flag.
`ssh_delete_file_on_remote`	Get the value of ssh_delete_file_on_remote.

Check the parameters of a queue.

Parameters:

queue (str) – The queue to check.
cores (int, optional) – The number of cores. Defaults to 1.
run_time_max (int, optional) – The maximum run time. Defaults to None.
memory_max (int, optional) – The maximum memory. Defaults to None.
active_queue (dict, optional) – The active queue. Defaults to None.

Returns:

A list of queue parameters [cores, run_time_max, memory_max].

Return type:

list

property config: dict#

Get the QueueAdapter configuration.

Returns:: The QueueAdapter configuration.
Return type:: dict

convert_path_to_remote(path: str)[source]#

Converts a local file path to a remote file path.

Parameters:: path (str) – The local file path to be converted.
Returns:: The converted remote file path.
Return type:: str

delete_job(process_id: int) → str | None#

Delete a job.

Parameters:: process_id (int) – The process ID.
Returns:: The result of the delete job command.
Return type:: str

enable_reservation(process_id: int)#

Enable reservation for a process.

Parameters:: process_id (int) – The process ID.
Returns:: The result of the enable reservation command.
Return type:: str

get_job_from_remote(working_directory: str) → None[source]#

Get the results of the calculation - this is necessary when the calculation was executed on a remote host.

Parameters:: working_directory (str) – The working directory where the calculation was executed.

get_queue_status(user: str | None = None) → DataFrame | None#

Get the status of the queue.

Parameters:: user (str) – The user to filter the queue status for.
Returns:: The queue status.
Return type:: pandas.DataFrame

get_status_of_job(process_id: int) → str | None#

Get the status of a job.

Parameters:: process_id (int) – The process ID.
Returns:: The status of the job.results_lst.append(df_selected.values[0])
Return type:: str

get_status_of_jobs(process_id_lst: list[int]) → list[str]#

Get the status of multiple jobs.

Parameters:: process_id_lst (list[int]) – List of process IDs.
Returns:: List of job statuses.
Return type:: list[str]

get_status_of_my_jobs() → DataFrame#

Get the status of the user’s jobs.

Returns:: The status of the user’s jobs.
Return type:: pandas.DataFrame

property queue_list: list#

Get the list of available queues.

Returns:: The list of available queues.
Return type:: list

property queue_view: DataFrame#

Get the Pandas DataFrame representation of the available queues.

Returns:: The Pandas DataFrame representation of the available queues.
Return type:: pandas.DataFrame

property queues#

Get the available queues.

Returns:: The available queues.
Return type:: Queues

property remote_flag: bool#

Get the value of remote_flag.

Returns:: The value of remote_flag.
Return type:: bool

property ssh_delete_file_on_remote: bool#

Get the value of ssh_delete_file_on_remote.

Returns:: The value of ssh_delete_file_on_remote.
Return type:: bool

Submit a job to the queue.

Parameters:

queue (str/None) – The queue to submit the job to.
job_name (str/None) – The name of the job.
working_directory (str/None) – The working directory for the job.
cores (int/None) – The number of cores required for the job.
memory_max (int/None) – The maximum memory required for the job.
run_time_max (int/None) – The maximum run time for the job.
dependency_list (list[str]/None) – List of job dependencies.
command (str) – The command to execute for the job.

Returns:

The job ID.

Return type:

int

transfer_file(file: str, transfer_back: bool = False, delete_file_on_remote: bool = False)[source]#

Transfer a file to a remote location.

Parameters:

file (str) – The path of the file to be transferred.
transfer_back (bool, optional) – Whether to transfer the file back after processing. Defaults to False.
delete_file_on_remote (bool, optional) – Whether to delete the file on the remote location after transfer. Defaults to False.

pysqa.base.config.QueueAdapterWithConfig

Contents

pysqa.base.config.QueueAdapterWithConfig#