pysqa.base.modular.ModularQueueAdapter

pysqa.base.modular.ModularQueueAdapter#

class pysqa.base.modular.ModularQueueAdapter(config: dict, directory: str = '~/.queues', execute_command: ~typing.Callable = <function execute_command>)[source]#

Bases: QueueAdapterWithConfig

A class representing a modular queue adapter.

Parameters:

config (dict) – The configuration dictionary.
directory (str, optional) – The directory path. Defaults to “~/.queues”.
execute_command (callable, optional) – The execute command function. Defaults to execute_command.

_queue_to_cluster_dict#

A dictionary mapping queues to clusters.

Type:: dict

Raises:: ValueError – If a cluster is not found in the list of clusters.

__init__(config: dict, directory: str = '~/.queues', execute_command: ~typing.Callable = <function execute_command>)[source]#

Methods

`__init__`(config[, directory, execute_command])
`check_queue_parameters`(queue[, cores, ...])	Check the parameters of a queue.
`convert_path_to_remote`(path)	Converts a local file path to a remote file path.
`delete_job`(process_id)	Delete a job.
`enable_reservation`(process_id)	Enable reservation for a process.
`get_job_from_remote`(working_directory)	Get the results of the calculation - this is necessary when the calculation was executed on a remote host.
`get_queue_status`([user])	Get the queue status.
`get_status_of_job`(process_id)	Get the status of a job.
`get_status_of_jobs`(process_id_lst)	Get the status of multiple jobs.
`get_status_of_my_jobs`()	Get the status of the user's jobs.
`submit_job`([queue, job_name, ...])	Submit a job to the queue.
`transfer_file`(file[, transfer_back, ...])	Transfer a file to a remote location.

Attributes

`config`	Get the QueueAdapter configuration.
`queue_list`	Get the list of available queues.
`queue_view`	Get the Pandas DataFrame representation of the available queues.
`queues`	Get the available queues.
`remote_flag`	Get the value of remote_flag.
`ssh_delete_file_on_remote`	Get the value of ssh_delete_file_on_remote.

Check the parameters of a queue.

Parameters:

queue (str) – The queue to check.
cores (int, optional) – The number of cores. Defaults to 1.
run_time_max (int, optional) – The maximum run time. Defaults to None.
memory_max (int, optional) – The maximum memory. Defaults to None.
active_queue (dict, optional) – The active queue. Defaults to None.

Returns:

A list of queue parameters [cores, run_time_max, memory_max].

Return type:

list

property config: dict#

Get the QueueAdapter configuration.

Returns:: The QueueAdapter configuration.
Return type:: dict

convert_path_to_remote(path: str)#

Converts a local file path to a remote file path.

Parameters:: path (str) – The local file path to be converted.
Returns:: The converted remote file path.
Return type:: str

delete_job(process_id: int)[source]#

Delete a job.

Parameters:: process_id (int) – The process ID.
Returns:: The output of the delete job command.
Return type:: str

enable_reservation(process_id: int)[source]#

Enable reservation for a process.

Parameters:: process_id (int) – The process ID.
Returns:: The output of the enable reservation command.
Return type:: str

get_job_from_remote(working_directory: str) → None#

Get the results of the calculation - this is necessary when the calculation was executed on a remote host.

Parameters:: working_directory (str) – The working directory where the calculation was executed.

get_queue_status(user: str | None = None) → DataFrame | None[source]#

Get the queue status.

Parameters:: user (str, optional) – The user name. Defaults to None.
Returns:: The queue status.
Return type:: pandas.DataFrame

get_status_of_job(process_id: int) → str | None#

Get the status of a job.

Parameters:: process_id (int) – The process ID.
Returns:: The status of the job.results_lst.append(df_selected.values[0])
Return type:: str

get_status_of_jobs(process_id_lst: list[int]) → list[str]#

Get the status of multiple jobs.

Parameters:: process_id_lst (list[int]) – List of process IDs.
Returns:: List of job statuses.
Return type:: list[str]

get_status_of_my_jobs() → DataFrame#

Get the status of the user’s jobs.

Returns:: The status of the user’s jobs.
Return type:: pandas.DataFrame

property queue_list: list#

Get the list of available queues.

Returns:: The list of available queues.
Return type:: list

property queue_view: DataFrame#

Get the Pandas DataFrame representation of the available queues.

Returns:: The Pandas DataFrame representation of the available queues.
Return type:: pandas.DataFrame

property queues#

Get the available queues.

Returns:: The available queues.
Return type:: Queues

property remote_flag: bool#

Get the value of remote_flag.

Returns:: The value of remote_flag.
Return type:: bool

property ssh_delete_file_on_remote: bool#

Get the value of ssh_delete_file_on_remote.

Returns:: The value of ssh_delete_file_on_remote.
Return type:: bool

Submit a job to the queue.

Parameters:

queue (str, optional) – The queue name. Defaults to None.
job_name (str, optional) – The job name. Defaults to None.
working_directory (str, optional) – The working directory. Defaults to None.
cores (int, optional) – The number of cores. Defaults to None.
memory_max (int, optional) – The maximum memory. Defaults to None.
run_time_max (int, optional) – The maximum run time. Defaults to None.
dependency_list (list[int], optional) – The list of dependencies. Defaults to None.
command (str, optional) – The command to execute. Defaults to None.

Returns:

The cluster queue ID.

Return type:

int

transfer_file(file: str, transfer_back: bool = False, delete_file_on_remote: bool = False)#

Transfer a file to a remote location.

Parameters:

file (str) – The path of the file to be transferred.
transfer_back (bool, optional) – Whether to transfer the file back after processing. Defaults to False.
delete_file_on_remote (bool, optional) – Whether to delete the file on the remote location after transfer. Defaults to False.

pysqa.base.modular.ModularQueueAdapter

Contents

pysqa.base.modular.ModularQueueAdapter#