Package mdp :: Package parallel :: Class Scheduler

Class Scheduler

Base class and trivial implementation for schedulers.

New tasks are added with add_task(data, callable).
get_results then returns the results (and locks if tasks are
pending).

In this simple scheduler implementation the tasks are simply executed in the
add_task method.

Instance Methods

[hide private]

__enter__(self)
Return self.

__exit__(self, type, value, traceback)
Shutdown the scheduler.

__init__(self, result_container=None, verbose=False)
Initialize the scheduler.

_process_task(self, data, task_callable, task_index)
Process the task and store the result.

_shutdown(self)
Hook method for shutdown to be used in custom schedulers.

_store_result(self, result, task_index)
Store a result in the internal result container.

add_task(self, data, task_callable=None)
Add a task to be executed.

get_results(self)
Get the accumulated results from the result container.

set_task_callable(self, task_callable)
Set the callable that will be used if no task_callable is given.

shutdown(self)
Controlled shutdown of the scheduler.

Inherited from unreachable.newobject: __long__, __native__, __nonzero__, __unicode__, next

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Properties

[hide private]

n_open_tasks
This property counts of submitted but unfinished tasks.

task_counter
This property counts the number of submitted tasks.

Inherited from object: __class__

Method Details

[hide private]

enter(self)

Return self.

exit(self, type, value, traceback)

Shutdown the scheduler.

It is important that all the calculations have finished
when this is called, otherwise the shutdown might fail.

init(self, result_container=None, verbose=False)
(Constructor)

Initialize the scheduler.

result_container -- Instance of ResultContainer that is used to store
    the results (default is None, in which case a ListResultContainer
    is used).
verbose -- If True then status messages will be printed to sys.stdout.

Overrides: object.__init__

_process_task(self, data, task_callable, task_index)

Process the task and store the result.

You can override this method for custom schedulers.

Warning: When this method is entered is has the lock, the lock must be
released here.

Warning: Note that fork has not been called yet, so the provided
task_callable must not be called. Only a forked version can be called.

_shutdown(self)

Hook method for shutdown to be used in custom schedulers.

_store_result(self, result, task_index)

Store a result in the internal result container.

result -- Result data
task_index -- Task index. Can be None if an error occured.

This function blocks to avoid any problems during result storage.

add_task(self, data, task_callable=None)

Add a task to be executed.

data -- Data for the task.
task_callable -- A callable, which is called with the data. If it is
    None (default value) then the last provided callable is used.
    If task_callable is not an instance of TaskCallable then a
    TaskCallableWrapper is used.

The callable together with the data constitutes the task. This method
blocks if there are no free recources to store or process the task
(e.g. if no free worker processes are available).

get_results(self)

Get the accumulated results from the result container.

This method blocks if there are open tasks.

set_task_callable(self, task_callable)

Set the callable that will be used if no task_callable is given.

Normally the callables are provided via add_task, in which case there
is no need for this method.

task_callable -- Callable that will be used unless a new task_callable
    is given.

shutdown(self)

Controlled shutdown of the scheduler.

This method should always be called when the scheduler is no longer
needed and before the program shuts down! Otherwise one might get
error messages.

Property Details

[hide private]

n_open_tasks

This property counts of submitted but unfinished tasks.

Get Method:: unreachable.n_open_tasks(self) - This property counts of submitted but unfinished tasks.

task_counter

This property counts the number of submitted tasks.

Get Method:: unreachable.task_counter(self) - This property counts the number of submitted tasks.

Class Scheduler

__enter__(self)

__exit__(self, type, value, traceback)

__init__(self, result_container=None, verbose=False) (Constructor)

_process_task(self, data, task_callable, task_index)

_shutdown(self)

_store_result(self, result, task_index)

add_task(self, data, task_callable=None)

get_results(self)

set_task_callable(self, task_callable)

shutdown(self)

n_open_tasks

task_counter

enter(self)

exit(self, type, value, traceback)

init(self, result_container=None, verbose=False)
(Constructor)