pwtools.batch.ParameterStudy

class pwtools.batch.ParameterStudy(machines=[], templates=[], params_lst=[], study_name='calc', db_name='calc.db', db_table='calc', calc_root='.', calc_dir_prefix='calc')[source]

Bases: object

Class to represent a parameter study (a number of Calculations, based on template files).

The basic idea is to assemble all to-be-varied parameters in a script outside (params_lst) and pass these to this class along with a list of templates files, usually software input files. Then, a simple loop over the parameter sets is done and input files are written.

Calculation dirs are numbered automatically. The default is

calc_dir = <calc_root>/<calc_dir_prefix>_<machine.hostname>, e.g. ./calc_foo

and each calculation for each parameter set

./calc_foo/0
./calc_foo/1
./calc_foo/2
...

A sqlite database calc_dir_root/<db_name> is written. If this class operates on a calc_dir_root where such a database already exists, then the default is to append new calculations. The numbering of calc dirs continues at the end. This can be changed with the mode kwarg of write_input(). By default, a sql column “revision” is added which numbers each addition.

Examples

>>> # Here are some examples for constructing `params_lst`.
>>>
>>> # Vary two (three, ...) parameters on a 2d (3d, ...) grid: In fact, the
>>> # way you are constructing params_lst is only a matter of zip() and
>>> # comb.nested_loops()
>>> par1 = sql.sql_column('par1', [1,2,3])
>>> par2 = sql.sql_column('par2', ['a','b'])
>>> par3 = ...
>>> # 3d grid
>>> params_lst = comb.nested_loops([par1, par2, par3])
>>> # which is the same as:
>>> params_lst = []
>>> for par1 in [1,2,3]:
...     for par2 in ['a','b']:
...         for par3 in [...]:
...             params_lst.append([sql.SQLEntry(key='par1', sqlval=par1),
...                                sql.SQLEntry(key='par2', sqlval=par2),
...                                sql.SQLEntry(key='par3', sqlval=par3),
...                                ])
>>>
>>> # vary par1 and par2 together, and par3 -> 2d grid w/ par1+par2 on one
>>> # axis and par3 on the other
>>> params_lst = comb.nested_loops([zip(par1, par2), par3], flatten=True)

See examples/parameter_study/ for complete examples, as well as test/test_parameter_study.py.

See also

itertools.product, pwtools.comb.nested_loops(), pwtools.sql.sql_column()

__init__(machines=[], templates=[], params_lst=[], study_name='calc', db_name='calc.db', db_table='calc', calc_root='.', calc_dir_prefix='calc')[source]
Parameters:

  • machines (sequence or Machine) – List of Machine instances or a single Machine

  • templates (see Calculation)

  • params_lst (list of lists) –

    The “parameter sets”. Each sublist is a set of calculation parameters as SQLEntry instances:

    [[SQLEntry(...), SQLEntry(...), ...], # calc_foo/0
 [SQLEntry(...), SQLEntry(...), ...], # calc_foo/1
 ...
]

    For each sublist, a separate calculation dir is created and populated with files based on templates. The key attribute of each SQLEntry will be converted to a placeholder in each FileTemplate and an attempt to replacement in the template files is made. Thus, the way placeholders are created is defined in FileTemplate, not here! Note: Each sublist (parameter set) is flattened, so that it can in fact be a nested list, e.g. params_lst = the result of a complex comb.nested_loops() call. Also, sublists need not have the same length or key attributes per entry (“incomplete parameter sets”). The sqlite table header is compiled from all distinct `key`s found.

  • study_name (str, optional) – Name for the parameter study. From this, the calc_name for input files and job name etc. will be built. By default, a string “_run<idx>” is appended to create a unique name.

  • db_name (str, optional) – Basename of the sqlite database.

  • db_table (str, optional) – Name of the sqlite database table.

  • calc_root (str, optional) – Root of all calc dirs.

  • calc_dir_prefix (str, optional) – Prefix for the top calculation dir (e.g. ‘calc’ for ‘calc_foo’).

Methods

write_input([mode, backup, sleep, excl])

Create calculation dir(s) for each parameter set and write input files based on templates.