script_tools

library of functions helpful for cli script development and parallel computing particulary with subprocess calls.

clasp.script_tools.try_mkdir(s)[source]

silently ignore exceptions on mkdir

clasp.script_tools.arange(start, stop=None, step=1)[source]

like numpy.arange for integers

clasp.script_tools.int_rng(s)[source]

expand start:end:inc notation into range

clasp.script_tools.rm_dup(seq)[source]

removes duplicates from list while preserving order

clasp.script_tools.warn_match(kwargs, sargs)[source]
clasp.script_tools.kwarg_match(func, kwargs, debug=False)[source]

filters dict for keys used by func

clasp.script_tools.arg_match(func, kwargs, *args)[source]

filters dict for positional arguments used by func

clasp.script_tools.kwarg_arg(func, kwargs, skip=None)[source]

returns ordered list of optional arg values

clasp.script_tools.crossref(l1, l2)[source]

return all possible pairs of 2 lists

clasp.script_tools.crossref_all(l, followers=[])[source]

return all possible combos of list of lists

clasp.script_tools.subpipe(commands)[source]

parses special syntax in pipe expressions

$(some command) executes to a temporary file whose path is inserted in
the command.
$((expression)) evaluates a arithmetic expression in place +-*/()
clasp.script_tools.pipeline(commands, outfile=None, inp=None, close=False, cwd=None, writemode='w', forceinpfile=False, caperr=False)[source]

executes pipeline of shell commands (given as list of strings)

special syntax:

$(some command) executes to a temporary file whose path is inserted in
the command.
$((expression)) evaluates a arithmetic expression in place +-*/()
Parameters
  • commands (list) – list of commands to execute in order

  • outfile (writeable file object) – optional destination for stdout

  • inp (str or filebuffer) – string to feed to stdin at start of pipeline

  • close (bool) – if true closes file object before returning

  • cwd (str) – directory to execute pipeline (temp files and Popen cwd)

  • writemode (str) – passed to open() for outfile (‘w’, ‘wb’ for write or ‘a’ for append)

  • forceinpfile (bool) – always treat inp as a file, if a string, open the path for reading

Returns

out – returns stdout of pipeline (will be None if outfile is given)

Return type

str

clasp.script_tools.flat_list(l)[source]

flattens any depth list

clasp.script_tools.pool_call(func, args, kwargs={}, cwd=None, order=True, expand=False, handle=False, test=False)[source]

execute func with concurrent.futures return output

Parameters
  • func (python function) – function to execute

  • args (list of tuples) – each set is mapped to function

  • kwargs (dict) – constant keyword args for func

  • cwd (str) – directory in which to execute function calls

  • order (bool) – whether to maintain order of input

  • expand (bool) – whether to expand items in args to map to function args

  • handle (bool) – whether to return future objects or results

Returns

Return type

list of results unless handle=True then returns iterable of futures

clasp.script_tools.cluster_call(func, args, kwargs={}, timeout=0.1, cwd=None, debug=False)[source]

for backwards compatibility only

clasp.script_tools.read_epw(epw)[source]

read daylight sky data from epw or wea file

Returns

out – (month, day, hour, dirnorn, difhoriz, globhoriz, skycover)

Return type

tuple

clasp.script_tools.isnum(s)[source]

test if input can be converted to float

clasp.script_tools.try_float(s)[source]

attempt conversion to float

clasp.script_tools.coerce_data(datastr, i_vals, dataf, coerce=True)[source]

ensure all data points parsed are valid numbers

clasp.script_tools.get_i(i, d_vals)[source]

if (x, y) return y if x == i if y return y

clasp.script_tools.read_data_file(dataf, header=False, xheader=False, comment='#', delim='\t, ', coerce=True)[source]
clasp.script_tools.read_data(dataf, x_vals=[0], y_vals=[- 1], rows=False, header=False, weax=None, reverse=False, autox=None, comment='#', xheader=False, delim='\t, ', coerce=True, weatherfile=False, drange=None)[source]

read generic csv/tsv data file

Parameters
  • dataf (str) – file to read data from

  • x_vals (list of ints) – column (or row with rows=True) indices for x values

  • y_vals (list of ints) – column (or row with rows=True) indices for y values

  • rows (Boolean) – if True read data in rows

  • header (Boolean) – return first row (or column with rows=True) as series labels

  • weax (2 item list of ints) – idx for month and day to use day number as x_vals, if given ignores x_val

  • reverse (Boolean) – reverse order of data (use with autox)

  • autox (Boolean) – assigns integers (starting at 0) as x_vals

  • comment (str) – comment line signifiers (inserted in regex ^[comment].*)

  • delim (str) – delimeters for parsing data (inserted in regex [delim]+)

  • coerce (Boolean) – raise exception if all values are are not numbers

  • weatherfile (str of file path) – handles wea and epw file formates returning daylight parameters

  • drange (list of ints) – limit series output to given indices.

Returns

  • datax (list) – list of x_vals for each y_val (pads with last item if necessary) if there are more x_vals than y_vals does not return excess datax

  • datay (list) – list for each y_val

  • head (list) – if header=True list of labels for each y_val else []

clasp.script_tools.read_all_data(datafs, x_vals=[], y_vals=[], **kwargs)[source]

read multiple data files and pair x and y data call read_data

Parameters
  • datafs (list of str) – files to read data from

  • x_vals (list of ints or tuple int pairs) – (fileidx, colidx) or colidx to read from each file

  • y_vals (list of ints or tuple int pairs) – (fileidx, colidx) or colidx to read from each file

  • kwargs – optional arguments for read_data

Returns

  • datax (list) – list of x_vals for each y_val (pads with last item if necessary)

  • datay (list) – list for each y_val

  • head (list) – if header=True list of labels for each y_val else []

clasp.script_tools.clean_tmp(ctx)[source]
clasp.script_tools.expandpat(pat, s, mark=0)[source]

expand sglob pattern for each character option

Parameters
  • pat (regex) – regex pattern to split on

  • s (str) – string to split

  • mark (int) – 0: include splitting mark in output 1: skip splitting mark (assume 1 character in length)

Returns

allpat – list of strings enumerating all possible combinations of pattern

Return type

list of strings

clasp.script_tools.sglob(s)[source]

super glob includes [abc] notation + [!abc] exclude notation