script_tools¶
library of functions helpful for cli script development and parallel computing particulary with subprocess calls.
-
clasp.script_tools.
kwarg_match
(func, kwargs, debug=False)[source]¶ filters dict for keys used by func
-
clasp.script_tools.
arg_match
(func, kwargs, *args)[source]¶ filters dict for positional arguments used by func
-
clasp.script_tools.
kwarg_arg
(func, kwargs, skip=None)[source]¶ returns ordered list of optional arg values
-
clasp.script_tools.
crossref_all
(l, followers=[])[source]¶ return all possible combos of list of lists
-
clasp.script_tools.
subpipe
(commands)[source]¶ parses special syntax in pipe expressions
$(some command) executes to a temporary file whose path is inserted inthe command.$((expression)) evaluates a arithmetic expression in place +-*/()
-
clasp.script_tools.
pipeline
(commands, outfile=None, inp=None, close=False, cwd=None, writemode='w', forceinpfile=False, caperr=False)[source]¶ executes pipeline of shell commands (given as list of strings)
special syntax:
$(some command) executes to a temporary file whose path is inserted inthe command.$((expression)) evaluates a arithmetic expression in place +-*/()- Parameters
commands (list) – list of commands to execute in order
outfile (writeable file object) – optional destination for stdout
inp (str or filebuffer) – string to feed to stdin at start of pipeline
close (bool) – if true closes file object before returning
cwd (str) – directory to execute pipeline (temp files and Popen cwd)
writemode (str) – passed to open() for outfile (‘w’, ‘wb’ for write or ‘a’ for append)
forceinpfile (bool) – always treat inp as a file, if a string, open the path for reading
- Returns
out – returns stdout of pipeline (will be None if outfile is given)
- Return type
str
-
clasp.script_tools.
pool_call
(func, args, kwargs={}, cwd=None, order=True, expand=False, handle=False, test=False)[source]¶ execute func with concurrent.futures return output
- Parameters
func (python function) – function to execute
args (list of tuples) – each set is mapped to function
kwargs (dict) – constant keyword args for func
cwd (str) – directory in which to execute function calls
order (bool) – whether to maintain order of input
expand (bool) – whether to expand items in args to map to function args
handle (bool) – whether to return future objects or results
- Returns
- Return type
list of results unless handle=True then returns iterable of futures
-
clasp.script_tools.
cluster_call
(func, args, kwargs={}, timeout=0.1, cwd=None, debug=False)[source]¶ for backwards compatibility only
-
clasp.script_tools.
read_epw
(epw)[source]¶ read daylight sky data from epw or wea file
- Returns
out – (month, day, hour, dirnorn, difhoriz, globhoriz, skycover)
- Return type
tuple
-
clasp.script_tools.
coerce_data
(datastr, i_vals, dataf, coerce=True)[source]¶ ensure all data points parsed are valid numbers
-
clasp.script_tools.
read_data_file
(dataf, header=False, xheader=False, comment='#', delim='\t, ', coerce=True)[source]¶
-
clasp.script_tools.
read_data
(dataf, x_vals=[0], y_vals=[- 1], rows=False, header=False, weax=None, reverse=False, autox=None, comment='#', xheader=False, delim='\t, ', coerce=True, weatherfile=False, drange=None)[source]¶ read generic csv/tsv data file
- Parameters
dataf (str) – file to read data from
x_vals (list of ints) – column (or row with rows=True) indices for x values
y_vals (list of ints) – column (or row with rows=True) indices for y values
rows (Boolean) – if True read data in rows
header (Boolean) – return first row (or column with rows=True) as series labels
weax (2 item list of ints) – idx for month and day to use day number as x_vals, if given ignores x_val
reverse (Boolean) – reverse order of data (use with autox)
autox (Boolean) – assigns integers (starting at 0) as x_vals
comment (str) – comment line signifiers (inserted in regex ^[comment].*)
delim (str) – delimeters for parsing data (inserted in regex [delim]+)
coerce (Boolean) – raise exception if all values are are not numbers
weatherfile (str of file path) – handles wea and epw file formates returning daylight parameters
drange (list of ints) – limit series output to given indices.
- Returns
datax (list) – list of x_vals for each y_val (pads with last item if necessary) if there are more x_vals than y_vals does not return excess datax
datay (list) – list for each y_val
head (list) – if header=True list of labels for each y_val else []
-
clasp.script_tools.
read_all_data
(datafs, x_vals=[], y_vals=[], **kwargs)[source]¶ read multiple data files and pair x and y data call read_data
- Parameters
datafs (list of str) – files to read data from
x_vals (list of ints or tuple int pairs) – (fileidx, colidx) or colidx to read from each file
y_vals (list of ints or tuple int pairs) – (fileidx, colidx) or colidx to read from each file
kwargs – optional arguments for read_data
- Returns
datax (list) – list of x_vals for each y_val (pads with last item if necessary)
datay (list) – list for each y_val
head (list) – if header=True list of labels for each y_val else []
-
clasp.script_tools.
expandpat
(pat, s, mark=0)[source]¶ expand sglob pattern for each character option
- Parameters
pat (regex) – regex pattern to split on
s (str) – string to split
mark (int) – 0: include splitting mark in output 1: skip splitting mark (assume 1 character in length)
- Returns
allpat – list of strings enumerating all possible combinations of pattern
- Return type
list of strings