Top-Level | Table | Verbs | Op Functions | Expressions | Extensibility |
Add new functions for use in table expressions.
# aq.addFunction([name,] fn[, options]) · Source
Register a function for use within table expressions. If only a single argument is provided, it will be assumed to be a function and the system will try to extract its name. Throws an error if a function with the same name is already registered and the override option is not specified, or if no name is provided and the input function is anonymous. After registration, the function will be accessible via the op
object.
Also see the escape()
expression helper for a lightweight alternative that allows access to functions defined in an enclosing scope.
false
) indicating if the added function is allowed to override an existing function with the same name.Examples
// add a function named square, which is then available as op.square()
// if a 'square' function already exists, this results in an error
aq.addFunction('square', x => x * x);
// add a function named square, override any previous 'square' definition
aq.addFunction('square', x => x * x, { override: true });
// add a function using its existing name
aq.addFunction(function square(x) { return x * x; });
# aq.addAggregateFunction(name, def[, options]) · Source
Register a custom aggregate function. Throws an error if a function with the same name is already registered and the override option is not specified. After registration, the operator will be accessible via the op
object.
In addition to column values, internally aggregate functions are passed a state
object that tracks intermediate values throughout the aggregation. Each groupby group receives a different state object. The state object always has a count
property (total number of values, including invalid values) and a valid
property (number of values that are not null
, undefined
, or NaN
). Each aggregate operator may write intermediate values to the state
object. Follow the property naming convention of using the aggregate function name as a property name prefix to avoid namespace collisions! For example, the mean
aggregate function writes to the properties state.mean
and state.mean_d
.
The rem
function of an aggregate definition is used to support rolling window calculations. It is safe to define rem
as a no-op (() => {}
) if the aggregate is never used in the context of a rolling window frame.
false
) indicating if the added function is allowed to override an existing function with the same name.Examples
// add an aggregate function that computes a sum of squares
// the "rem" method is only needed for windowed aggregates
aq.addAggregateFunction('sumsq', {
create: () => ({
init: state => state.sumsq = 0,
add: (state, value) => state.sumsq += value * value,
rem: (state, value) => state.sumsq -= value * value,
value: state => state.sumsq
}),
param: [1, 0] // 1 field input, 0 extra parameters
});
aq.table({ x: [1, 2, 3] })
.rollup({ ssq: op.sumsq('x') } // { ssq: [14] }
# aq.addWindowFunction(name, def[, options]) · Source
Register a custom window function. Throws an error if a function with the same name is already registered and the override option is not specified. After registration, the operator will be accessible via the op
object.
false
) indicating if the added function is allowed to override an existing function with the same name.Examples
// add a window function that outputs the minimum of a field value
// and the current sorted window index, plus an optional offset
aq.addWindowFunction('idxmin', {
create: (offset = 0) => ({
init: () => {}, // do nothing
value: (w, f) => Math.min(w.value(w.index, f), w.index) + offset
}),
param: [1, 1] // 1 field input, 1 extra parameter
});
aq.table({ x: [4, 3, 2, 1] })
.derive({ x: op.idxmin('x', 1) }) // { x: [1, 2, 3, 2] }
Add new table-level methods or verbs. The addTableMethod function registers a new function as an instance method of tables only. The addVerb method registers a new transformation verb with both tables and serializable queries.
# aq.addTableMethod(name, method[, options]) · Source
Register a custom table method, adding a new method with the given name to all table instances. The provided method must take a table as its first argument, followed by any additional arguments.
This method throws an error if the name argument is not a legal string value.
To protect Arquero internals, the name can not start with an underscore (_
) character. If a custom method with the same name is already registered, the override option must be specified to overwrite it. In no case may a built-in method be overridden.
false
) indicating if the added method is allowed to override an existing method with the same name. Built-in table methods can not be overridden; this flag applies only to methods previously added using the extensibility API.Examples
// add a table method named size, returning an array of row and column counts
aq.addTableMethod('size', table => [table.numRows(), table.numCols()]);
aq.table({ a: [1,2,3], b: [4,5,6] }).size() // [3, 2]
# aq.addVerb(name, method, params[, options]) · Source
Register a custom transformation verb with the given name, adding both a table method and serializable query support. The provided method must take a table as its first argument, followed by any additional arguments. The required params argument describes the parameters the verb accepts. If you wish to add a verb to tables but do not require query serialization support, use addTableMethod.
This method throws an error if the name argument is not a legal string value.
To protect Arquero internals, the name can not start with an underscore (_
) character. If a custom method with the same name is already registered, the override option must be specified to overwrite it. In no case may a built-in method be overridden.
"Options"
, the descriptor can include an additional object-valued props property to describe any non-literal values, for which the keys are property names and the values are parameter types.false
) indicating if the added method is allowed to override an existing method with the same name. Built-in verbs can not be overridden; this flag applies only to methods previously added using the extensibility API.Parameter Types. The supported parameter types are:
"Expr"
: A single table expression, such as the input to filter()
."ExprList"
: A list of column references or expressions, such as the input to groupby()
."ExprNumber"
: A number literal or numeric table expression, such as the weight option of sample()
."ExprObject"
: An object containing a set of expressions, such as the input to rollup()
."JoinKeys"
: Input join keys, as in join()
."JoinValues"
: Output join values, as in join()
."Options"
: An options object of key-value pairs. If any of the option values are column references or table expressions, the descriptor should include a props property with property names as keys and parameter types as values."OrderKeys"
: A list of ordering criteria, as in orderby
."SelectionList"
: A set of columns to select and potentially rename, as in select
."TableRef"
: A reference to an additional input table, as in join()
."TableRefList"
: A list of one or more additional input tables, as in concat()
.Examples
// add a bootstrapped confidence interval verb that
// accepts an aggregate expression plus options
aq.addVerb(
'bootstrap_ci',
(table, expr, options = {}) => table
.params({ frac: options.frac || 1000 })
.sample((d, $) => op.round($.frac * op.count()), { replace: true })
.derive({ id: (d, $) => op.row_number() % $.frac })
.groupby('id')
.rollup({ bs: expr })
.rollup({
lo: op.quantile('bs', options.lo || 0.025),
hi: op.quantile('bs', options.hi || 0.975)
}),
[
{ name: 'expr', type: 'Expr' },
{ name: 'options', type: 'Options' }
]
);
// apply the new verb
aq.table({ x: [1, 2, 3, 4, 6, 8, 9, 10] })
.bootstrap_ci(op.mean('x'))
Extend Arquero with a bundle of functions, table methods, and/or verbs.
# aq.addPackage(bundle[, options]) · Source
Register a bundle of extensions, which may include standard functions, aggregate functions, window functions, table methods, and verbs. If the input bundle has a key named "arquero_package"
, the value of that property is used; otherwise the bundle object is used directly. This method is particularly useful for publishing separate packages of Arquero extensions and then installing them with a single method call.
A package bundle has the following structure:
const bundle = {
functions: { ... },
aggregateFunctions: { ... },
windowFunctions: { ... },
tableMethods: { ... },
verbs: { ... }
};
All keys are optional. For example, functions
or verbs
may be omitted. Each sub-bundle is an object of key-value pairs, where the key is the name of the function and the value is the function to add.
The lone exception is the verbs
bundle, which instead uses an object format with method and params keys, corresponding to the method and params arguments of addVerb:
const bundle = {
verbs: {
name: {
method: (table, expr) => { ... },
params: [ { name: 'expr': type: 'Expr' } ]
}
}
};
The package method performs validation prior to adding any package content. The method will throw an error if any of the package items fail validation. See the addFunction, addAggregateFunction, addWindowFunction, addTableMethod, and addVerb methods for specific validation criteria. The options argument can be used to specify if method overriding is permitted, as supported by each of the aforementioned methods.
false
) indicating if the added method is allowed to override an existing method with the same name. Built-in table methods or verbs can not be overridden; for table methods and verbs this flag applies only to methods previously added using the extensibility API.Examples
// add a package
aq.addPackage({
functions: {
square: x => x * x,
},
tableMethods: {
size: table => [table.numRows(), table.numCols()]
}
});
// add a package, ignores any content outside of "arquero_package"
aq.addPackage({
arquero_package: {
functions: {
square: x => x * x,
},
tableMethods: {
size: table => [table.numRows(), table.numCols()]
}
}
});
// add a package from a separate library
aq.addPackage(require('arquero-arrow'));