eb

compression_safeguards.safeguards.stencil.qoi.eb

Stencil quantity of interest (QoI) error bound safeguard.

Classes:

StencilQuantityOfInterestErrorBoundSafeguard 

StencilQuantityOfInterestErrorBoundSafeguard(
    qoi: StencilQuantityOfInterestExpression,
    neighbourhood: Sequence[
        dict[str, JSON] | NeighbourhoodBoundaryAxis
    ],
    type: str | ErrorBound,
    eb: int | float | str | Parameter,
    qoi_dtype: str | ToFloatMode = lossless,
)

Bases: StencilSafeguard

flowchart LR compression_safeguards.safeguards.stencil.qoi.eb.StencilQuantityOfInterestErrorBoundSafeguard[StencilQuantityOfInterestErrorBoundSafeguard] compression_safeguards.safeguards.stencil.abc.StencilSafeguard[StencilSafeguard] compression_safeguards.safeguards.abc.Safeguard[Safeguard] compression_safeguards.safeguards.stencil.abc.StencilSafeguard --> compression_safeguards.safeguards.stencil.qoi.eb.StencilQuantityOfInterestErrorBoundSafeguard compression_safeguards.safeguards.abc.Safeguard --> compression_safeguards.safeguards.stencil.abc.StencilSafeguard click compression_safeguards.safeguards.stencil.qoi.eb.StencilQuantityOfInterestErrorBoundSafeguard href "" "compression_safeguards.safeguards.stencil.qoi.eb.StencilQuantityOfInterestErrorBoundSafeguard" click compression_safeguards.safeguards.stencil.abc.StencilSafeguard href "" "compression_safeguards.safeguards.stencil.abc.StencilSafeguard" click compression_safeguards.safeguards.abc.Safeguard href "" "compression_safeguards.safeguards.abc.Safeguard"

The StencilQuantityOfInterestErrorBoundSafeguard guarantees that the pointwise error type on a derived quantity of interest (QoI) over a neighbourhood of data points is less than or equal to the provided bound eb.

The quantity of interest is specified as a non-constant expression, in string form, over the neighbourhood tensor X that is centred on the pointwise value x. For example, to bound the error on the four-neighbour box mean in a 3x3 neighbourhood (where x = X[I]), set qoi="(X[I[0]-1, I[1]] + X[I[0]+1, I[1]] + X[I[0], I[1]-1] + X[I[0], I[1]+1]) / 4". Note that X can be indexed absolute or relative to the centred data point x using the index array I.

Tip: Finite Differences

The stencil QoI safeguard can also be used to bound the pointwise error of the finite-difference-approximated derivative (of arbitrary order, accuracy, and grid spacing) over the data by using the finite_difference function in the qoi expression.

Tip: Monotonic Sequences

The stencil QoI safeguard can also be used to preserve the monotonicity of a sequence of values, i.e. to guarantee that a sequence that was originally strictly/weakly monotonically increasing/decreasing/constant still is. The sequence can be arbitrary within the stencil neighbourhood, e.g. along a single axis, in a zigzag, etc. Preserving the monotonicity of multiple sequences, e.g. along several axes, requires multiple stencil QoI safeguards. For instance, to guarantee that strictly increasing/decreasing sequences along a single axis stay strictly increasing/decreasing, use the following qoi expression with an absolute error bound of zero (more monotonicity QoIs, including strict vs weak monotonicity and constant sequences, can be found in test_monotonicity.py):

all([
    # strictly decreasing sequences stay strictly decreasing
    all(X[1:] < X[:-1]) == all(C["$X"][1:] < C["$X"][:-1]),
    # strictly increasing sequences stay strictly increasing
    all(X[1:] > X[:-1]) == all(C["$X"][1:] > C["$X"][:-1]),
])

The shape of the data neighbourhood is specified as an ordered list of unique data axes and boundary conditions that are applied to these axes. If the safeguard is applied to data with an insufficient number of dimensions, it raises an exception. If the safeguard is applied to data with additional dimensions, it is indendently applied along these extra axes. For instance, a 2d QoI is applied to independently to all 2d slices in a 3d data cube.

If the data neighbourhood uses the valid boundary condition along an axis, the safeguard is only applied to data neighbourhoods centred on data points that have sufficient points before and after to satisfy the neighbourhood shape, i.e. it is not applied to all data points. If the axis is smaller than required by the neighbourhood along this axis, the safeguard is not applied. Using a different BoundaryCondition ensures that the safeguard is always applied to all data points.

If the derived quantity of interest for a data neighbourhood evaluates to an infinite value, this safeguard guarantees that the quantity of interest on the corrected data neighbourhood produces the exact same infinite value. For a NaN quantity of interest, this safeguard guarantees that the quantity of interest on the corrected data neighbourhood is also NaN, but does not guarantee that it has the same bit pattern.

The error bound can be verified by evaluating the QoI in the floating-point data type selected by qoi_dtype parameter using the evaluate_qoi method.

Please refer to the StencilQuantityOfInterestExpression for the EBNF grammar that specifies the language in which the quantities of interest are written.

The implementation was originally inspired by:

Pu Jiao, Sheng Di, Hanqi Guo, Kai Zhao, Jiannan Tian, Dingwen Tao, Xin Liang, and Franck Cappello. (2022). Toward Quantity-of-Interest Preserving Lossy Compression for Scientific Data. Proceedings of the VLDB Endowment. 16, 4 (December 2022), 697-710. Available from: doi:10.14778/3574245.3574255.

Parameters:
  • qoi (StencilExpr) –

    The non-constant expression for computing the derived quantity of interest over a neighbourhood tensor X.

  • neighbourhood (Sequence[dict[str, JSON] | NeighbourhoodBoundaryAxis]) –

    The non-empty axes of the data neighbourhood for which the quantity of interest is computed. The neighbourhood window is applied independently over any additional axes in the data.

    The per-axis boundary conditions are applied to the data in their order in the neighbourhood, i.e. earlier boundary extensions can influence later ones.

  • type (str | ErrorBound) –

    The type of error bound on the quantity of interest that is enforced by this safeguard.

  • eb (int | float | str | Parameter) –

    The value of or late-bound parameter name for the error bound on the quantity of interest that is enforced by this safeguard.

    The error bound is applied relative to the values of the quantity of interest evaluated on the original data.

    If eb is a late-bound parameter, its late-bound value must be broadcastable to the shape of the data to be safeguarded, not the shape of the QoI that has been evaluated (even though it is only applied to the QoI). The expand_qoi_to_data_shape method can be used to expand an error-bound from the QoI shape to the data shape.

  • qoi_dtype (str | ToFloatMode, default: lossless ) –

    The floating-point data type in which the quantity of interest is evaluated. By default, the smallest floating-point data type that can losslessly represent all input data values is chosen.
Raises:
  • TypeCheckError –

    if any parameter has the wrong type.

  • SyntaxError –

    if the qoi is not a valid stencil quantity of interest expression.

  • ValueError –

    if the neighbourhood is empty.

  • ValueError –

    if any neighbourhood.axis is not unique.

  • ValueError –

    if type does not name a valid error bound, or the qoi_dtype does not name a valid floating-point data type.

  • ValueError –

    if eb is an invalid error bound value for the error bound type.

  • ... –

    if instantiating a neighbourhood boundary axis raises an exception.

Methods:

kind class-attribute 

kind: str = 'qoi_eb_stencil'

late_bound property 

late_bound: Set[Parameter]

The set of late-bound parameters that this safeguard has.

Late-bound parameters are only bound when checking and applying the safeguard, in contrast to the normal early-bound parameters that are configured during safeguard initialisation.

Late-bound parameters can be used for parameters that depend on the specific data that is to be safeguarded.

compute_check_neighbourhood_for_data_shape 

compute_check_neighbourhood_for_data_shape(
    data_shape: tuple[int, ...],
) -> tuple[dict[BoundaryCondition, NeighbourhoodAxis], ...]

Compute the shape of the data neighbourhood for data of a given shape. Boundary conditions of the same kind are combined, but separate kinds are tracked separately.

An empty dict is returned along dimensions for which the stencil QoI safeguard does not need to look at adjacent data points.

This method also checks that the data shape is compatible with the stencil QoI safeguard.

Parameters:
  • data_shape (tuple[int, ...]) –

    The shape of the data.
Returns:
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in data_shape.

  • IndexError –

    if any neighbourhood axis is duplicate.

evaluate_qoi 

evaluate_qoi(
    data: ndarray[S, dtype[T]], late_bound: Bindings
) -> ndarray[tuple[int, ...], dtype[F]]

Evaluate the derived quantity of interest on the data in the floating-point data type selected by the qoi_dtype parameter.

The quantity of interest may have a different shape if the valid boundary condition is used along any axis.

Parameters:
  • data (ndarray[S, dtype[T]]) –

    Data for which the quantity of interest is evaluated.

  • late_bound (Bindings) –

    Bindings for late-bound constants in the quantity of interest.
Returns:
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in data.

  • IndexError –

    if any neighbourhood axis is duplicate.

  • TypeError –

    if the data could not be losslessly cast to qoi_dtype.

  • LateBoundParameterResolutionError –

    if any neighbourhood axis.constant_boundary is late-bound but its late-bound parameter is not in late_bound.

  • ValueError –

    if any neighbourhood axis.constant_boundary is late-bound but not a scalar.

  • TypeError –

    if any neighbourhood axis.constant_boundary is floating-point but the data is integer.

  • ValueError –

    if any neighbourhood axis.constant_boundary could not be losslessly converted to the data's type.

  • LateBoundParameterResolutionError –

    if any of the qoi's late-bound constants is not contained in the bindings.

  • ValueError –

    if any late-bound constant could not be broadcast to the data's shape.

  • TypeError –

    if any late-bound constant is floating-point but the data is integer.

  • ValueError –

    if not all values for all late-bound constants could be losslessly converted to the data's type.
Referenced by:

check_pointwise 

check_pointwise(
    data: ndarray[S, dtype[T]],
    approximation: ndarray[S, dtype[T]],
    *,
    late_bound: Bindings,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> ndarray[S, dtype[bool]]

Check which elements in the approximation array satisfy the error bound for the quantity of interest over a neighbourhood on the data.

Parameters:
  • data (ndarray[S, dtype[T]]) –

    Original data array, relative to which the approximation is checked.

  • approximation (ndarray[S, dtype[T]]) –

    Approximation of the data array.

  • late_bound (Bindings) –

    Bindings for late-bound parameters, including for this safeguard.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only check at data points where the condition is True.
Returns:
  • ok( ndarray[S, dtype[bool]] ) –

    Pointwise, True if the check succeeded for this element.
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in data.

  • IndexError –

    if any neighbourhood axis is duplicate.

  • TypeError –

    if the data could not be losslessly cast to qoi_dtype.

  • LateBoundParameterResolutionError –

    if any neighbourhood axis.constant_boundary is late-bound but its late-bound parameter is not in late_bound.

  • ValueError –

    if any neighbourhood axis.constant_boundary is late-bound but not a scalar.

  • TypeError –

    if any neighbourhood axis.constant_boundary is floating-point but the data is integer.

  • ValueError –

    if any neighbourhood axis.constant_boundary could not be losslessly converted to the data's type.

  • LateBoundParameterResolutionError –

    if any of the qoi's late-bound constants is not contained in the bindings.

  • ValueError –

    if any late-bound constant could not be broadcast to the data's shape.

  • TypeError –

    if any late-bound constant is floating-point but the data is integer.

  • ValueError –

    if not all values for all late-bound constants could be losslessly converted to the data's type.

  • LateBoundParameterResolutionError –

    if the error bound eb is late-bound but its late-bound parameter is not in late_bound.

  • ValueError –

    if the late-bound eb could not be broadcast to the data's shape.

  • ValueError –

    if the late-bound eb is non-finite, i.e. infinite or NaN, or an invalid error bound value for the error bound type.

compute_safe_intervals 

compute_safe_intervals(
    data: ndarray[S, dtype[T]],
    *,
    late_bound: Bindings,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> IntervalUnion[T, int, int]

Compute the intervals in which the error bound is upheld with respect to the quantity of interest over a neighbourhood on the data.

Parameters:
  • data (ndarray[S, dtype[T]]) –

    Data for which the safe intervals should be computed.

  • late_bound (Bindings) –

    Bindings for late-bound parameters, including for this safeguard.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only compute the safe intervals at pointwise checks where the condition is True.
Returns:
  • intervals( IntervalUnion[T, int, int] ) –

    Union of intervals in which the error bound is upheld.
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in data.

  • IndexError –

    if any neighbourhood axis is duplicate.

  • TypeError –

    if the data could not be losslessly cast to qoi_dtype.

  • LateBoundParameterResolutionError –

    if any neighbourhood axis.constant_boundary is late-bound but its late-bound parameter is not in late_bound.

  • ValueError –

    if any neighbourhood axis.constant_boundary is late-bound but not a scalar.

  • TypeError –

    if any neighbourhood axis.constant_boundary is floating-point but the data is integer.

  • ValueError –

    if any neighbourhood axis.constant_boundary could not be losslessly converted to the data's type.

  • LateBoundParameterResolutionError –

    if any of the qoi's late-bound constants is not contained in the bindings.

  • ValueError –

    if any late-bound constant could not be broadcast to the data's shape.

  • TypeError –

    if any late-bound constant is floating-point but the data is integer.

  • ValueError –

    if not all values for all late-bound constants could be losslessly converted to the data's type.

  • LateBoundParameterResolutionError –

    if the error bound eb is late-bound but its late-bound parameter is not in late_bound.

  • ValueError –

    if the late-bound eb could not be broadcast to the data's shape.

  • ValueError –

    if the late-bound eb is non-finite, i.e. infinite or NaN, or an invalid error bound value for the error bound type.

compute_footprint 

compute_footprint(
    foot: ndarray[S, dtype[bool]],
    *,
    late_bound: Bindings,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> ndarray[S, dtype[bool]]

Compute the footprint of the foot array, e.g. for expanding data points into the pointwise checks that they contribute to.

The footprint usually extends beyond foot & where.

Parameters:
  • foot (ndarray[S, dtype[bool]]) –

    Array for which the footprint is computed.

  • late_bound (Bindings) –

    Bindings for late-bound parameters, including for this safeguard.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only compute the footprint at pointwise checks where the condition is True.

    Conceptually, where is applied to footprint at the end.
Returns:

compute_inverse_footprint 

compute_inverse_footprint(
    foot: ndarray[S, dtype[bool]],
    *,
    late_bound: Bindings,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> ndarray[S, dtype[bool]]

Compute the inverse footprint of the foot array, e.g. for expanding pointwise check fails into the points that could have contributed to the failures.

The inverse footprint usually extends beyond foot & where.

Parameters:
  • foot (ndarray[S, dtype[bool]]) –

    Array for which the inverse footprint is computed.

  • late_bound (Bindings) –

    Bindings for late-bound parameters, including for this safeguard.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only compute the inverse footprint at pointwise checks where the condition is True.

    Conceptually, where is applied to foot at the start.
Returns:

truncate_data_to_qoi_shape 

truncate_data_to_qoi_shape(
    data: ndarray[tuple[int, ...], dtype[T]],
) -> ndarray[tuple[int, ...], dtype[T]]

Truncate the data array to the shape of the qoi array that would be evaluated from it.

For neighbourhoods without any valid boundaries, this method simply copies the data array. Along axes with a valid boundary, the data is truncated before and after by as many elements as the boundary cuts off.

This method is the lossy inverse of expand_qoi_to_data_shape.

Parameters:
Returns:
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in data.

  • IndexError –

    if any neighbourhood axis is duplicate.
Referenced by:

expand_qoi_to_data_shape 

expand_qoi_to_data_shape(
    qoi: ndarray[tuple[int, ...], dtype[T]],
) -> ndarray[tuple[int, ...], dtype[T]]

Expand the qoi array to the shape of the data it was evaluated from.

The qoi array is assumed to have been produced by the evaluate_qoi method.

For neighbourhoods without any valid boundaries, this method simply copies the qoi array. Along axes with a valid boundary, the qoi is expanded before and after by as many elements as the boundary cut off. These new elements are filled with zeros.

This method can be used to produce the array for the late-bound eb parameter by first deriving the error bound from an evaluated QoI array (shape) and then expanding it to the data shape.

This method is the lossy inverse of truncate_data_to_qoi_shape.

Parameters:
Returns:
Raises:
  • IndexError –

    if any neighbourhood axis is out of bounds in qoi.

  • IndexError –

    if any neighbourhood axis is duplicate.
Referenced by:

get_config 

get_config() -> dict[str, JSON]

Returns the configuration of the safeguard.

Returns:
  • config( dict[str, JSON] ) –

    Configuration of the safeguard.