api

compression_safeguards.api

Implementation of the Safeguards, which compute the correction needed to satisfy a set of Safeguards.

Classes:

Safeguards 

Safeguards(
    *,
    safeguards: Collection[dict[str, JSON] | Safeguard],
    _version: None | str | Version = None,
)

Collection of Safeguards.

Parameters:
  • safeguards (Collection[dict[str, JSON] | Safeguard]) –

    The safeguards that will be applied. They can either be passed as a safeguard configuration dict or an already initialized Safeguard.

    Please refer to the SafeguardKind for an enumeration of all supported safeguards.

  • _version (..., default: None ) –

    The version of the safeguards. Do not provide this parameter explicitly.
Raises:
  • ValueError –

    if the safeguards are instantiated with a configuration from an incompatible version of the safeguards.

  • NotImplementedError –

    if the safeguards contain an unsupported kind of safeguard (currently: pointwise and stencil safeguards are supported).

  • ... –

    if instantiating a safeguard raises an exception.

Methods:

  • supported_dtypes –

    The set of numpy dtypes that the safeguards support.

  • correction_dtype_for_data –

    Compute the dtype of the correction for data of the provided dtype.

  • check –

    Check if the approximation array upholds the properties enforced by the safeguards with respect to the data array.

  • compute_correction –

    Compute the correction required to make the approximation array satisfy the safeguards relative to the data array.

  • apply_correction –

    Apply the correction to the approximation to satisfy the safeguards for which the correction was computed.

  • compute_required_stencil_for_chunked_correction –

    Compute the shape of the stencil neighbourhood around chunks of the complete data that is required to compute the chunked corrections.

  • check_chunk –

    Check if the approximation_chunk array chunk upholds the properties enforced by the safeguards with respect to the data_chunk array chunk.

  • compute_chunked_correction –

    Compute the correction required to make the approximation_chunk array chunk satisfy the safeguards relative to the data_chunk array chunk.

  • get_config –

    Returns the configuration of the safeguards.

  • from_config –

    Instantiate the safeguards from a configuration dict.

safeguards property 

safeguards: Collection[Safeguard]

The collection of safeguards.

late_bound property 

late_bound: Set[Parameter]

The set of late-bound parameters that the safeguards have.

Late-bound parameters are only bound when computing the correction, in contrast to the normal early-bound parameters that are configured during safeguard initialisation.

Late-bound parameters can be used for parameters that depend on the specific data that is to be safeguarded.

builtin_late_bound property 

builtin_late_bound: Set[Parameter]

The set of built-in late-bound constants that the safeguards provide automatically, which include $x and $X.

version property 

version: Version

The semantic version 1 of the safeguards provided by this package, which covers

  • the guarantees provided by the safeguards (can only be weakened in a new breaking major release)
  • the configurations of the safeguards (can be extended backwards- compatibly in a new minor release)
  • the format of the safeguards corrections (can only be changed in a new breaking major release)

The Safeguards can only load configurations for and apply corrections produced by safeguards with a compatible semantic version.

Note that the version of the safeguards may be different from the version of the compression-safeguards package, which may make changes to the implementation or programmatic API without needing to increase the version of the safeguards.

  1. https://semver.org ↩

supported_dtypes staticmethod 

supported_dtypes() -> frozenset[dtype[number]]

The set of numpy dtypes that the safeguards support.

Referenced by:

correction_dtype_for_data 

correction_dtype_for_data(dtype: dtype[T]) -> dtype[C]

Compute the dtype of the correction for data of the provided dtype.

The correction dtype is the corresponding binary unsigned integer data type with the same bit size.

Parameters:
  • dtype (dtype[T]) –

    The dtype of the data.
Returns:
  • correction( dtype[C] ) –

    The dtype of the correction.

check 

check(
    data: ndarray[S, dtype[T]],
    approximation: ndarray[S, dtype[T]],
    *,
    late_bound: Mapping[str | Parameter, Value]
    | Bindings = EMPTY,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> bool

Check if the approximation array upholds the properties enforced by the safeguards with respect to the data array.

The data array must contain the complete data, i.e. not just a chunk of data, so that non-pointwise safeguards are correctly applied. Please use the check_chunk method instead when working with individual chunks of data.

Parameters:
  • data (ndarray[S, dtype[T]]) –

    The data array, relative to which the safeguards are checked.

  • approximation (ndarray[S, dtype[T]]) –

    The approximation array for which the safeguards are checked.

  • late_bound (Mapping[str | Parameter, Value] | Bindings, default: EMPTY ) –

    The bindings for all late-bound parameters of the safeguards.

    The bindings must resolve all late-bound parameters and include no extraneous parameters.

    The safeguards automatically provide the $x and $X built-in constants, which must not be included.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only check at data points where the condition is True.
Returns:
  • ok( bool ) –

    True if the check succeeded.
Raises:
  • TypeSetError –

    if the data uses an unsupported data type.

  • ValueError –

    if the data's dtype or shape do not match the approximation's.

  • RuntimeError –

    if the data array is chunked.

  • LateBoundParameterResolutionError –

    if late_bound does not resolve all late-bound parameters of the safeguards or includes any extraneous parameters.

  • ... –

    if checking a safeguard raises an exception.
Referenced by:

compute_correction 

compute_correction(
    data: ndarray[S, dtype[T]],
    approximation: ndarray[S, dtype[T]],
    *,
    late_bound: Mapping[str | Parameter, Value]
    | Bindings = EMPTY,
    where: Literal[True] | ndarray[S, dtype[bool]] = True,
) -> ndarray[S, dtype[C]]

Compute the correction required to make the approximation array satisfy the safeguards relative to the data array.

The data array must contain the complete data, i.e. not just a chunk of data, so that non-pointwise safeguards are correctly applied. Please use the compute_chunked_correction method instead when working with individual chunks of data.

The correction is defined as as_bits(corrected) - as_bits(approximation) using wrapping unsigned integer arithmetic. It has the corresponding binary unsigned integer data type with the same bit size. The correction can be applied using apply_correction to get the corrected approximation.

Parameters:
  • data (ndarray[S, dtype[T]]) –

    The data array, relative to which the safeguards are enforced.

  • approximation (ndarray[S, dtype[T]]) –

    The approximation array for which the correction is computed.

  • late_bound (Mapping[str | Parameter, Value] | Bindings, default: EMPTY ) –

    The bindings for all late-bound parameters of the safeguards.

    The bindings must resolve all late-bound parameters and include no extraneous parameters.

    The safeguards automatically provide the $x and $X built-in constants, which must not be included.

  • where (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only compute the correction at data points where the condition is True.
Returns:
Raises:
  • TypeSetError –

    if the data uses an unsupported data type.

  • ValueError –

    if the data's dtype or shape do not match the approximation's.

  • RuntimeError –

    if the data array is chunked.

  • LateBoundParameterResolutionError –

    if late_bound does not resolve all late-bound parameters of the safeguards or includes any extraneous parameters.

  • ... –

    if computing the correction for a safeguard raises an exception.
Referenced by:

apply_correction 

apply_correction(
    approximation: ndarray[S, dtype[T]],
    correction: ndarray[S, dtype[C]],
) -> ndarray[S, dtype[T]]

Apply the correction to the approximation to satisfy the safeguards for which the correction was computed.

The approximation must be bitwise equivalent to the approximation that was used to compute the correction.

This method is guaranteed to work for chunked data as well, i.e. applying a chunk of the correction to the corresponding chunk of the approximation produces the correct result.

The correction is applied with from_bits(as_bits(approximation) + correction) using wrapping unsigned integer arithmetic.

Parameters:
  • approximation (ndarray[S, dtype[T]]) –

    The approximation array for which the correction has been computed.

  • correction (ndarray[S, dtype[C]]) –

    The correction array.
Returns:
  • corrected( ndarray[S, dtype[T]] ) –

    The corrected array, which satisfies the safeguards.
Raises:
  • ValueError –

    if the correction's shape dos not match the approximation's, or if the correction's dtype does not match the correction dtype for the approximation's dtype.
Referenced by:

compute_required_stencil_for_chunked_correction cached 

compute_required_stencil_for_chunked_correction(
    data_shape: tuple[int, ...],
) -> tuple[
    tuple[Literal[valid, wrap], NeighbourhoodAxis], ...
]

Compute the shape of the stencil neighbourhood around chunks of the complete data that is required to compute the chunked corrections.

For each data dimension, the stencil might require either a valid or wrapping boundary condition.

This method also checks that the data shape is compatible with the safeguards.

Parameters:
  • data_shape (tuple[int, ...]) –

    The shape of the complete data.
Returns:
Raises:
  • ... –

    if computing the stencil neighbourhood for a safeguard raises an exception.
Referenced by:

check_chunk 

check_chunk(
    data_chunk: ndarray[S, dtype[T]],
    approximation_chunk: ndarray[S, dtype[T]],
    *,
    data_shape: tuple[int, ...],
    chunk_offset: tuple[int, ...],
    chunk_stencil: tuple[
        tuple[Literal[valid, wrap], NeighbourhoodAxis], ...
    ],
    late_bound_chunk: Mapping[str | Parameter, Value]
    | Bindings = EMPTY,
    where_chunk: Literal[True]
    | ndarray[S, dtype[bool]] = True,
) -> bool

Check if the approximation_chunk array chunk upholds the properties enforced by the safeguards with respect to the data_chunk array chunk.

Both the data_chunk and approximation_chunk contain the stencil around the chunk, and the shape of this applied stencil is specified in the chunk_stencil parameter. This stencil must be compatible with the required stencil returned by compute_required_stencil_for_chunked_correction(data_shape): - a wrapping boundary is always compatible with a valid boundary. - a larger stencil is always compatible with a smaller stencil. - a smaller stencil is sometimes compatible with a larger stencil, iff the smaller stencil is near the entire data boundary and still includes all required elements; for instance, providing the entire data as a single chunk with no stencil is always compatible with any stencil

This advanced method should only be used when working with individual chunks of data, for non-chunked data please use the simpler and more efficient check method instead.

Parameters:
  • data_chunk (ndarray[S, dtype[T]]) –

    A stencil-extended chunk from the data array, relative to which the safeguards are checked.

  • approximation_chunk (ndarray[S, dtype[T]]) –

    The corresponding stencil-extended chunk from the approximation array for which the safeguards are checked.

  • data_shape (tuple[int, ...]) –

    The shape of the entire data array, i.e. not just the chunk, without any stencil.

  • chunk_offset (tuple[int, ...]) –

    The offset of the non-stencil-extended chunk inside the entire array. For arrays going from left to right, bottom to top, ..., the offset is the index of the bottom left element in the entire array.

  • chunk_stencil (tuple[tuple[Literal[valid, wrap], NeighbourhoodAxis], ...]) –

    The shape of the stencil neighbourhood that was applied around the chunk in data_chunk and approximation_chunk.

  • late_bound_chunk (Mapping[str | Parameter, Value] | Bindings, default: EMPTY ) –

    The bindings for all late-bound parameters of the safeguards.

    The bindings must resolve all late-bound parameters and include no extraneous parameters.

    If a binding resolves to an array, it must be the corresponding chunk of the entire late-bound array.

    The safeguards automatically provide the $x and $X built-in constants, which must not be included.

  • where_chunk (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only check at data points where the condition is True.
Returns:
  • chunk_ok( bool ) –

    True if the check succeeded for the chunk.
Raises:
  • TypeSetError –

    if the data_chunk uses an unsupported data type.

  • ValueError –

    if the data_chunk's dtype or shape do not match the approximation_chunk's, or if the length of the data_shape, chunk_offset, or chunk_stencil do not match the dimensionality of the data.

  • ValueError –

    if the chunk_stencil is not compatible with the required stencil.

  • LateBoundParameterResolutionError –

    if late_bound_chunk does not resolve all late-bound parameters of the safeguards or includes any extraneous parameters.

  • ValueError –

    if any late_bound_chunk array could not be broadcast to the data_chunk's shape.

  • ... –

    if checking a safeguard raises an exception.
Referenced by:

compute_chunked_correction 

compute_chunked_correction(
    data_chunk: ndarray[S, dtype[T]],
    approximation_chunk: ndarray[S, dtype[T]],
    *,
    data_shape: tuple[int, ...],
    chunk_offset: tuple[int, ...],
    chunk_stencil: tuple[
        tuple[Literal[valid, wrap], NeighbourhoodAxis], ...
    ],
    any_chunk_check_failed: bool,
    late_bound_chunk: Mapping[str | Parameter, Value]
    | Bindings = EMPTY,
    where_chunk: Literal[True]
    | ndarray[S, dtype[bool]] = True,
) -> ndarray[tuple[int, ...], dtype[C]]

Compute the correction required to make the approximation_chunk array chunk satisfy the safeguards relative to the data_chunk array chunk.

Both the data_chunk and approximation_chunk contain the stencil around the chunk, and the shape of this applied stencil is specified in the chunk_stencil parameter. This stencil must be compatible with the required stencil returned by compute_required_stencil_for_chunked_correction(data_shape): - a wrapping boundary is always compatible with a valid boundary. - a larger stencil is always compatible with a smaller stencil. - a smaller stencil is sometimes compatible with a larger stencil, iff the smaller stencil is near the entire data boundary and still includes all required elements; for instance, providing the entire data as a single chunk with no stencil is always compatible with any stencil

This advanced method should only be used when working with individual chunks of data, for non-chunked data please use the simpler and more efficient compute_correction method instead.

The (chunked) correction is defined as as_bits(corrected) - as_bits(approximation) using wrapping unsigned integer arithmetic. It has the corresponding binary unsigned integer data type with the same bit size. The chunked correction can be applied using apply_correction to get the corrected chunked approximation.

Parameters:
  • data_chunk (ndarray[S, dtype[T]]) –

    A stencil-extended chunk from the data array, relative to which the safeguards are enforced.

  • approximation_chunk (ndarray[S, dtype[T]]) –

    The corresponding stencil-extended chunk from the approximation array for which the correction is computed.

  • data_shape (tuple[int, ...]) –

    The shape of the entire data array, i.e. not just the chunk.

  • chunk_offset (tuple[int, ...]) –

    The offset of the non-stencil-extended chunk inside the entire array. For arrays going from left to right, bottom to top, ..., the offset is the index of the bottom left element in the entire array.

  • chunk_stencil (tuple[tuple[Literal[valid, wrap], NeighbourhoodAxis], ...]) –

    The shape of the stencil neighbourhood that was applied around the chunk in data_chunk and approximation_chunk.

  • any_chunk_check_failed (bool) –

    If any chunk failed its check, this chunk requires corrections; if no chunk failed its check, this chunk does not require corrections. If no check across all chunks was performed, this option must be set to True.

  • late_bound_chunk (Mapping[str | Parameter, Value] | Bindings, default: EMPTY ) –

    The bindings for all late-bound parameters of the safeguards.

    The bindings must resolve all late-bound parameters and include no extraneous parameters.

    If a binding resolves to an array, it must be the corresponding chunk of the entire late-bound array.

    The safeguards automatically provide the $x and $X built-in constants, which must not be included.

  • where_chunk (Literal[True] | ndarray[S, dtype[bool]], default: True ) –

    Only compute the correction at data points where the condition is True.
Returns:
  • correction_chunk( ndarray[tuple[int, ...], dtype[C]] ) –

    The correction array chunk. The correction chunk is truncated to remove the stencil, i.e. it only contains the correction for the non-stencil-extended chunk.
Raises:
  • TypeSetError –

    if the data_chunk uses an unsupported data type.

  • ValueError –

    if the data_chunk's dtype or shape do not match the approximation_chunk's, or if the length of the data_shape, chunk_offset, or chunk_stencil do not match the dimensionality of the data.

  • ValueError –

    if the chunk_stencil is not compatible with the required stencil.

  • LateBoundParameterResolutionError –

    if late_bound_chunk does not resolve all late-bound parameters of the safeguards or includes any extraneous parameters.

  • ValueError –

    if any late_bound_chunk array could not be broadcast to the data_chunk's shape.

  • ... –

    if computing the correction for a safeguard raises an exception.
Referenced by:

get_config 

get_config() -> dict[str, JSON]

Returns the configuration of the safeguards.

Returns:
  • config( dict[str, JSON] ) –

    Configuration of the safeguards.

from_config classmethod 

from_config(config: dict[str, JSON]) -> Self

Instantiate the safeguards from a configuration dict.

Parameters:
  • config (dict[str, JSON]) –

    Configuration of the safeguards.
Returns:
  • safeguards( Self ) –

    Collection of safeguards.