zea.data.convert.echonet

Script to convert the EchoNet database to zea format.

Note

Will segment the images and convert them to polar coordinates.

For more information about the dataset, resort to the following links:

• The original dataset can be found at this link.

• The project page is available here.

Functions

accept_shape(tensor)	Acceptance algorithm that determines whether to reject an image based on left and right corner data.
cartesian_to_polar_matrix(cartesian_matrix)	Function that converts a timeseries of a cartesian cone to a polar representation that is more compatible with CNN's/action selection.
convert_echonet(args)	Convert an EchoNet dataset into zea files, organizing results into train/val/test/rejected splits.
count_init(shared_counter[, processor])	Initialize the module-level shared counter used by worker processes.
find_split_for_file(file_dict, target_file)	Locate which split contains a given filename.
load_split(split_path)	Load an EchoNet dataset split, as written by convert_echonet().
rotate_coordinates(data_points, degrees)	Function that rotates the datapoints by a certain degree.
segment(tensor[, number_erasing, min_clip])	Segments the background of the echonet images by setting it to 0 and creating a hard edge.

Classes

H5Processor(path_out_h5[, num_val, ...])

Stores a few variables and paths to allow for hyperthreading.

zea.data.convert.echonet.COUNTER: Synchronized[int]

class zea.data.convert.echonet.H5Processor(path_out_h5, num_val=500, num_test=500, range_from=(0, 255), range_to=(-60, 0), splits=None)

Bases: object

Stores a few variables and paths to allow for hyperthreading.

__call__(avi_file)

Convert a single AVI file into a zea dataset entry. Loads the AVI, validates and rescales pixel ranges, applies segmentation, assigns a data split (train/val/test/rejected), converts accepted frames to polar coordinates and saves as a zea HDF5 file via File.create.

Parameters:

avi_file (pathlib.Path) – Path to the source .avi file to process.

Raises:

ValueError – If the converted image has non-finite values.

get_split(hdf5_file, sequence)

Determine the dataset split label for a given file and its image sequence.

This method checks acceptance based on the first frame of sequence. If explicit splits were provided to the processor, it returns the split found for hdf5_file, even when the acceptance check disagrees: the check thresholds decoded pixels, so a different decoder can flip borderline videos, and a published split should not change because of that. Disagreements are logged. If no explicit splits are provided, rejected sequences are labeled “rejected”. Accepted sequences increment a shared counter and are assigned “val”, “test”, or “train” according to the processor’s num_val and num_test quotas.

Parameters:

• hdf5_file (str) – Filename or identifier used to look up an existing split when splits are provided.

• sequence (array-like) – Time-ordered sequence of images; the first frame is used for acceptance checking.

Returns:

One of “train”, “val”, “test”, or “rejected” indicating the assigned split.

Return type:

str

validate_split_copy(split_file)

Validate that a generated split YAML matches the original splits provided to the processor.

Reads the YAML at split_file and compares its train, val, test, and rejected lists (or other split keys present in self.splits) against self.splits; logs confirmation when a split matches and logs which entries are missing or extra when they differ. If the processor was not initialized with splits, validation is skipped and a message is logged.

Parameters:

split_file (str or os.PathLike) – Path to the YAML file containing the generated dataset splits.

zea.data.convert.echonet.PROCESSOR: H5Processor

zea.data.convert.echonet.accept_shape(tensor)

Acceptance algorithm that determines whether to reject an image based on left and right corner data.

Parameters:

tensor (ndarray) – Input image (sc) with 2 dimensions. (112, 112)

Returns:

Whether or not the tensor should be rejected.

Return type:

decision (bool)

zea.data.convert.echonet.cartesian_to_polar_matrix(cartesian_matrix, tip=(61, 7), r_max=107, angle=0.79, interpolation='nearest')

Function that converts a timeseries of a cartesian cone to a polar representation that is more compatible with CNN’s/action selection.

Parameters:

• cartesian_matrix (-) – (rows, cols) scan-converted (Cartesian) image, or a (n_frames, rows, cols) time sequence of them.

• tip (-) – coordinates (in indices) of the tip of the cone. Defaults to (61, 7).

• r_max (-) – expected radius of the cone. Defaults to 107.

• angle (-) – expected angle of the cone, will be used as (-angle, angle). Defaults to 0.79.

• interpolation (-) – can be [nearest, linear, cubic]. Defaults to ‘nearest’.

Returns:

polar conversion of the input, same shape as the input.

Return type:

polar_matrix (ndarray)

zea.data.convert.echonet.convert_echonet(args)

Convert an EchoNet dataset into zea files, organizing results into train/val/test/rejected splits.

Parameters:

args (argparse.Namespace) –

An object with the following attributes.

• src (str|Path): Path to the source archive or directory containing .avi files.

  Will be unzipped if needed.

• dst (str|Path): Destination directory for generated zea files

  per-split subdirectories (train, val, test, rejected) and a split.yaml are created or updated.

• split_path (str|Path|None, optional): The split to reproduce, in any format

  load_split() reads. Defaults to the published split, ECHONET_SPLIT_PATH.

• new_split (bool, optional): Create a new random split instead of reproducing

  one. Defaults to False.

• no_hyperthreading (bool): When false, processing uses a ProcessPoolExecutor

  with a shared counter; when true, processing runs sequentially.

Note

• May unzip the source into a working directory.

• Writes zea files into dst.

• Writes a split.yaml into dst summarizing produced files per split.

• Logs progress and validation results, including files whose acceptance check disagrees with the given split.

Raises:

RuntimeError – If any file failed to convert. The split.yaml is still written for the files that succeeded.

zea.data.convert.echonet.count_init(shared_counter, processor=None)

Initialize the module-level shared counter used by worker processes.

Parameters:

• shared_counter (multiprocessing.Value) – A process-shared integer Value that will be assigned to the module-global COUNTER for coordinated counting across processes.

• processor (H5Processor, optional) – Processor that _process_file() runs in this worker.

zea.data.convert.echonet.find_split_for_file(file_dict, target_file)

Locate which split contains a given filename.

Parameters:

• file_dict (dict) – Mapping from split name (e.g., “train”, “val”, “test”, “rejected”) to an iterable of filenames.

• target_file (str) – Filename to search for within the split lists.

Returns:

The split name that contains target_file, or “rejected” if the file is not found.

Return type:

str

zea.data.convert.echonet.load_split(split_path)

Load an EchoNet dataset split, as written by convert_echonet().

Parameters:

split_path (str, Path) – A split.yaml file, a folder containing one, or an hf:// path to either. The file maps train, val, test and rejected to lists of file names. The default split is published on zeahub/echonet-dynamic.

Returns:

The split, mapping split names to sets of .hdf5 file names.

Return type:

dict

Raises:

RuntimeError – If an hf:// split cannot be fetched, e.g. without a connection.

zea.data.convert.echonet.rotate_coordinates(data_points, degrees)

Function that rotates the datapoints by a certain degree.

Parameters:

• data_points (ndarray) – tensor containing [N,2] (x and y) datapoints.

• degrees (int) – angle to rotate the datapoints with

Returns:

the rotated data_points.

Return type:

rotated_points (ndarray)

zea.data.convert.echonet.segment(tensor, number_erasing=0, min_clip=0)

Segments the background of the echonet images by setting it to 0 and creating a hard edge.

Parameters:

• tensor (ndarray) – Input image (sc) with 3 dimensions. (N, 112, 112)

• number_erasing (float, optional) – number to fill the background with.

• min_clip (float, optional) – If > 0, values on the computed cone edge will be clipped to be at least this value. Defaults to 0.

Returns:

Segmented matrix of same dimensions as input

Return type:

tensor (ndarray)