topostats.processing#

Functions for processing data.

Attributes#

LOGGER

Functions#

run_filters(→ None)

Filter and flatten an image. Optionally plots the results, returning the flattened image.

run_grains(→ None)

Identify grains (molecules) and optionally plots the results.

run_grainstats(→ None)

Calculate grain statistics for an image and optionally plots the results.

run_disordered_tracing(→ None)

Skeletonise and prune grains, adding results to statistics data frames and optionally plot results.

run_nodestats(→ None)

Analyse crossing points in grains adding results to statistics data frames and optionally plot results.

run_ordered_tracing(→ None)

Order coordinates of traces, adding results to statistics data frames and optionally plot results.

run_splining(→ None)

Smooth the ordered trace coordinates and optionally plot results.

run_curvature_stats(→ None)

Calculate curvature statistics for the traced DNA molecules.

get_out_paths(image_path, base_dir, output_dir, ...[, ...])

Determine components of output paths for a given image and plotting config.

process_scan(→ tuple[str, pandas.DataFrame, ...)

Process a single image, filtering, finding grains and calculating their statistics.

process_filters(→ tuple[str, bool])

Filter an image return the flattened images and save to ''.topostats''.

process_grains(→ tuple[str, bool])

Detect grains in flattened images and save to ''.topostats''.

process_grainstats(→ tuple[str, bool])

Calculate grain statistics in an image where grains have already been detected.

check_run_steps(→ None)

Check options for running steps (Filter, Grain, Grainstats and DNA tracing) are logically consistent.

completion_message(→ None)

Print a completion message summarising images processed.

Module Contents#

topostats.processing.LOGGER#
topostats.processing.run_filters(topostats_object: topostats.classes.TopoStats, filter_out_path: pathlib.Path, core_out_path: pathlib.Path, filter_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Filter and flatten an image. Optionally plots the results, returning the flattened image.

Parameters:

  • topostats_object (TopoStats) – TopoStats object to be filtered.

  • filter_out_path (Path) – Output directory for step-by-step flattening plots.

  • core_out_path (Path) – General output directory for outputs such as the flattened image.

  • filter_config (dict) – Dictionary of configuration for the Filters class to use when initialised.

  • plotting_config (dict) – Dictionary of configuration for plotting output images.

topostats.processing.run_grains(topostats_object: topostats.classes.TopoStats, grain_out_path: pathlib.Path | None, core_out_path: pathlib.Path, plotting_config: dict | None = None, grains_config: dict | None = None) → None[source]#

Identify grains (molecules) and optionally plots the results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object for grain detection.

  • grain_out_path (Path) – Output path for step-by-step grain finding plots.

  • core_out_path (Path) – General output directory for outputs such as the flattened image with grain masks overlaid.

  • plotting_config (dict) – Dictionary of configuration for plotting images.

  • grains_config (dict) – Dictionary of configuration for the Grains class to use when initialised.

topostats.processing.run_grainstats(topostats_object: topostats.classes.TopoStats, grain_out_path: pathlib.Path | None, core_out_path: pathlib.Path, grainstats_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Calculate grain statistics for an image and optionally plots the results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object post grain detection for which statistics are to be calculated.

  • grain_out_path (Path) – Directory to save optional grain statistics visual information to.

  • core_out_path (Path) – General output directory for outputs such as the flattened image with grain masks overlaid.

  • grainstats_config (dict, optional) – Dictionary of configuration for the GrainStats class to be used when initialised.

  • plotting_config (dict, optional) – Dictionary of configuration for plotting images.

topostats.processing.run_disordered_tracing(topostats_object: topostats.classes.TopoStats, core_out_path: pathlib.Path, tracing_out_path: pathlib.Path, disordered_tracing_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Skeletonise and prune grains, adding results to statistics data frames and optionally plot results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object for processing, should have had grain detection performed prior to disordered tracing otherwise there are no grains to trace.

  • core_out_path (Path) – Path to save the core disordered trace image to.

  • tracing_out_path (Path) – Path to save the optional, diagnostic disordered trace images to.

  • disordered_tracing_config (dict) – Dictionary configuration for obtaining a disordered trace representation of the grains.

  • plotting_config (dict) – Dictionary configuration for plotting images.

topostats.processing.run_nodestats(topostats_object: topostats.classes.TopoStats, core_out_path: pathlib.Path, tracing_out_path: pathlib.Path, nodestats_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Analyse crossing points in grains adding results to statistics data frames and optionally plot results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object for processing, should have had disordered tracing performed first.

  • core_out_path (Path) – Path to save the core NodeStats image to.

  • tracing_out_path (Path) – Path to save optional, diagnostic NodeStats images to.

  • nodestats_config (dict) – Dictionary configuration for analysing the crossing points.

  • plotting_config (dict) – Dictionary configuration for plotting images.

topostats.processing.run_ordered_tracing(topostats_object: topostats.classes.TopoStats, core_out_path: pathlib.Path, tracing_out_path: pathlib.Path, ordered_tracing_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Order coordinates of traces, adding results to statistics data frames and optionally plot results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object for processing, should have had nodestats processing performed first.

  • core_out_path (Path) – Path to save the core ordered tracing image to.

  • tracing_out_path (Path) – Path to save optional, diagnostic ordered trace images to.

  • ordered_tracing_config (dict) – Dictionary configuration for obtaining an ordered trace representation of the skeletons.

  • plotting_config (dict) – Dictionary configuration for plotting images.

topostats.processing.run_splining(topostats_object: topostats.classes.TopoStats, core_out_path: pathlib.Path, splining_config: dict | None = None, plotting_config: dict | None = None, tracing_out_path: str | pathlib.Path | None = None) → None[source]#

Smooth the ordered trace coordinates and optionally plot results.

Parameters:

  • topostats_object (TopoStats) – TopoStats object to be splined.

  • core_out_path (Path) – Path to save the core ordered tracing image to.

  • splining_config (dict) – Dictionary configuration for obtaining an ordered trace representation of the skeletons.

  • plotting_config (dict) – Dictionary configuration for plotting images.

  • tracing_out_path (str | Path) – Directory to save images from splining to. The splining directory will be created within and images saved there.

topostats.processing.run_curvature_stats(topostats_object: topostats.classes.TopoStats, core_out_path: pathlib.Path, tracing_out_path: pathlib.Path, curvature_config: dict | None = None, plotting_config: dict | None = None) → None[source]#

Calculate curvature statistics for the traced DNA molecules.

Currently only works on simple traces, not branched traces.

Parameters:

  • topostats_object (TopoStats) – TopoStats object post splining, all Molecules within the grain_crops attribute (a dictionary of GrainCrop should have splined_coords attributes populated.

  • core_out_path (Path) – Path to save the core curvature image to.

  • tracing_out_path (Path) – Path to save the optional, diagnostic curvature images to.

  • curvature_config (dict) – Dictionary of configuration for running the curvature stats.

  • plotting_config (dict) – Dictionary of configuration for plotting images.

topostats.processing.get_out_paths(image_path: pathlib.Path, base_dir: pathlib.Path, output_dir: pathlib.Path, filename: str, plotting_config: dict, grain_dirs: bool = True)[source]#

Determine components of output paths for a given image and plotting config.

Parameters:

  • image_path (Path) – Path of the image being processed.

  • base_dir (Path) – Path of the data folder.

  • output_dir (Path) – Base output directory for output data.

  • filename (str) – Name of the image being processed.

  • plotting_config (dict) – Dictionary of configuration for plotting images.

  • grain_dirs (bool) – Whether to create the grains and dnatracing sub-directories, by default this is True but it should be set to False when running just Filters.

Returns:

Core output path for general file outputs, filter output path for flattening related files and grain output path for grain finding related files.

Return type:

tuple

topostats.processing.process_scan(topostats_object: topostats.classes.TopoStats, base_dir: str | pathlib.Path, filter_config: dict, grains_config: dict, grainstats_config: dict, disordered_tracing_config: dict, nodestats_config: dict, ordered_tracing_config: dict, splining_config: dict, curvature_config: dict, plotting_config: dict, output_dir: str | pathlib.Path = 'output') → tuple[str, pandas.DataFrame, topostats.classes.TopoStats, pandas.DataFrame, pandas.DataFrame, pandas.DataFrame, pandas.DataFrame][source]#

Process a single image, filtering, finding grains and calculating their statistics.

Parameters:

  • topostats_object (TopoStats) – A dictionary with keys ‘image’, ‘img_path’ and ‘pixel_to_nm_scaling’ containing a file or frames’ image, it’s path and it’s pixel to namometre scaling value.

  • base_dir (str | Path) – Directory to recursively search for files, if not specified the current directory is scanned.

  • filter_config (dict) – Dictionary of configuration options for running the Filter stage.

  • grains_config (dict) – Dictionary of configuration options for running the Grain detection stage.

  • grainstats_config (dict) – Dictionary of configuration options for running the Grain Statistics stage.

  • disordered_tracing_config (dict) – Dictionary configuration for obtaining a disordered trace representation of the grains.

  • nodestats_config (dict) – Dictionary of configuration options for running the NodeStats stage.

  • ordered_tracing_config (dict) – Dictionary configuration for obtaining an ordered trace representation of the skeletons.

  • splining_config (dict) – Dictionary of configuration options for running the splining stage.

  • curvature_config (dict) – Dictionary of configuration options for running the curvature stats stage.

  • plotting_config (dict) – Dictionary of configuration options for plotting figures.

  • output_dir (str | Path) – Directory to save output to, it will be created if it does not exist. If it already exists then it is possible that output will be over-written.

Returns:

Tuple of filename, grain statistics dataframe, TopoStats, image statistics dataframe, disordered tracing dataframe, matched branch statistics dataframe and molecule statistics dataframe.

Return type:

tuple[str, pd.DataFrame, TopoStats, pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]

topostats.processing.process_filters(topostats_object: dict, base_dir: str | pathlib.Path, filter_config: dict, plotting_config: dict, output_dir: str | pathlib.Path = 'output') → tuple[str, bool][source]#

Filter an image return the flattened images and save to ‘’.topostats’’.

This function serves as an entry point to run just the first key step of flattening images to remove noise, tilt and optionally scars saving to ‘’.topostats’’ for subsequent processing and analyses.

Parameters:

  • topostats_object (dict[str, Union[npt.NDArray, Path, float]]) – A dictionary with keys ‘image’, ‘img_path’ and ‘pixel_to_nm_scaling’ containing a file or frames’ image, it’s path and it’s pixel to namometre scaling value.

  • base_dir (str | Path) – Directory to recursively search for files, if not specified the current directory is scanned.

  • filter_config (dict) – Dictionary of configuration options for running the Filter stage.

  • plotting_config (dict) – Dictionary of configuration options for plotting figures.

  • output_dir (str | Path) – Directory to save output to, it will be created if it does not exist. If it already exists then it is possible that output will be over-written.

Returns:

A tuple of the image and a boolean indicating if the image was successfully processed.

Return type:

tuple[str, bool]

topostats.processing.process_grains(topostats_object: dict, base_dir: str | pathlib.Path, grains_config: dict, plotting_config: dict, output_dir: str | pathlib.Path = 'output') → tuple[str, bool][source]#

Detect grains in flattened images and save to ‘’.topostats’’.

This function serves as an entry point to run grain detection on flattened images to identify grains and save data to ‘’.topostats’’ for subsequent processing and analyses.

Parameters:

  • topostats_object (dict[str, Union[npt.NDArray, Path, float]]) – A dictionary with keys ‘image’, ‘img_path’ and ‘pixel_to_nm_scaling’ containing a file or frames’ image, it’s path and it’s pixel to namometre scaling value.

  • base_dir (str | Path) – Directory to recursively search for files, if not specified the current directory is scanned.

  • grains_config (dict) – Dictionary of configuration options for running the Filter stage.

  • plotting_config (dict) – Dictionary of configuration options for plotting figures.

  • output_dir (str | Path) – Directory to save output to, it will be created if it does not exist. If it already exists then it is possible that output will be over-written.

Returns:

A tuple of the image and a boolean indicating if the image was successfully processed.

Return type:

tuple[str, bool]

topostats.processing.process_grainstats(topostats_object: topostats.classes.TopoStats, base_dir: str | pathlib.Path, grainstats_config: dict, plotting_config: dict, output_dir: str | pathlib.Path = 'output') → tuple[str, bool][source]#

Calculate grain statistics in an image where grains have already been detected.

This function serves as an entry point to run just the grainstats processing step and optionally saving to .topostats for subsequent processing and analyses.

Parameters:

  • topostats_object (TopoStats) – A TopoStats object.

  • base_dir (str | Path) – Directory to recursively search for files, if not specified the current directory is scanned.

  • grainstats_config (dict) – Dictionary of configuration options for running the Filter stage.

  • plotting_config (dict) – Dictionary of configuration options for plotting figures.

  • output_dir (str | Path) – Directory to save output to, it will be created if it does not exist. If it already exists then it is possible that output will be over-written.

Returns:

A tuple of the image and a boolean indicating if the image was successfully processed.

Return type:

tuple[str, pd.DataFrame]

topostats.processing.check_run_steps(filter_run: bool, grains_run: bool, grainstats_run: bool, disordered_tracing_run: bool, nodestats_run: bool, ordered_tracing_run: bool, splining_run: bool) → None[source]#

Check options for running steps (Filter, Grain, Grainstats and DNA tracing) are logically consistent.

This checks that earlier steps required are enabled.

Parameters:

  • filter_run (bool) – Flag for running Filtering.

  • grains_run (bool) – Flag for running Grains.

  • grainstats_run (bool) – Flag for running GrainStats.

  • disordered_tracing_run (bool) – Flag for running Disordered Tracing.

  • nodestats_run (bool) – Flag for running NodeStats.

  • ordered_tracing_run (bool) – Flag for running Ordered Tracing.

  • splining_run (bool) – Flag for running DNA Tracing.

topostats.processing.completion_message(config: dict, img_files: list, summary_config: dict, images_processed: int) → None[source]#

Print a completion message summarising images processed.

Parameters:

  • config (dict) – Configuration dictionary.

  • img_files (list) – List of found image paths.

  • summary_config (dict) – Configuration for plotting summary statistics.

  • images_processed (int) – Pandas DataFrame of results.