samgis_core.utilities package

Submodules

samgis_core.utilities.constants module

Project constants

samgis_core.utilities.create_folders_if_not_exists module

samgis_core.utilities.create_folders_if_not_exists.create_folder_if_not_exists(pathname)

Create a folder given its path.

Parameters:

pathname (Path | str) – folder Path or string

Returns:

samgis_core.utilities.create_folders_if_not_exists.folders_creation(folders_map=None, ignore_errors=True)

Create all folders listed within the folders_map argument (this argument can be a dict or a json string). If folders_map is None the function will try to load the ‘FOLDERS_MAP’ env variable, then will load that json into dict. Once loaded and parsed the folders_map variable, the function will loop over the dict to create the folders using the create_folder_if_not_exists() function.

Parameters:

• folders_map (dict | str, default: None) – dict or string map of folder string

• ignore_errors (bool, default: True) – bool needed to eventually ignore errors on folder creation

Returns:

samgis_core.utilities.create_folders_if_not_exists.stats_pathname(pathname)

samgis_core.utilities.frontend_builder module

samgis_core.utilities.frontend_builder.assert_envs(envs_list)

Assert env variables are not empty.

Parameters:

envs_list (ListStr) – list of env variables

Return type:

None

Returns:

samgis_core.utilities.frontend_builder.build_frontend(project_root_folder, input_css_path, output_dist_folder=PosixPath('/opt/buildhome/.cache/pypoetry/virtualenvs/samgis-VmB9M1Nz-py3.12/lib/python3.12/site-packages/static/dist'), index_page_filename='index.html', output_css_filename='output.css', force_build=False)

Build a static [Vue js](https://vuejs.org/), [tailwindcss](https://tailwindcss.com/) frontend. If force_build is False, the function also check if index_page_filename and output_css_filename already exists: in this case skip the build.

Parameters:

• project_root_folder (str | Path) – Project folder that contains the static frontend

• input_css_path (str | Path) – file path pointing to the input css file

• output_dist_folder (Path, default: PosixPath('/opt/buildhome/.cache/pypoetry/virtualenvs/samgis-VmB9M1Nz-py3.12/lib/python3.12/site-packages/static/dist')) – dist folder path where to write the frontend bundle

• index_page_filename (str, default: 'index.html') – index html filename

• output_css_filename (str, default: 'output.css') – output css filename

• force_build (bool, default: False) – useful to skip the frontend build

Return type:

bool

Returns:

state of the build (True in case of build completed, False in case of build skipped)

samgis_core.utilities.frontend_builder.read_std_out_err(std_out_err, output_type, command)

Capture the standard output or standard error content for a given system command pipe.

Parameters:

• std_out_err (str) – str

• output_type (str) – output type (stdout or stderr)

• command (ListStr) – command executed

Return type:

None

Returns:

samgis_core.utilities.frontend_builder.run_command(commands_list, capture_output=True, text=True, check=True)

Run a system command and capture its output. See https://docs.python.org/3.11/library/subprocess.html#subprocess.run for more details

Parameters:

• commands_list (ListStr) – list of single string commands

• capture_output (bool, default: True) – if true, stdout and stderr will be captured

• text (bool, default: True) – if true, capture stdout and stderr as strings

• check (bool, default: True) – If check is true, and the process exits with a non-zero exit code, a CalledProcessError exception will be raised. Attributes of that exception hold the arguments, the exit code, and stdout and stderr if they were captured.

Return type:

None

Returns:

samgis_core.utilities.plot_images module

samgis_core.utilities.plot_images.helper_imshow_output_expected(img_list, titles_list, cmap='gist_rainbow', plot_size=5, show=False, debug=False, close_after=0.0)

Simple way to display a list of images with their titles, color map. Should work also in an automate environments, like tests (use a close_after argument > 0)

Parameters:

• img_list (list[ndarray]) – ndarray images to display

• titles_list (ListStr) – title images

• cmap (str, default: 'gist_rainbow') – color map

• plot_size (int, default: 5) – figure plot size

• show (default: False) – fire plt.show() action if needed

• debug (bool, default: False) – workaround useful in an interactive context, like Pycharm debugger

• close_after (float, default: 0.0) – close after give seconds (useful in tests, contrasted to ‘debug’ option)

Return type:

tuple[Figure, Axes]

Returns:

tuple of matplotlib Figure, Axes

samgis_core.utilities.plot_images.imshow_raster(raster, title, cmap='gist_rainbow', interpolation=None, alpha=None, transform=None, plot_size=5, show=False, debug=False, close_after=0.0, backend=None)

Displays raster images lists/arrays with titles, legend, alpha transparency, figure sizes and geographic transformations, if not none (leveraging rasterio.plot)

Parameters:

• raster – image to display

• title – title image

• cmap (str, default: 'gist_rainbow') – color map

• interpolation (str, default: None) – interpolation type

• alpha (default: None) – alpha transparency

• transform (default: None) – geographic transform, eventually used for map representation by rasterio

• plot_size (default: 5) – figure plot size

• show (default: False) – fire plt.show() action if needed

• debug (bool, default: False) – workaround useful in an interactive context, like Pycharm debugger

• close_after (float, default: 0.0) – close after give seconds (useful in tests, contrasted to ‘debug’ option)

• backend (MatplotlibBackend, default: None) – matplotlib backend string

Return type:

tuple[Figure, Axes]

Returns:

tuple of matplotlib Figure, Axes

samgis_core.utilities.serialize module

Serialize objects

samgis_core.utilities.serialize.serialize(obj, include_none=False)

Return the input object into a serializable one

Parameters:

• obj (any) – Object to serialize

• include_none (bool, default: False) – bool to indicate if include also keys with None values during dict serialization

Returns:

serialized object

samgis_core.utilities.session_logger module

samgis_core.utilities.session_logger.drop_color_message_key(_, __, event_dict)

Uvicorn logs the message a second time in the extra color_message, but we don’t need it. This processor drops the key from the event dict if it exists.

Return type:

MutableMapping[str, Any]

samgis_core.utilities.session_logger.rename_event_key(_, __, event_dict)

Log entries keep the text message in the event field, but Datadog uses the message field. This processor moves the value from one field to the other. See https://github.com/hynek/structlog/issues/35#issuecomment-591321744

Return type:

MutableMapping[str, Any]

samgis_core.utilities.session_logger.setup_logging(json_logs=False, log_level='INFO')

Enhance the configuration of structlog. Needed for correlation id injection with fastapi middleware in samgis-web. After the use of logging_middleware() in samgis_web.web.middlewares, add also the CorrelationIdMiddleware from ‘asgi_correlation_id’ package. (See ‘tests/web/test_middlewares.py’ in samgis_web). To change an input parameter like the log level, re-run the function changing the parameter (no need to re-instantiate the logger instance: it’s a hot change)

Parameters:

• json_logs (bool, default: False) – set logs in json format

• log_level (str, default: 'INFO') – log level string

Returns:

samgis_core.utilities.type_hints module

custom type hints

class samgis_core.utilities.type_hints.DictStr

Bases: dict[str]

class samgis_core.utilities.type_hints.DictStrAny

Bases: dict[str, any]

class samgis_core.utilities.type_hints.DictStrInt

Bases: dict[str, int]

class samgis_core.utilities.type_hints.EmbeddingImage

Bases: TypedDict

image_embedding: ndarray

original_size: TupleInt

transform_matrix: ndarray

class samgis_core.utilities.type_hints.EmbeddingPILImage

Bases: TypedDict

image_embedding: ndarray

original_size: NewType(TupleInt, tuple[int, int])

resized_size: NewType(TupleInt, tuple[int, int])

class samgis_core.utilities.type_hints.ListDict(iterable=(), /)

Bases: list[dict]

class samgis_core.utilities.type_hints.ListFloat(iterable=(), /)

Bases: list[float]

class samgis_core.utilities.type_hints.ListInt(iterable=(), /)

Bases: list[int]

class samgis_core.utilities.type_hints.ListStr(iterable=(), /)

Bases: list[str]

class samgis_core.utilities.type_hints.LlistFloat(iterable=(), /)

Bases: ListFloat

class samgis_core.utilities.type_hints.MatplotlibBackend(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

agg = 'agg'

cairo = 'cairo'

gtk3agg = 'gtk3agg'

gtk3cairo = 'gtk3cairo'

gtk4agg = 'gtk4agg'

gtk4cairo = 'gtk4cairo'

macosx = 'macosx'

nbagg = 'nbagg'

notebook = 'notebook'

pdf = 'pdf'

pgf = 'pgf'

ps = 'ps'

qt5agg = 'qt5agg'

qt5cairo = 'qt5cairo'

qtagg = 'qtagg'

qtcairo = 'qtcairo'

svg = 'svg'

template = 'template'

tkagg = 'tkagg'

tkcairo = 'tkcairo'

webagg = 'webagg'

wx = 'wx'

wxagg = 'wxagg'

wxcairo = 'wxcairo'

class samgis_core.utilities.type_hints.StrEnum(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

class samgis_core.utilities.type_hints.TupleFloat(iterable=(), /)

Bases: tuple[float]

class samgis_core.utilities.type_hints.TupleFloatAny(iterable=(), /)

Bases: tuple[float, any]

class samgis_core.utilities.type_hints.TupleInt(iterable=(), /)

Bases: tuple[int]

class samgis_core.utilities.type_hints.TupleNdarrayFloat(iterable=(), /)

Bases: tuple[ndarray, float]

class samgis_core.utilities.type_hints.TupleNdarrayInt(iterable=(), /)

Bases: tuple[ndarray, int]

samgis_core.utilities.update_requirements_txt module

samgis_core.utilities.update_requirements_txt.get_args(current_args)

Return type:

Namespace

samgis_core.utilities.update_requirements_txt.get_dependencies_freeze()

get a ‘freeze.txt’-like dict of ‘name’:’version’ metadata about installed packages.

Return type:

dict

samgis_core.utilities.update_requirements_txt.get_requirements_txt(requirements_no_versions_filename, requirements_output_filename)

Write on disk a requirements.txt file with an updated list of dependencies from installed python packages. Both input and output requirements files must be within the folder from which the current command is executed.

Parameters:

• requirements_no_versions_filename (str | Path) – input requirements filename with no specified versions

• requirements_output_filename (str | Path) – output requirements.txt filename

Returns:

samgis_core.utilities.update_requirements_txt.sanitize_path(filename, strict_on_filename=True)

Return type:

Path

samgis_core.utilities.utilities module

Various utilities (logger, time benchmark, args dump, numerical and stats info)

samgis_core.utilities.utilities.apply_coords(coords, embedding)

Expects a numpy np_array of length 2 in the final dimension. Requires the original image size in (H, W) format.

Args:

coords: coordinates ndarray embedding: PIL image embedding dict

Returns:

coordinates ndarray

samgis_core.utilities.utilities.base64_decode(s)

Decode base64 strings

Parameters:

s – input string

Returns:

decoded string

samgis_core.utilities.utilities.base64_encode(sb)

Encode input strings or bytes as base64

Parameters:

sb (str | bytes) – input string or bytes

Return type:

bytes

Returns:

base64 encoded bytes

samgis_core.utilities.utilities.convert_ndarray_to_pil(pil_image)

Check if an image is a ndarray and then convert to a PIL Image instance.

Parameters:

pil_image (Image | ndarray) – PIL image or ndarray

Returns:

PIL Image

samgis_core.utilities.utilities.hash_calculate(arr)

Return computed hash from input variable (typically a numpy array).

Parameters:

arr – input variable

Return type:

str | bytes

Returns:

computed hash from input variable

samgis_core.utilities.utilities.normalize_array(arr, new_h=255.0, type_normalization='int')

Normalize numpy array between 0 and ‘new_h’ value. Default dtype of output array is int

Parameters:

• arr (ndarray) – input numpy array

• new_h (int | float, default: 255.0) – max value of output array

• type_normalization (str, default: 'int') – default dtype of output array

Return type:

ndarray

Returns:

numpy array

Module contents

various helpers functions