API

This package has the following modules:

build

This module contains functions for building the metadata file.

miagis_schema

This module contains the schema for validating user input.

validate

This module contains functions to validate the metadata file.

user_input_checking

This module contains functions to validate user input.

print_map_layers

This module contains the code to run the print_map_layers command.

user_input_checking

This module contains the functions used to read in and validate various inputs.

miagis.user_input_checking.additional_args_checks(args: dict)

Run some checks on args that jsonschema can’t do.

This assumes that args has been validated with a JSON schema and does some further checking to make sure the values entered by the user make sense. Prints a message and exits the program if problems are found.

Parameters:

args (dict) – the arguments entered into the program by the user.

miagis.user_input_checking.additional_json_schemas_checks(schema_list: list)

Check that all input schemas are valid JSON Schema.

If any schema are not valid JSON Schema then print a message and exit.

Parameters:

schema_list (list) – list of dictionaries of properties for JSON schemas.

miagis.user_input_checking.load_json(filepath: str) → dict

Adds error checking around loading a json file.

Parameters:

filepath (str) – filepath to the json file

Returns:

json read from file in a dictionary

Raises:

Exception – If file opening has a problem will raise an exception.

Return type:

dict

miagis.user_input_checking.read_in_resource_properties(resource_properties_path: str, exact_matching: bool) → Tuple[dict, dict]

Read in resource_properties and put it in expected dict form.

resource_properties can be csv, xlsx, or JSON, so if it is one of the tabular forms some of the fields have to be read in special.

Parameters:

• resource_properties_path (str) – filepath to the resource_properties file.

• exact_matching (bool) – if True resource names will not be modified. if False resource names are stripped, lowered, and spaces replaced with underscores.

Returns:

The final dictionary of resource_properties, and a mapping of the original resource name and new name.

Return type:

Tuple[dict, dict]

miagis.user_input_checking.validate_arbitrary_schema(dict_to_validate: dict, schema: dict)

Validate any arbitrary JSON Schema.

Wraps around jsonschema.validate() to give more human readable errors for most validation errors.

Parameters:

• dict_to_validate (dict) – instance to validate.

• schema (dict) – JSON Schema to validate the instance with.

Raises:

jsonschema.ValidationError – any validation errors that aren’t handled reraise the original.

build

This module contains the code to build the metadata file.

miagis.build.build(resource_properties_path: str, exact_matching: bool = False, remove_optional_fields: bool = True, add_resources: bool = True, overwrite_format: bool = False, overwrite_fairness: bool = False, base_metadata: dict | None = None, entry_version: int = 1, entry_id: str = '', base_description: str = '', products: list | None = None, schema_list: list | None = None) → dict

Build a metadata file from the input settings in the current directory.

Loop over files in the folders of the current directory, ignoring files in the top level of the current directory, and fill in what information can be about the file in the metadata. If resource_properties are provided then resources are matched to those in resource_properties and more information is able to be automatically put into the metadata.

Parameters:

• resource_properties_path (str) – filepath to a file containing file properties.

• exact_matching (bool) – if True file names are matched exactly. if False names are modified and matched fuzzy.

• remove_optional_fields (bool) – if True optional metadata fields that are empty or null are removed.

• add_resources (bool) – if True add resources that aren’t in the metadata after looping over the files to the metadata.

• overwrite_format (bool) – if True overwrite the default format determined from the file extension for files with what is in resource_properties.

• overwrite_fairness (bool) – if True overwrite the default value of “FAIR” for files with what is in resource_properties.

• base_metadata (dict | None) – update the metadata dict with this input dict.

• entry_version (int) – version number of the metadata.

• entry_id (str) – unique id for the metadata.

• base_description (str) – description of the metadata.

• products (list | None) – a list of resource ids.

• schema_list (list | None) – a list of dicitonaries with information about how to detect if a JSON file is using that schema, and how to decode that schema to fill in the metadata.

Returns:

The metadata JSON as a python dict.

Return type:

dict

miagis.build.determine_json_fields(schema_list: list, input_json: dict, file_path: str) → Tuple[dict, dict | str]

Determine the types of the feature properties in the JSON file.

Based on the ESRIJSON and GEOJSON formats there are 2 types of schema styles, mapping and testing. The mapping style has types already identified with the properties in the JSON, but the types are not the same as those defined in the MIAGIS schema, so a mapping between the types is necessary. This style’s dict entry looks like:

{“style”:”mapping”, “schema”:valid_jsonschema, “field_path”:path_to_fields,

“name_key”:key_to_name, “type_key”:key_to_type, “type_map”:type_mapping}

Actual ESRIJSON example:

{“style”:”mapping”, “schema”:miagis_schema.arcgis_schema,

“field_path”:’[“layers”][0][“layerDefinition”][“fields”]’, “name_key”:”name”, “type_key”:”type”, “type_map”:arcgis_type_map}

The testing style doesn’t have types listed directly in the JSON so each field’s type must be tested. This style’s dict entry looks like:

{“style”:”testing”, “schema”:valid_jsonschema,

“features_path”:path_to_features, “properties_key”:key_to_properties}

Actual GEOJSON example:

{“style”:”testing”, “schema”:miagis_schema.geojson_feature_schema,

“features_path”:””, “properties_key”:”properties”, “schema_URL”:”https://datatracker.ietf.org/doc/html/rfc7946”}

Note that the features_path is assumed to lead to a dict or list, and will print a warning if does is not.

Parameters:

• schema_list (list) – list of dictionaries where each dictionary describes a schema to look for and how to determine field types.

• input_json (dict) – the JSON to determine field types for.

• file_path (str) – path to the JSON file, used for printing error messages.

Returns:

A dictionary of the fields and their types. Ex field_name : {“name”:field_name, “type”:field_type}. If “schema_URL” is in the schema dictionary then this is returned, otherwise return the jsonschema.

Return type:

Tuple[dict, dict | str]

miagis.build.determine_table_fields(extension: str, path_to_read_file: Path) → dict

Read in the tabular file and determine the names and types of the columns.

Parameters:

• extension (str) – the extension of the file without the period, used to read in the file correctly.

• path_to_read_file (Path) – filepath to read in the file from.

Returns:

Keys are the field name or column number and values are the name, type, and column number for each field. Ex: field_name : {“name”:field_name, “type”:field_type, “identifier”:column_number+1, “identifier%type”:”column”}.

Return type:

dict

miagis.build.find_resource_properties(resource_properties: dict, resource_properties_keys: list, exact_matching: bool, filename_minus_extension: str, relative_location: str) → Tuple[dict, list, str]

Match filename to an entry in resource_properties and pull out properties.

The point of this function is to find the match in resource_properties and format all of the information so it can be easily added to metadata without more logic after returning from this function.

Parameters:

• resource_properties (dict) – dictionary where the keys are filenames and values are properties of the file to go in the metadata.

• resource_properties_keys (list) – list of the resource_properties_keys, it is given as an input so the list isn’t created every time the function runs.

• exact_matching (bool) – if True filename is matched as is, if False the filename is stripped, lowered, and spaces replaced with underscores before matching

• filename_mins_extension – the filename to match with the extension removed.

• relative_location (str) – location of the file relative to the current directory, input here so it can be added to alternate_locations.

• filename_minus_extension (str) –

Returns:

The properties of the matched filename from resource_properties. If alternate_locations is in resource_properties return it, else return an empty list, but always add the relative_location to the list. The resource_name in resource_properties that was matched to filename_minus_extension.

Return type:

Tuple[dict, list, str]

validate

This module contains the code to validate the metadata file.

miagis.validate.validate(metadata: dict)

Validate input metadata against the MIAGIS schema.

Use jsonschema to validate input metadata, and then do some additional checking that jsonschema alone cannot, such as checking that all layers for each map exist in the layers section.

The specific JSON schema used is the metadata_schema in the miagis_schema module.

Parameters:

metadata (dict) – input dictionary of metadata.

Raises:

jsonschema.ValidationError – any validation errors that aren’t handled reraise the original.

print_map_layers

This module contains the code to print and save map structure.

miagis.print_map_layers.print_map_layers(metadata: dict, save_path: str = '')

Find the maps and layers in metadata and pretty print them.

Parameters:

• metadata (dict) – A valid metadata to find maps in.

• save_path (str) – path to save the output to a file, doesn’t save if it is an empty string.