{
"cells": [
{
"cell_type": "markdown",
"id": "cb3afcaa",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/casangi/graphviper/blob/main/docs/graph_building_tutorial.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "ed122f09-ead5-4a22-a70b-7702d556e7fe",
"metadata": {},
"source": [
"# GraphVIPER Tutorial\n",
"\n",
"This tutorial provides examples of how `GraphVIPER` can be used to build [Dask](https://www.dask.org/) graphs by mapping a dictionary-based container of [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) to [Dask](https://www.dask.org/) graph nodes, followed by a reduction step. The dictionary of [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) used in this tutorial is referred to as a [Processing Set](https://github.com/casangi/xradio/blob/main/src/xradio/vis/_processing_set.py), although any dictionary containing [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) can be used. Using the `GraphVIPER` [map](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/map/index.html#module-contents) and [reduce](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/reduce/index.html#module-contents) functions can be thought of as a generalization of [xarray.map_blocks](https://docs.xarray.dev/en/stable/generated/xarray.map_blocks.html) that can be applied to more than one [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html). Both [map](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/map/index.html#module-contents) and [reduce](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/reduce/index.html#module-contents) build [Dask](https://www.dask.org/) graphs using [dask.delayed](https://docs.dask.org/en/stable/delayed.html).\n",
"\n",
"The following types of mapping are supported:\n",
"\n",
"- Partitions defined by any combination of the coordinates in the [Processing Set](https://github.com/casangi/xradio/blob/main/src/xradio/vis/_processing_set.py).\n",
"- More than one [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) can be assigned to a single mapping node.\n",
"- [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) partitions assigned to different nodes can have coordinates that overlap.\n",
"\n",
"The tutorial will cover the following examples:\n",
"\n",
"- Frequency Map Reduce: This example explains the concepts of `parallel_coords` and `node_task_data_mapping` that define parallelism.\n",
"- Overlapping Frequency Map Reduce.\n",
"- Baseline and Frequency Map Reduce.\n",
"- Time Map Reduce.\n",
"\n",
"`GraphVIPER` provides improvements over the [CNGI prototype](https://cngi-prototype.readthedocs.io/en/stable/development.html):\n",
"\n",
"- There is a clear separation between the concurrency layer ([GraphVIPER](https://graphviper.readthedocs.io/en/latest/)) and the domain layer (science code, [AstroVIPER](https://github.com/casangi/astroviper)).\n",
"- The memory backpressure issue was solved by incorporating the loading of data into the compute nodes. An example of the memory backpressure issue is cube imaging where large in-memory image cubes have to be created, which [Dask](https://www.dask.org/) is not aware of, causing [Dask](https://www.dask.org/) to be overeager in loading data from disk into memory. In the future, [Dask](https://www.dask.org/) might provide an alternative solution where graph nodes can be annotated with expected memory usage.\n",
"- The number of graph nodes has been minimized; this was also solved by incorporating the loading of data into the compute nodes. When [Xarray](https://docs.xarray.dev/) backed [Dask](https://www.dask.org/) datasets are used, a node is created for each data variable, and since Radio Astronomy datasets have numerous data variables, it led to a bloated graph that impacted scaling performance.\n",
"- Multiple [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) can be processed together with overlap. This cannot be done with the current [Xarray](https://docs.xarray.dev/) functionality, such as [xarray.map_blocks](https://docs.xarray.dev/en/stable/generated/xarray.map_blocks.html).\n",
"- Using a [Dask plugin](https://distributed.dask.org/en/latest/plugins.html), the [Dask Scheduler](https://docs.dask.org/en/stable/scheduler-overview.html) has been modified so that data can be cached to a local disk when multiple passes over larger-than-memory data have to be done. This reduces clustered file system or binary object store access (see [GraphVIPER Client](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/dask/client/index.html)).\n"
]
},
{
"cell_type": "markdown",
"id": "ffbf2b16",
"metadata": {},
"source": [
"## Install GraphVIPER"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "937bd155",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"GraphVIPER version 0.0.39 already installed.\n"
]
}
],
"source": [
"import os\n",
"import dask\n",
"import toolviper\n",
"\n",
"import numpy as np\n",
"\n",
"from importlib.metadata import version\n",
"\n",
"try:\n",
" import graphviper\n",
"\n",
" print(\"GraphVIPER version\", version(\"graphviper\"), \"already installed.\")\n",
"except ImportError as e:\n",
" print(e)\n",
" print(\"Installing GraphVIPER\")\n",
"\n",
" os.system(\"pip install graphviper\")\n",
"\n",
" import graphviper\n",
"\n",
" print(\"GraphVIPER version\", version(\"graphviper\"), \" installed.\")"
]
},
{
"cell_type": "markdown",
"id": "2682bb1c-c377-47f2-9faa-271108954d8f",
"metadata": {},
"source": [
"## Setup Dask Cluster\n",
"To simplify things we are going to start of by just using a single process (everything will run in serial)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f4e50179-3486-4454-9036-ff55d24346fc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-04-06 15:38:38,938\u001b[0m] \u001b[38;2;255;160;0m WARNING\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m It is recommended that the local cache directory be set using the \u001b[38;2;50;50;205mdask_local_dir\u001b[0m parameter. \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:38,938\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m Running client in synchronous mode. \n"
]
}
],
"source": [
"# Code to start a Dask cluster with two workers and 1 thread each.\n",
"from toolviper.dask.client import local_client\n",
"\n",
"# viper_client = local_client(cores=2, memory_limit=\"4GB\",autorestrictor=True)\n",
"viper_client = local_client(serial_execution=True)\n",
"viper_client"
]
},
{
"cell_type": "markdown",
"id": "e0aa8d3c-6fd9-474f-9223-6da8856ab948",
"metadata": {},
"source": [
"## Download and Convert Dataset"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d4f2b674-78e3-4dbb-9079-2b46825e1f3a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\u001b[38;2;128;05;128m2026-04-06 15:38:38,947\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m Initializing download... \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:38,948\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m File already exists: /Users/jsteeb/Dropbox/viper_dev/graphviper/docs/Antennae_North.cal.lsrk.split.ms \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:39,446\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m Updated partition scheme used: ['DATA_DESC_ID', 'OBS_MODE', 'OBSERVATION_ID'] \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:39,453\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m Number of partitions: 4 \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:39,453\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m OBSERVATION_ID [0], DDI [0], STATE [23, 24, 25, 30, 31, 32, 33, 34, 37], FIELD [0, 1, 2], SCAN [9, 17, 21, 25], EPHEMERIS [None] \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:39,803\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m OBSERVATION_ID [1], DDI [0], STATE [23, 24, 25, 30, 31, 32, 33, 34, 37], FIELD [0, 1, 2], SCAN [26, 34, 38, 42], EPHEMERIS [None] \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:40,021\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m OBSERVATION_ID [2], DDI [0], STATE [32, 33, 34], FIELD [0, 1, 2], SCAN [43], EPHEMERIS [None] \n",
"[\u001b[38;2;128;05;128m2026-04-06 15:38:40,298\u001b[0m] \u001b[38;2;50;50;205m INFO\u001b[0m\u001b[38;2;112;128;144m client: \u001b[0m OBSERVATION_ID [3], DDI [0], STATE [39, 40, 41, 46, 47, 48, 49, 50, 53], FIELD [0, 1, 2], SCAN [48, 56, 60, 64], EPHEMERIS [None] \n"
]
}
],
"source": [
"toolviper.utils.data.download(file=\"Antennae_North.cal.lsrk.split.ms\")\n",
"\n",
"from xradio.measurement_set.convert_msv2_to_processing_set import convert_msv2_to_processing_set\n",
"\n",
"# The chunksize on disk. Chunksize can be specified for any of the following dimensions :\n",
"# time, baseline_id (interferometer) / antenna_id (single dish), frequency, and polarization.\n",
"chunks_on_disk = {\"frequency\": 3}\n",
"infile = \"Antennae_North.cal.lsrk.split.ms\"\n",
"outfile = \"Antennae_North.cal.lsrk.split.ps.zarr\"\n",
"convert_msv2_to_processing_set(\n",
" in_file=infile,\n",
" out_file=outfile,\n",
" parallel_mode=\"none\",\n",
" persistence_mode=\"w\",\n",
" main_chunksize=chunks_on_disk,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "33a13d40-5654-4011-9467-d269fe601826",
"metadata": {},
"source": [
"## Inspect the Processing Set\n",
"\n",
"The [open_processing_set](https://github.com/casangi/xradio/blob/main/src/xradio/measurement_set/open_processing_set.py) is a lazy function, so no data is loaded into memory; only metadata is loaded (the [load_processing_set](https://github.com/casangi/xradio/blob/main/src/xradio/measurement_set/load_processing_set.py) will load everything into memory). Metadata is defined as everything that is not an [xarray.DataArray](https://docs.xarray.dev/en/stable/generated/xarray.DataArray.html). Note that a [Processing Set](https://github.com/casangi/xradio/blob/main/src/xradio/measurement_set/processing_set.py) does not have to be used with `GraphVIPER`, and any dictionary of [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) can be used."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "774b2011-8921-48fb-90dc-f56689862110",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
{'base': {'correlated_data': 'VISIBILITY', 'date': '2026-04-06T19:38:39.541311+00:00', 'description': "Data group derived from the data column 'VISIBILITY' of an MSv2 converted to MSv4", 'field_and_source': 'field_and_source_base_xds', 'flag': 'FLAG', 'uvw': 'UVW', 'weight': 'WEIGHT'}}
"
],
"text/plain": [
"\n",
"Group: /Antennae_North.cal.lsrk.split_0\n",
"│ Dimensions: (time: 50, baseline_id: 45, frequency: 8,\n",
"│ polarization: 2, uvw_label: 3)\n",
"│ Coordinates:\n",
"│ * time (time) float64 400B 1.307e+09 ... 1.307e+09\n",
"│ field_name (time) \n",
"│ scan_name (time) \n",
"│ * baseline_id (baseline_id) int64 360B 0 1 2 3 ... 41 42 43 44\n",
"│ baseline_antenna1_name (baseline_id) \n",
"│ baseline_antenna2_name (baseline_id) \n",
"│ * frequency (frequency) float64 64B 3.439e+11 ... 3.44e+11\n",
"│ * polarization (polarization) \n",
"│ FLAG (time, baseline_id, frequency, polarization) bool 36kB dask.array\n",
"│ TIME_CENTROID (time, baseline_id) float64 18kB dask.array\n",
"│ UVW (time, baseline_id, uvw_label) float64 54kB dask.array\n",
"│ VISIBILITY (time, baseline_id, frequency, polarization) complex64 288kB dask.array\n",
"│ WEIGHT (time, baseline_id, frequency, polarization) float32 144kB dask.array\n",
"│ Attributes:\n",
"│ creation_date: 2026-04-06T19:38:39.489741+00:00\n",
"│ creator: {'software_name': 'xradio', 'version': '1.1.3'}\n",
"│ data_groups: {'base': {'correlated_data': 'VISIBILITY', 'date': '20...\n",
"│ observation_info: {'execution_block_UID': 'uid://A002/X1ff7b0/Xb', 'obse...\n",
"│ processor_info: {'sub_type': 'ALMA_CORRELATOR_MODE', 'type': 'CORRELAT...\n",
"│ schema_version: 4.0.0\n",
"│ type: visibility\n",
"├── Group: /Antennae_North.cal.lsrk.split_0/antenna_xds\n",
"│ Dimensions: (antenna_name: 10, cartesian_pos_label: 3,\n",
"│ receptor_label: 2)\n",
"│ Coordinates:\n",
"│ * antenna_name (antenna_name) \n",
"│ station_name (antenna_name) \n",
"│ telescope_name (antenna_name) \n",
"│ * cartesian_pos_label (cartesian_pos_label) \n",
"│ Data variables:\n",
"│ ANTENNA_DISH_DIAMETER (antenna_name) float64 80B dask.array\n",
"│ ANTENNA_POSITION (antenna_name, cartesian_pos_label) float64 240B dask.array\n",
"│ ANTENNA_RECEPTOR_ANGLE (antenna_name, receptor_label) float64 160B dask.array\n",
"│ Attributes:\n",
"│ overall_telescope_name: ALMA\n",
"│ relocatable_antennas: True\n",
"│ type: antenna\n",
"├── Group: /Antennae_North.cal.lsrk.split_0/field_and_source_base_xds\n",
"│ Dimensions: (field_name: 3, sky_dir_label: 2)\n",
"│ Coordinates:\n",
"│ * field_name (field_name) \n",
"│ * sky_dir_label (sky_dir_label) \n",
"│ SOURCE_DIRECTION (field_name, sky_dir_label) float64 48B dask.array\n",
"│ Attributes:\n",
"│ type: field_and_source\n",
"└── Group: /Antennae_North.cal.lsrk.split_0/weather_xds\n",
" Dimensions: (station_name: 2, time_weather: 259,\n",
" cartesian_pos_label: 3)\n",
" Coordinates:\n",
" * station_name (station_name) \n",
" PRESSURE (station_name, time_weather) float64 4kB dask.array\n",
" REL_HUMIDITY (station_name, time_weather) float64 4kB dask.array\n",
" STATION_POSITION (station_name, cartesian_pos_label) float64 48B dask.array\n",
" TEMPERATURE (station_name, time_weather) float64 4kB dask.array\n",
" WIND_DIRECTION (station_name, time_weather) float64 4kB dask.array\n",
" WIND_SPEED (station_name, time_weather) float64 4kB dask.array\n",
" Attributes:\n",
" type: weather"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ms_xds = ps_xdt[\n",
" \"Antennae_North.cal.lsrk.split_0\"\n",
"]\n",
"\n",
"ms_xds"
]
},
{
"cell_type": "markdown",
"id": "9a2e880b",
"metadata": {},
"source": [
"## Nomenclature\n",
"\n",
"- `input_data`: A dictionary of [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) or a [processing_set](https://github.com/casangi/xradio/blob/main/src/xradio/vis/_processing_set.py).\n",
"- `n_datasets`: The number of [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html) in the input_data.\n",
"- `i_dim`: The $\\text{i}^{\\text{th}}$ dimension name.\n",
"- `n_dims`: The number of dimensions over which parallelism will occur.\n",
"- `n_dim_i_chunks`: Number of chunks into which the dimension coordinate `dim_i` has been divided.\n",
"- `n_nodes`: Number of nodes in the mapping stage of a MapReduce graph.\n",
"- `_{}`: If curly brackets are preceded by an underscore, it indicates a subscript and not a dictionary value."
]
},
{
"cell_type": "markdown",
"id": "aa7e11c7-18f8-4174-a713-a020a8f91273",
"metadata": {},
"source": [
"## How Graph Parallelism is Specified: ```parallel_coords```\n",
"\n",
"The `parallel_coords` is a dictionary where the keys are dimensions over which parallelism will occur and can be any of the dimension coordinate names present in the input data. For the `MS v4` [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html), the options include time, baseline_id (interferometer) / antenna_id (single dish), frequency, and polarization. Each dimension coordinate name is associated with a dictionary that describes the data selection for that dimension in each node of the mapping stage of the graph.\n",
"\n",
"The structure of the `parallel_coordinates`:\n",
"```\n",
" parallel_coords = {\n",
" dim_0: {\n",
" 'data': 1D list/np.ndarray of Number,\n",
" 'data_chunks': {\n",
" 0 : 1D list/np.ndarray of Number,\n",
" ⋮\n",
" n_dim_0_chunks-1 : ...,\n",
" }\n",
" 'data_chunk_edges': 1D list/np.ndarray of Number,\n",
" 'dims': (dim_0,), \n",
" 'attrs': measure attribute,\n",
" }\n",
" ⋮\n",
" dim_{n_dims-1}: ...\n",
" }\n",
"```\n",
"\n",
"The `dim_i` dictionaries keys have the following meanings:\n",
"\n",
"- `data`: An array containing all the coordinate values associated with that dimension. These values do not necessarily have to match the values in the coordinates of the input data, as those are interpolated onto these values. The minimum and maximum values can be respectively larger or smaller than the values in the coordinates of individual [xarray.Datasets](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html); this will simply exclude that data from being processed. It's important to note that the `parallel_coords` and the input data coordinates must have the same measures attributes (reference frame, units, etc.).\n",
"- `data_chunks`: A dictionary where the values are chunks of the data and the keys are integers. This chunking determines the parallelism of the graph. The values in the chunks can overlap.\n",
"- `data_chunks_edges`: An array with the start and end values of each chunk.\n",
"- `dims`: The dimension coordinate name.\n",
"- `attrs`: The `XRADIO` measures attributes of the data (refer to [XRADIO documentation](https://docs.google.com/spreadsheets/d/14a6qMap9M5r_vjpLnaBKxsR9TF4azN5LVdOxLacOX-s/edit#gid=1504318014)).\n",
"\n",
"The combinations of all the chunks in `parallel_coords` determine the parallelism of the graph. For example, if you have `parallel_coords` with 5 `time` and 3 `frequency` chunks, you would have 15-way parallelism (5x3).\n",
"\n",
"This description may seem somewhat convoluted, but the following examples should help clarify things."
]
},
{
"cell_type": "markdown",
"id": "86fa2f48",
"metadata": {},
"source": [
"## Frequency Map Reduce"
]
},
{
"cell_type": "markdown",
"id": "33094606-edf4-414b-800c-589146c4a55a",
"metadata": {},
"source": [
"### Create Parallel Coordinates\n",
"\n",
"GraphVIPER offers a convenient function, [make_parallel_coord](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/coordinate_utils/index.html#graphviper.graph_tools.coordinate_utils.make_parallel_coord), that converts any [XRADIO measures](https://docs.google.com/spreadsheets/d/14a6qMap9M5r_vjpLnaBKxsR9TF4azN5LVdOxLacOX-s/edit#gid=1504318014) to a `parallel_coord`. In this case, we will use the frequency coordinate of one of the datasets in the [processing_set](https://github.com/casangi/xradio/blob/main/src/xradio/vis/_processing_set.py). It's worth noting that all datasets in this [processing_set](https://github.com/casangi/xradio/blob/main/src/xradio/vis/_processing_set.py) have the same frequency coordinates but differing time coordinates. This is the case because they represent the same spectral window but different fields in a Mosaic observation."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "a33a781e-9b81-46c6-9c16-7510d0f4eae3",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
],
"text/plain": [
""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from graphviper.graph_tools.coordinate_utils import make_parallel_coord\n",
"\n",
"parallel_coords = {}\n",
"n_chunks = 3\n",
"parallel_coords[\"frequency\"] = make_parallel_coord(\n",
" coord=ms_xds.frequency, n_chunks=n_chunks\n",
")\n",
"\n",
"toolviper.utils.display.DataDict.html(parallel_coords[\"frequency\"])"
]
},
{
"cell_type": "markdown",
"id": "a679dd6a",
"metadata": {},
"source": [
"The display of the frequency `parallel_coords` clearly shows how the data was split into 3 chunks. All the chunks must have the same number of values, except the last chunk, which can have fewer. `GraphVIPER` also has a convenience functions that can create [frequency](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/coordinate_utils/index.html#graphviper.graph_tools.coordinate_utils.make_frequency_coord) and [time](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/coordinate_utils/index.html#graphviper.graph_tools.coordinate_utils.make_time_coord) coordinate measures:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "078f9152-a57d-4c26-bd32-7736dffa850a",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"
"
],
"text/plain": [
""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from graphviper.graph_tools.coordinate_utils import make_frequency_coord\n",
"\n",
"n_chunks = 3\n",
"\n",
"coord = make_frequency_coord(\n",
" freq_start=343928096685.9587,\n",
" freq_delta=11231488.981445312,\n",
" n_channels=8,\n",
" velocity_frame=\"lsrk\",\n",
")\n",
"parallel_coords[\"frequency\"] = make_parallel_coord(coord=coord, n_chunks=n_chunks)\n",
"toolviper.utils.display.DataDict.html(parallel_coords[\"frequency\"])"
]
},
{
"cell_type": "markdown",
"id": "f3b99bcd",
"metadata": {},
"source": [
"### Create Node Task Data Mapping\n",
"\n",
"Now, the coordinates in the input data must be mapped onto the `parallel_coords`. This is achieved using the `interpolate_data_coords_onto_parallel_coords` function, which produces the `node_task_data_mapping`. It is a dictionary where each key is a node id of one of the nodes in the mapping stage of the graph.\n",
"\n",
"Structure of node_task_data_mapping:\n",
"```\n",
" node_task_data_mapping = {\n",
" 0 : {\n",
" 'chunk_indices': tuple of int,\n",
" 'parallel_dims': (dim_0, ..., dim_{n_dims-1}),\n",
" 'data_selection': {\n",
" dataset_name_0: {\n",
" dim_0: slice,\n",
" ⋮\n",
" dim_(n_dims-1): slice\n",
" }\n",
" ⋮\n",
" dataset_name_{n_dataset-1}: ...\n",
" }\n",
" 'task_coords': \n",
" dim_0:{\n",
" 'data': list/np.ndarray of Number,\n",
" 'dims': str,\n",
" 'attrs': measure attribute,\n",
" }\n",
" ⋮\n",
" dim_(n_dims-1): ...\n",
" }\n",
" ⋮\n",
" n_nodes-1 : ...\n",
" }\n",
"```\n",
"\n",
"Each `node_id` dictionary has the keys with the following meaning:\n",
"\n",
"- `chunk_indices`: The indices assigned to the data chunks in the `parallel_coords`. There must be an index for each `parallel_dims`.\n",
"- `parallel_dims`: The dimension coordinates over which parallelism will occur.\n",
"- `data_selection`: A dictionary where the keys are the names of the datasets in the `processing_set`, and the values are dictionaries with the coordinates and accompanying slices. If a coordinate is not included, all values will be selected.\n",
"- `task_coords`: The chunk of the parallel_coord that is assigned to this node."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4ffb8a9b",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
],
"text/plain": [
""
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from graphviper.graph_tools.coordinate_utils import (\n",
" interpolate_data_coords_onto_parallel_coords,\n",
")\n",
"\n",
"node_task_data_mapping = interpolate_data_coords_onto_parallel_coords(\n",
" parallel_coords, ps_xdt\n",
")\n",
"\n",
"toolviper.utils.display.DataDict.html(node_task_data_mapping)"
]
},
{
"cell_type": "markdown",
"id": "bbbea156-e5e0-45eb-8a40-ca1f11c97e1b",
"metadata": {},
"source": [
"### Create a chunk function and map graph\n",
"\n",
"The [map](https://graphviper.readthedocs.io/en/latest/_api/autoapi/graphviper/graph_tools/map/index.html#graphviper.graph_tools.map.map) function combines a `node_task_data_mapping` and a `node_task` to create the map portion of the graph. The `node_task` must be a function with a single dictionary input and a single output as is the `my_func` in the example below. The `map` function will pass the `input_params` dictionary to the `node_task` and add the following items from the `node_task_data_mapping`:\n",
"\n",
"- chunk_indices\n",
"- parallel_dims\n",
"- data_selection\n",
"- task_coords\n",
"- task_id\n",
"\n",
"If local caching is enabled the following will also be included with the `input_params` dictionary:\n",
"\n",
"- date_time\n",
"- viper_local_dir"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "93a5da8a-6b40-4912-903b-e9c9ee196fbd",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAFACAYAAABqYdEUAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3dd1hUZ9o/8PvMMDB0kCYo0gRFBWtQo0ZBjV2j2dgx0Y3GTYzJvjExyZuNm91k45tNsiZuosmm0RSFWIMVKWJDQQWxgCIoUpQi0pl2//7wx6yEIsg855lh7s918YfDMM9XeTzfM2fOeY6AiAiEEEIIO8clvBMQQgjp/qhsCCGEMEdlQwghhDkTMQc7d+4c5OXliTmkwXF1dYVx48bxjqE3SkpK4Pjx47xj6L158+aBiYmo/5312q5du0ClUvGOodeCgoLA09NTvAFRRC+++CICAH218zVt2jQxfyV6Ly4ujvvvxBC+qquref+q9IqFhQX334m+f/30009i/kqSRT+MNm3aNEBE+mrl68UXXxT712Ewqquruf9+9PErLi6O969Gb/3000/cfz/6+mVhYSH674M+syGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzFHZEEIIYY7KhhBCCHNUNoQQQpijsiGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzFHZEEIIYY7KhhBCCHNUNoQQQpijsiGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzFHZEEIIYY7KhhBCCHNUNoQQQpijsiGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzFHZEEIIYY7KhhBCCHNUNoQQQpijsiGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzFHZEEIIYY7KhhBCCHNUNoQQQpijsiGEEMIclQ0hhBDmqGwIIYQwR2VDCCGEOSobQgghzJmIPWBJSQns3LlT7GENQn5+PlhYWPCOoZd2794NZmZmvGPonQsXLvCOoLfOnTsHlpaWvGPoJbVaLfqYopfNhQsXYMGCBWIPazCmTZvGO4JeWrZsGe8IxMBs2bIFtmzZwjsG+f8ERETeIXg5fPgwlJaWwtKlS3lHIQZCpVLBn/70J/jPf/7DOwoxIGvWrIGPP/4Y7OzseEfh5bhRf2azfft2iIyM5B2DGJCjR49CeHg43L9/n3cUYiAUCgVs27YN9uzZwzsKV0ZbNg0NDRAbGwvx8fFw79493nGIgYiKigKFQgGxsbG8oxADcfDgQbh//z5ERETwjsKV0ZbNgQMHoK6uDgAAdu3axTkNMQR1dXXauRIeHs45DTEU27ZtA0EQICkpCe7evcs7DjdGWzaRkZEglUoBEWnDQTpk//790NDQAAAAJ0+ehMLCQs6JiL6rra2Fffv2ASKCRCIx6jNxjbJsqqqqIC4uDlQqFWg0Gjhz5gzk5+fzjkX0XFRUFEilUgAAMDExgR07dnBORPTdnj17QKFQAMDD042NecfWKMtm165doFKptH82MTGBmJgYjomIvqusrIRDhw5p541KpYKwsDDOqYi+i4yMBInk4WYWESEtLQ1u3LjBORUfRlk2ERERIAiC9s9KpZI2HKRdsbGxoNFotH9GRMjMzIScnByOqYg+q6iogPj4+GY7tjKZzGgPpRld2ZSWlkJycnKLK2gvX74MV65c4ZSK6LvWziQyNTWF6OhoDmmIIdi5cyf8/jJGpVIJP//8M6dEfBld2ezYsaPZu5omMpmMjsGTVhUXF8OJEyda7KAoFAp6R0zaFBER0aJsAABu3LgBly5d4pCIL6Mrm/Dw8FbXBWra4zDiBRVIG6Kjo7XH3X/v5s2bcP78eZETEX1XVFQEp0+fbnbotYmpqSls376dQyq+jKpsbt++DWlpaW0WSkFBAaSlpYmciui7tnZQAIx3w0Hat23bNu2Zi7+nUCjgl19+MbodW6Mqm6ioKDAxaXvtUZlMRhsO0kxubi5kZGS0uWFoOpTW2h4sMV7t7aAAPDw0e/r0aRET8WdUZRMeHg5KpbLN7yuVSoiIiOCy/DbRT9u2bWt3BwXg4UknJ06cECkR0XfXrl2DS5cutfvOxRh3bI2mbC5fvgzXrl0DqVQKMpkMZDIZmJiYgImJifbPUqkUysrKIDk5mXdcoiciIiJApVK1OWdkMhkIgmB0Gw7StqYzFB+dJ7+fN2q1GrZt29bstOjuTvT72fBSVFQEq1atavZYUlISNDQ0wNSpU5s9XllZKWY0oqcqKiogODgYgoODtY/dvn0bDh06BMuXLweZTKZ93N7enkdEoofkcnmLbc1PP/0EY8aMgX79+jV7/M6dO+Dp6SliOn6M+n42y5cvh3v37kFcXBzvKMRAHDhwAGbMmAHV1dVgZWXFOw4xEFZWVrB582ZYvnw57yi8GPf9bAghhIiDyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5kx4BxBLeXm59qu2thYUCgVIpVKwtbWF3377DeRyOVhYWICDg4P2SxAE3rEJR7W1tc3mDSJCVlYWjB49Gg4cOAA9evQAExOTZnPGzMyMd2zCkUKh0M6XiooKUCgUUFNTA8OGDYNbt27BkSNHQCKRgJ2dHTg6OoKDgwNYW1vzji0KARGRdwhdUSgUkJmZCRkZGZCZmQk5OTmQn58P+fn50NDQ0KnXMjMzgz59+oCnpyf4+flBYGAgBAYGwpAhQ0AulzP6GxAecnNz4cKFC3Dp0iW4fPky5OXlQX5+PlRUVHTqdQRBAFdXV/Dy8gJvb28YNGgQDB48GIYOHQrOzs6M0hMeKioq4Pz585CZmQlZWVlw/fp1yM/Ph6KiItBoNJ16LVtbW/D09AQvLy8YMGAADB48GAIDA6Ffv37daYf3uEGXjUqlgvT0dEhMTITExEQ4efIk1NbWgoWFBQwcOBAGDBgAXl5e4OnpCb169QJHR0dwdHQES0tLMDU1BUtLSwAAqK+vh4aGBqirq4Py8nIoKyuDwsJCbVFdvXoVsrKyoLq6GuRyOYwePRqCg4MhODgYRo4cCTKZjPO/BOmMvLw87ZxJSEiAoqIikEql4OPjAwEBAeDt7Q2enp7g4eEBDg4O2j1QQRDAzs4OAB7OverqalCpVFBWVgbl5eVw79497ZzJycmBrKwsKCwsBACAgQMHQkhICAQHB8P48eOhR48ePP8JSCdVVVVBcnKydt5kZmaCRqMBFxcXCAwMBF9fX+22xsXFRftO19TUFKysrLTbiAcPHoBGo4H79+9DWVkZlJWVwe3btyE/Px/y8vIgKysLcnJyQKVSgbOzs3Y7ExwcDH5+fpz/FbrE8Mqmvr4edu/eDTt27ICkpCSoqqoCV1dX7S9k3Lhx4OvrCxKJbj+OQkS4efMmnDx5EhISEiAhIQEKCgrA0tISxo0bBwsWLIDnn3/eaN4SGxJEhNOnT0NkZCQcPHgQ8vPzwcLCAp5++mntvBk8eDBYWFjofOyKigpITU3VFtvFixcBEWHw4MEwZ84cWLp0Kfj4+Oh8XNJ1BQUFEBkZCXv27IH09HTQaDQQEBAAISEhEBISAiNHjmTyjrWxsREyMzMhOTkZEhISICUlBWpqaqBXr14wdepUWLJkCYwfP17n2zjGjgMaALVajUlJSbhixQq0sbFBExMTnDFjBn7zzTd45coVbrmuX7+O33//Pc6dOxfNzMzQwsIClyxZgocOHUKVSsUtF3no5s2b+NFHH2Hfvn0RAHDQoEG4YcMGTE5OxoaGBi6ZKioqcM+ePbhmzRrs2bMnCoKAY8eOxe+++w7v37/PJRP5r+rqavzll18wJCQEJRIJOjg44CuvvIIxMTFYWlrKJZNSqcSTJ0/ixx9/jMOHD0cAwD59+uD777+PV69e5ZLpCSTrddlUVVXhxo0b0cPDAwEAhw0bhps2bcK7d+/yjtZCRUUFbtmyBZ9++mkEAHR1dcUPP/wQy8rKeEczKhqNBvfu3YvPPPMMCoKALi4u+Oabb+L58+d5R2tBpVLhgQMHcNGiRWhubo5yuRwXLVqEFy5c4B3N6Fy9ehWXL1+OlpaWaGZmhnPnzsXdu3djY2Mj72gtXL58Gd99913s3bs3AgAGBQXh9u3b9X0HVz/LpqysDD/88EO0t7dHGxsbfOuttzArK4t3rA67fv06fvDBB+jg4IBWVla4bt06LC4u5h2rW1OpVLh9+3YMCAhAQRBw9uzZGBcXh0qlkne0Dnnw4AH++OOPOGTIEBQEAWfOnImnTp3iHavbu3DhAr7wwgsokUjQ398fv/nmGywvL+cdq0PUajXGx8fjggULUCqVop+fH/7000+oUCh4R2uNfpVNSUkJrlu3Dq2srNDBwQH/9re/YUVFBe9YT6y6uho///xzdHV1Rblcjq+99hrevn2bd6xuRalU4o8//oi+vr4olUpx0aJFmJmZyTvWE9NoNPjbb79p3yGHhIRgYmIi71jdzunTp3HGjBkoCAIOHToUY2JiUK1W8471xHJycnDFihVoamqKHh4e+O9//5vboeI26EfZqNVqDAsLQwcHB3R2dsYNGzZgZWUl71g609jYiGFhYdi3b180NzfHDRs26NtEMEjnzp3Dp556CmUyGYaGhuK1a9d4R9KplJQUnDlzJgIAzpw5E/Pz83lHMnjl5eW4atUqFAQBn376ady3bx9qNBresXTm1q1buHbtWrSwsMC+ffviwYMHeUdqwr9s0tPTMSgoCGUyGa5duxarq6t5R2JGoVDgpk2b0MrKCn19ffHw4cO8IxmkiooKXLt2LUokEgwODuZ6kogYEhIS0N/fHy0sLHDDhg16+TmCvmvaoXV0dMRevXphWFgY70hM3blzB0NDQ7U7Krdu3eIdiV/ZVFRU4KpVq1AikeDEiRO73V5pe/Lz83HOnDkIALh48WIsKSnhHckgaDQa/P7777FHjx7Yu3dv3LlzJ+9IomloaMC//e1vaG5ujgMHDsTjx4/zjmQw0tLScMSIESiTyXD9+vVYU1PDO5JoDhw4gD4+PmhtbY1ffvklz5MI+JTNmTNn0MPDA93c3HD79u08IuiF/fv3o5eXF7q4uGB8fDzvOHqtvLwcZ8+ejSYmJrhu3bpu/Q64Pbm5uThjxgyUSqW4YcMGfT8DiSuNRoNffPEFymQynDBhQrd/B9yW+vp63LBhA8rlcgwJCeF1spK4ZaPRaHDTpk1oamqKkyZNoj16fHh69+LFi1EQBFy/fj1tPFpx7tw59Pb2xt69e2NKSgrvOHohLCwMLSwscPz48VhYWMg7jt6prKzEP/zhD2hiYoIbNmww6A//deXy5cs4YMAAdHJywkOHDok9vHhlQ7/89tHGoyXaOWnf5cuXceDAgejk5KRPHwRzd/bsWfTy8sLevXvjiRMneMfRK4/u3K5du1bMSwPEKZurV6+ip6cnuru748mTJ8UY0iBdvHgR/fz80MXFBc+dO8c7Dlf19fX4hz/8AWUyGX722Wfd6owhXaqqqsJFixahRCLBzz//nHcc7n744QeUyWQ4c+ZMuqC6HVu3bkW5XI7PPvssVlVViTEk+7I5c+YMOjo64pgxY+iX3wFVVVU4depUtLa2xqNHj/KOw0VlZSVOmDAB7e3tMTk5mXccg/Dll1+iRCLBdevWGW0xf/LJJygIAn744YdG+2/QGenp6dizZ08cMWKEGKuysC2b+Ph4tLa2xpkzZ2JtbS3LoboVpVKpvUDL2E6gKCkpwaFDh2LPnj1p2ZZO+vXXX9HMzAyXLl2qr1eRM6HRaHDdunUoCAJ++eWXvOMYlJs3b6Kvry96e3vj9evXWQ7FrmyioqJQJpMZ3cTXFY1Gg2+//TYKgmA0h0dEnPjd1rFjx9Da2honTZok1uERrhobG3HRokVoamqK0dHRvOMYpJKSEhw2bBjrHTw2ZRMZGWn0b+l1ZePGjSgIAm7atIl3FKby8/OxV69eOGLECLx37x7vOAYtNTUVHR0dMSQkpFuvVKFSqfD5559Ha2trunSgix48eIDBwcHYo0cPVutQ6r5s4uPj0czMDN9++21dv7TR+uyzz1AQBPzll194R2GitLQU+/fvjwEBAQa9Fp4+yczMRHt7e5wzZ063PZ3+9ddfR7lcTp/r6UhdXR0+88wz6ObmxmJpJN2WzdmzZ9HKygqXLl1K72h0bN26dSiTybrdKa61tbU4evRo9Pb2xqKiIt5xupWkpCSUy+W4evVq3lF07i9/+QtKpVKMjY3lHaVbefDgAQ4ZMgR9fX11fdKA7somJycHnZ2dccaMGQazrLsh0Wg0uHz5crSwsOg2S88rFAqcOnUqOjk5GdVyRWLau3cvmpiY4EcffcQ7is5s2bIFBUHAH374gXeUbqmwsBC9vLzwqaee0uVKHbopm/LycvT09MQxY8bQWWcMKRQKnD59Ojo7O2NBQQHvOF3WdOfV9PR03lG6te+++w4FQcDIyEjeUbosLi4OJRIJ/uMf/+AdpVvLzs5GJycnfO6553R1lKrrZaPRaHDWrFno7u5O19GIoKamBv39/XHs2LEG/Q4yLCwMBUHA/fv3845iFN566y20srIypNsIt3D79m10cHDAl156iXcUo5CSkoImJia6Op2862Xz+eefo4mJCS0LIaKsrCy0sLDA9957j3eUJ5KdnY3W1tZ0EomIlEolPv300zho0CCDPPqgVCpx7Nix6OfnZxSndOuLTz/9FGUymS4O3XetbFJTU9HU1BT/7//+r6tBSCf98MMPKAgC7tu3j3eUTqmvr8fBgwdjUFAQ3ZdFZE3vDFauXMk7SqetX78e5XI5Xrx4kXcUo6LRaHD27NnYp0+frh65evKyqaioQA8PD5wxYwadecZJaGgoOjk5GdTCnS+//DL26NFDH27mZJT27t2LgiBgVFQU7ygddvDgQZRIJPjTTz/xjmKUSktLsXfv3jh37tyuvMyTl82qVavQ1dWVPqfhqKamBn19fXHevHm8o3TI0aNHEQBw9+7dvKMYtTfeeAN79OhhEBfPVldXo7u7Oy5ZsoR3FKN2/PhxlEgkXVml4cnK5uzZsyiRSIxu3S591LQB/+2333hHaVdjYyP279/fYIqxO6utrUUPDw9csWIF7yiP9fbbb6O9vb0YC0WSx1i5ciX27NkTKysrn+THO182arUag4KCcPz48XT4TE/Mnz8ffXx8sL6+nneUNn388cdoYWGBeXl5vKMQRIyJiUFBEDApKYl3lDZdvnwZZTIZbt26lXcUgg8vcXFycsL/+Z//eZIf73zZfP3112hqamq0t1jVR8XFxWhra4t//etfeUdp1a1bt9DS0hI3btzIOwp5xPTp03HQoEF6uVCuRqPB4OBgHDFiRLddbscQ/ec//0ETE5MnOVGjc2VTUlKCtra2+P7773d2IMLYF198gXK5HHNzc3lHaWH27Nk4YMAAOvtMz+Tk5KCZmZleLvIaERGBUqkU09LSeEchj1Cr1Th69GgcO3ZsZ3+0c2Wzbt06dHV1Ncjz9Ls7pVKJ/fr1w+XLl/OO0szZs2cRAHjc85x0wPr169HZ2Vmv/k8rlUr08fHBl19+mXcU0opz586hIAidXaex42VTXl6O1tbWRnNvFUP0888/o0wm06vPRebMmYNBQUG8Y5A2lJWVoZWVFX711Ve8o2hFRkaiVCrFnJwc3lFIG6ZNm4ajR4/uzI90vGw+/PBDdHBwoKt39ZhCoUBPT0987bXXeEdBxIcf8EokEty7dy/vKKQdb7zxBvbu3Vsv7n2j0Whw0KBBGBoayjsKacfp06cRADpzgknHyqaqqgrt7e3x73//+5OnI6L45ptv0MzMTC8u9Fy0aBEOHDgQ1Wo17yikHUVFRSiXy/H777/nHQV37dqFgiDgpUuXeEchjzFhwgScPHlyR5/esbLZuHEj2tjY0I2tDEB9fT26urriW2+9xTXH9evXUSqV0rVYBmLVqlXo7e3NfXHXESNG0LVYBqLpGr/U1NSOPD1ZAo+BiLB161ZYtWoV2NvbP+7phDO5XA5vvPEG/PTTT9DY2Mgtx3fffQd9+vSBF154gVsG0nHvvPMO5OXlweHDh7llOHv2LKSlpcE777zDLQPpuEmTJsHw4cPh22+/7dDzH1s2ycnJkJ+fDy+++GKXwxFxhIaGQlVVFfz2229cxtdoNLB9+3ZYtmwZSKVSLhlI5/j4+MDYsWMhIiKCW4bw8HDw9/eHkSNHcstAOufFF1+E2NhYqKmpeexzH1s2ERERMHz4cBg0aJBOwhH23NzcYOLEidw2HEeOHIHCwkJYvHgxl/HJkwkNDYW9e/dCZWWl6GMrFArYsWMHLFu2TPSxyZNbvHgxKJVK2Lt372Of227Z1NfXw6+//gqhoaE6C0fEERoaCgcOHIDS0lLRx46IiIAxY8aAn5+f6GOTJ7dgwQIQBAFiY2NFHzsuLg4qKipgyZIloo9NnpyDgwNMmTKlQzu27ZbN7t27oba2FhYuXKizcEQcc+fOBblcDjt27BB13KqqKtizZw/toBggGxsbmDVrFpd3xBERERASEgLu7u6ij026JjQ0FI4ePQqFhYXtPq/dstmxYwdMmTIFXFxcdBqOsGdpaQlz586F6OhoUcfdt28fqNVqmD9/vqjjEt0IDQ2FlJSUx244dKm6uhri4uJg6dKloo1JdGfWrFlgY2Pz2HfEbZaNSqWCpKQkmD59us7DEXFMmzYNUlNToaqqSrQxjx07BqNHj6YzFw1USEgImJqaQkJCgmhjHj9+HJRKJUydOlW0MYnuyOVyCAkJeeycabNs0tPToaqqCkJCQnQejogjJCQE1Go1nDhxQrQxExMTITg4WLTxiG5ZWFjAqFGjIDExUbQxExMTYeDAgXQExYAFBwdDcnIyqNXqNp/TZtkkJCSAq6sr9OvXj0k4wp6zszMMHDhQtA1Hbm4u3Lp1i8rGwAUHB8OxY8dEGy8xMZF2ag1cSEgIPHjwANLT09t8Tptl0zQBBEFgEo6IoyNvb3UlMTERLCwsICgoSJTxCBshISFw+/ZtyM3NZT5WZWUlZGRk0A6KgRswYAC4ubm1u61ptWwUCgWcOnWKJkA3EBwcDBcvXoSysjLmYyUmJsLYsWPBzMyM+ViEnZEjR4KlpaUoOymJiYmAiDB+/HjmYxG2JkyY0O5RlFbLJjMzE2pra2Hs2LHMghFxPPPMM6DRaODcuXPMxzp9+jSMGzeO+TiELVNTUxg1ahScPn2a+VhnzpyBQYMG0Qkl3cC4cePgzJkzgIitfr/Vsrl48SJYWlqCr68v03CEvR49eoC7uztkZmYyHaeqqgry8/Nh8ODBTMch4hg8eDDzOQMAkJGRAUOGDGE+DmFv8ODB2u1Aa1otm0uXLsGgQYNAInnsajbEAAQEBMClS5eYjpGZmQmICIGBgUzHIeIICAiAK1eutHt2kS5kZmZCQEAA0zGIOAICAkAQhDZ3Ulptk+zsbOjfvz/TYEQ8/v7+cO3aNaZj5OTkgIWFBfTp04fpOEQc/v7+UF9fD7dv32Y2RnV1NRQXF4O/vz+zMYh4rKyswN3dHbKzs1v9fqtlk5+fD15eXkyDEfF4enq2+dZWV5rmDJ292D14enoCADCdN3l5eQAAtK3pRtrb1rQoG0SEW7duaScbMXxeXl5QXl7OdCWBvLw8mjPdiIuLC1haWmoLgYWmjZKHhwezMYi4PD0925wzLcrm/v370NDQAL169WIeTF/cu3cPLCwsQBCEbnkrhd69ewMAQHFxMbMxiouLwc3Njdnr66PuPm969eoFRUVFzF6/uLgY7O3twdLSktkY+qa7z5nevXu3uZ1pUTZN12M4OTmxTaVHTp48CfX19fDpp59CVlYW7zg65+joCAAA5eXlzMYoKyszqjkD0P3njYODA/M50zQ3jUV3nzOOjo5tXtPXomyaJpeDgwPbVHrk/v37AAAwbNgwUcZDxDbPRWeh6XfJcsNRXl5uVHMGoPvPG0dHR+ZzxtjKprvPmfZ2UFqUTdPtPa2trdmmgoe3FL1x4wa8/PLL4O7uDiEhIRAZGQkAAF9++SUMHz4cnJ2dYdq0aXD9+nXtz3344Ycwbtw4uHnzZquvOWXKFFCpVB3K8O6778K//vUvAAD44IMP4KWXXgIAgGXLlrW65PnGjRth3Lhx2tdfuXIlrFmzBoqKimDx4sXg4eEBPj4+sGLFCqitrW32sxkZGTBp0iSws7MDCwsLGDlyJBw8eLBDObtCLpeDqakpVFdXMxujtrZWlDmzcuVKmjcizRsbGxumc6ampgasrKyYvX4TmjPizpmGhgZQKpUtvteibBobGwHg4VXErB05cgTGjx8Pp0+fhuDgYDh58iQsW7YMpk+fDuvXr4fevXvD2LFjISEhASZNmgQajQYAAPr37w8nTpyAmJiYZq9369YtCA8PB3t7ezAxMelQBjc3N3B2dgYAAHd3d+2Hlenp6a0uKnf9+nU4ceKENsvFixchLi4OgoKCoKCgABYuXAju7u7w888/N7vFbVJSEowaNQquXbsGf/zjH2Hx4sWQnZ0Ns2fPhlOnTnX+H6+TTE1Ntb9bFhoaGkSZMxcvXqR5I9K8YT1nGhsbRVnaiOaMeHOm6fepUChafhN/JzY2FgEAVSrV77+lcwCAH3/8sfbPBw4cQABAc3NzzM7O1j7+4osvIgBoH6upqUErKyscMWJEs9f74osvEABw//79ncrx448/IgBgSkqK9rEBAwZg//79Wzx3xYoVCADY2NiIiIgjRoxAAMD169ejRqNBRES1Wo3Dhg1DW1tb7Z8HDx6Mtra2eP36de1rXb16FQVBwCVLlnQq75NwcHDALVu2MHt9qVSK27ZtY/b6TZr+vWnesJ83q1evxpCQEGavv3DhQpw3bx6z129Cc0a8ORMfH48AgGUTuS0AAB/eSURBVOXl5b//VjLXJQKkUim8/fbb2j83LXUSEhLS7P71EyZMAACAK1euAMB/70KZlpbW7JzumJgYcHR0hClTprAP/whzc3P461//qr3GRCKRwJgxY+DBgwdw584duHDhAmRkZMBzzz0Hffv21f5c//794euvvxZllWRBEJgfuxXrGhuaN+LMGzHmjFhozog3ZwBA+27sUS3KpulQSGvH3HTNzc2t2aEXuVyuffxRUqkUAJq/NWs6xtl0K9KCggJITU2FBQsWgEwmY5r795ydnbXZmzQtLFhTUwM3btwAAGh1WY41a9bA2rVrmWdsaGhgesjC1NS09bfODNC8EWfeNDQ0tMinS6ampqJsZwBozgCIM2eaDru2Nm9alE27x9x0rK3z6zuyJtvEiROhZ8+e2gkQGxsLiAhLlizRacbfq6ioaPGYubl5m89HRCgtLQUA4HrtkkKhYL7hEKtsaN6IQ6FQdJsdFJoz4uhU2TSdHcLyLBRdkEqlsHDhQjh79iwUFBRATEwM+Pj4wOjRo3Xy+oIgtPpWsK11f9rTdGV9ampqi++Fh4fDL7/80unX7IzGxkZQKBRML56ztrbW+zkDQPOmM6qrq5meLWZlZcV0VQtdoTnTcTU1NWBmZtbqSRMtyqbpWgkxbrbVVUuXLgVEhE2bNsGZM2cgNDRUZ6/dtMbPo2/zL1++rH2b2hlPPfUUmJubt7gZ1ZUrV+Cll16C5OTkLudtT9PvkuU1DawvANQlmjcdw/qiS5ozD3WnOVNaWtrmnGlRNk1PNISyGT58OPTv3x82bdoEANDs9L+uGjlyJCgUCnjppZcgKSkJfvjhB3juuefA1ta206/l4uICb775JmRmZsLq1ashLS0NwsPDYdGiRWBiYgKrV6/WWe7WiFU2hjBnAGjedFRZWRnTC3Xbu9pc39Cc6Zj2Lu5u8V7H3t4ezMzMmK6JpEtLly6FDz74AJ599lmdrh771ltvwenTp2Hbtm2wbds26NWrl3ZvZuPGjZ1+vb///e+AiPDPf/4TvvvuOwAAcHV1haioKBg5cqTOcremaa0iFxcXZmO4uroazJwBoHnTEcXFxUznTM+ePeH+/ftQX1/f7mcR+oLmzOMVFRVBz549W/9ma+dK9+3bF//2t7+xOA1b53bt2oUAgL/++iuT17937x5euHBBe157V9XU1OCpU6cwKytLe/48a99++y3a29szHeP999/HgIAApmPoEs2b9pWWliIA4NGjR5mNceHCBQQAvHr1KrMxdInmzONNmDABX3nllda+ldzqpa9eXl7M73+iKz/++CP06tULZs+e3ezxV199tUM/Hxoa2u4HfU5OTjpdYNLS0lJnHyx2VH5+PvPl/728vJguR69rNG/a1/S7ZDlvml47Ly/PIG7WSHPm8fLz89u89qjVsvHz84Pz588zDdVVn3zyCRQWFsKBAwfg66+/bnH2Q3BwcIdex9XVlUU8vXL16tVmF66x4OfnBzU1NXDnzh3tLQ30Ec2bjrl27RqYmZkxvdeMnZ0dODs7w7Vr12DatGnMxukqmjMdU1dXB7dv3257W9Pa+53vvvsOra2tdfZ2joU+ffpgjx49cOXKlaK9RTRUHh4e+MknnzAd4/79+ygIAsbFxTEdp6to3nTMunXrcOjQoczHmTx5Mi5fvpz5OF1Bc6ZjUlNTEQCaLZPziNYPowUGBkJ1dTXcvHkTfHx8GHbhk7t16xbvCAbhwYMHcPv27VavKNYlOzs7cHd3h8zMTJg+fTrTsbqC5k3HXLp0ifmcAXh4pXtSUhLzcbqC5kzHXLp0CSwtLcHb27vV77d6+eyQIUNALpfDyZMnmYYj7J04cQIAQJT110aOHKkdjxgulUoFp0+fZn7mEsDDOZOZmWkQF3eS9qWkpEBQUFCbqzK0+qhcLodRo0ZBYmIi03CEvcTERBg0aBDTU1ibBAcHw/Hjx0Vb74qwkZaWBlVVVRASEsJ8rODgYFCr1ZCSksJ8LMJWUlJSu59ftbkwUHBwMBw7doxJKCKehIQEUTYaAA/nTHV1dav35iCGIyEhAVxdXUU5Q8zJyQkGDRpEO7YG7vr163Dr1q12tzVtlk1ISAgUFBRAbm4uk3CEvYqKCsjIyOjw2TJd1b9/f+jVqxdtOAxcYmIiTJw4UbTxgoODWyyvQgxLQkICWFpawlNPPdXmc9osm6CgILC0tKRJYMASExNBEAR45plnRBtzwoQJVDYGTKFQwKlTp0TbQQF4WDYZGRkGs3QNaSkxMRGeeeaZdu/W22bZmJqawrhx40S5bzVh49ChQzBs2DDt/S7EMHHiRDhx4oRBrABNWkpKSoK6ujrRDr0CPNxBkUgkcPToUdHGJLqjVCrh2LFjj50z7d7MYf78+RAXF2cwK7OS/2poaIDY2FhYsGCBqOM+99xzoNFo4NdffxV1XKIbkZGRMHLkSOYrTjzKzs4Onn32WYiKihJtTKI7hw4dgvLycnj++efbfV67ZfPCCy+AqakpxMTE6DQcYW///v1QVVUFCxcuFHVce3t7mDlzJkRERIg6Lum62tpa2L17t06Xz++o0NBQOHz4MJSUlIg+NumaiIgIGD9+/GMXJ223bKysrGDOnDm04TBAERERMHnyZC537AsNDYWkpCQoKCgQfWzy5Hbt2gWNjY2ivxsGePiO2MrKCqKjo0Ufmzy5Bw8ewG+//dahHZTH3hM1NDQUTp06BTk5OToJR9grLy+Hw4cPc9lDBQCYPn069OjRAyIjI7mMT55MREQEzJgxg+l9j9oil8vh+eefpx1bA7Nz505ARJg3b95jn/vYspk0aRK4ubnRJDAgUVFRYGpqCs899xyX8WUyGSxcuBDCw8MBEblkIJ1z584dSEhIgKVLl3LLsGzZMjh//jxkZmZyy0A6JywsDObMmQN2dnaPfe5jy0YqlcLKlSthy5YtUFNTo5OAhB2lUgn/+te/YNmyZWBpacktxyuvvALZ2dmwf/9+bhlIx33xxRfQs2dPmDVrFrcM48aNg4EDB8I///lPbhlIx506dQpOnjzZ8bt/dmQ1z/LycrS2tsbPP/9cl4uEEgZ+/vlnlMlkmJeXxzsKzpkzB4OCgnjHII9RVlaGVlZWuGnTJt5RMDIyEqVSKebk5PCOQh5j+vTpOGrUqI4+PblDZYP4cMnxnj17Yl1d3ZMlI8ypVCrs16+f3izZnp6ejoIgML3bI+m69957Dx0dHbG6upp3FFSpVOjr64urVq3iHYW048KFCygIAh44cKCjP9LxsikpKUFzc3P89ttvnywdYS46OhqlUileu3aNdxStyZMn44QJE3jHIG2orKxEOzs7/PTTT3lH0fr+++9RJpPhrVu3eEchbZg3bx4OHTq0M/c863jZICK+9tpr6OHhQTcQ0kNqtRoDAwNx4cKFvKM0k5ycjACAx48f5x2FtOKjjz5Ce3t7fPDgAe8oWo2Njeju7o5r167lHYW0IisrCyUSCf7666+d+bHOlc3t27fR0tKS+V0fSedt3boVZTIZXr58mXeUFiZNmoTDhg1DlUrFOwp5hD7/f9bn+WzsJk6ciMOGDUO1Wt2ZH+tc2SAi/uMf/0Bzc3O8efNmZ3+UMFJWVoaOjo64bt063lFalZ2djWZmZvjvf/+bdxTyiHnz5mHfvn2xvr6ed5QW1Go1jhw5EseNG6fXt6c3Ntu3b0eJRIKnTp3q7I92vmwaGxvR398f58yZ09kfJYysWLECe/furRcf8Lbl3XffRRsbGywqKuIdhSDi4cOHEQA68wGv6NLS0lAqlWJkZCTvKAQRq6qqsFevXk968kbnywbx4XF4QRBw//79T/LjRIdSU1NRIpFgTEwM7yjtqq2tRS8vL1y2bBnvKEavoaEB+/Xrh/Pnz+cd5bH+9Kc/oYuLC96/f593FKP35z//GXv06IGlpaVP8uNPVjaIiIsXL0YvLy+sqqp60pcgXdTQ0ICBgYH47LPP8o7SIXv27EFBEDA+Pp53FKP2wQcfoLW1Nd65c4d3lMeqqKhAZ2dnXL16Ne8oRi09PR1NTEzwhx9+eNKXePKyKSkpQRcXF707+8mYrFmzBm1tbTE3N5d3lA6bP38+urq6YklJCe8oRikhIQGlUilu2bKFd5QO27lzJwqCgLGxsbyjGKUHDx5g3759ceLEiZ09KeBRT142iP+duFu3bu3Ky5AnEBMTgwCAUVFRvKN0SlVVFfr5+WFwcDCdnSayu3fvopubG/7hD3/gHaXTVq9ejXZ2dga1Y9VdLFmyBJ2dnbv6eWvXygYR8S9/+QvK5XI8f/58V1+KdNCNGzfQ1tYWX331Vd5RnkhaWhqamZnh3//+d95RjIZarcbJkyejj48PVlZW8o7TaQ0NDTh06FAcMWIENjQ08I5jNL755huUSCR45MiRrr5U18tGpVLhhAkT0M/Pjz6/EUFDQwMOHz4chwwZopenrHbU119/jVKpFBMTE3lHMQofffQRmpmZYVpaGu8oTyw7Oxutra3pYk+RnD9/HuVyOX744Ye6eLmulw0iYlFREbq4uODUqVNRoVDo4iVJK9RqNS5cuBBtbGwMfqFCjUaD8+bNQycnJ71aXqc72rFjB0okEty8eTPvKF22bds2FASBDt0zVlBQgO7u7ro83K2bskFEzMjIQFtbW1y0aFFXPkQi7fjzn/+MpqamePjwYd5RdKKurg7HjBmDvXv3pnWwGElISEAzMzODPeTamo8++gglEgnu3LmTd5RuqbKyEgMDA3HgwIFYXl6uq5fVXdkg/ndiv/baa7p8WYKIH3/8MUokEtyxYwfvKDpVVlaG/v7+up7YBB/uANrZ2eHChQu73Q7gG2+8gaamprr4LIE8oq6uDseOHctiB1C3ZYP4cOVhiUSCGzdu1PVLG63w8HAUBEEv7jfCwp07d9DDwwNHjRqFNTU1vON0Czdu3MCePXtiSEhIt/xAXa1W4/z589HGxgbT09N5x+kWVCoVzp07Fx0cHPDKlSu6fnndlw3iww9/BUHAzz77jMXLG5Uff/wRpVIpfvDBB7yjMJWVlYU9evTAiRMn0okmXXT16lXs06cPjhgxolv/WzY0NODEiRPRyckJz507xzuOQWtoaMB58+ahpaUlnj59msUQbMoGEXHz5s0okUhw7dq13e4tvFg2bdqEgiDg+vXrjWIxwkuXLmGvXr1w+PDhePfuXd5xDNLZs2fRyckJg4KCnnRZEYNSU1OD06ZNQ0tLSzx48CDvOAapuroaJ0+ejHZ2dpicnMxqGHZlg4gYGxuLZmZmuGTJEjpLrRM0Gg2uW7cOBUHAL774gnccUd28eRN9fX3Ry8sLr1+/zjuOQYmPj0dra2ucPHmyXi/KqmtKpRJXrFiBpqamuH37dt5xDEpxcTEOHToUe/bsiRcuXGA5FNuyQXy4uqyVlRVOmzbNqP4DPKmGhgZctGgRmpmZGe3ZNiUlJTh06FB0dXVl/R+g2wgPD0eZTIYvvfQSKpVK3nFEp9Fo8M0330SJRIJfffUV7zgGIScnB729vbFfv36Yl5fHejj2ZYP4cGViJycn9Pf3x0uXLokxpEHKzc3FESNGoI2NDR47dox3HK4ePHiAISEhaGFhgT/++CPvOHqroaEBX3/9dQQAfPfdd43icGt7Nm7ciIIg4LJly+hkk3bExsaira2tmIdbxSkbxIcXCY0dOxblcnm3PauqK3bv3o329vY4dOhQg79gU1eUSiVu2LABJRIJLl26lN4Z/05+fj6OGjUKra2tcdu2bbzj6I2jR4+ii4sL9uvXDzMyMnjH0SsNDQ24du1aFAQBQ0NDsba2VqyhxSsbRNp4tIbjL99gxMfHY8+ePWnj8YhHd07os62WCgoKcNy4cbRz+4hHd044fLYlbtk0OXjwIDo5OaGvry8ePXqURwS9cPLkSRw8eDDa2NgY7eczHdW08bCwsMBPP/0UGxsbeUfi4u7du/jiiy+iIAj42muvdctraHRFqVTie++9hxKJBJ9//nksKCjgHYkLlUqFmzdvRhsbGxw2bBjeuHGDRww+ZYOIWFhYiPPmzUMAwAULFhjEjZx0pbS0FFesWIGCIOCzzz5Ly6Z3kFKpxH/84x9oYWGB/v7+RvW5llqtxm+//Rbt7e2xT58+uHfvXt6RDEZ8fDz6+vqilZUVfvbZZ0Z1Zuzp06dx6NChaGpqiu+//z7PnRN+ZdPk2LFj2L9/f7S0tMQNGzZ06z1WjUaDYWFh6OTkhG5ubhgWFsY7kkG6c+cOhoaGIgDgzJkz8fbt27wjMZWeno6jRo1CmUyGa9eu7dYXarKiUChw06ZNaGVlhX5+ft1+mZuKigpcu3YtSqVSnDBhAl6+fJl3JP5lg4hYX1+PGzZsQLlcjn5+fvjzzz93q70PtVqN0dHRGBgYiDKZDNetW0efV+nA/v370cvLC62trfGdd97pdnf/zMjIwPnz56NEIsHg4GAWS4gYndzcXJwxYwYKgoCzZs3CM2fO8I6kU+Xl5bhhwwa0t7dHNzc3jI6O5h2piX6UTZPc3Fxcvnw5ymQy9PT0xG+//dag79miUCjwl19+wX79+qFUKsWFCxfqwx5Gt1JXV4efffYZuri4oLm5Ob7++usG/04nNTUVZ8+ejYIg4JAhQ+h2yAzExcXhqFGjEABw0qRJBn9fpZKSEnznnXfQ2toaHRwc8KOPPtK3d8D6VTZN8vPz8dVXX0W5XI6urq64ceNGLCws5B2rw0pLS/Grr75CLy8vlMlkuHz5cszOzuYdq1urq6vDzZs3Y58+fdDU1BT/+Mc/YmpqKu9YHaZQKHDfvn04efJkBAAcNWoU7t+/3+ivm2EtPj4eg4ODEQBwzJgxuHPnToPawc3IyMA1a9agubk5uri44GeffaavR030s2yaFBUV4VtvvYX29vYolUpxypQpGBERoZcXa9XX12NMTAzOnj0bZTIZWltb46uvvor5+fm8oxmVxsZG/OGHH9Df3x8BAPv3748ff/yx3v4eUlNT8fXXX0cnJyeUSCQ4adIkjI+P5x3L6Jw8eRJnzZqFUqkU7ezscNWqVZiSkqKXZV9YWIiff/45BgYGIgCgj48Pbt68Gevq6nhHa49+l02ThoaGZhtyKysrXLZsGUZHR3NdsLG8vBx//fVXfPnll9HOzk5biJGRkXS9jB44e/Zssw35+PHj8euvv8asrCxumRQKBaakpOCGDRuwf//+2kL85JNP6AZyeqCoqKjZhtzb2xvff/99TEhI4PqO59q1a7hlyxacMmWKQRRiK5IFREQwIGVlZbB9+3bYsWMHpKamglqthoEDB0JISAiEhITAuHHjoEePHkzGrqqqghMnTkBCQgIkJCRARkYGCIIAw4YNgwULFsDixYvB1dWVydjkySmVSjh06BBERkbCkSNHoLKyElxcXCA4OBhCQkJgwoQJ4Ovry2zsjIwMSEhIgMTEREhJSYHa2lro06cPzJkzB0JDQ+Gpp55iMjbpmszMTAgPD4c9e/ZAbm4umJubw+jRo7XbmuHDh4OpqSmTsfPz8yEpKUm7rSksLAQrKyuYNGkSLFmyBGbOnAlyuZzJ2IwcN7iyeVR1dTUcP34cEhMTtRt/jUYD7u7uEBAQAP7+/uDl5QWenp7g7u4ODg4O4ODg0OYvqbGxEcrLy6G8vBwKCwshLy8P8vPz4erVq5CVlQV5eXkgCEKzchs/fjzY2dmJ/DcnT0qtVsP58+e1c+bEiRNQW1sLNjY2EBAQAIMGDQJvb2/w8vICd3d3cHZ2BkdHR7CxsWn19RBRO2fu3bsHeXl5kJeXBzdu3ICsrCy4cuUKKBSKZuUWHBwMffv2FflvTrri1q1b2jnTtPGXyWTQr18/GDRoEPTr1w88PT3By8sLnJ2dwcHBARwdHUEikbT6ejU1NVBWVgb37t2DgoICyM/Ph9zcXLhy5QpkZmbC/fv3QS6Xw9NPP62dM0FBQWBiYiLy31xnDLtsfq+iogJSU1MhMzMTMjMzITs7G/Lz86G8vLzZ8yQSCdja2jZ7rKqqCtRqdbPH7OzswMvLC3x9fWHw4MEQEBAAI0eOBGdnZ+Z/FyIOpVIJaWlpkJmZCRkZGXDlyhW4efMmFBUVtZgP1tbWzf6zNzQ0QH19fbPnmJmZgYeHB3h7e0NgYCAEBATAsGHDwN/fHwRBEOXvRNjLycmB8+fPQ0ZGBmRlZUFubi7k5eVBQ0NDs+fJ5XIwNzfX/lmj0cCDBw+aPUcQBHBzcwNvb2/w9/eHwMBACAwMhKeeesrQ3r20p3uVTVuqqqqguLhYuwdaX18PlZWVUFdXB/X19eDg4AC2trYgl8u1735cXV3pHYsRUygUUFxcDKWlpVBWVgbV1dVQU1MDSqUSbt++DX369AFzc3MwNzfXzhlHR0dwdXWlUjFixcXFUFZW1mxbU1dXp50zUqkUbGxswMrKChwcHMDJyQnc3NzAzMyMd3TWjKNs2vKf//wHSktL4f333+cdhRiIiooKmDp1Kpw9e5Z3FGJApkyZAps3bwY/Pz/eUXg53voBRSMRGRkJ4eHhvGMQA7Jz5044d+4cXLlyhXcUYiDu3r0Lx44dg+3bt/OOwpXRlk1xcTGcOHECsrOz4dKlS7zjEAPRtHNi7BsO0nE7d+4EtVoNYWFhvKNwZbRlEx0dDRKJBGQyGW04SIcUFBTAmTNnAAAgLCwMjPgINOmEiIgIEAQB8vLy4Pz587zjcGO0ZRMWFgZqtRqUSiX88ssvtOEgjxUdHQ1SqRQAHhYPfW5DHufWrVuQlpYGiAimpqZGvWNrlGWTm5sLmZmZ2oIpLi6G06dPc05F9F14eLj2dGhj33CQjomKitKeLq9QKCAsLAw0Gg3nVHwYZdk8OgEAgA6lkce6du0aZGVlaXdQFAoFREZGtrgWh5BHhYeHg1Kp1P65tLQUUlJSOCbixyjLJiIiotkEUCqVEBkZCSqVimMqos+2bdsGMpms2WPl5eWQlJTEJxDRe1euXIHs7Oxmj8lkMti2bRunRHwZXdlcuHABbty40eLxyspKOHbsGIdExBD8fgcFwLg3HOTxIiMjW+ygKJVKiI6OBoVCwSkVP0ZXNtu3b2918TzacJC2nDt3DvLz81s83rTh+P0SJYQgYqs7KAAP13Q8cuQIh1R8GVXZICJERka2ulehVCohJiamxVpXhGzfvr3FHmqT+vp6OHTokMiJiL47c+YM3Llzp9XvSaVSiIqKEjkRf0ZVNikpKVBcXNzm9xsbGyEuLk7ERETfaTQaiIyMbHUPFcB4NxykfW0dQQEAUKlUsGfPHqipqRE5FV9GVTbtTQCAh6tBR0ZGipiI6LukpCQoLS1t8/sqlQr27dsHVVVVIqYi+kytVkNUVFS7n8soFAr47bffREzFn9GUTUc+mFOpVHDgwIEWS4AT49XeIbQmSqUS9u7dK1Iiou8SEhKgoqKi3ecIgmB0O7ZGUzZHjx6FysrKxz5PqVTCrl27REhE9J1CoYCYmJg2D6E1QUS6TotodWQuqNVqOHLkyGNLqTsx2Nu+dVa/fv3g6NGjzR775z//CZWVlfDJJ580e7xXr15iRiN6SqFQQGxsbLPHzp49C//7v/8L+/bta3ZTrMe9+yHGY+XKlbB48eJmj82aNQvWrFkDU6ZM4ZSKP6MpGx8fH/Dx8Wn2WNNKApMmTeKUiuizpnu+P6rpMGxwcDBYWVnxiEX03OjRo1s8JpVKYcCAAUa9rTGaw2iEEEL4obIhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkqG0IIIcxR2RBCCGGOyoYQQghzVDaEEEKYo7IhhBDCHJUNIYQQ5qhsCCGEMEdlQwghhDkBEVGswd544w2Ijo4Wa7jHamxsBEQEuVzOO4rWpEmTICoqincMvXHs2DFYvHgx7xhaarUa6uvrwdLSEgRB4B1HKy8vDywsLHjH0Bve3t5QW1vLO4ZWbW0tmJmZgYmJCe8oWl999RUsXLhQrOGOi/o3f/DgATg6OsLq1avFHNZgREVFwf3793nH0CuNjY1w7949+Pzzz8HU1JR3HL1z5coV2Lp1K2g0Gt5R9Mrdu3dh7ty5MHLkSN5R9NK6deugvr5e1DFFr1kPDw94/fXXxR7WIKSnp8O9e/d4x9BLr7zyClhZWfGOoXcOHDgAW7du5R1DL02cOBGWL1/OO4Zeevfdd0Ufkz6zIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMUdkQQghhjsqGEEIIc1Q2hBBCmKOyIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMUdkQQghhjsqGEEIIc1Q2hBBCmKOyIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMUdkQQghhjsqGEEIIc1Q2hBBCmKOyIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMUdkQQghhjsqGEEIIc1Q2hBBCmKOyIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMUdkQQghhjsqGEEIIc1Q2hBBCmKOyIYQQwhyVDSGEEOaobAghhDBHZUMIIYQ5KhtCCCHMmYg94LFjx8DFxUXsYQ1CVVUVBAcH846hl7y9vUEQBN4x9I5CoeAdQW+9+eab8O677/KOoZfq6upEH1PUslm6dCkMHz5czCENjqenJ+8IemXgwIHw9ddf846h98zMzHhH0CtffPEFKJVK3jH02qhRo0QdT0BEFHVEQgghxuY4fWZDCCGEOSobQgghzFHZEEIIYe7/AdTL/D5Z7hMAAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#Iff an error is given about graphviz not being installed, please install it using the following command:\n",
"# conda install Graphviz\n",
"\n",
"%load_ext autoreload\n",
"%autoreload 2\n",
" \n",
"from graphviper.graph_tools.map import map\n",
"from graphviper.graph_tools.generate_dask_workflow import generate_dask_workflow\n",
"\n",
"def my_func(input_params):\n",
" toolviper.utils.display.DataDict.html(input_params)\n",
"\n",
" print(\"*\" * 30)\n",
" return input_params[\"test_input\"]\n",
"\n",
"\n",
"input_params = {}\n",
"input_params[\"test_input\"] = 42\n",
"\n",
"viper_graph = map(\n",
" input_data=ps_xdt,\n",
" node_task_data_mapping=node_task_data_mapping,\n",
" node_task=my_func,\n",
" input_params=input_params,\n",
")\n",
"\n",
"dask_graph = generate_dask_workflow(viper_graph)\n",
"dask.visualize(dask_graph, filename=\"map_graph\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "2dced77b",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"