DOWNLOAD CASES FROM AZURE

Summary

Download Cases from Azure downloads GeoClaw simulation results for your release point features from your Microsoft Azure^® Batch account’s associated Storage account to your specified working directory.

To learn more about the Liquids HCA Tool in general, please see Liquids HCA Tool Frequently Asked Questions.

To learn more about the structure of the Liquids HCA Tool project geodatabase, please see Liquids HCA Tool Data Dictionary.

Usage

Before running Download Cases from Azure, you should make sure that all tasks in the Azure batch job for your submitted GeoClaw cases have completed successfully. You can use Monitor Azure Resources to identify the status of tasks submitted with Run Cases on Azure. You can also use Update GeoClaw Case Status to update case status in the OSANALYSIS column in your release point feature class. Unless GeoClaw itself fails, you can generally simply reactivate failed tasks using Reactivate GeoClaw Cases. When GeoClaw itself fails, you’ll need to generate new cases for those release points with updated GeoClaw parameters using Create GeoClaw Cases. (You’ll need to rerun them, too, using Run Cases on Azure.)

Download Cases from Azure uses the following tool parameters:

Working Directory – This must be the same working directory that you used in running the Create GeoClaw Cases and Run Cases on Azure tools.
Input Release Point Features – This should be the same feature layer that you used in the Create GeoClaw Cases and Run Cases on Azure tools. However, if you wish to download only a subset of your GeoClaw cases, you may do so either by applying either a definition query or a feature selection to your release point feature layer.
Release Point Identifier Field – The field that uniquely identifies your input release point features. This should be the same as the field name specified in the Create GeoClaw Cases and Run Cases on Azure The default value is the POINT_ID field.
Azure Credential Method – You can either use the encrypted, passcode-protected Azure credential file that you created using the Create Encrypted Azure Credential File tool or you can manually input your Azure Batch and associated Storage account credentials. When you select “Encrypted file” for this parameter, the following required dependent parameters are exposed:
- Encrypted Credential File – This parameter is the path to your encrypted, passcode-protected credential file. By default, the tool looks for a credential file named azure_cred.bin in the scratch folder for the active ArcGIS Pro project.
- Encrypted Credential File Passcode – This parameter requires you to specify the passcode that you supplied when creating your encrypted Azure credential file. The encrypted Azure credential file can only be unencrypted for use when you supply this passcode. You should apply the same security measures for safeguarding this passcode that you apply to your Windows password.

The following required dependent parameters are exposed when you select “Manual input” for the Azure Credential Method parameter:

- Azure Batch Account Name – Your Azure Batch account name.
- Azure Batch Account Key – The primary or secondary access key for your Azure Batch account.
- Azure Batch Account URL – The URL for your Azure Batch account.
- Azure Storage Account Name – The name of the Azure Storage account associated with your Azure Batch account.
- Azure Storage Account Key – Key1 or Key2 for the Azure Storage account associated with your Azure Batch account.
Use synchronization mode – This checkbox parameter allows you to specify whether the tool will overwrite existing files in the release point case folder, or ‘synchronize’ files between the case folder and Azure. When synchronization mode is enabled, existing files in the case folder that are newer than those on Azure are not overwritten. When synchronization is disabled, Azure will overwrite existing files in the release point case folder. By default, file synchronization is enabled.
Ignore GeoClaw raw data – This checkbox parameter allows you to specify whether to download GeoClaw raw output data. When this parameter is enabled, Azure will not download raw GeoClaw output to the release point case folder. By default, this parameter is enabled.
Ignore GeoClaw topography and hydrography rasters – This checkbox parameter allows you to specify whether to download the topography and hydrography ASCII format raster files created when Land Spill downloads topography and hydrography data from the United States Geological Survey (USGS) or Esri web servers. When this parameter is enabled, Azure will not download topography and hydrography ASCII format raster files to the release point case folder. By default, this parameter is enabled.
Ignore missing results on Azure – This checkbox parameter allows you to specify whether to continue the download process even if results folders are missing from the Azure Storage account for one or more release points. When this parameter is enabled, Azure will continue the download process even when results folders on the Azure Storage account are missing for one or more release points. When this parameter is disabled, the download process will abort the first time a missing results folder on the Azure Storage account is encountered. By default, this parameter is disabled.

In a typical Liquids HCA Tool workflow, you run Download Cases from Azure after Azure processing for all cases submitted with Run Cases on Azure is complete.

For visual reference on Liquids HCA Tool execution order, see Liquids HCA Tool Process Flow Diagrams.

Syntax

DownloadCasesFromAzure_ (working_dir, rupture_point, case_name_field, cred_type, {cred_file}, {passcode}, {azure_batch_name}, {azure_batch_key}, {azure_batch_URL}, {azure_storage_name}, {azure_storage_key}, sync_mode, ignore_raw, ignore_asc, ignore_noexist)

Parameter	Explanation	Data Type
working_dir	Dialog Reference Specify the working directory storing your release point case folders. There is no Python reference for this parameter.	Workspace
rupture_point	Dialog Reference Specify the release point features associated with your GeoClaw cases. There is no Python reference for this parameter.	Feature Layer
case_name_field	Dialog Reference Select the field storing your release point unique identifier values. There is no Python reference for this parameter.	Field
cred_type	Dialog Reference Select the Azure authentication method. There is no Python reference for this parameter.	String
cred_file (Optional)	Dialog Reference Select your encrypted, passcode-protected Azure credential file. There is no Python reference for this parameter.	File
passcode (Optional)	Dialog Reference Specify the passcode for your encrypted, passcode-protected Azure credential file. There is no Python reference for this parameter.	String Hidden
azure_batch_name (Optional)	Dialog Reference Specify the name of your Azure Batch account. There is no Python reference for this parameter.	String Hidden
azure_batch_key (Optional)	Dialog Reference Specify your Azure Batch account key. There is no Python reference for this parameter.	String Hidden
azure_batch_URL (Optional)	Dialog Reference Specify your Azure Batch account URL. There is no Python reference for this parameter.	String Hidden
azure_storage_name (Optional)	Dialog Reference Specify your Azure Storage account name. There is no Python reference for this parameter.	String Hidden
azure_storage_key (Optional)	Dialog Reference Specify your Azure Storage account key. There is no Python reference for this parameter.	String Hidden
sync_mode	Dialog Reference Check this checkbox to enable file synchronization mode between Azure and your case folders. There is no Python reference for this parameter.	Boolean
ignore_raw	Dialog Reference Check this checkbox to skip the download of GeoClaw raw results data. There is no Python reference for this parameter.	Boolean
ignore_asc	Dialog Reference Check this checkbox to skip the download of topography and hydrography rasters used by GeoClaw. There is no Python reference for this parameter.	Boolean
ignore_noexist	Dialog Reference Check this checkbox to continue the download process if missing results folders are encountered on Azure. There is no Python reference for this parameter.	Boolean

Code sample

The following script demonstrates how to use Download Cases from Azure with an encrypted passcode-protected file:

import arcpy

arcpy.ImportToolbox(r”C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\liquidshca\esri\toolboxes\LiquidsHCA.pyt”)

in_dir = r”C:\data”

arcpy.env.workspace = r”C:\data\test.gdb”

release_points = “OSPointM”

point_id_field = “POINT_ID”

file_path = r”C:\data\azure_cred.bin”

passcode = XXXXXXX

cred_type = “Encrypted file”

batch_name = None

batch_key = None

batch_url = None

storage_name = None

storage_key = None

sync_mode = True

ignore_geoclaw = True

ingore_topo = True

ignore_case_found = False

arcpy.liquidshca.DownloadCasesFromAzure(in_dir, release_points, point_id_field, cred_type, file_path, passcode, batch_name, batch_key, batch_url, storage_name, storage_key, sync_mode, ignore_geoclaw, ingore_topo, ignore_case_found)

Environments

Scratch Workspace

Licensing information

This tool requires a valid Liquids HCA Tool user license or subscription. Please see the Request License and Register License tool help topics for details on obtaining and registering a Gas HCA Tool software license.

Credits

geoclaw-azure-launcher:

Use limitations

There are no access and use limitations for this item.