Requirement	Map Layers
Section	3.2.5.2
JIRA Task	EIR-72 - Getting issue details... STATUS

Reviewed For	Date
Conventional spacing between sections	2016-08-31

Introduction

Geographic information associated with records collected using the Enter module is placed on the global coordinate system in Epi Map in layers, referred to as Data Layers. In contrast, reference information from map servers, shapefiles, or KML/KMZ files is associated with the map in Base Layers. There are two categories of Data Layers: Choropleth and Dot Density maps associate data records to regions or features by name and rely on geospatial data from external sources; Spot and case-cluster maps use latitude and longitude coordinates stored with each data record. The coordinates may be entered directly during data collection or determined from a street address or other geographical reference point using a GEOCODE command embedded in the Check Code of the data collection form. All Base and Data Layers are listed in the Map Layers tab along with their parameters. The tab enables the user to edit basic Data Layer parameters, move the layer forward and backward in this display stack, and access the layer's configuration panel.

Base Layers

Overview

Base Layers typically contain reference information to aid in the interpretation of project data, although they contain no project data themselves. Epi Map supports the construction of Base Layers from three different data sources: shapefiles, KML files, and map servers.

Shapefile

Shapefile is an open vector data format for geospatial information used by GIS software. Defining a shapefile resource requires a minimum of three files with a common root name and the extensions .shp (shape file, containing feature geometry), .shx (shape file index, containing offsets to features in shape file), .dbf (database file, a Dbase IV file containing a table of shape attributes). In Epi Map, shapefiles are read from the local file system.

KML

KML stands for "Keyhole Markup Language", an open XML standard for geospatial data developed by Keyhole, Inc. In addition to coordinates for point locations, KML can store polygons for describing regions (using connected points to create "shapes"), descriptive text, altitude, azimuth, tilt angle, and many other data types. File content is human-readable and can also be edited graphically using Google Earth software. In Epi Map, local KML files are selected using the MS Windows file browser. The map servers and remote KML files are identified by URL. Using a map server, the user shall provide an additional parameter, "feature", to select the specific resource for downloading. KML files can be downloaded in native (.kml) format or the Zip 2.0 compressed (.kmz) format.

Map Server

The United States, other national governments and non-governmental organizations operate web services that provide structured geographic data in response to web queries. These data can be transmitted as "Map Server" data directly to Epi Map for use in data analysis. For example, the U.S. Geological Survey (USGS) runs nationalmap.gov, an extensive resource for public information with a geographic component. In addition to maps of physical features and political boundaries, the USGS provides maps of critical infrastructure such as ground and air transportation, pipelines, electrical distribution, and Internet services. Data can be accessed through a variety of web protocols including REST and SOAP. Queries are posted in a format such as JSON and results are returned in HTML, JSON, KMZ, or streamed image data. The USGS uses ArcGIS, which has an extensive REST API. The documentation also catalogs its services, URLs, and other information needed to retrieve data for Base Layers in Epi Map.

Functional Requirements

When initiated with a mouse click on Icon 5 (refer to the Icons table in Section 3.2.5.1) in the Epi Map Toolbar, the Base Layer UI shall enable the user to define a new Base Layer by:

browsing for a Shapefile on the local filesystem;
entering a URL for a Map Server, connecting to it, and selecting the desired features from a server-supplied menu; and
entering a URL referencing a KML file (.kml or .kmz) or browsing for a file locally.

Data Layers

Overview

Data Layers can be categorized by how Epi Info™ 7 records are associated with locations. This can occur in two ways. First, a record may be position on the map directly, using latitude and longitude values stored in the record itself to create Spot and Case Cluster maps. Alternatively, the association can be indirect, with the record referring to a location by name; a second resource contains the coordinates of the spot or region indexed by that name, allowing the record to be located on the map. These indirect relationships are created with the same data resources used in defining Base Layers: shapefiles, KML files, and Map Servers. Since these map resources typically contain geographic regions (national, state, or local jurisdictional boundaries), this category of Data Layer only supports Choropleth and Dot Density maps.

Choropleth and Dot-Density Maps

As described above, Choropleth and Dot Density maps plot numerical values to regions using a color scale or number of dots per region, respectively, to indicate magnitude.

Spot and Case-Cluster Maps

Spot maps are created by marking locations with a specific shape and color. Case Cluster maps resemble Spot maps at high spatial resolution, with locations identified by colored circles containing an integer (usually "1"). The integer, which indicates the number of records associated with the location of the circle, starts out small when the map is zoomed in (e.g., when 1" on the screen equals 100 yards on the map), but grows as the map is zoomed out (when 1" on the screen corresponds hundreds of miles). When the map is fully zoomed out, a layer contains hundreds of subjects spread over a small state may collapse into a single cluster.

Configuration Options for Data Layers

Data Layer	Configuration	Parameters	Summary	User Guide
Choropleth	Data Source	Data Source^[1] (Browser) Boundaries Shapefile (Browser) Map Server URL (Connect) Select feature (Menu) KML/KMZ File URL or File (Browser)	The Choropleth map colors map regions, identified by a Key Field, with colors chosen according a Value Field. The Key Field identifies a region by name in reference data that also contains the coordinates for defining the regional boundaries on the map.	CM1, C M2
	Variables	Data Source Data key* (Menu) Value field* (Menu) Boundaries Feature key* (Menu)
	Data Filters	Data Filters (Filter Configuration) Value of Field Name Operator Value Filter List Join operator (`AND` or `OR`)
	Display	Legend Title Colors Start color (selector)^[2] End color (selector) Opacity (slider) Ranges Classes [2-10] `[ ]` Quantiles (Range Assignments^[3])
Dot Density	Data Source	Data Source* (Browser) Boundaries* Shapefile (Browser) Map Server URL (Connect) Select feature (Menu) KML/KMZ File URL or File (Browser)	A Dot Density map is similar to a Choropleth map, however the density of randomly placed dots is used to indicate the magnitude of the value field, instead of a color scale.	DD
	Variables	Data Source Data key* (Menu) Value field* (Menu) Boundaries Feature key* (Menu)
	Data Filters	Data Filters (Filter Configuration) Value of Field Name Operator Value Filter List Join operator (`AND` or `OR`)
	Display	Select color (Color chooser) Dot value `[int]`
Case Cluster	Data Source	Data Source* (Browser/Explorer)	Like a Spot map, a Case Cluster map plots the location of a record using the coordinates in the record itself. The Case Cluster map differs in the behavior of the position markers in response to changes in scale. At the highest map resolution, only records with identical latitude and longitude will be grouped into a single spot, with a number indicating the record count at that location. As the map is zoomed out, records (cases) begin to coalesce as more cases are localized to a virtual circle with a diameter proportional to the scale. The circles marking the location will grow in diameter in proportion to the number of cases localized to it.	CC
	Variables	Latitude Field* (Menu) Longitude Field* (Menu)
	Display	Description Legend description `[text]` Select a Color Color representation (Color chooser)
	Data Filters	Data Filters (Filter Configuration) Value of Field Name (Menu) Operator (Menu) Value Filter List Join operator (`AND` or `OR`)
Spot Map	Data Source	Data Source* (Browser/Explorer)	Like the Case Cluster map, the Spot map marks the locations of record based on their coordinates. The behavior of the marker is simpler; regardless of zoom level, the associated shape will maintain the same size on the screen.
	Variables	Latitude Field* (Menu) Longitude Field* (Menu)
	Display	Description Legend description `[text]` Color Select a color (Color chooser) Styles Select shape (Menu), options: Circle Square Cross Diamond Triangle
	Data Filters	Data Filters (Filter Configuration) Value of Field Name Operator Value Filter List Join operator (`AND` or `OR`)

Notes:

An asterisk following a parameter name indicates that the parameter is required.
A color selector resembles a square filled with a particular color. A left-mouse button click brings up window containing a chooser for 48 basic colors plus 16 custom colors (initially all white). A color may be chosen from this palette by clicking on the color and selecting "OK". Alternatively, the user may choose "Define Custom Colors", which enlarges the pop-up window to include a color selector consisting of square with hue changing from 0 - 255 on the horizontal axis and saturation changing from 0 - 255 on the vertical axis. The right is a gradient slider representing luminosity, changing from 0 - 255, bottom to top. There is a rectangle below the color selection square showing the current color and, to the right, its definition, characterized by three parameters (values, 0 - 255) for each of the two methods for defining color: hue, saturation, and luminosity (HSL); and red, green, and blue (RGB). Clicking the "Add to Custom Colors" button, immediately below the rectangle and parameter triplets, saves the color to an unused square in the "Custom colors" grid. This becomes the selected color when "OK" is selected to close the pop-up window.
See Range Assignment table for more information.

Choropleth Range Assignment

Bin	Color Selector	Range			Legend Text
0	Gray				Missing / Excluded
1	White		`x <`	`[numeric₁]`	`[text]`
...	Light Blue	`[numeric₁]`	`≤ x <`	`[numeric_n]`	`[text]`
n	Dark Blue	`[numeric_n]`	`≤ x`		`[text]`

The interface for defining ranges and assigning colors to choropleth maps can be configured to have 2 - 10 classes (n in the Bin column). When the Quantiles box is checked, the range boundary values are pre-calculated based on the mean and standard deviation of the selected variable. If unchecked, the boundaries can be user-defined. Elements in brackets will accept user input. Bin 0 is reserved for missing/excluded data. Bin 1 defines the color for regions with a score below the cut-off value (i.e., x < value). The subscript on the numeric fields indicates that fields with the same subscript are linked so that the values are always equal, ensuring than the range has continuous coverage. The colors in the Color Selector column represent the default values, which can be modified using the selector. The user is not restricted to gradients of the same hue; an arbitrary color may be selected for each bin.

Functional Requirements

Clicking on the Data Layer Icon (Icon 4) shall enable the user to create a new Data Layer by selecting one option from a list of the supported types:
1. Choropleth,
2. Dot Density,
3. Case Cluster, and
4. Spot Map.
Having selected a specific Data Layer type, the system shall open a configuration panel to enable the user to enter parameters^[1] to define the appearance and behavior of the new layer.

Note:

For each Data Layer type, the specific parameters and their conceptual organization are defined in the table Configuration Options for Data Layers.

Time Lapse

Overview

The Time Lapse functionality allows a single Case Cluster Data Layer to be animated based on a time field, typically a date, in the data records. Clicking on the Time Lapse Icon (Icon 6) initiates the process. The user is prompted to select the time field from a menu in a pop-up dialog box. The menu contains a list of all potential date fields in the source data. Selecting a field and clicking "OK" closes the dialog and opens another small window consisting of histogram of records, binned by day, above a timeline with a scroll bar. There is a start/stop control on the left of the timeline and two opposing arrows on its right. Clicking on the control on the left (initially, a right-facing arrow) will initiate the Time Lapse animation (the arrow will change to the pause icon, represented by two vertical bars). The histogram scale will shrink to a single day (bin), corresponding to the cases displayed on the map. The animation progressively adds days at a rate of one per second, increasing the number of bins displayed in the histogram along with the number of records displayed on the map. At the conclusion, the histogram displays all bins, from the first event to the last; the map displays all records (with map locations) in the data set. The arrow controls on the right enable the user to step forward and backward through the animation, one day at a time. The scroll bar enables the user to move quickly to arbitrary points in the animation sequence.

Functional Requirements

The Time Lapse function in Epi Map shall enable the user to:

choose a time field from the source data used in the Case Cluster map;
view a histogram of data records binned by days determined by the time field;
manipulate the end-point of the X-axis of the histogram with a scroll bar, controlling the amount of data displayed;
view the date of the first and last event currently visible on the timeline;
synchronize the records visible on the map with the records in the bins visible on the histogram;
initiate an animation in which the number of days (bins) is increased progressively, beginning with the first (earliest) day;
pause and restart the animation;
step through the animation, one date at a time;
use the scroll bar to move to arbitrary positions on the animation timeline.

Future Development

Functional Requirements

There needs to be a way to turn off a Data Layer without removing it completely. The time required to recreate a layer is significant, making it impossible to quickly toggle layers on and off.
When the Time Lapse histogram opens, the scroll bar and dates represent the first day of the outbreak chronology, yet the graph represents the entire interval, from first event to last. This is a bug.
It is easy to crash Epi Map by selecting the wrong time field when setting up Time Lapse. Although the user is provided with a limited number of choices, they are apparently not vetted beyond having the appropriate variable type. Example: CommunityHealthAssessment.prj:
1. if time field = DOB, it sometimes generates an error "Too many time stops..." (see Requirement 4);
2. if time field = Tdate, it crashes with an exception, possible reason: all records have the same value.
Time Lapse appears to be hard-coded to deal with an outbreak taking place over, at most, a few months. Selecting a field that has values spanning years (e.g., DOB) creates problems with poor and erratic performance. The tool should be designed to bin the data into a number of intervals that makes sense for a video clip: 2 - 3 minutes, maximum, regardless of the chronology's duration. This would make it usable for illnesses that develop over a long period of time such as cancer.
Epi Map shall enable the user to create a video file from a Time Lapse display. Enabling the system to render a video would allow the user to create much more complex and data intensive Time Lapse displays without the performance issues encountered when the Time Lapse is produced on the fly. The process could be forked off as a batch job, allowing it to take the CPU time required to produce a quality video. This would make it easier for users to include Time Lapse displays in presentations, increasing the visibility of the application.

Browser not supported