Mainnull/-/wiki/608060172024-03-29T02:09:03Z2024-03-29T02:09:03ZEclipse Data Documentation 1.1Kristen Weavernull/-/wiki/60806017/Eclipse+Data+Documentation2024-03-07T19:59:38Z2024-03-07T19:48:35Z<p>During solar eclipses, the ability to enter current air temperature data has been/will be made available to users of the app who have not gone through GLOBE training, but are using the GLOBE Eclipse tool. For eclipses after the 2017 event, the appearance of the GLOBE Eclipse tool has been constrained geographically to roughly the area experiencing the eclipse (notes about region with each individual eclipse below.) </p><p>Air temperatures reported through the Eclipse tool (by trained or untrained users) can be distinguished by the field labeled “airtempsComments” in the API and “air temps:comments” in ADAT. Data points with comments in the format {"temp":"36.2","units":"C"} were entered via the Eclipse tool. The unit value will be C or F depending on whether the app user reported measurements in Celsius or Fahrenheit although temperatures are converted as needed and stored in Celsius only in the “airtempsCurrentTemp” or “air temps:current temp (deg C)” fields for the API/ADAT respectively. Other values in this comments field may indicate automated weather stations, such as (Phase 4 Email Data Entry), Earth Networks, or Weatherbug. Trained GLOBE users entering data via other means (online web forms or the Data Entry app, which was incorporated into GLOBE Observer for the atmosphere protocols in July 2021) also have the opportunity to enter custom comments associated with the data, including observations about current conditions. </p><p>The Eclipse tool was/will be opened to users during these four previous and one upcoming eclipse events: </p><p>Total Solar Eclipse 21 August 2017 (North America): </p><ul><li>Limited testing with a few data points in the database beginning 15 August, full availability for data submission 18 August through 22 August</li><li>No geographic limitation on visibility of Eclipse tool</li></ul><p>Total Solar Eclipse 02 July 2019 (South America): </p><ul><li>Data submission available on 01 July and 02 July</li><li>Eclipse tool visibility: -65 to 20 latitude, -100 to -35 longitude </li></ul><p>Total Solar Eclipse 14 December 2020 (South America): </p><ul><li>Data submission available from 07 December to 15 December</li><li>Eclipse tool visibility: -70 to 3 latitude, -100 to -25 longitude</li></ul><p>Annular Solar Eclipse 14 October 2023 (North, Central and South America): </p><ul><li>Data submission available only on 14 October</li><li>Eclipse tool visibility: polygon represented in this <a href="https://observer.globe.gov/documents/19589576/51589943/2023EclipseArea_ShapeForApp.geojson">GeoJSON file for 2023</a></li></ul><p>Total Solar Eclipse 08 April 2024 (North and Central America): </p><ul><li>Data submission planned to be available from 01 April 2024 to 08 April 2024</li><li>Eclipse tool visibility: polygon represented in this <a href="https://observer.globe.gov/documents/19589576/51589943/2024EclipseArea_ShapeForApp.geojson">GeoJSON file for 2024</a></li></ul>Kristen Weaver2024-03-07T19:48:35ZLandcover Supplementary Photos added 1.0David Overoyenull/-/wiki/60806017/Landcover+Supplementary+Photos+added2024-02-22T17:35:11Z2024-02-22T17:35:08Z<p>On 12/22/2023 the Observer app / Landcover module was modified to allow users to include up to 4 "special feature" photos. The intent is to allow users to photograph up to 4 features which may exist in the landcover site being surveyed. These photos are optional and are not required to be taken in the cardinal directions. Directional information is obtained from the mobile device when available and is stored with the photos as "extra data" which can be used by researchers. </p>David Overoye2024-02-22T17:35:08ZPython Notebook Samples 1.1Jonas Choinull/-/wiki/60806017/Python+Notebook+Samples2023-03-03T00:20:50Z2023-03-02T17:45:51Z<p>Below are Python Notebooks that provide models to pull and analyze data from GLOBE's API. </p>Jonas Choi2023-03-02T17:45:51ZChanges Incorporated in Volume 2 1.1Jonas Choinull/-/wiki/60806017/Changes+Incorporated+in+Volume+22023-03-03T00:20:15Z2023-03-02T17:45:16Z<p>The following announcements were incorporated into Volume 2 and are kept here for archival purposes. </p>Jonas Choi2023-03-02T17:45:16Z2019-10-09 Latitude and longitude data-1 1.1Jonas Choinull/-/wiki/60806017/2019-10-09+Latitude+and+longitude+data-12023-03-02T17:50:01Z2021-11-19T23:21:22Z<h2 id="section-2019-10-09+Latitude+and+longitude+data-1-Latitude+and+longitude+data"> Latitude and longitude data </h2><p>9 October 2019 </p><p>Effective September 15, 2019, the site location as defined by the Military Grid Reference System (MGRS) and the true latitude and longitude of an observation made with the NASA GLOBE Observer mobile app are both returned by the <a href="https://datasearch.globe.gov/">GLOBE Advanced Data Access Tool (ADAT)</a>,<a href="https://vis.globe.gov/GLOBE/">GLOBE Visualization System (Vis)</a>, and<a href="https://www.globe.gov/globe-data/globe-api">GLOBE Application Programming Interface (API)</a> in the downloaded dataset. GLOBE Observer data are impacted (i.e., observations made with the Observer app). Need help? Contact <a href="https://www.globe.gov/globe-data/help@globe.gov">help@globe.gov</a>. </p><h3 id="section-2019-10-09+Latitude+and+longitude+data-1-Background"> Background </h3><p>GLOBE uses the Military Grid Reference System (MGRS) as a reference geographic coordinate system. You can read more about the MGRS grid in the <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.globe.gov_globe-2Ddata_globe-2Ddata-2Duser-2Dguide&d=DwMFaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=FtSlwI_LClWCqzpohB6uXm1LFxCOJ56AHzAW4oBRc-s&m=TKnmmgn9YEChdip22fvvVdckvI5xaJ-RZNhUIGgi95E&s=gfG4fOwV-kKo2Hw_stxo-ZwrnnPkgi9BPeXm4JHNNRc&e=">GLOBE Data User Guide</a>. If a user queried the GLOBE database, the user would see a latitude, longitude, and elevation associated with each GLOBE observation in the downloaded dataset. For data submitted via the GLOBE Online Data Entry Form, GLOBE Data Entry app, or GLOBE Email Data Entry, the latitude, longitude, and elevation in the downloaded dataset were the location data input by the participant. For data submitted via the GLOBE Observer mobile app, the latitude, longitude, and elevation in the downloaded dataset were defined at the lower left corner of the associated MGRS gridbox and were based upon the first observation submitted from any participant and any protocol that fell within the associated MGRS gridbox. </p><p><img src="https://lh4.googleusercontent.com/EEb3e8vB_yUxVrcGADPIJ3lBvi0EhsfI3-9fL_z6av0a7V79ezpENv6n-h23_vWp05pdOlJcCBRrryk5MoMNNK7IcLqvM9JWXLVTwSygOMqS8MDersflT9NQqarhUYXiLu549cZm" /> </p><h3 id="section-2019-10-09+Latitude+and+longitude+data-1-What+changed?"> What changed? </h3><p>Effective September 15, 2019, the site location as defined by the MGRS grid and the true latitude and longitude of an observation made with the NASA GLOBE Observer mobile app are both returned by the <a href="https://datasearch.globe.gov/">GLOBE Advanced Data Access Tool (ADAT)</a>,<a href="https://vis.globe.gov/GLOBE/">GLOBE Visualization System (Vis)</a>, and<a href="https://www.globe.gov/globe-data/globe-api">GLOBE Application Programming Interface (API)</a> in the downloaded dataset. Data submitted via the GLOBE Online Data Entry Form, GLOBE Data Entry app, or GLOBE Email Data Entry are not impacted. This change only affects GLOBE Observer data. </p><p>Effective September 15, 2019, satellite matching for GLOBE Observer Clouds is now utilizing the true latitude and longitude. Previous GLOBE Observer data was matched coincident to MGRS. </p><p>The “true” location of an Observer measurement is the latitude and longitude recorded by the Global Position System (GPS) reading from a person’s mobile device. The GLOBE Observer app does not convey GPS elevation, but instead a Google web service is used that determines elevation based on WGS84 EGM96 geoid height. </p><p>Below is an example of the new, updated data output from ADAT . The example is for land cover data. The site latitude, longitude, and latitude associated as defined by the lower left corner of the MGRS grid box are in columns E, F, and G, respectively. </p><p><img src="https://lh3.googleusercontent.com/l3uB4ODlKgVp0awWzC4Q1qeSLhSOuu76N5TWbiDmz5BPsPAVB-CGEYjok37qPhafU_tLz_0nBV0seyzP1xTBNKZXAPXfVaDNMo0H1Jtqc1YlCyxlxbWL7LOKiIdtaTapX0SKw0cP" /> </p><p>The true GPS latitude, longitude, and elevation of GLOBE Observer measurements are in columns AS, AT, and AU respectively. </p><p><img src="https://lh5.googleusercontent.com/UxUKMSbO0_V2CM1uCcuTbvBWV-B8NnY1u5yU59L0ppoxrw95SW8TUiP9lLnG8_GvlYcZTTfaCgadb0sDRmYWk9n0PfYB8DgXFr5NYFuNBOW6ON-XH1a6nl12pATAbRp2BwaEMn5E" /> </p><p>Here is an example of land cover data output from the GLOBE API. The variables "latitude", "longitude", "elevation" highlighted yellow are the site location as defined by the lower left corner of the MGRS grid box. </p><p><img src="https://lh3.googleusercontent.com/qd7ftGD1RAZqlG9tsbFI12HQBCsFsPj39loQrLBfznjHUo0eHoYep2u3Wxg_qLOtSnqRfaWpWLhuFTR0gbHLhGYTvAezX6Dr9_zymBuK2Qyag2Tw2f7RDocsKHvjOMlJnCCl_Q9m" /> </p><p>The variables "landcoversMeasurementLatitude", "landcoversMeasurementLongitude", "landcoversMeasurementElevation" are the true location of an Observer measurement. </p><p><img src="https://lh5.googleusercontent.com/RN5nfE7N6WNlf3VNyMa_Iu05Z5lEXPJkniKDE4UXKt7XklnRnkFOHtuLjzfdvDV6zKCuyvtNpqN7tH62ANq7InkBs1XPm-86FYPtnDBjkg4hnz5-01mon5aiqFbrbWaxsP4HOT_7" /> </p>Jonas Choi2021-11-19T23:21:22Z2019-09-23 GLOBE Observer elevation data-1 1.1Jonas Choinull/-/wiki/60806017/2019-09-23+GLOBE+Observer+elevation+data-12023-03-02T17:49:45Z2021-11-19T22:51:15Z<h2 id="section-2019-09-23+GLOBE+Observer+elevation+data-1-GLOBE+Observer+elevation+data"> GLOBE Observer elevation data </h2><p>23 September 2019 </p><p>GLOBE's GPS protocol states that a site's elevation is in relation to mean sea level (MSL). In 1995, the thinking was that sites would be only defined over land and so the elevation would be based upon the topographic elevation difference from MSL. Most GPS devices in the 1990s and early 2000s used the elipsoid model to determine elevation, which could be as much as 100m off from the geoid height. GLOBE's former database converted the GPS elevation values to geoid heights and presented those geoid heights on the legacy Vis system. In 2014, GLOBE discontinued this practice because most newer GPS devices are capable of determining elevation using the geoid height themselves. The GLOBE Observer app does not convey GPS elevation, but instead a Google web service is used that determines the height from MSL based upon the WGS84 EGM96 geoid height. It includes both bathymetry and topography values. The consumer of GLOBE data can assume that a returned elevation value for a site is based upon the geoid height of the landform from MSL. In other words, if the coordinates are over dry land it will be the topographic height. If the coordinates are over the ocean or sea, then elevation is reported is the distance of the seafloor from MSL expressed as a negative value. </p>Jonas Choi2021-11-19T22:51:15Z2019-10-01 GLOBE Observer land cover launch date correction-1 1.1Jonas Choinull/-/wiki/60806017/2019-10-01+GLOBE+Observer+land+cover+launch+date+correction-12023-03-02T17:49:14Z2021-11-19T22:49:56Z<h2 id="section-2019-10-01+GLOBE+Observer+land+cover+launch+date+correction-1-GLOBE+Observer+land+cover+launch+date+correction"> GLOBE Observer land cover launch date correction </h2><p>1 October 2019 </p><p>The land cover tool in the GLOBE Observer mobile app was launched in September 2018. The GLOBE Data User Guide v1.0 incorrectly stated the land cover tool launched in October 2018. </p>Jonas Choi2021-11-19T22:49:56Z2021-05-28 Filter by Teams Available 1.2Jonas Choinull/-/wiki/60806017/2021-05-28+Filter+by+Teams+Available2023-03-02T17:52:03Z2021-05-28T20:41:57Z<p>GLOBE's Advanced Data Access Tool (ADAT) and API (ElasticSearch) now support the ability to find measurements by GLOBE team name. </p><p>With the API, you will need to know the exact spelling of the team name. A new search by feature can be seen on the API landing page -<a href="https://api.globe.gov/search"> https://api.globe.gov/search</a>: </p><p><a href="https://api.globe.gov/search/swagger-ui.html#/operations/v-1-controller/findByProtocolAndMeasuredDateAndGlobeTeamsUsingGET">/v1/measurement/protocol/measureddate/globeteams/</a><br/>Find measurements by protocol, measured date range and GLOBE team name. </p><p>With ADAT the new Teams search is available under the site filters: School/Teacher/Partner/Team. This filter supports an auto fill capability. Begin typing the name of the team and then select from the provided list. </p><p>Since the teams capability is relatively recent, no data will be returned associated with a GLOBE team prior to 2019. </p>Jonas Choi2021-05-28T20:41:57Z2020-07-08 Python Notebook for Non-Linear Regression MHM Protocols 1.4Jonas Choinull/-/wiki/60806017/2020-07-08+Python+Notebook+for+Non-Linear+Regression+MHM+Protocols2023-03-02T17:50:46Z2020-07-08T17:33:03Z<p>Python notebook used to compare Mosquito Habitat Mapper protocol bundle data. Uses linear regression as initial parameters for non-linear regression between two protocols. </p><p><a href="https://www.globe.gov/documents/10157/67125873/requirements.txt/b02a6591-e988-4ec1-b53c-482decdf09a4">requirements.txt</a> </p><p><a href="https://www.globe.gov/documents/10157/67125873/protocols.txt/63331f1f-f5d8-4552-b80b-e69c1207c60b">protocols.txt</a> </p><p><a href="https://www.globe.gov/documents/10157/67125873/README.md/ba900f6d-1486-4c1d-a28f-b4b5002dc298">README.md</a> </p><p><a href="https://www.globe.gov/documents/10157/67125873/regressions.py/55b61f2f-38e7-48eb-8b13-81be6fc1f5cb"></a><a href="https://www.globe.gov/documents/10157/67125873/regressions.ipynb/6901d6b6-b01f-4e54-883e-43085e835d6d">regression.p</a><a href="https://www.globe.gov/documents/10157/67125873/regressions.py/55b61f2f-38e7-48eb-8b13-81be6fc1f5cb">y</a> </p><p><a href="https://www.globe.gov/documents/10157/67125873/jupyter_notebook_config.py/01bca505-6091-4d8e-b820-14def18e59fb">jupyter_notebook_config.py</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/bundler.ipynb/4db6c5b1-46f8-4024-8197-114ebca1736f"></a><a href="https://www.globe.gov/documents/10157/67125873/regressions.ipynb/6901d6b6-b01f-4e54-883e-43085e835d6d">regression.ipynb</a> </p><p>Download all files here: <a href="https://www.globe.gov/documents/10157/67125873/regressions.zip/385f8d23-a64c-4ce8-8ac4-b0d375e65129">regression.zip</a> </p><p><a href="https://www.globe.gov/documents/10157/67125873/regressions.ipynb/6901d6b6-b01f-4e54-883e-43085e835d6d"></a><a href="https://www.globe.gov/documents/10157/67539727/bundler.ipynb/4db6c5b1-46f8-4024-8197-114ebca1736f"></a> </p>Jonas Choi2020-07-08T17:33:03Z2020-04-08 Python Notebook for MHM 1.1Jonas Choinull/-/wiki/60806017/2020-04-08+Python+Notebook+for+MHM2023-03-02T17:47:08Z2020-07-02T18:30:44Z<p><a href="">2020-04-08 Python Notebook for MHM (files updated 7-1-2020)</a> </p>Jonas Choi2020-07-02T18:30:44Z2020-06-24 Python Notebook for MHM bundle data 1.1Jonas Choinull/-/wiki/60806017/2020-06-24+Python+Notebook+for+MHM+bundle+data2023-03-02T17:47:20Z2020-07-01T19:23:23Z<p><a href="">2020-06-24 Python Notebook for Selecting Data on a Map</a> </p>Jonas Choi2020-07-01T19:23:23Z2020-06-24 Python Notebook for Selecting Data on a Map 1.2Jonas Choinull/-/wiki/60806017/2020-06-24+Python+Notebook+for+Selecting+Data+on+a+Map2023-03-02T17:48:02Z2020-06-24T23:18:29Z<p>Python notebook used to pull all Mosquito Habitat Mapper protocol bundle data from a geographic region defined by the user. User draws a region on a map and all bundle data is returned for that region. </p><p>zip file for all documents below - <a href="https://www.globe.gov/documents/10157/67539727/bundler.zip/aac24949-9eb1-4bca-bcc8-15336c9b03bd">bundler.zip</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/requirements.txt/3338c8a8-0868-45a8-a5c9-93397800dcaf">requirements.txt</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/protocols.txt/71301779-885f-4245-91a9-e2480be355e1">protocols.txt</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/README.md/6ee98773-5ef0-441f-867c-83ea47675bc4">README.md</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/bundler.py/73c31ba9-ef76-497a-a0ef-cf06df6fb582">bundler.py</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/jupyter_notebook_config.py/ce29052d-bbcb-44fc-a258-e00344808154">jupyter_notebook_config.py</a> </p><p><a href="https://www.globe.gov/documents/10157/67539727/bundler.ipynb/4db6c5b1-46f8-4024-8197-114ebca1736f">bundler.ipynb</a> </p>Jonas Choi2020-06-24T23:18:29Z2020-04-08 Python Notebook for MHM (files updated 7-1-2020) 1.1Jonas Choinull/-/wiki/60806017/2020-04-08+Python+Notebook+for+MHM+%28files+updated+7-1-2020%292023-03-02T17:47:37Z2020-06-12T20:19:03Z<p><a href="">2020-04-08 Python Notebook to retrieve and graph MHM data</a> </p>Jonas Choi2020-06-12T20:19:03Z2020-04-08 Python Notebook to retrieve and graph MHM data 1.4Jonas Choinull/-/wiki/60806017/2020-04-08+Python+Notebook+to+retrieve+and+graph+MHM+data2023-03-02T17:48:26Z2020-04-14T18:59:28Z<p>Files Updated 2020-07-01 </p><p>There is now a beta version of a python-based Jupyter notebook, which can be used to pull data from the GLOBE API for Mosquito Habitat Mapper. The notebook includes code and examples of how to pull the data, remove outliers, and graph and plot the results. </p><p>The full .zip package is here: <a href="https://www.globe.gov/documents/10157/66011846/mosquitoes.zip/4f3d0855-3194-4035-86df-129a362de2a4">mosquitoes.zip</a> </p><p>The package includes: </p><p><a href="https://www.globe.gov/documents/10157/66011846/Mosquitoes_notebook_instructions.docx/dc0c2b10-205a-40bc-9018-a5607fae4e12">R</a><a href="https://www.globe.gov/documents/10157/66011846/README.md/6fa08628-9c62-4da6-aed1-49707c1e79d9">EADME.md</a> </p><p><a href="https://www.globe.gov/documents/10157/66011846/requirements.txt/00a66161-a8ef-422e-8d4e-0cc32a9816c7">Requirements</a> </p><p><a href="https://www.globe.gov/documents/10157/66011846/jupyter_notebook_config.py/8f4df5a6-5a07-4fac-ab30-8e99b5dc3e0e">Jupyter_Notebook_Config.py</a> </p><p><a href="https://www.globe.gov/documents/10157/66011846/mosquitoes.ipynb/600d5bf2-23d1-407a-a01b-e4ce16a997ba">mosquitoes.ipynb</a> </p><p><a href="https://www.globe.gov/documents/10157/66011846/mosquitoes.py/371f8dc2-057a-4958-b39f-1456fa325dfa">mosquitoes.py</a> </p><p><a href="https://www.globe.gov/documents/10157/66011846/postBuild/759f1816-d17a-4c32-b1d4-451621b410e7">postBuild</a> </p>Jonas Choi2020-04-14T18:59:28Z2020-01-26 GLOBE API - Change Announcement 1.2Jonas Choinull/-/wiki/60806017/2020-01-26+GLOBE+API+-+Change+Announcement2023-03-02T17:50:35Z2020-04-14T18:58:10Z<p><em>26 January 2020</em> </p><p><strong>Summary of Changes: </strong><br/>We are modifying the GLOBE API so that variables currently output as strings are output as integers, floats and booleans. This will provide data type consistency between the source files, ADAT data, and Elasticsearch data stores. We are upgrading the servers associated with the GLOBE API to improve performance and stability. </p><p><strong>Affected Systems:</strong> <br/>Primarily - GLOBE API<br/>Secondarily - The GLOBE database will be under greater than usual load which may impact overall system performance.<br/>Secondarily - The GLOBE Advanced Data Access Tool (ADAT) may return incomplete data for a given protocol when that protocol is being processed. Users will be informed which protocol is being processed on the ADAT main page. </p><p><strong>Affected Protocols:</strong><br/>All GLOBE protocols </p><p><strong>Affected Users: </strong><br/>All users who currently consume data from the GLOBE API may be affected. Users of GLOBE's ADAT tool may be intermittently affected. </p><p><strong>Planned Timeline:</strong> <br/>Data Migration in Production: 22 January 2020 - 26 January 2020<br/>Test and Validation of new server environment: 26 January 2020 - 30 January 2020<br/>New API server "live" in production - 30 January 2020 </p><p>Send Comments to: help@globe.gov </p><p><strong>Discussion</strong><br/>The current implementation of Elasticsearch (which drives the GLOBE API) returns data in GeoJSON or JSON formats. Both formats have the capability to allow JSON values to have the simple data types string (with double-quotes), number (integers and floats), boolean (true or false, without double-quotes), and null. However, all JSON values within the GLOBE API "data" subset are being returned as strings (which include numbers and booleans). For example, the userId is currently output as a string when it should be an integer. The planned changes will modify records which are currently incorrectly output as strings to their appropriate numeric or boolean format. </p><p>For example, JSON is currently being returned that looks like:<br/>{"data": {<br/> "string_field": "GLOBE",<br/> "integer_field": "100",<br/> "float_field": "23.45",<br/> "boolean_field": "f",<br/> "boolean_field_2": "false"}<br/>}<br/>After this change, it will look like:<br/>{"data": {<br/> "string_field": "GLOBE",<br/> "integer_field": 100,<br/> "float_field": 23.45,<br/> "boolean_field": false,<br/> "boolean_field_2": false}<br/>} </p><p><strong>Impact </strong><br/>It is possible if you have written a program to read from the API that this change may impact your program's ability to read the data. </p>Jonas Choi2020-04-14T18:58:10ZFrontPage 1.6Kristen Weavernull/-/wiki/60806017/FrontPage2023-01-10T15:06:21Z2019-10-10T18:52:37Z<h2 id="section-FrontPage-User+Guide+Updates"> User Guide Updates </h2><p>The GLOBE Data User Guide v2.0 has now been posted, effective 30 December 2022 (see link above). Going forward, this page will include updates that will be incorporated into v3.0. A new version of the User Guide is only planned to be released at approximately annual intervals. This page is to provide the GLOBE community and end users with updates in a timely manner. </p><p>Questions or comments should be addressed to: <a href="mailto:help@globe.gov">globehelp@ucar.edu</a> </p>Kristen Weaver2019-10-10T18:52:37Z