How to Download Dust Observations Reported through GLOBE
Dust event on 10 July 2019. Photo credit: GLOBE
Citizen scientists from around the world have been reporting dust events using the NASA GLOBE Observer app. You can learn about how to get involved here.
This blog offers step-by-step instructions on how to download GLOBE dust observations. There are two options. Option 1 is to download the data to a spreadsheet. Option 2 is to download the data to a JSON or GeoJSON file. For both options, you need to download clouds data first because dust data are part of the clouds data set, and then filter for dust observations.
Also available is a python script (see attachment at the bottom of this page), courtesy of Dr. Helen Amos, to download GLOBE clouds data from the GLOBE API between given start and end dates, and extract observations reporting dust. Dust observations are plotted on a map and as a timeseries. Find tips for the script at the very bottom of this blog.
Tip: ADAT is best if you want smaller batches of data (i.e., less than 50,000 measurements). If you want to download a large batch of data (e.g., more than 50,000), go to Option 2. On the order of 5,000 to 10,000 cloud observations are submitted per month.
Let’s get started!
Option 1. Download dust observations from GLOBE’s Advanced Data Access Tool (ADAT)
This is a great option if you want to download the data to a spreadsheet.
GLOBE’s Advanced Data Access Tool (ADAT; pronounced aye-dat) allows you to search for data by date, geographic region, school or organization, and protocol type. ADAT returns data in a comma separated value (CSV) file.
To access ADAT, go to globe.gov.
Click on GLOBE Data > Retrieve Data (ADAT)
Tip: Looking for documentation about GLOBE data? It’s in the menu on the left. Click on GLOBE Data User Guide.
On the ADAT landing page, click on Enter the Data Access Tool
Click on Data Filters > Select Protocols
Check the box next to Clouds and click Add to Filter
Click on Data Range and enter your desired date range. In this example, we are going to download data from 1 July 2019 to 1 August 2019. Click Add to Filter.
If you want to refine your data search further, you can search for specific sites, schools, countries, or geographic regions. In this example, we want all the data, so these search options are left blank.
Click Apply Filter
Click Obtain Measurement Data
Click Download Measurement Data
ADAT may return the data directly as CSV file or as a compressed (“zipped”) file depending on your computer’s operating system. If ADAT returns a CSV file, you can skip the next two steps. Follow the next two steps if ADAT returns the data as a zip file.
Save the zip file to your desired location.
Navigate to the folder finder on your desktop. Double click on the saved zip file to unzip it.
Double click on the CSV file to open it in Excel.
A spreadsheet like this will open in Excel. This contains all GLOBE clouds observations submitted between 1 July 2019 and 1 August 2019.
Now we are going to filter for just the GLOBE clouds observations reporting dust. Start by deleting the second row, so we can apply Excel’s filter function.
Select the first row. Then click on Sort & Filter in the top right corner. Select Filter from the dropdown menu.
Scroll over to column AD sky conditions:dust. Click on the down arrow on the header name. A pop-up window will appear. Uncheck the box next to (Blanks). Then close the pop-up window. What now appears in the spreadsheet are all the dust events reported to GLOBE between 1 July 2019 to 1 August 2019.
Click Try it out
Scroll down in the Protocols for search menu and select sky_conditions. Enter 2019-07-01 as the Start d0ate. Enter 2019-08-01 as the End date. Note dates must be entered as YYYY-MM-DD. The default file format is GeoJSON. In this example, I want the data returned as a JSON file. Select FALSE from the geojson dropdown menu. Click Execute.
Executing this call returns the Curl, request URL, and a sample of 10 observations is displayed in the Response body. Take moment to spot check the sample of data returned in the Response body. If the data looks okay, you can retrieve the full dataset.
To retrieve the full desired dataset, copy-past the Request URL into a new window in your web browser. After pasting the URL, edit sample=TRUE to sample=FALSE. Hit return on your keyboard to execute the search.
All GLOBE clouds data from 01 July 2019 to 01 August 2019 will be returned in JSON format to your web browser window. In this example, I am using Firefox because it displays JSON data nicely. Selecting Raw Data > Pretty Print makes it easier to read the data on the screen.
If you search (ctrl-f) the term “dust”, you see GLOBE clouds observations reporting dust.
To download the data, click Save.
Name your file and save to your desired location.
Now you can work with the JSON (or GeoJSON) data in the analysis or plotting software your normally use.
Python Script to download GLOBE clouds data from GLOBE API
Here is a python script to download GLOBE clouds data from the GLOBE API between given start and end dates, and extract observations reporting dust. Dust observations are plotted on a map and as a timeseries.
Tip: You’ll need the Anaconda distribution installed for the script to work. You can download it here.
When the script runs successfully, you’ll see the following output:
-- Downloading from API...
-- https://api.globe.gov/search/v1/measurement/protocol/measureddate/?protocols=sky_conditions&startdate=2019-07-01&enddate=2019-08-19&geojson=TRUE&sample=FALSE
-- Download successful. Saved to:
-- sky_conditions_20190701_20190819.json
-- Reading JSON from sky_conditions_20190701_20190819.json...
-- Interpreting file as JSON...
-- Parsing JSON as observations: 100% 11354/11354 [00:00<00:00, 541523.61it/s]
-- Preparing observations: 100% 97/97 [00:00<00:00, 265739.70it/s]
