Evaluation

Evaluator  -- Horizon Research, Inc. led by Eric Banilower

 

Nuggets posted at SMDEPO.org:

 

AREOKATS Highlighted in 2019 STEM For All Video Showcase

Acquire - Analyze - Apply (A3) 00:45 - 01:02

 

Mapping the Invisible: Practicing STEM Literacies   01:28 - 02:50

 

The Goal of AEROKATS and ROVER Education Network (AREN) is to train the next generation of scientists, engineers, and other professionals to observe and understand our planet Earth through experiential learning using NASA technology and data in real-world settings.

Towards this goal, the five-year AREN objectives are to:

  1. Apply NASA remote sensing and in-situ observation concepts, technology, and data in formal and informal learning settings for all ages and socioeconomic backgrounds.
  2. Apply NASA operations, NASA AEROKATS and ROVER technologies, and Earth Science concepts into a wide range of formal and informal STEM learning for educators, students and citizen scientists.
  3. Affordably implement AREN approaches, learning plans, and specific tools into the GLOBE Program.
  4. Increase participation in the GLOBE Program through involvement in the AREN Project and new AREN measurement protocols.
  5. Make affordably licensed AEROKATS and ROVER technologies and learning materials to the public through a distribution network.

 

The AREN objectives are consistent with those of the NASA Science Mission Directorate (SMD).

The four-fold Objectives of the NASA SMD Science Education (SE) Award are:

  1. Enable STEM Education (evaluated through the AREN Project),
  2. Improve U.S. Scientific Literacy,
  3. Advance National Education Goals, and
  4. Leverage Efforts Through Partnerships (evaluated through the AREN Project).

 

 

Each year, the AREN Project provides annual reports to NASA.  The annual reports can be found below:

Annual Report for 2018

Yearly Quad Chart for 2018

Annual Report for 2017

Yearly Quad Chart for 2017

Top Level Goals 1 and 4 for SMD - 2017

Evidence of Results / Impact for 2017

Evidence-Based Highlight - 2017 Data from the AREN Project

AREN Project Logic Model - 2017

 

Top Level Metrics: Presentations, Documents, Follow-On Actions and Conversation from the SMD Science Activation Annual Meeting, 2017 -- Full report from all 27 CAN Awardees.

 

Annual Report for 2016

 

Each month, the AREN Project provides a monthly update "Quad Chart/Status Report" to NASA.  The monthly "Quad Charts/Status Reports" can be found below:

August, 2019  Quad Chart/Status Report

July, 2019  Quad Chart/Status Report

June, 2019  Quad Chart/Status Report

May, 2019  Quad Chart/Status Report

April, 2019 Quad Chart/Status Report

March, 2019 Quad Chart/Status Report

February, 2019 Quad Chart/Status Report

January, 2019 Quad Chart/Status Report

September, 2018  Quad Chart/Status Report

August, 2018 Quad Chart/Status Report

July, 2018  Quad Chart/Status Report

June, 2018  Quad Chart/Status Report

May, 2018 Quad Chart/Status Report

April, 2018 Quad Chart/Status Report

March, 2018 Quad Chart/Status Report

February, 2018 Quad Chart/Status Report

January, 2017 Quad Chart

February, 2017 Quad Chart

March, 2017 Quad Chart

April, 2017 Quad Chart

May, 2017 Quad Chart

June, 2017 Quad Chart

July, 2017 Quad Chart

August, 2017 Quad Chart

September, 2017 Quad Chart

 

The AREN Project is regularly gathering data on workshops, presentations and the like.  The view this data please visit Events 2017 and Events 2018 on the AREN Project website.  The AREN Project regularly evaluates its progress.  All AREN partners can use the embedded word document or embedded pdf, to send in quarterly evaluations of progress.

 

 

Evaluation for this project is defined as a systematic investigation of the merit, worth or effectiveness of the project or curriculum material on teachers, students and their communities. Evaluation focuses on two main themes:

● Level of satisfaction of the participants

● Impact of the project on the targeted audience

 

Primary Questions Driving Our Work

Primarily, the following identified outcomes of the AREN project will be evaluated:

1. Did the Students and Teachers learn how remotely sensed data can be used to study Earth systems, and were they able to acquire remotely sensed data and produce meaningful information (including vegetation index, biomass and Land Use and Land Cover change over time) from that data?

2. Did the Students and Teachers learn how to collect and process remotely sensed imagery using AEROKATS kite-borne instrumentation?

3. Did the Students and Teachers learn how to collect and process in-situ atmospheric data using AEROKATS kite-borne instrumentation?

4. Did the Students and Teachers learn how to collect and process in-situ hydrologic data using ROVER aquatic vehicles?

5. Were the Students and Teachers able to utilize appropriate analysis and mapping tools to process and share their observation data via mapping tools (GIS), as well as contribute data where appropriate to the GLOBE database?

6. Were the Students able to participate in GLOBE field measurement campaigns and contribute observation data?

7. Were the Teachers able to become familiar with NGSS practices, disciplinary core ideas and cross cutting concepts in the disciplines of Earth science, biology, physics and chemistry?

8. Were the Teachers able to understand the fundamental concepts of Project Based Learning (PBL) and were they also able to implement an instructional unit using a PBL methodology grounded in inquiry and Student led investigations?

9. Did the Students and Teachers have access to technology through affordable hardware and software solutions and did they receive Benefits of augmentation of STEM engagement for traditionally underrepresented populations?

10. Was AREN able to establish a sustainable nationwide network to provide affordable Earth science data collection instruments and experiences, with ongoing professional development opportunities and project support?

 

The evaluation process, towards meeting the objectives of the program, will use an adaptation of the Kilpatrick and Kilpatrick (2006) four-level evaluation methodology which includes:

1. Reaction - How well did the learners (students) like the learning process?

2. Learning - To what extent did the learners (students) gain knowledge and skills?

3. Behavior - How did acquired knowledge and skills impact the behavior of the students and what was the consequential impact on science, society and the planet earth?

4. Results - What are the overall tangible impacts of the program in terms of the student knowledge and skills, development of student interest in earth science, changes in student perception or behavior, and projected long-term impact on society and the planet earth?