Participants in CREA Mont-Blanc’s citizen science program Phénoclim make repeat observations of tree phenology. To help us study whether or not our data is reliable, it is useful to know if two volunteer observers independently make the same evaluation of the same tree. Here are the results and lessons learned from the experiment we carried out with students from Franklin University Switzerland this fall.
Students from Franklin University Switzerland observing the phenology of larch trees © CREA Mont-Blanc
Phénoclim and the differences between observers
In the citizen science program, Phénoclim, volunteer observers participate in monitoring plant phenology to help our understanding of the impacts of climate change on living things. The phenological event observed depends on the season, and in the fall, Phenoclim focuses on color change in leaves and needles. Over the course of the season, volunteers observe different tree species including birch, beech, mountain-ash and larch, recording the date when 10% and 50% of leaves or needles change color.
Making these observations is not as easy as it might seem, and a few key questions have arisen: is there consistency among observations made by different observers? When looking at the same tree, do volunteers note the same percentage of color change?
The protocol
To help us answer these questions, 20 students from Franklin University Switzerland headed out into the field in Vallorcine (1300 meters) and Loriaz (1900 meters) with the CREA Mont-Blanc team. In each of the study areas, 30 larch trees had been pre-identified. Students visited the two study areas and evaluated the percentage of color change in each of the 30 trees.
Results of their observations:
The interactive graphic below presents the results of the observations made by the students. Each point represents a single student’s observation of an individual tree. The observations from Vallorcine (1300 meters) appear in blue, while the observations made at Loriaz (1900 meters) appear in red.
This graphic shows the variability in the observations made by different observers for each tree. For example, the percentage of color change observed in tree #1 at Loriaz was recorded between 85 and 100%, depending on the observer. Another example coming from Vallorcine, where tree #1 was recorded as having between 5 and 25% color change depending on the observer.
Unsurprisingly, the observations made by different volunteers are rarely identical (when it is the case, the dots are superposed on one another for each tree in each zone). However, can we really say that they are that different? To respond to this question, we compared the observations of each volunteer with those of each of the other volunteers. For each observer-pair, we evaluated whether or not the difference in the percentage of color change recorded were statistically significant (pair in disagreement) or not (par in agreement).
While a few observer-pairs disagreed (11-14% of observer pairs), the majority were in agreement (86-89%).
Why these results interest us
This study demonstrated there is a high probability that two different observers will record the same observation when looking at the same tree. This is particularly reassuring when we consider the reliability of data because, in Phénoclim, a single observer is responsible for making observations of the plants in their own study zone.
What are potential next steps for taking this line of study further? We came up with a few study ideas, but we’re always open to suggestions! One possibility would be to try the same type of test on other tree species that we study in Phénoclim. Another would be to test observer variability for other, harder-to-observe phenological stages such as budburst in springtime.
A big thank you to the students who participated in this experiment!
Written by: Colin Van Reeth
Leave a Reply