Insights for Informal Science Institutions from Citizen Science Projects

Today we have a guest post from Dr. Ryan Cook, Citizen Science Learning Researcher at the Adler Planetarium.  Ryan earned his PhD in socio-cultural anthropology from the University of Chicago.  His research interests include ethnographic  investigations in Mexico and the US on the intersection of science and religion.

It has been my pleasure to be a researcher for Zooniverse, based at the Adler Planetarium in Chicago, since May 2012. This position has exercised my anthropologist’s curiosity about how people understand and engage with science, taking it in an interesting and very productive new direction that I plan to continue. Thus I am pleased to have a chance to share my work on this blog.

At this writing I am close to completing my portion of a federally-funded project studying Zooniverse volunteers.  I have benefitted greatly in this research from the assistance of your esteemed edu-bloggers, Kelly and Laura, as well as my former Adler colleague Jason Reed and former supervisor Karen Carney. Specifically, we tried to determine whether and how much volunteers’ conceptions of and attitudes towards science changed through their participation in virtual citizen science projects.

This week, I presented some of our findings at the Visitor Studies Association’s annual conference in the town that beer built: Milwaukee, Wisconsin. Outfitted with a snazzy poster and a pile of official Zooniverse postcards and stickers, I argued for the relevance of our studies of Zoo volunteers to museums and science centers that want visitors to their websites to learn about science.

To know what could possibly be learned about science in Zooniverse, Karen, Kelly, and I put together a model of understanding science to guide us. We based the model’s criteria on what scholars who theorize, research, and teach science claimed as central characteristics of the sciences — for instance, relying on sense experience, proceeding methodically or logically, and revising knowledge in light of new evidence.

I then spent several months combing through Zooniverse databases and Google Analytics tables, trying to create a quantitative picture of how volunteers engaged with the tasks, blogs, and forums making up each Zoo. Figure 1 shows an example of the data by which we quantified and compared engagement among Zoos.

Fig. 1 - Old Weather visitor flow, Google Analytics
Fig. 1 – Old Weather visitor flow, Google Analytics

Following the lead of some preliminary statistics, Kelly and I applied our model to mapping out opportunities for learning about science in a subset of mature Zoos (i.e., those launched before the shift to an all-in-one-page design strategy). The Zoos were chosen in pairs with similar tasks but different levels of volunteer engagement:

[*Since the Supernovae Zoo was retired during the course of our project, it was included in the engagement variables but left out of subsequent research stages.]

Upon matching these engagement statistics to the range and type of learning opportunities we identified, three main patterns emerged:

  1. Opportunities for science learning were unevenly distributed within and across Zoos’ webpages. Talk and the Forums, for instance, allow a wide range of volunteers to engage in rich communication with each other and with moderators, administrators, and the science teams regarding the scientific import of the Zoos.
  2. The parts of the Zoos where volunteers went in the greatest numbers and spent the most time were typically those with the fewest, most limited, and least obvious learning opportunities. High-traffic, low-opportunity pages included the classification, marking, and transcription tasks at the core of each Zoo, as we can see in Figure 2.
Fig. 2 - average time on page by page type
Fig. 2 – average time on page by page type

1. Of the more than 700,000 volunteers to visit these Zoos at the time of our analysis, only a small percentage stayed long enough or reached enough pages to encounter many of the learning opportunities we identified.

Each of these findings makes sense if we bear in mind that Zooniverse did not start out as a platform for volunteers to learn about science, but rather as a tool for scientists to carry out certain kinds of data-intensive research.

I contended in my VSA presentation that this mismatch offered museums and science centers some guidance in how to (re)design their websites to improve the chances that visitors would encounter opportunities to learn what the institutions decided was important. Laura, Kelly, and the Zooniverse team have been testing out ways to design more learning opportunities into the “stickiest” parts of the Zoos.

And as for me, I have followed up this quantitative work with a series of in-depth interviews of heavily involved volunteers. By coding their responses based on an extended version of our science learning model, I aim to find out what they feel they learned from their Zooniverse engagement and how it helps us to determine how one segment of volunteers engaged with the science learning opportunities we identified. This interview material will appear along with the engagement data and the science learning model in my report, which should be completed by late September. Stay tuned: you will hear about it first!

3 thoughts on “Insights for Informal Science Institutions from Citizen Science Projects”

  1. This is terrific, can’t wait to read the paper that you’ll be publishing Ryan!

    Somewhat tangentially, I have an observation the anthropologist you may find curious: it is likely I meet your criteria for being a “heavily involved volunteer“, and as such, I seem to be of great interest to researchers like you … the number of approaches I have had (and surveys into ‘citizen scientists, their motivations, etc’ (or similar) I have taken part in) is now north of ten. My curiosity is aroused: assuming my experience is not atypical, what makes us “heavily involved volunteers” so desirable?

    Back to your post: the “Zooniverse did not start out as a platform for volunteers to learn about science“. Presumably that has now changed, otherwise why is there a Zooniverse Education Blog? And what can Laura, Kelly, and the Zooniverse team share with us, so far, concerning their work to test “ways to design more learning opportunities into the “stickiest” parts of the Zoos“? Do ordinary zooites, whether heavily involved or not, have an opportunity to become involved in identifying and characterizing such ways?

    Finally, from your work – in general, not just this study – would you say that Zooniverse Education would benefit (however you choose to define that) by having an active public forum, one in which true discussions can take place?

    1. Hi Jean! Great points. Some answers to some of your questions below….

      1.) One reason why you’ve received many invitations to participate as a heavily involved volunteer is that, frankly, this a small percentage of the overall Zooniverse volunteer demographics. It’s simply a numbers game – there are way fewer of the super-involved Zooniverse volunteers than there are of users who check out a project once and make a few classifications. By better understanding the motivations across a variety of our users, we’re hoping to learn how to craft experiences tailored to individual needs and interests of different types of Zooniverse volunteers.

      2.) Zooniverse projects remain first and foremost scientific research projects. There are a few benchmarks used to evaluate whether a proposed citizen science project will be built by the Zooniverse development team. One is that the task, whether identifying animals in Snapshot Serengeti or transcribing ships logs Old Weather, cannot be a task that could be automated. All task require human attention. We place a high value on our volunteer’s time. Second, each project has definite science goals that researchers are trying to understand. Classifications are not collected solely for the purposes of building a dataset that “could someday be useful to someone”. They are answers to questions that the science teams identify as key in advancing research in the particular field.

      As much as it pains me as educator to say this, educational research and educational opportunities are a secondary benefit to these active research projects. We started the education blog out of conversations we started having when we brought these citizen science projects to events like the National Science Teacher Association meeting. We knew there were teachers using Zooniverse projects in their classrooms, but had no idea how many. This blog and educational resources like ZooTeach act as way of communicating to other education practitioners the resources we have available and to encourage other to share ideas and resources they’ve developed.

      As for the “stickier” places on Zooniverse project sites, it seems like Talk is one of the areas of a project where Zooniverse volunteers, yet more volunteers than not don’t use this tool. Is it because how to use it is not inherently clear? Is it because that many of these volunteers simply don’t care to participate in this aspect of the project? We don’t know. Based on Ryan’s work and work of researchers, we’ve begun internal discussions on ways to make Talk more accessible and understandable. One possible solution may be to introduce a simple and optional tutorial giving interested volunteers an overview of how their Zooniverse peers use Talk.

      3.) I think the short answer to your question about an active public forum for Zooniverse Education like those on each of the Zooniverse projects is “Yes, but we haven’t figured out the best place and way to implement this.” Do we put it on ZooTeach? Do we have an education thread in each individual project’s Talk sites. We’re not sure yet.

      1. Thanks Kelly.

        I think this will likely be my last comment here; I am quite interested in engaging in discussions on the broad topic of zooites and Zooniverse projects, beyond mere classification, but will await a decent forum for such (and no, Talk would most definitely NOT be such a forum!)

        I think Ryan (et al.) may have missed something important. I’ll explain by example.

        Modulo poor memory (a very real bias!), my involvement in Galaxy Zoo goes back almost to the very beginning of that project: I began classifying not long after its launch. However, I did so under a completely different name/handle/avatar (I don’t even remember what it was), on a different computer than any I now use, a different ISP, etc, etc, etc. I was an avid reader of a great many of the threads in the Galaxy Zoo forum. At some point I wrote my first post (quite possibly to the ‘interim forum’, set up to allow continuity while the main forum was repaired; this forum was never merged back), which I have long since forgotten. Fast forward, and for reasons that have little to do with GZ or the Zooniverse, at some point I started classifying, and posting, under Jean Tate. By then I’d learned a great deal about galaxies, etc, and was totally bitten. One thing led to another, and I became the mod for that forum’s Object of the Day (OotD), and eventually its sole writer (effectively).

        I have come to know several of the regular readers of my OotD threads, but realize those are a small fraction of the total readership; the GZ forum has a ‘Views’ counter, and it often climbs to over 100 within a week (or at least it did; I haven’t written any for a while now); the most heavily read ones have well over 5k ‘Views’.

        Are the zooites who have been actively classifying galaxies – perhaps for several years’ now – and who have been avid readers of the GZ forum for just as long, BUT who have never (or rarely) written a post not “heavily involved volunteers”, *by definition*? What about all those who are enthusiastic but who lack the confidence in their ability to write English to venture to write a post? How would Ryan – or anyone – know how many micro-blogs there are, in Chinese, based on GZ work (to take just one possible example)? And so on.

        In short, I think there’s a great thirst among (a great many) ordinary zooites, to learn more. Given the right environment – and I recognize that those simple words hide great complexity, and much that is unknown – how many of those ‘eager but passive’ zooites would become ‘heavily involved volunteers’?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s