All posts by The Zooniverse

Online citizen science projects. The Zooniverse is doing real science online,.

Science Scribbler: Key2Cat Update from Nanoparticle Picking Workflow

Science Scribbler: Key2Cat Update from Nanoparticle Picking Workflow

Hi!

This is the Science Scribbler Team with some exciting news from our latest project: Key2Cat! We have been blown away by the incredible support of this community – hundreds of you have taken part in the Key2Cat project (https://www.zooniverse.org/projects/msbrhonclif/science-scribbler-key2cat) and helped to pick nanoparticles in our electron microscopy images of catalyst nanoparticles. In just 1 week, over 50,000 classifications were completed on 10,000 subjects and 170,000 nanoparticles and clusters were found!

Thank you for this huge effort!

We went through the data and prepared everything for the next step: classification. Getting the central coordinates of our nanoparticles and clusters with the correct class will allow us to improve our deep learning approach. But before getting into the details of the next steps, let’s recap what has been done so far using the gold on germanium (Au/Ge) data as an example.

PICKING CATALYST PARTICLES

In the first workflow, you were asked to pick out both nanoparticles and clusters using a marking tool, which looked something like this:

As you might have realized, each of the images was only a small piece of a whole image. We tiled the images so that they wouldn’t be so overwhelming and time-consuming for an individual volunteer to work with. We also built in some overlap between the tiles so that if a nanoparticle fell on the edge in one image, it would be in the centre in another. Each tile was then shown to 5 different volunteers so that we could form a consensus on the centres of nanoparticles and clusters.

CRUNCHING THROUGH THE DATA

With your enormous speed, the whole Au/Ge dataset (94 full size images) was classified in just a few days! We have collected all of your marks and sorted them into their corresponding tiles. If we consider just a single tile that has been looked at by 5 volunteers, this is what the output data looks like:


With some thinking and coding we can recombine all the tiles that make up a single image, including the marks placed by all volunteers that contributed to the image:

Recontructed marked image

Wow, you all are really good at picking out the catalyst particles! Seeing how precisely all centres have been picked out in this visualisation is quite impressive. You may notice that there are more than 5 marks per nanoparticle – this is because of the overlap that we mentioned earlier. When taking the overlap into consideration, this means that each nanoparticle should be seen (at least partially!) by 20 volunteers.

The next step is to combine all of the marks to find a consensus centre point for each nanoparticle so that we have one set of coordinates to work with. There are numerous ways of doing this. One of the first that has given us good results is an unsupervised k-means algorithm [1]. This algorithm looks at all of the marks on the image and tries to find clusters of marks that are close to each other. It then joins these marks up into a single mark by finding a weighted average of their placements. You can think of it like tug-of-war where the algorithm finds the centre point because more marks are pulling it there.  

Reconstructed image with centroids of marks

As you can see, the consensus based on your marks almost perfectly points at the centres of individual nanoparticles or nanoparticle clusters. We don’t yet know from this analysis if the nanoparticle is a part of a cluster or not, and in some cases, we also get marks in areas which are not nanoparticles as shown in the orange and red boxes above. Since only small parts of the overall image were shown in the marking task, the artifact in the orange box was mistaken as a nanoparticle and in the case of the red box, there is a mark at the very edge and on a very small dot-like instance where some of you might have been suspicious about another nanoparticle. This is expected, especially since we asked volunteers to place marks if they were unsure – we wanted to capture all possible instances of nanoparticles in this first step!

REFINING THE DATA

This is the part where the second workflow comes into play. Using the marks from the first workflow, we createda new dataset showing just a small area around the mark to collect more information.In this workflow we ask a few questions to help identify exactly what we see at each of the marks


With this workflow, we hope to classify all the nanoparticles and clusters of both the Au/Ge and Pd/C catalyst systems, while potential false marks can be cleaned up! Once this is accomplished, we’ll have all the required inputs to improve our deep learning approach.

We’re currently collecting classifications on the Au/Ge data and will soon switch over to the Pd/C data, so if you have a few spare minutes, we would be very happy if you left some classifications in our project! https://www.zooniverse.org/projects/msbrhonclif/science-scribbler-key2cat/classify

-Kevin & Michele


Got your interest? Do you have questions? Get in touch!

Talk: https://www.zooniverse.org/projects/msbrhonclif/science-scribbler-key2cat/talk

References:

[1]: M. Ahmed, R. Seraj, S. M. S. Islam, Electronics (2020), 9 (8), 1295.

A Sky Full of Chocolate Sauce: Citizen Science with Aurora Zoo

by Dr. Liz MacDonald and Laura Brandt

Viewing the aurora in person is a magnificent experience, but due to location (or pesky clouds) it’s not always an option. Fortunately, citizen science projects like Aurorasaurus and Zooniverse’s Aurora Zoo make it easy to take part in aurora research from any location with an internet connection. 

The Aurorasaurus Ambassadors group was excited to celebrate Citizen Science Month by inviting Dr. Daniel Whiter of Aurora Zoo to speak at our April meeting. In this post we bring you the highlights of his presentation, which is viewable in full here

To ASK the Sky for Knowledge

Far to the north on the Norwegian island of Svalbard, three very sensitive scientific cameras gaze at a narrow patch of sky. Each camera is tuned to look for a specific wavelength of auroral light, snapping pictures at 20 or 32 frames per second. While the cameras don’t register the green or red light that aurora chasers usually photograph, the aurora dances dynamically across ASK’s images. Scientists are trying to understand more about what causes these small-scale shapes, what conditions are necessary for them to occur, and how energy is transferred from space into the Earth’s atmosphere. ASK not only sees night-time aurora, but also special “cusp aurora” that occur during the day but are only visible in extremely specific conditions (more or less from Svalbard in the winter.)

Still from Dr. Whiter’s presentation. The tiny blue square on the allsky image (a fisheye photo looking straight up) represents the field of view of the ASK cameras. The cameras point almost directly overhead. 

The setup, called Auroral Structure and Kinetics, or ASK, sometimes incorporates telescopes, similar to attaching binoculars to a camera. Project lead Dr. Daniel Whiter says, “The magnification of the telescopes is only 2x; the camera lenses themselves already provide a small field of view, equivalent to about a 280mm lens on a 35mm full frame camera. But the telescopes have a large aperture to capture lots of light, even with a small field of view.”

The challenge is that ASK has been watching the aurora for fifteen years and has amassed 180 terabytes of data. The team is too small to look through it all for the most interesting events, so they decided to ask for help from the general public. 

Visiting the Aurora Zoo

Using the Zooniverse platform, the Aurora Zoo team set up a project with which anyone can look at short clips of auroras to help highlight patterns to investigate further. The pictures are processed so that they are easier to look at. They start out black and white, but are given “false color” to help make them colorblind-friendly and easier for citizen scientists to work with. They are also sequenced into short video clips to highlight movement. To separate out pictures of clouds, the data is skimmed by the scientists each day and run through an algorithm.

Aurora Zoo participants are then asked to classify the shape, movement, and “fuzziness,” or diffuse quality, of the aurora. STEVE fans will be delighted by the humor in some of the options! For example, two of the more complex types are affectionately called “chocolate sauce” and “psychedelic kaleidoscope.” So far, Aurora Zoo citizen scientists have analyzed 7 months’ worth of data out of the approximately 80 months ASK has been actively observing aurora. Check out Dr. Whiter’s full presentation for a walkthrough on how to classify auroras, and try it out on their website!

Some of the categories into which Zooniverse volunteers classify auroral movement. Credit: Dr. Daniel Whiter.

What can be learned from Aurora Zoo is different from other citizen science projects like Aurorasaurus. For example, when several arc shapes are close to one another, they can look like a single arc to the naked eye or in a photo, but the tiny patch of sky viewed through ASK can reveal them to be separate features. These tiny details are also relevant to the study of STEVE and tiny green features in its “picket fence”.

Early (Surprising!) Results

Aurora Zoo participants blew through the most recent batch of data, and fresh data is newly available. The statistics they gathered show that different shapes and movements occur at different times of day. For example, psychedelic kaleidoscopes and chocolate sauce are more common in the evening hours. The fact that the most dynamic forms show up at night rather than in the daytime cusp aurora reveals that these forms must be connected to very active aurora on the night side of the Earth. 

Aurora Zoo participants also notice other structures. Several noted tiny structures later termed “fragmented aurora-like emissions,” or FAEs. Because of the special equipment ASK uses, the team was able to figure out that the FAEs they saw weren’t caused by usual auroral processes, but by something else. They published a paper about it, co-authored with the citizen scientists who noticed the FAEs. 

Still from Dr. Whiter’s presentation, featuring FAEs and Aurora Zoo’s first publication.

What’s next? Now that Aurora Zoo has a lot of classifications, they plan to use citizen scientists’ classifications to train a machine learning program to classify more images. They also look forward to statistical studies, and to creating new activities within Aurora Zoo like tracing certain shapes of aurora. 

STEVE fans, AuroraZoo hasn’t had a sighting yet. This makes sense, because ASK is at a higher latitude than that at which STEVE is usually seen. However, using a similar small-field technique to examine the details of STEVE has not yet been done. It might be interesting to try and could potentially yield some important insights into what causes FAEs.

Citizen Science Month, held during April of each year, encourages people to try out different projects. If you love the beautiful Northern and Southern Lights, you can help advance real aurora science by taking part in projects like Aurora Zoo and Aurorasaurus

About the authors of this blog post: Dr. Liz MacDonald and Laura Brandt lead a citizen science project called Aurorasaurus. While not a Zooniverse project, Aurorasaurus tracks auroras around the world via real-time reports by citizen scientist aurora chasers on its website and on Twitter. Aurorasaurus also conducts outreach and education across the globe, often through partnerships with local groups of enthusiasts.  Aurorasaurus is a research project that is a public-private partnership with the New Mexico Consortium supported by the National Science Foundation and NASA. Learn more about NASA citizen science here

Engaging Faith-based Communities in Citizen Science through Zooniverse

Engaging Faith-based Communities in Citizen Science through Zooniverse was an initiative designed to broaden participation in people-powered research (also referred to as citizen science) among religious and interfaith communities by helping them to engage with science through Zooniverse. Citizen science is a powerful way to build positive, long-term relationships across diverse communities by “putting a human face” on science and scientists. Participating in real scientific research is a great way to learn about the process of science as well as the scientists who conduct research.

The Engaging initiative provided models for how creative partnerships can be formed between scientific and religious communities that empower more people to become collaborators in the quest for knowledge. It included integrating Zooniverse projects into seminary classes as well as adult, youth, and intergenerational programs of religious communities; and promoting Zooniverse among interfaith communities concerned with environmental justice. Among other things, the project’s evaluation highlighted the need for scientists to do a better job of engaging with religious audiences in order to address racial and gender disparities in science. I encourage Zooniverse research teams to check out the series of short videos recently released by the AAAS Dialogue on Science, Ethics, and Religion to help scientists engage more effectively with communities of faith. By interacting personally with these communities and helping to “put a human face” on science, you may not only increase participation in your research projects, but help in the effort to diversify science in general.

Despite the difficulties imposed by the pandemic, I’m encouraged by what the Engaging initiative achieved, and the possibilities for expanding its impact in the future! The summary article of this project was published on March 28, 2022 by the AAAS Dialogue on Science, Ethics, and Religion.

Grace Wolf-Chase, Ph.D.

The project team thanks the Alfred P. Sloan Foundation for supporting this project. Any opinions, findings, or recommendations expressed are those of the project team and do not necessarily reflect the views of the Sloan Foundation.

THE RESULTS ARE IN – Grant’s Great Leaving Challenge

The time has come to announce the winners of Grant’s Great Leaving Challenge! Many thanks to all who submitted classifications for our four featured projects over the past week. Your efforts have absolutely wowed us at the Zooniverse – not only did you meet the 100,000 classifications goal, you blew right through it. All in all, you submitted a whopping 293,692 classifications – nearly 3x our goal!

This classification challenge was a massive push forward for the projects involved, and the research teams are incredibly grateful. Grant himself was touched – he had this to say about the results of his namesake challenge:

“Over the last decade I’ve constantly been blown away by the amazing effort and commitment from Zooniverse volunteers, and yet again they have surpassed all expectations! I want to thank them for all they have done, both for this challenge, and over the entire lifetime of the project. THANK YOU!”

Here’s some data to back up just how successful this challenge was:

Figure 1. The x-axes show each day the challenge ran, while the y-axes mark the percent change in classifications from the week prior. For example, this means that for Penguin Watch, there was a 100% increase in classifications on Tuesday March 22nd compared to Tuesday March 15th.

Figure 2. Here, each plot shows the date on the x-axis and the total number of classifications for that day on the y-axis. The shaded areas indicate which days were part of the challenge, and the non-shaded white areas prior are data from the preceding week. Note that the y-axes are unequal across plots because they’ve been scaled to fit their own data.

While, in this case, I do really think the figures speak for themselves, here are some highlights:

Just two days into the challenge, daily classifications for Dingo? Bingo! more than doubled compared to one week prior. A short two days later, they reached a 300% increase from the same day the previous week. All in all, Dingo? Bingo! volunteers submitted an incredible 112,505 classifications!

Planet Hunters NGTS volunteers rode a hefty 200% increase in classifications for the first two days of the challenge. On the fifth day, they peaked at an incredible 300% increase! Overall, volunteers submitted a whopping 115,388 classifications over the course of the 6 day challenge. Remarkable!

Penguin Watch volunteers readily doubled classifications from the week prior, with a peak on the fourth day when classifications were up more than 200% from the preceding week. By the end of the challenge, volunteers had submitted a grand total of 55,787 classifications!

On day two of the challenge, Weather Rescue at Sea volunteers submitted an astonishing 350% more classifications than one week prior. On the final two days of the challenge, classifications were up by nearly 400% from the preceding week! Overall, volunteers submitted an awesome 10,012 classifications.

When pulling together this data, we were just absolutely amazed by how much effort the volunteers put into Grant’s Great Leaving Challenge. What an awesome example of the power of citizen science. From all of us at the Zooniverse and from the project teams who took part in the challenge – thank you. This has been such a fun way to send off Grant, who will be greatly missed by all!

Grant’s Great Leaving Challenge

If you subscribe to our newsletters, the name “Grant” probably sounds familiar to you. Grant (our Project Manager and basically the ‘backbone of the Zooniverse’) has been with us for nearly 9 years, and with a heavy heart we’re sad to report he’s finally moving on to his next great adventure.

To mark his departure, we’ve announced “Grant’s Great Leaving Challenge”. The goal of this challenge is to collect 100,000 new classifications for the four Featured Projects on the homepage. Starting yesterday, if you submit at least 10 classifications total for these projects your name will automatically be entered to win one of three prizes. Importantly, you must be logged-in while classifying to be eligible for the draw. The challenge will end on Sunday, March 27th at midnight (GMT), and the winners will be announced on Tuesday, March 29th.

While we aren’t divulging what the prizes are, it might tempt you to hear that they’ll be personalised by Grant himself…

Read on to learn about the four featured projects, and what you can do to help them out.

Penguin Watch
Penguins – globally loved, but under threat. Research shows that in some regions, penguin populations are in decline; but why? Begin monitoring penguins to help us answer this question. With over 100 sites to explore, we need your help now more than ever!

Planet Hunters NGTS
The Next-Generation Transit Survey have been searching for transiting exoplanets around the brightest stars in the sky. We need your help sifting through the observations flagged by the computers to search for hidden worlds that might have been missed in the NGTS team’s review. Most of the planets in the dataset have likely been found already, but you just might be the first to find a new exoplanet not known before!

Dingo? Bingo!
The Myall Lakes Dingo Project aims to develop and test non-lethal tools for dingo management, and to further our understanding and appreciation of this iconic Australian carnivore. We have 64 camera-traps across our study site, and need your help to identify the animals they detect – including dingoes.

Weather Rescue at Sea
The aim of the Weather Rescue At Sea project is to construct and extended the global surface temperature record back to the 1780s, based on the air temperature observations recorded across the planet. This will be achieved by crowd-sourcing the recovery (or data rescue) of the weather observations from historical ship logbooks, station records, weather journals and other sources, to produce a longer, and more consistent dataset of global surface temperature.

Let’s send Grant off with a bang. Happy classifying!

Zooniverse Volunteers Discover More New Worlds

The volunteers on our Planet Hunters TESS project have helped discover another planetary system! The new system, HD 152843, consists of two planets that are similar in size to Neptune and Saturn in our own solar system, orbiting around a bright star that is similar to our own Sun. This exciting discovery follows on from our validation of the long-period planet around an evolved (old) star, TOI-813, and from our recent paper outlining the discovery of 90 Planet Hunters TESS planet candidates, which gives us encouragement that there are a lot more exciting systems to be found with your help!

Figure: The data obtained by NASA’s Transiting Exoplanet Survey Satellite which shows two transiting planets. The plot shows the brightness of the star HD 152843 over a period of about a month. The dips appear where the planets passed in front of the star and blocked some of its light from getting to Earth.


Multi-planet systems, like this one, are particularly interesting as they allow us to study how planets form and evolve. This is because the two planets that we have in this system must have necessarily formed out of the same material at the same time, but evolved in different ways resulting in the different planet properties that we now observe.


Even though there are already hundreds of confirmed multi-planet systems, the number of multi-planet systems with stars that are bright enough such that we can study them using ground-based telescopes remains very small. However, the brightness of this new citizen science found system, HD 152843, makes it an ideal target for follow-up observations, allowing us to measure the planet masses and possibly even probe their atmospheric composition.


This discovery was made possibly with the help of tens of thousands of citizen scientists who helped to visually inspect data obtained by NASA’s Transiting Exoplanet Survey Satellite, in the search for distant worlds. We thank all of the citizen scientists taking part in the project who continue to help with the discovery of exciting new planet systems and in particular to Safaa Alhassan, Elisabeth M. L. Baeten, Stewart J. Bean, David M. Bundy, Vitaly Efremov, Richard Ferstenou, Brian L. Goodwin, Michelle Hof, Tony Hoffman, Alexander Hubert, Lily Lau, Sam Lee, David Maetschke, Klaus Peltsch, Cesar Rubio-Alfaro, Gary M. Wilson, the citizen scientists who directly helped with this discovery and who have become co-authors of the discovery paper.


The paper has been published by the Monthly Notices of the Royal Astronomical Society (MNRAS) journal and you can find a version of it on arXiv at: https://arxiv.org/pdf/2106.04603.pdf.

Corporate Digital Engagement and volunteering through Zooniverse

Over the years a growing number of companies have included Zooniverse in their digital engagement and volunteer efforts, connecting their employee network with real research projects that need their help.

It’s been lovely hearing the feedback from employees:

“This was an awesome networking event where we met different team members and also participated in a wonderful volunteer experience. I had so much fun!”

“This activity is perfectly fitted to provide remote/virtual support. You can easily review photos from anywhere. Let’s do this again!”

“Spotting the animals was fun; a nice stress reliever!’

The impact of these partnerships on employees and on Zooniverse has been tremendous. For example, in 2020 alone, 10,000+ Verizon employees contributed over a million classifications across dozens of Zooniverse projects. With companies small to large incorporating Zooniverse into their volunteer efforts, this new stream of classifications has been a tremendous boon for helping propel Zooniverse projects towards completion and into the analysis and dissemination phases of their efforts. And the feedback from employees has been wonderful — participants across the board express their appreciation for having a meaningful way to engage in real research through their company’s volunteer efforts. 

A few general practices that have helped set corporate volunteering experiences up for success:

  • Focus and choice: Provide a relatively short list of recommended Zooniverse projects that align with your company’s goals/objectives (e.g., topic-specific, location-specific, etc.), but also leave room for choice. We have found that staff appreciate when a company provides 3-6 specific project suggestions (so they can dive quickly into a project), as well as having the option to choose from the full list of 70+ projects at zooniverse.org/projects
  • Recommend at least 3 projects: This is essential in case there happens to be a media boost for a given project before your event and the project runs out of active data*. Always good to have multiple projects to choose from. 
  • Team building: Participation in Zooniverse can be a tremendous team building activity. While it can work well to just have people participate individually, at their own convenience, it also can be quite powerful to participate as a group. We have created a few different models for 1-hour, 3-hour, and 6-hour team building experiences. The general idea is that you start the session as a group to learn about Zooniverse and the specific project you’ll be participating in. You then set a Classification Challenge for the hour (e.g., as a group of 10, we think we can contribute 500 classifications by the end of the hour). You play music in the background while you classify and touch base halfway through to see how you’re doing towards your goal (by checking your personal stats at zooniverse.org) and to share interesting, funny, and/or unusual images you’ve classified. At the end of the session, you celebrate reaching your group’s Classification Challenge goal and talk through a few reflection questions about the experience and other citizen science opportunities you might explore in the future. 
  • Gathering stats: Impact reports have been key in helping a company tell the story of the impact of their corporate volunteering efforts, both internally to their employee network and externally to their board and other stakeholders. 
    • Some smaller companies (or subgroups within a larger company) manually gather stats about their group’s participation in Zooniverse. They do this by taking advantage of the personal stats displayed within the Zooniverse.org page (e.g., number of classifications you’ve contributed). They request that their staff register and login to Zooniverse before participating and send a screenshot of their Zooniverse.org page at the end of each session. The team lead then adds up all the classifications and records the hours spent as a group participating in Zooniverse. 
    • If manual stats collection is not feasible for your company, don’t hesitate to reach out to us at contact@zooniverse.org to explore possibilities together. 

We’ve also created a variety of bespoke experiences for companies who are interested in directly supporting the Zooniverse. Please email contact@zooniverse.org if you’re interested in exploring further and/or have any questions. 

If you’re a teacher, school administrator, student, or anyone else who might be interested in having Zooniverse help you in fulfilling student volunteer or service hour requirements, please check out https://blog.zooniverse.org/2020/03/26/fulfilling-service-hour-requirements-through-zooniverse/ 

*Zooniverse project datasets range in size; everything from a project’s dataset being fully completed within a couple weeks (e.g., The Andromeda Project) to projects like Galaxy Zoo and Snapshot Serengeti that have run and will continue to run for many years. But even for projects that have data that will last many months or years, standard practice is to upload data in batches, lasting ~2-4 months. When a given dataset is completed, this provides an opportunity for the researchers to share updates about the project, interim results, etc. and encourage participation in the next cycle of active data. 

New Results for Milky Way Project Yellowballs!

What are “Yellowballs?” Shortly after the Milky Way Project (MWP) was launched in December 2010, volunteers began using the discussion board to inquire about small, roundish “yellow” features they identified in infrared images acquired by the Spitzer Space Telescope. These images use a blue-green-red color scheme to represent light at three infrared wavelengths that are invisible to our eyes. The (unanticipated) distinctive appearance of these objects comes from their similar brightness and extent at two of these wavelengths: 8 microns, displayed in green, and 24 microns, displayed in red. The yellow color is produced where green and red overlap in these digital images. Our early research to answer the volunteers’ question, “What are these `yellow balls’?” suggested that they are produced by young stars as they heat the surrounding gas and dust from which they were born. The figure below shows the appearance of a typical yellowball (or YB) in a MWP image.  In 2016, the MWP was relaunched with a new interface that included a tool that let users identify and measure the sizes of YBs. Since YBs were first discovered, over 20,000 volunteers contributed to their identification, and by 2017, volunteers catalogued more than 6,000 YBs across roughly 300 square degrees of the Milky Way. 

New star-forming regions. We’ve conducted a pilot study of 516 of these YBs that lie in a 20-square-degree region of the Milky Way, which we chose for its overlap with other large surveys and catalogs. Our pilot study has shown that the majority of YBs are associated with protoclusters – clusters of very young stars that are about a light-year in extent (less than the average distance between mature stars.) Stars in protoclusters are still in the process of growing by gravitationally accumulating gas from their birth environments. YBs that represent new detections of star-forming regions in a 6-square-degree subset of our pilot region are circled in the two-color (8 microns: green, 24 microns: red) image shown below. YBs present a “snapshot” of developing protoclusters across a wide range of stellar masses and brightness. Our pilot study results indicate a majority of YBs are associated with protoclusters that will form stars less than ten times the mass of the Sun.

YBs show unique “color” trends. The ratio of an object’s brightness at different wavelengths (or what astronomers call an object’s “color”) can tell us a lot about the object’s physical properties. We developed a semi-automated tool that enabled us to conduct photometry (measure the brightness) of YBs at different wavelengths. One interesting feature of the new YBs is that their infrared colors tend to be different from the infrared colors of YBs that have counterparts in catalogs of massive star formation (including stars more than ten times as massive as the Sun). If this preliminary result holds up for the full YB catalog, it could give us direct insight into differences between environments that do and don’t produce massive stars. We would like to understand these differences because massive stars eventually explode as supernovae that seed their environments with heavy elements. There’s a lot of evidence that our Solar System formed in the company of massive stars.

The figure below shows a “color-color plot” taken from our forthcoming publication. This figure plots the ratios of total brightness at different wavelengths (24 to 8 microns vs. 70 to 24 microns) using a logarithmic scale. Astronomers use these color-color plots to explore how stars’ colors separate based on their physical properties. This color-color plot shows that some of our YBs are associated with massive stars; these YBs are indicated in red. However, a large population of our YBs, indicated in black, are not associated with any previously studied object. These objects are generally in the lower right part of our color-color plot, indicating that they are less massive and cooler then the objects in the upper left. This implies there is a large number of previously unstudied star-forming regions that have been discovered by MWP volunteers. Expanding our pilot region to the full catalog of more than 6,000 YBs will allow us to better determine the physical properties of these new star-forming regions.

Volunteers did a great job measuring YB sizes!  MWP volunteers used a circular tool to measure the sizes of YBs. To assess how closely user measurements reflect the actual extent of the infrared emission from the YBs, we compared the user measurements to a 2D model that enabled us to quantify the sizes of YBs. The figure below compares the sizes measured by users to the results of the model for YBs that best fit the model. It indicates a very good correlation between these two measurements. The vertical green lines show the deviations in individual measurements from the average. This illustrates the “power of the crowd” – on average, volunteers did a great job measuring YB sizes!

Stay tuned…  Our next step is to extend our analysis to the entire YB catalog, which contains more than 6,000 YBs spanning the Milky Way. To do this, we are in the process of adapting our photometry tool to make it more user-friendly and allow astronomy students and possibly even citizen scientists to help us rapidly complete photometry on the entire dataset.

Our pilot study was recently accepted for publication in the Astrophysical Journal. Our early results on YBs were also presented in the Astrophysical Journal, and in an article in Frontiers for Young Minds, a journal for children and teens.

Researchers working to improve participant learning through Zooniverse

Our research group at Syracuse University spends a lot of time trying to understand how participants master tasks given the constraints they face. We conducted two studies as a part of a U.S. National Science Foundation grant to build Gravity Spy, one of the most advanced citizen science projects to date (see: www.gravityspy.org). We started with two questions: 1) How best to guide participants through learning many classes? 2) What type of interactions do participants have that lead to enhanced learning?  Our goal was to improve experiences on the project. Like most internet sites, Zooniverse periodically tries different versions of the site or task and monitors how participants do.

We conducted two Gravity Spy experiments (the results were published via open access: article 1 and article 2). Like in other Zooniverse projects, Gravity Spy participants supply judgments to an image subject, noting which class the subject belongs to. Participants also have access to learning resources such as the field guide, about pages, and ‘Talk’ discussion forums. In Gravity Spy, we ask participants to review spectrograms to determine whether a glitch (i.e., noise) is present. The participant classifications are supplied to astrophysicists who are searching for gravitational waves. The classifications help isolate glitches from valid gravitational-wave signals.

Gravity Spy combines human and machine learning components to help astrophysicists search for gravitational waves. Gravity Spy uses machine learning algorithms to determine the likelihood of a glitch belonging to a particular glitch class (currently, 22 known glitches appear in the data stream); the output is a percentage likelihood of being in each category.

Figure 1. The classification interface for a high level in Gravity Spy

Gradual introduction to tasks increases accuracy and retention. 

The literature on human learning is unclear about how many classes people can learn at once. Showing too many glitch class options might discourage participants since the task may seem too daunting, so we wanted to develop training while also allowing them to make useful contributions. We decided to implement and test leveling, where participants can gradually learn to identify glitch classes across different workflows. In Level 1, participants see only two glitch class options; in Level 2, they see 6; in Level 3, they see 10, and in Level 4, 22 glitch class options. We also used the machine learning results to route more straightforward glitches to lower levels and the more ambiguous subjects to higher workflows. So participants in Level 1 only saw subjects that the algorithm was confident a participant could categorize accurately. However, when the percentage likelihood was low (meaning the classification task became more difficult), we routed these to higher workflows.

We experimented to determine what this gradual introduction into the classification task meant for participants. One group of participants were funneled through the training described above (we called it machine learning guided training or MLGT);  another group of participants was given all 22 classes at once.  Here’s what we found:  

  • Participants who completed MLGT were more accurate than participants who did not receive the MLGT (90% vs. 54%).  
  • Participants who completed MLGT executed more classifications than participants who did not receive the MLGT (228 vs. 121 classifications).
  • Participants who completed MLGT had more sessions than participants who did not receive the MLGT (2.5 vs. 2 sessions). 

The usefulness of resources changes as tasks become more challenging

Anecdotally, we know that participants contribute valuable information on the discussion boards, which is beneficial for learning. We were curious about how participants navigated all the information resources on the site and whether those information resources improved people’s classification accuracy. Our goal was to (1) identify learning engagements, and (2) determine if those learning engagements led to increased accuracy. We turned on analytics data and mined these data to determine which types of interactions (e.g., posting comments, opening the field guide, creating collections) improved accuracy. We conducted a quasi-experiment at each workflow, isolating the gold standard data (i.e., the subjects with a known glitch class). We looked at each occasion a participant classified a gold standard subject incorrectly and determined what types of actions a participant made between that classification and the next classification of the same glitch class. We mined the analytics data to see what activities existed between Classification A and Classification B. We did some statistical analysis, and the results were astounding and cool. Here’s what we found:  

  • In Level 1, no learning actions were significant. We suspect this is because the tutorial and other materials created by the science team are comprehensive, and most people are accurate in workflow 1 (~97%).
  • In Level 2 and Level 3, collections, favoriting subjects, and the search function was most valuable for improving accuracy. Here, participants’ agency seems to help to learn. Anecdotally, we know people collect and learn from ambiguous subjects.
  • In Level 4, we found that actions such as posting comments and, viewing the collections created by other participants were most valuable for improving accuracy. Since the most challenging glitches are administered in workflow 4, participants seek feedback from others.

The one-line summary of this experiment is that when tasks are more straightforward, learning resources created by the science teams are most valuable; however, as tasks become more challenging, learning is better supported by the community of participants through the discussion boards and collections. Our next challenge is making these types of learning engagements visible to participants.

Note: We would like to thank the thousands of Gravity Spy participants without whom this research would not be possible. This work was supported by a U.S. National Science Foundation grant No. 1713424 and 1547880. Check out Citizen Science Research at Syracuse for more about our work.

Fixed Cross-Site Scripting Vulnerability on Zoomapper App

On 9 November 2020, a security researcher notified us of a cross-site scripting (XSS) vulnerability on our zoomapper application. This service hosts tile sets that are used to render maps for a small number of other Zooniverse applications, but is not connected to any critical Zooniverse infrastructure. This XSS vulnerability could have allowed users to execute malicious code on the zoomapper application in the browser.

We were able to remediate the vulnerability within hours of the report by disabling the browser GUI for zoomapper (see PR #6). The GUI had been turned on by default for the zoomapper app, but is not necessary to fulfill the app’s intended role.

Additional notes on the incident:

  • The vulnerability existed since the app was first deployed on September 15th 2020.
  • The vulnerability was located in the underlying Tileserver-GL dependency.
  • No Zooniverse user or project data was vulnerable or exposed by this vulnerability.

We’d like to thank Rachit Verma (@b43kd00r) for bringing this issue to our attention and for following responsible disclosure by reporting it to us in private, as requested on our security page.