We recently uncovered a couple of bugs in the Zooniverse code which meant that the wrong question text may have been shown to some volunteers on Zooniverse projects while they were classifying. They were caught and a fix was released the same day on 29th November 2018.
The bugs only affected some projects with multiple live workflows from 6th-12th and 20th-29th November.
One of the bugs was difficult to recreate and relied on a complex timing of events, therefore we think it was rare and probably did not affect a significant fraction of classifications, so it hopefully will not have caused major issues with the general consensus on the data. However, it is not possible for us to say exactly which classifications were affected in the timeframe the bug was active.
We have apologised to the relevant science teams for the issues this may cause with their data analysis, but we would also like to extend our apologies to all volunteers who have taken part in these projects during the time the bugs were in effect. It is of the utmost importance to us that no effort is wasted on our projects and when something like this happens it is taken very seriously by the Zooniverse team. Since we discovered these bugs we worked tirelessly to fix them, and we have taken actions to make sure nothing like this will happen in the future.
We hope that you accept our most sincere apologies and continue the amazing work you do on the Zooniverse. If you have any questions please don’t hesitate to contact us at email@example.com.
Hi all, I am Coleman Krawczyk and for the past year I have been working on tools to help Zooniverse research teams work with their data exports. The current version of the code (v1.3.0) supports data aggregation for nearly all the project builder task types, and support will be added for the remaining task types in the coming months.
What does this code do?
This code provides tools to allow research teams to process and aggregate classifications made on their project, or in other words, this code calculates the consensus answer for a given subject based on the volunteer classifications.
The code is written in python, but it can be run completely using three command line scripts (no python knowledge needed) and a project’s data exports.
The first script is the uses a project’s workflow data export to auto-configure what extractors and reducers (see below) should be run for each task in the workflow. This produces a series of `yaml` configuration files with reasonable default values selected.
Next the extraction script takes the classification data export and flattens it into a series of `csv` files, one for each unique task type, that only contain the data needed for the reduction process. Although the code tries its best to produce completely “flat” data tables, this is not always possible, so more complex tasks (e.g. drawing tasks) have structured data for some columns.
The final script takes the results of the data extraction and combine them into a single consensus result for each subject and each task (e.g. vote counts, clustered shapes, etc…). For more complex tasks (e.g. drawing tasks) the reducer’s configuration file accepts parameters to help tune the aggregation algorithms to best work with the data at hand.
At the moment this code is provided in its “offline” form, but we testing ways for this aggregation to be run “live” on a Zooniverse project. When that system is finished a research team will be able to enter their configuration parameters directly in the project builder, a server will run the aggregation code, and the extracted or reduced `csv` files will be made available for download.
Occasionally we run studies in collaboration with external researchers in order to better understand our community and improve our platform. These can involve methods such as A/B splits, where we show a slightly different version of the site to one group of volunteers and measure how it affects their participation, e.g. does it influence how many classifications they make or their likelihood to return to the project for subsequent sessions?
One example of such a study was the messaging experiment we ran on Galaxy Zoo. We worked with researchers from Ben Gurion University and Microsoft research to test if the specific content and timing of messages presented in the classification interface could help alleviate the issue of volunteers disengaging from the project. You can read more about that experiment and its results in this Galaxy Zoo blog post https://blog.galaxyzoo.org/2018/07/12/galaxy-zoo-messaging-experiment-results/.
As the Zooniverse has different teams based at different institutions in the UK and the USA, the procedures for ethics approval differ depending on who is leading the study. After recent discussions with staff at the University of Oxford ethics board, to check our procedure was up to date, our Oxford-based team will be changing the way in which we gain approval for, and report the completion of these types of studies. All future study designs which feature Oxford staff taking part in the analysis will be submitted to CUREC, something we’ve been doing for the last few years. From now on, once the data gathering stage of the study has been run we will provide all volunteers involved with a debrief message.
The debrief will explain to our volunteers that they have been involved in a study, along with providing information about the exact set-up of the study and what the research goals were. The most significant change is that, before the data analysis is conducted, we will contact all volunteers involved in the study allow a period of time for them to state that they would like to withdraw their consent to the use of their data. We will then remove all data associated with any volunteer who would not like to be involved before the data is analysed and the findings are presented. The debrief will also contain contact details for the researchers in the event of any concerns and complaints. You can see an example of such a debrief in our original post about the Galaxy Zoo messaging experiment here https://blog.galaxyzoo.org/2015/08/10/messaging-test/.
As always, our primary focus is the research being enabled by our volunteer community on our individual projects. We run experiments like these in order to better understand how to create a more efficient and productive platform that benefits both our volunteers and the researchers we support. All clicks that are made by our volunteers are used in the science outcomes from our projects no matter whether they are part of an A/B split experiment or not. We still strive never to waste any volunteer time or effort.
We thank you for all that you do, and for helping us learn how to build a better Zooniverse.
Irma has brought widespread devastation to many islands in the Caribbean over the last few days, and now Hurricane Jose is a growing threat in the same region.
By analysing images of the stricken areas captured by ESA’s Sentinel-2 satellites, Zooniverse volunteers can provide invaluable assistance to rescue workers. Rescue Global are a UK-based disaster risk reduction and response charity who are deploying a team to the Caribbean and will use the information you provide to help them assess the situation on the ground.
Breaking news… Zooniverse volunteers on Exoplanet Explorers have discovered a new 4-planet system!
Congratulations to all* who directly classified the light curves for this system, bringing it to the attention of the research team. And an enormous *thank you* to the 14,000+ volunteers who provided over 2 million classifications in just three days to make this discovery possible. This is equivalent to 3.4 years of full time effort. I *heart* people-powered research! It’s also amazing how quickly we were able to get these data to the eyes of the public — the Kepler Space Satellite observed this star between December 15 and March 4, 2017. Data arrived on Earth on March 7th and Zooniverse volunteers classified it April 3-5, 2017. I *heart* Zooniverse.
ExoplanetExplorers.org was the featured project for our inaugural ABC Australia Stargazing Live 3-day, prime-time TV event, which just ended yesterday and through which this discovery was made. Over the years, we’ve partnered with the BBC as part of their Stargazing Live event in the UK. On night 1, Chris Lintott, our intrepid leader, invites the million+ viewers to participate in that year’s featured Zooniverse project, on night 2 he highlights interesting potential results coming through the pipeline, and on night 3, if science nods in our favor, he has the pleasure of announcing exciting discoveries you all, our volunteers, have made (for example, last year’s pulsar discovery and the supernova discovery from a couple years back).
This year we partnered with both the UK’s BBC and Australia’s ABC TV networks to run two Stargazing Live series in two weeks. We’re exhausted and exhilarated from the experience! We can imagine you all are as well (hats off to one of our volunteers who provided over 15,000 classifications in the first two days)!
Stargazing Live epitomizes many of our favorite aspects of being a member of the Zooniverse team – it’s a huge rush, filled with the highs and lows of keeping a site up when thousands of people are suddenly providing ~7000 classifications a minute at peak. We’re so proud of our web development team and their amazing effort; their smart solutions, quick thinking, and teamwork. The best part is that we collectively get to experience the joy, wonder, and discovery of the process of science right alongside the researchers. Each year the research teams leading each project have what is likely among the most inspiring (and intense) 3-days of their careers, carrying out the detective work of following up each potential discovery at breakneck speed.
Brad Tucker and his team leading PlanetNineSearch.org featured in the BBC Stargazing Live event this year checked and rechecked dozens of Planet 9 candidates orbital parameters and against known object catalogs, making sure no stone was left unturned. We were bolstered throughout with re-discoveries of known objects, including many known asteroids and Chiron, a minor planet in the outer Solar System, orbiting the Sun between Saturn and Uranus.
Even though Planet 9 hasn’t been discovered yet, it’s huge progress for that field of research to have completed a thorough search through this Skymapper dataset, which allows us to probe out to certain distances and sizes of objects across a huge swath of the sky. Stay tuned for progress at planetninesearch.org and through the related BackyardWorlds.org project, searching a different parameter space for Planet 9 in WISE data.
Also, and very importantly, the BBC Stargazing Live shows gave the world an essential new member of the Twitterverse:
The Exoplanet Explorers team, led by Ian Crossfield, Jessie Christiansen, Geert Barentsen, Tom Barclay, and more were also up through much of each night of the event this week, churning through the results. Because the Kepler Space Telescope K2 dataset is so rich, there were dozens of potential candidates to triple check in just 3 days. Not only did our volunteers discover the 4-planet system shown above, but 90 new and true candidate exoplanets! That’s truly an amazing start to a project.
Once you all, our amazing community, have classified all the images in this project and the related PlanetHunters.org, the researchers will be able to measure the occurrence rates of different types of planets orbiting different types of stars. They’ll use this information to answer questions like — Are small planets (like Venus) more common than big ones (like Saturn)? Are short-period planets (like Mercury) more common than those on long orbits (like Mars)? Do planets more commonly occur around stars like the Sun, or around the more numerous, cooler, smaller “red dwarfs”?
There’s also so much to learn about the 4-planet system itself. It’s particularly interesting because it’s such a compact system (all orbits are well within Mercury’s distance to our Sun) of potentially rocky planets. If these characteristics hold true, we expect they will put planet formation theories to the test.
A fun part of our effort for the show was to create visualizations for this newly discovered system. Simone, one of our developers, used http://codepen.io/anon/pen/RpOYRw to create the simulation shown above. We welcome all to try their hand using this tool or others to create their favorite visualization of the system. Do post your effort in the comments below. To set you on the right path, here are our best estimates for the system so far:
The star is in the constellation of Aquarius (see if can get the WWT), with ra, dec = 23:15:47.77, -10:50:58.91.
Host star (V=12): 0.8 Rsol, 0.9 Msol. Late G or early K.
We predict there may be more planets further out, with similar resonances as the inner planets. The predictions for outer planets are 20d, 30.7d, 47d, etc. (assuming Per_x = 3.56 * 1.538^x.). Planet number 11 would be ~264d, planet 12 ~405d.
There are 73 other previously discovered exoplanet systems with 4 or more planets known.
In 2372 years, on July 9, 4388AD, all four planets will transit at the same time.
If you’re standing on planet e, the nearest planet would appear bigger than the full moon on the sky. Apparent size of other planets while standing on e = 10 arcmin, 16 arcmin, 32 arcmin.
If you’re on planet e, the star barely appears to rotate: you see the same side of it for many “years,” because the star rotates just as quickly as planet “e” goes around it.
This post wouldn’t be complete without a thank you to Edward Gomez for following up candidates with the Los Cumbres Observatory Robotic Telescope Network. Not only is LCO a great research tool, but it provides amazing access to telescopes and quality curricular materials for students around the world.
*And a special thanks to the following volunteers who correctly identified at least one the planets in the newly discovered 4-planet system:
Alan Patricio Zetina Floresmarhx
Anastasios D. Papanastasiou
We’re testing out a new feature of our interface, which means if you’re classifying images on Comet Hunters you may see occasional pop-up messages like the one pictured above.
The messages are designed to give you more information about the project. If you do not want to see them, you have the option to opt-out of seeing any future messages. Just click the link at the bottom of the pop-up.
You can have a look at this new feature by contributing some classifications today at www.comethunters.org.
We’re cleaning up our email list to make sure that we do not email anyone who does not want to hear from us. You will have got an email last week asking you if you want to stay subscribed. If you did not click the link in that email, then you will have received one today saying you have been unsubscribed from our main mailing list. Don’t worry! If you still want to receive notifications from us regarding things like new projects, please go to www.zooniverse.org/settings/email and make sure you’re subscribed to general Zooniverse email updates.
NOTE: This has not affected emails you get from individual Zooniverse projects.
The AsteroidZoo community has exhausted the data that are available at this time. With all the data examined we are going to pause the experiment, and before users spend more time we want to make sure that we can process your finds through the Minor Planet Center and get highly reliable results.
We understand that it’s frustrating when you’ve put in a lot of work, and there isn’t a way to confirm how well you’ve done. But please keep in mind that this was an experiment – How well do humans find asteroids that machines cannot?
Often times in science an experiment can run into dead-ends, or speed-bumps; this is just the nature of science. There is no question that the AsteroidZoo community has found several potential asteroid candidates that machines and algorithms simply missed. However, the conversion of these tantalizing candidates into valid results has encountered a speed bump.
What’s been difficult is that all the processing to make an asteroid find “real” has been based on the precision of a machine – for example the arc of an asteroid must be the correct shape to a tiny fraction of a pixel to be accepted as a good measurement. The usual process of achieving such great precision is hands-on, and might take takes several humans weeks to get right. On AsteroidZoo, given the large scale of the data, automating the process of going from clicks to precise trajectories has been the challenge.
While we are paused, there will be updates to both the analysis process, and the process of confirming results with the Minor Planet Center. Updates will be posted as they become available.
We’re getting through the first round of Penguin Watch data- it’s amazing and it’s doing the job we wanted, which is to revolutionise the collection and processing of penguin data from the Southern Ocean – to disentangle the threats of climate change, fishing and direct human disturbance. The data are clearly excellent, but we’re now trying to automate processing them so that results can more rapidly influence policy.
In “PenguinWatch 2.0”, people will be able to see the results of their online efforts to monitor and conserve Antarctica’s penguins colonies. The more alert among you will notice that it’s not fully there yet, but we’re working on it!
We have loads of ideas on how to integrate this with the penguinwatch.org experience so that people are more engaged, learn more and realise what they are contributing to!
For now, we’re doing this the old-fashioned way; anyone such as schools who want to be more engaged, can contact us (firstname.lastname@example.org) and we’ll task you with a specific colony and feedback on that.
Recently my class of 8-9 year old kids from ZŠ Brno, Jihomoravské náměstí (a primary school in the Czech Republic) took part in several Zooniverse projects.
First, they were just talking about their dreams – what they would like to achieve in life. Mostly, they wanted to become a sports star or music celebrity, but some actually considered becoming a scientist!
Then they were introduced to the Zooniverse and citizen science. Fascinated by the idea than they can actually contribute to real science (so someone’s dream can come true), they dived into the list of projects on the Zooniverse website. All the cover images and project names were really attractive to them, sadly, only two projects are available in Czech. Anyway, the first project they started – Snapshots at Sea – was in English only. This project focusing on marine animals, especially cetaceans, is very simple though. The only task is to say whether there are any animals present in the picture. They learned the English question very quickly and classified over 200 images on their own. They asked various questions about those fascinating animals and looked hungry for more answers. Initially, they didn’t want to stop classifying, but when they heard the name of the following project to try – Penguin Watch, they were totally into it!
This project, available in Czech, shows wintery images of remote locations in Antarctica, usually crowded with nesting penguins. The tasks here are to mark adult penguins, chicks, or their eggs, and any predators, if present. They took turns marking, trying to mark at least 30 penguins as quickly as possible so they could see another image. They couldn’t wait to find an egg. And after only 9 images they succeeded!
They were curious about Antarctica, as well as about penguins. They wondered, why it is so cold there, and how are long polar days and nights likely to happen. Answering their last question would have been a great step to lead into trying a space project, as many of them are available on Zooniverse. But, they decided to try another wildlife project, Chimp & See, a project monitoring wild animals in Africa, especially chimpanzees and their behaviour. This project wasn’t as easy for them, as they were asked to identify unfamiliar animals in short video clips (they had to learn their names in English during classification) and then to describe their behaviour using a list of options. Surprisingly, they didn’t mind the language barrier much. After a short while, all of them were standing in front of the screen and everyone wanted to touch it! They seemed to be totally hooked.
The researchers from Chimp&See were so kind to offer them the chance to choose a name for a currently unidentified juvenile chimp, captured on 4 different video sequences! The kids were really excited by such an opportunity and suggested a lot of names to choose from. They were voting in the end and all agreed on a single name – Kibu!
When the lesson ended, many of them asked to create their own accounts, so they could participate on their own from home. Next time, we are going to try Plankton Portal and Floating Forests.
Zooniverse projects are really a great opportunity for kids to learn about nature, they bring them to the real science, and not to forget, they are great fun!
__ By Zuzana Macháčková, a primary school teacher in Brno and Zooniverse volunteer.