If you’ve ever wanted to build new Citizen Science projects, and live in Chicago, then now is probably a good time to visit the Zooniverse jobs page.
Category Archives: News
Work at the Zooniverse : SETI Scientist post

I’m pleased to announce that the Zooniverse team is expanding once again, and that we have a postdoc available at the Adler Planetarium. We’re looking for someone to help lead a joint project with the SETI institute, looking for interesting signals in data from the Allan Telescope Array. The post is funded by TED, and we’re keen to get started so please do apply or drop me a line if you’re interested.
Chris (chris AT zooniverse.org)
P.S. Here’s the formal job ad.
Recent downtime at The Zooniverse
Last week the majority of The Zooniverse websites suffered their first major outage for over 2 years. Firstly I wanted to reassure you that the valuable classifications you all provide was always secure but perhaps more importantly I wanted to say sorry – we should have recovered more quickly from this incident and we’ll be working hard this week to put better systems in place to enable us to recover more rapidly in the future.
For those who’d like more of the gory details, read on…
Some background
Many of you will know the story of the launch of the original Galaxy Zoo (in July 2007); just a few hours after launch the website hosted by the SDSS web team and Johns Hopkins University crashed under the strain of thousands of visitors to the site. Thankfully due to the heroic efforts of the JHU team (involving a new web server being built) the Galaxy Zoo site recovered and a community of hundreds of thousands of zooties was born.
Fast-forward to February 2009 and we were planning for the launch of Galaxy Zoo 2. This time we knew that we were going to have a busy launch day – Chris was once again going to be on BBC breakfast. So that we could keep the site running well during the extremely busy periods we looked to commercial solutions for scalable web hosting. There are a number of potential choices in this arena but by far the most popular and reliable service was that offered by Amazon (yes the book store) and their Web Services platform. While I won’t dig into the technical details of Amazon Web Services (AWS) here, the fundamental difference when running web sites on AWS is that you have a collection of ‘virtual machines’ rather than physical servers.
If any of you have ever run Virtual PC or VMWare on your own computer then you’ll already realise that using machine virtualisation it’s possible to run a number of virtual machines on a single physical computer. This is exactly what AWS do except they do it at a massive scale (millions of virtual machines) and have some fantastic tools to help you build new virtual servers. One particularly attractive feature of using these virtual machines is that you can essentially have as many as you want and you only pay by the hour. At one point on the Galaxy Zoo 2 launch day we had 20 virtual machines running the Galaxy Zoo website, API and databases. 2 days later we were running only 3. The ability to scale up (and down) in realtime the number of virtual machines means that we are able to cope with huge variations in the traffic that a particular Zooniverse site may be receiving.
The outage last week
As I write this blog post we currently have 22 virtual servers running on AWS. That includes all of The Zooniverse projects, the database servers, caching servers for Planet Hunters, our blogs, the forums and Talk and much more. Amazon have a number of hosting ‘regions’ that are essentially different geographical locations where they have datacenters. We happen to host in the ‘us-east’ region in Virginia – conveniently placed for both Europe and American traffic.
We have a number of tools in place that monitor the availability of our web sites and last Thursday at about 9am GMT I received a text-message notification that our login server (login.zooiverse.org) was down. We have a rota within the dev team for keeping an eye on the production web servers and last week it was my turn to be on call.
I quickly logged on to our control panel and saw that there was a problem with the virtual machine and attempted a reboot. At this point I also started to receive notifications that a number of the Zooniverse project sites were also unavailable. At this point realising that something rather unusual was going on I checked the Amazon status page which was ‘all green’, i.e. no known issues. Amazon can be a little slow to update this page so I also checked Twitter (https://twitter.com/#!/search/aws). Twitter was awash with people complaining that their sites were down and that they couldn’t access their virtual machines. Although this wasn’t ‘good’ news, it’s always helpful to understand in a situation such as this if the issue is with the code that we run on the servers or the servers themselves.
Waiting for a fix?
At this point we rapidly put up holding pages on our project sites and reviewed the status of each project site and service that we run. As the morning progressed it became clear that the outage was rather serious and actually became significantly worse for The Zooniverse as in turn, each of the database servers that we run became inaccessible. We take great care to execute nightly backups of all of our databases and so when the problems started the oldest backup was 4 hours old. With hindsight when the problems first started we should have immediately moved The Zooniverse servers to a different AWS region (this is actually what we did do with login.zooniverse.org) and booted up new database servers with the backup from the night before however we were reluctant to do this because of the need to reintegrate the classifications made by the community during the outage. But hindsight is always 20-20 and this isn’t what we did. Instead, believing that a fix for the current situation was only a matter of hours away we waited for Amazon to fix the problem for us.
As the day progressed a number of the sites became available for short periods and then inaccessible again. It wasn’t a fun day to be on operations duty with servers continually going up and down. Worse, our blogs were also unavailable so we only had Twitter to communicate what was going on. At about 11pm on the first evening a number of The Zooniverse projects had been up for a number of hours and things looked to be improving. Chris and I spoke on the phone and we agreed that if things weren’t completely fixed by the morning we’d move the web stack early Friday morning.
Friday a.m.
Friday arrived and the situation was slightly improved but the majority of the our projects were still in maintenance mode so I set about rebuilding The Zoonivere web stack. Three out of five of our databases were accessible again and so I took a quick backup of all of the three and booted up replacement databases and web servers. This was all we needed to restore the majority of the projects and by lunchtime on Friday we pretty much had a fully working Zooniverse again. The database server used by the blogs and forums took a little longer to recover and so it was Friday evening before they were back up.
A retrospective
We weren’t alone in having issues with AWS last week. Sites such as Reddit and Foursquare were also affected as were thousands of other users of the service. I think the team at the Q&A site Quora put it best when they said ‘We’d point fingers, but we wouldn’t be where we are today without EC2.’ on their holding page and this is certainly true of The Zooniverse.
Over the past 2 years we’ve only been able to deliver the number of projects that we have and the performance and uptime that we all enjoy due to the power, flexibility and reliability of the AWS platform. Amazon have developed a number of services that mean that it was possible (in theory) to protect against the failure they experienced in the US-east region last week. Netflix was notably absent from the list of AWS hosted sites affected by the AWS downtime and this is because they’ve gone to huge effort (and expense) to protect themselves against such a scenario (http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html).
Unfortunately with the limited resources available to The Zooniverse we’re not able to build a resilient web stack as Netflix however there are a number of steps we’ll be taking this to make sure that we’re in a much better position to recover from a similar incident in the future so that we experience downtime of minutes and hours rather than days.
Cheers
Arfon & The Tech Team
Zooniverse up and down
Many of you will have noticed we’ve been having trouble keeping the Zooniverse up this morning. Like a lot of the internet, we host our sites on Amazon Web Services, who are having serious problems at their North Virginia data center. Using cloud hosting allows us to do many amazing things – primarily produce sites that automatically spin up new servers when we’re busy (like last week, when Planet Hunters hit the top of www.time.com – but in this case it’s bitten us rather.
All of the other Amazon data centers seem fine, and the team are currently busily moving us to California, but it’s taking us some time. In the long run, we’ll see what we can do to protect ourselves from single data center outages; we can’t afford to host in many places just in case, but maybe we should sprinkle the projects around a little more so there’s always something up.
Please bear with us, and rest assured that your hard work in the form of classifications is backed up and secure.
Zooniverse Roundup Podcast 1
In the first of a new series of audio blog entires, Chris Lintott chats about some current goings on in the Zooniverse. In this edition, Chris talks about Aida’s Green Blob (click here to see the discussion on the Galaxy Zoo Forum) with Bill Keel. You can see the image they are talking about below:

You can either listen using the player above, or by grabbing this link to the MP3 file.
Zooniverse Groups
Today we let loose a new feature called Zooniverse Groups. We’re always being told how our projects are used by schools, open evenings and other groups of people. This is great, and we encourage the use of our projects in education. We are often asked by teachers if it is possible for them to manage a group of students and keep track of their classifications. Today’s release is the first step in allowing this type of management by educators and others who want to share the Zooniverse.

You can access Groups from your account settings on Zooniverse Home. By default you will see the options to create a group. You do this by by giving the group a name and quick description. Once a group has been created (you will be the first member) you can then invite others to join in. You do this by viewing the group’s settings page (the cog icon next the group’s name). Here you can invite people, either by knowing their Zooniverse user ID, or by sending them a unique group link.
As an example, if you follow this link – http://www.zooniverse.org/account/new?group=ab55cbbd – you’ll join my example group called ‘Rob’s Example Group’. If you already have a Zooniverse ID you can then sign in and join the group, or you can create an account and join as a new Zooniverse user.

As the owner of a group you can see how many classifications each user has made since joining the group – and we have provided a couple of charts showing this data. We’d really like to hear from you about the other features you’d like to see included as a group owner. As a group member you get to see who else is in the group and the contact details of the group owner. Note to group owners: your email address is shared with members of the group.
Groups currently only tracks your Galaxy Zoo activity. We had to start somewhere and we get the most requests for groups from schools using Galaxy Zoo. We plan to add in support for other projects as time goes by. Group members can leave a group at any time, and group owners can remove members at any time. Group owners can delete a group at any time and this will delete all membership records – we don’t store your group activity after deletion.
Groups is in an early stage and we’re keen to hear what else you’d like to use it for. If you have feedback please get in touch via team@zooniverse.org.
‘Citizen’ science and ‘real’ science
While the old maxim about all publicity being good publicity isn’t exactly true, it is always a good feeling when one of our projects gets mentioned in the press, and so I was delighted to see the New York Times highlighting everyone’s favourite Voorwerp (and its discoverer) today.
The rest of the article highlights some of the other well known examples of ‘citizen science’, from SETI@home to Fold.it, but also includes some more critical comments. For example, David Weinberger from the Berkman Center for the Internet and Society at Harvard comments ‘These people are not doing the work of scientists…They are doing the work of scientific instruments.’
From the way the article’s written, it seems that it was supposed to refer to more traditional citizen science projects which involve collecting data, rather than our kind that involves analysis. Without more context, it’s also difficult to know whether this was meant pejoratively, but this sort of criticism – that projects like Galaxy Zoo don’t offer participants the chance to behave as ‘proper’ scientists – comes up a lot, and I think it raises some interesting questions. For starters, before we can answer the question of whether Galaxy Zoo meets such a criteria we need to decide what ‘doing the work of scientists’ is.
I’m well aware that there’s a whole literature on this subject, which I don’t intend to review here, but for the sake of argument let’s say that a ‘proper’ scientist is someone who, informed by a knowledge of existing understanding in a particular field (we hope, anyway), comes up with an idea, and then, through experiment, theory or computer modelling, or increasingly by exploring existing data, seeks to test that idea before reporting the results in a journal or at a conference.
If that’s our model, it’s clear that a random visitor to a Zooniverse project isn’t functioning as a proper scientist. It was the Galaxy Zoo science team who read the literature and realised that sorting galaxies by their shape would be of interest, and who take the results of the ‘experiment’ (in this case, running a citizen science project), interpret them and write them up. If you push the analogy further then, sure, the ‘scientific instrument’ used to investigate galaxy shape includes not only the website, but also the visitors to it.
That seems to confirm the lowly status of the Zooites – no longer citizen scientists, just ‘high-functioning cogs in a distributed machine’ as the article has it. Except that that’s exactly how scientists behave a lot of the time. There may be scientists out there who only think grand thoughts, whose particular genius requires only, in the reverse of Edison’s famous formula, one percent perspiration and ninety-nine percent inspiration.
I’ve never met any of them. When students ask me whether they should do a PhD or not, my answer is likely to be influenced by whether they’ve come to terms with the idea that a lot of the day to day effort of science involves not seeking flashes of inspiration, but hard, repetitive work. It might be sitting in the field waiting for the lesser mongolian tree frog to do something interesting, or it might be attempting to understand why your simulation of star formation just won’t compile, but it’ll be there. Before Galaxy Zoo came along, individual scientists classified the galaxies themselves.
Perhaps they too were just cogs in the scientific machine. But this is now an argument about semantics, rather that status. Galaxy Zoo and projects like it open up part of the scientific process to participation by anyone, and I don’t think the wonder of that idea is diminished by the fact that for most people, most of the time we need professionals for the rest of it.
When I was a kid, I used to count meteors and send it the results of my count to the British Astronomical Association. The wonder at the idea that I could do something that in some tiny way contributed to our knowledge of the Universe was totally unaffected by the realisation that it would be others who analysed the results.
This highlights an important difference between some of the new citizen science projects, and older endeavours such as meteor watching. The Galaxy Zoo site provides enough information for those who are interested to take control of the entire scientific process. Links to professional archives are available for each galaxy, the data set is made available to all (albeit after a delay), and we are building a suite of tools to lower the barriers to this more advanced participation. There are a steady stream of volunteers appearing as authors on Zoo publications because of their contributions, working alongside the science team. The investigations of things like the Galaxy Zoo peas are being driven by prompting from our ever-alert community of volunteers. In testing and refining our projects, successive generations of volunteers are involved in designing future ‘experiments’. I know of several Zooites who have gone back to formal education, inspired to increase their level of scientific knowledge by participation in the project.
In other words, if you need to run your own projects, or to acquire a publication record to be a ‘citizen scientist’, then consider it an aspirational label. The Zooniverse provides everything you need to do that, although, for now, the barriers are still high. Otherwise, if you contribute to our understanding of the Universe in however minor a fashion, then I’ll call you a scientist, and I look forward to being able to drop the distinction between professional and citizen.
Happy Holidays!
We’ve had a lot of fun with our 2010 Zooniverse Advent Calendar (and hope you now know what Rick Rolling is if you didn’t before). Over the past 24 days we have launched the ninth and tenth Zooniverse projects: the Milky Way Project and Planet Hunters – which are both running along very well. You can find the whole list of Zooniverse projects on our main site. We have also significantly updated Galaxy Zoo, created a host of author posters for your enjoyment and had a bit of fun along the way.
As well as clicking away on your favourite Zooniverse projects this holiday period, you might also want to look up and catch the International Space Station as flies by. The ISSwave runs from today until New Year’s Eve and is hoping to get as many people as possible out to wave at the people aboard the ISS as it flies overhead. It’s a wonderful project and we hope you’ll try and take part. There is more information about ISSwave and hoe to get involved over on the official site.

For our UK Zooites, there is a special event in early January that you might want to look out for. On BBC Two from January 3-5th three special Stargazing Live programmes are taking place. Each day will focus on a different aspect of astronomy, covering a different special event in the sky. The Zooniverse are involved and we hope you will be too!
Have a very merry Christmas – or whatever you are doing over the holidays – and we hope you’ll find a Zooniverse project to keep you amused between mince pies.
Planet Hunters
We are pleased to announce the debut of (another!) new Zooniverse project: Planet Hunters! This time we want you to help us find planets around other stars (exoplanets) using data from NASA’s Kepler mission.

NASA’s Kepler spacecraft is one of the most powerful tools in the hunt for extrasolar planets. The Kepler data set is unprecedented and has incredible photometric precision. Before Kepler, the only star monitored this precisely was our own Sun. The lightcurves reveal subtle variability that has never before been documented. Kepler lightcurves are were made publicly available with the first data release this past June and the next release scheduled for February 2011. We are very excited here at Planet Hunters to get our hands on them!

The Kepler Team computers are sifting through the data, but we at Planet Hunters are betting that there will be planets which can only be found via the remarkable human ability for pattern recognition. This is a gamble, a bet, if you will, on the ability of humans to beat machines just occasionally. It may be that no new planets are found or that computers have the job down to a fine art. That’s ok. For science to progress sometimes we have to do experiments, and although it may not seem like it at the time negative results are as valuable as positive ones. Most of the lightcurves will be flat, devoid of transit signals but it’s possible that you might be the first to know that a star somewhere out there in the Milky Way has a companion, just as our Sun does.
Fancy giving it a try? If you do, you could be the first to spot an new planet – it may be a Jupiter-size behemoth or even an Earth-sized rock. If you want to take part in our amazing experiment you’ll be playing with cutting-edge web technology. You’ll need one of the most modern browsers around (Safari, Chrome, Firefox or Opera) and you’ll need an up-to-date version if possible. We are testing the limits of citizen science on the web and hope that you’ll come along for the ride. We hope to bring support for older browsers in early 2011.
So, come join our adventure and log on to Planet Hunters now!
350,000 Zooniverse Users
What an amazing festive present for us at Zooniverse HQ: the Zooniverse has signed up its 350,000th user! That’s a lot of people! There are now more people in the Zooniverse than there are in Iceland (or Belize or nearly the Bahamas)! We soon hope to take on Malta.
350,000 is roughly the amount the world’s population increases in two days! This means that the Zooniverse is big enough to fill nearly 4 Wembley Stadiums and over 8.5 Wrigley Fields! We are now more than twice the size of the world’s largest stadium in North Korea!
On a tactical note, there are now only 12 armies in the world that are larger than the Zooniverse user base (China, USA, India, North Korea, Russia, South Korea, Pakistan, Iran, Turkey, Egypt, Vietnam and Mynamar). That means we can take on any nation in Europe – though not Iceland as they don’t have an army. Irony.
Standing side-by-side the Zooniverse could now fill the Vatican or hand-in-hand we could get half way around Saturn’s moon Mimas. We weigh the same as about 70 fully-loaded 747s and output a combined 42 Megawatts of energy – so we’ve a way to go before we can rival most power stations.
Assuming Santa delivers to 6.5 billion people this Christmas – it will take him just over 7 seconds to deliver to 350,000 people. Imagine how good he’s be at classifying galaxies! If only he had the time…
[Thanks to Creative Commons and Flickr user Tochis for portions of the image used in this post, which itself contains 350,000 pixels. Also: anyone noticed that the 14th is a really big day on the advent calendar?]

