All posts by The Zooniverse

Online citizen science projects. The Zooniverse is doing real science online,.
News

NEEMO

5 Comments

We’re inviting you – for a limited time only! – to take part in a big experiment! For our first ‘Zooniverse Lab’, we’ve partnered with NASA, who are training astronauts in an underwater environment. For the next two weeks they’re sending six astronauts, researchers and habitat technicians to live in an undersea habitat and work as part of the NASA Extreme Environment Mission Operations (NEEMO). We’re interested to see if having many eyes on the problem will help the scientists and engineers working on NEEMO.

The mission is taking place 43 feet underwater at the NOAA ‘Aquarius’ facility, three miles off the coast of Key Largo, USA. The goal of this NEEMO-15 mission is to understand how NASA may one day explore and operate on asteroids, using techniques that are orders of magnitude more capable and efficient than methods are in use today.

For the Zooniverse this is the first of a new breed of project for us – one where the experiment is the interface itself! The project will only run for two weeks and in that time we hope to collect sufficient data to work out if an interface where people confirm or reject each other’s classifications can prove more efficient than our current approach of purely independent classification. NASA science teams will be using the data collected during NEEMO-15, alongside your classifications, to help develop different data sampling techniques for future asteroid missions.

Although we can’t promise that this project will result in academic papers, it will aid the NEEMO project and we’re convinced that it can help us better understand the limits of citizen science online.

Check out http://neemo.zooniverse.org and let us know what you think.

News, Technical

The New Zooniverse Home

1 Comment

If you visit Zooniverse Home today you may notice that a few things have changed. For a while now we’ve wanted to improve the design of the site to better handle the growing range of projects that are part of the Zooniverse. With the updates released today we’ve completely reworked the look and feel of the site. We’ve organised projects into different categories, starting with Space, Climate and Humanities. There are lots of new categories coming soon.

Today we’re also launching an entirely new type of project as part of ‘Zooniverse Labs’. To date all Zooniverse projects come with the guarantee that your work will contribute directly toward real research output (usually academic papers), however this can make it hard to try new ideas. As the name suggests, Zooniverse Labs is about experimentation and we’re excited about launching new, often smaller-scale websites that continue to stretch the boundaries of science online. More news on this very soon.

Another new part of the redesigned Zooniverse homepage is the ‘My Projects’ section that allows you to see your contributions to the Zooniverse, and what else you might like to try out. This part of the site is very much in beta, and we hope to add to it in the coming months.

We hope you like the new look, and that it helps you explore more of what the Zooniverse has to offer.

News

Voorwerpje roundup – we have a paper!

7 Comments

Eighteen thousand candidate active galactic nuclei. One hundred ninety-nine Zooites. A hundred fifty-four possible galaxies with clouds, of which 49 became targets for spectra. And finally, nineteen certified Voorwerpjes – giant clouds of gas ionized by a central active nucleus, like Hanny’s Voorwerp but smaller (and sometimes not all that much smaller) and dimmer. Of these clouds, many (including the largest) are new discoveries.

That’s a short description of the current status of the Galaxy Zoo Voorwerpje Hunt project. We just submitted a 28-page manuscript with all the gory details to the Monthly Notices of the Royal Astronomical Society (we’ll post the whole thing once it passes muster with the referee). Probably the most exciting result is that about half of these have gas too highly ionized too far from the nucleus to account for by the nucleus we see (even including far-infrared results to tell how much radiation is being absorbed by dust), so they may be additional, less dramatic instances of the AGN fading over time spans of 100,000 years or so. This large fraction suggests that at least Seyfert nuclei may constantly be brightening and fading over times of a few hundred thousands years (a time span about which we’ve previously had almost no information).

Here are some of the kinds of data we’re presenting. For a few galaxies in the right redshift ranges, we have new images with filters that isolate [O III] or Hα emission, like this Hα image for UGC 7342 after subtracting the ordinary starlight:

UGC 7342 clouds in H-alpha
UGC 7342 clouds in H-alpha

This is even more impressive when you know that the gas extends more than 200,000 light-years end to end.

This graph compares the spectrum of its nucleus (upper trace) to one of the giant clouds. The key points are in the relative intensities of the emission peaks, especially from Ne4+ and He+ (whose locations are marked with the red dotted lines). Starlight doesn’t have enough far-ultraviolet or X-rays to make gas that highly ionized, but an active galactic nucleus does. Furthermore, the ratios of these lines let us estimate how intense this radiation is when it reaches a cloud.

UGC 7342 nuclear and cloud spectra
UGC 7342 nuclear and cloud spectra

From our spectra, we also get information on the Doppler shifts across the galaxy and its gas. Here’s what we see for UGC 7342. The horizontal axis is distance along the spectrograph slit – each little tick is a kiloparsec, 3,200 light-years, at the galaxy. The vertical axis is the amount of Doppler shift in km/s relative to the galaxy core (this was taken out of a montage so the labels got clipped). The colored traces at the bottom show the intensity of starlight and of each emission line along the slit, to show how velocity features line up with the galaxy and clouds. Even though UGC 7342 is pretty chewed up because of an interaction with at least one companion, the gas motions aren’t as chaotic as they might be – the gas isn’t orbiting retrograde or anything.

UGC 7342 velocity slices
UGC 7342 velocity slices

We see some patterns emerge. Most of these ionized clouds are fond in galaxies which are interacting or merging. That makes sense (and fits as well with IC 2497 and Hanny’s Voorwerp) – tidal disturbances can pull streams of gas well out of the galaxy, and out of the pane of a spiral’s disk, giving a distant target to show us whether there is strong radiation there. About half of these galaxies show two ionized clouds on opposite sides, tracing the emerging radiation – it can get out on both sides of the central accretion disk, and as long as there is enough gas, we’ll see it on both sides.

Of course, we want to know more. Answers tend to multiply questions. Hubble observations are scheduled, and (with a little luck) X-ray measurements with ESA’s XMM-Newton observatory. We’ve managed to interest some of the people at ASTRON in the Netherlands in using the Westerbork array to examine the cold hydrogen around these galaxies. In addition, we’re doing new observations of various samples of active and “nonactive” galaxies to look for fainter, and maybe older, gas clouds. Special thanks to everyone who participated in this project, either through the targeted hunt or the complementary forum search for clouds in galaxies not listed as AGN. Stay tuned!

News

Open call for proposals

1 Comment

Brief version : This call for proposals means we’ll be producing many more cool projects – click if you’re a researcher and would like yours to be one of them.

Zooniverse

Longer version :

The Zooniverse was born from the conviction that the sheer enthusiasm of our original Galaxy Zoo volunteers could make a significant difference in other fields too. A few years and many projects on, I’m more convinced than ever that that’s true, and it’s time to step up a gear.

Until now, we’ve typically only been able to develop projects that came with their own funding for development. We’re grateful for their support, but I’ve always been worried that we were turning away some really amazing projects, and putting the Zooniverse out of reach of more junior researchers who are often on the front line of dealing with the flood of data threatening to engulf scientists.

Thanks to amazing support from the Alfred P. Sloan foundation we’re expanding our team and can now support a proper call for proposals. So if you’re a researcher – in any field – who would benefit from assistance of tens of thousands of people, it might be easier than you think. Just click here.

Chris

News

Work at the Zooniverse : SETI Scientist post

Leave a comment
Image by Jcolbyk at en.wikipedia
One of the dishes of the Allan Telescope Array

I’m pleased to announce that the Zooniverse team is expanding once again, and that we have a postdoc available at the Adler Planetarium. We’re looking for someone to help lead a joint project with the SETI institute, looking for interesting signals in data from the Allan Telescope Array. The post is funded by TED, and we’re keen to get started so please do apply or drop me a line if you’re interested.

Chris (chris AT zooniverse.org)

P.S. Here’s the formal job ad.

News, Technical

Recent downtime at The Zooniverse

Leave a comment

Last week the majority of The Zooniverse websites suffered their first major outage for over 2 years. Firstly I wanted to reassure you that the valuable classifications you all provide was always secure but perhaps more importantly I wanted to say sorry – we should have recovered more quickly from this incident and we’ll be working hard this week to put better systems in place to enable us to recover more rapidly in the future.

For those who’d like more of the gory details, read on…

Some background

Many of you will know the story of the launch of the original Galaxy Zoo (in July 2007); just a few hours after launch the website hosted by the SDSS web team and Johns Hopkins University crashed under the strain of thousands of visitors to the site. Thankfully due to the heroic efforts of the JHU team (involving a new web server being built) the Galaxy Zoo site recovered and a community of hundreds of thousands of zooties was born.

Fast-forward to February 2009 and we were planning for the launch of Galaxy Zoo 2. This time we knew that we were going to have a busy launch day – Chris was once again going to be on BBC breakfast. So that we could keep the site running well during the extremely busy periods we looked to commercial solutions for scalable web hosting. There are a number of potential choices in this arena but by far the most popular and reliable service was that offered by Amazon (yes the book store) and their Web Services platform. While I won’t dig into the technical details of Amazon Web Services (AWS) here, the fundamental difference when running web sites on AWS is that you have a collection of ‘virtual machines’ rather than physical servers.

If any of you have ever run Virtual PC or VMWare on your own computer then you’ll already realise that using machine virtualisation it’s possible to run a number of virtual machines on a single physical computer. This is exactly what AWS do except they do it at a massive scale (millions of virtual machines) and have some fantastic tools to help you build new virtual servers. One particularly attractive feature of using these virtual machines is that you can essentially have as many as you want and you only pay by the hour. At one point on the Galaxy Zoo 2 launch day we had 20 virtual machines running the Galaxy Zoo website, API and databases. 2 days later we were running only 3. The ability to scale up (and down) in realtime the number of virtual machines means that we are able to cope with huge variations in the traffic that a particular Zooniverse site may be receiving.

The outage last week

As I write this blog post we currently have 22 virtual servers running on AWS. That includes all of The Zooniverse projects, the database servers, caching servers for Planet Hunters, our blogs, the forums and Talk and much more. Amazon have a number of hosting ‘regions’ that are essentially different geographical locations where they have datacenters. We happen to host in the ‘us-east’ region in Virginia – conveniently placed for both Europe and American traffic.

We have a number of tools in place that monitor the availability of our web sites and last Thursday at about 9am GMT I received a text-message notification that our login server (login.zooiverse.org) was down. We have a rota within the dev team for keeping an eye on the production web servers and last week it was my turn to be on call.

I quickly logged on to our control panel and saw that there was a problem with the virtual machine and attempted a reboot. At this point I also started to receive notifications that a number of the Zooniverse project sites were also unavailable. At this point realising that something rather unusual was going on I checked the Amazon status page which was ‘all green’, i.e. no known issues. Amazon can be a little slow to update this page so I also checked Twitter (https://twitter.com/#!/search/aws). Twitter was awash with people complaining that their sites were down and that they couldn’t access their virtual machines. Although this wasn’t ‘good’ news, it’s always helpful to understand in a situation such as this if the issue is with the code that we run on the servers or the servers themselves.

Waiting for a fix?

At this point we rapidly put up holding pages on our project sites and reviewed the status of each project site and service that we run. As the morning progressed it became clear that the outage was rather serious and actually became significantly worse for The Zooniverse as in turn, each of the database servers that we run became inaccessible. We take great care to execute nightly backups of all of our databases and so when the problems started the oldest backup was 4 hours old. With hindsight when the problems first started we should have immediately moved The Zooniverse servers to a different AWS region (this is actually what we did do with login.zooniverse.org) and booted up new database servers with the backup from the night before however we were reluctant to do this because of the need to reintegrate the classifications made by the community during the outage. But hindsight is always 20-20 and this isn’t what we did. Instead, believing that a fix for the current situation was only a matter of hours away we waited for Amazon to fix the problem for us.

As the day progressed a number of the sites became available for short periods and then inaccessible again. It wasn’t a fun day to be on operations duty with servers continually going up and down. Worse, our blogs were also unavailable so we only had Twitter to communicate what was going on. At about 11pm on the first evening a number of The Zooniverse projects had been up for a number of hours and things looked to be improving. Chris and I spoke on the phone and we agreed that if things weren’t completely fixed by the morning we’d move the web stack early Friday morning.

Friday a.m.

Friday arrived and the situation was slightly improved but the majority of the our projects were still in maintenance mode so I set about rebuilding The Zoonivere web stack. Three out of five of our databases were accessible again and so I took a quick backup of all of the three and booted up replacement databases and web servers. This was all we needed to restore the majority of the projects and by lunchtime on Friday we pretty much had a fully working Zooniverse again. The database server used by the blogs and forums took a little longer to recover and so it was Friday evening before they were back up.

A retrospective

We weren’t alone in having issues with AWS last week. Sites such as Reddit and Foursquare were also affected as were thousands of other users of the service. I think the team at the Q&A site Quora put it best when they said ‘We’d point fingers, but we wouldn’t be where we are today without EC2.’ on their holding page and this is certainly true of The Zooniverse.

Over the past 2 years we’ve only been able to deliver the number of projects that we have and the performance and uptime that we all enjoy due to the power, flexibility and reliability of the AWS platform. Amazon have developed a number of services that mean that it was possible (in theory) to protect against the failure they experienced in the US-east region last week. Netflix was notably absent from the list of AWS hosted sites affected by the AWS downtime and this is because they’ve gone to huge effort (and expense) to protect themselves against such a scenario (http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html).

Unfortunately with the limited resources available to The Zooniverse we’re not able to build a resilient web stack as Netflix however there are a number of steps we’ll be taking this to make sure that we’re in a much better position to recover from a similar incident in the future so that we experience downtime of minutes and hours rather than days.

Cheers
Arfon & The Tech Team

News

Zooniverse up and down

Leave a comment

Many of you will have noticed we’ve been having trouble keeping the Zooniverse up this morning. Like a lot of the internet, we host our sites on Amazon Web Services, who are having serious problems at their North Virginia data center. Using cloud hosting allows us to do many amazing things – primarily produce sites that automatically spin up new servers when we’re busy (like last week, when Planet Hunters hit the top of www.time.com – but in this case it’s bitten us rather.

All of the other Amazon data centers seem fine, and the team are currently busily moving us to California, but it’s taking us some time. In the long run, we’ll see what we can do to protect ourselves from single data center outages; we can’t afford to host in many places just in case, but maybe we should sprinkle the projects around a little more so there’s always something up.

Please bear with us, and rest assured that your hard work in the form of classifications is backed up and secure.

Technical

Spring (theme) Cleaning

Leave a comment

It’s that time of year again. Here in the Northern Hemisphere the flowers are starting to pop up, the ground hogs are coming out of hibernation, and signs everywhere indicate that it’s time to invest in the latest styles. Here in the Zooniverse, I’m not sure we can generally be accused of clinging to the newest fashions, but security is always on our mind, and with the release of WordPress 3.1 we did need to do a few updates. While I was inside the guts of our server, I admit I went a little nuts and decided (in conversations with a bunch of people, including maybe you) that it might be nice to give our blogs matching outfits.

The new Zoo theme you see before you comes from one of my favourite WordPress design companies: Woo Themes. The theme is an adaption of their social media ready MyStream, and it’s my hope that all the happy little links to classifying and social media will form a central jumping off point to see what’s new, to share what you like, and to do science..

There are still a few things wanting. The Mendeley pages are, um, empty, but that will get fixed soon. The posts are also missing a lot of comments. This is where you come in. We know there are a ton of people contributing to science through the Zooniverse, and we’d like at least a large chunk of that ton reading this blog. Next time someone asks you about what you’re doing online when you’re clicking away, send them to the blog. Get them involved one blog post and one click at a time.

See you in the comments…

News, Zoo Roundup

Zooniverse Roundup Podcast 1

3 Comments

In the first of a new series of audio blog entires, Chris Lintott chats about some current goings on in the Zooniverse. In this edition, Chris talks about Aida’s Green Blob (click here to see the discussion on the Galaxy Zoo Forum) with Bill Keel. You can see the image they are talking about below:

aidasgreenblob

You can either listen using the player above, or by grabbing this link to the MP3 file.