Tag Archives: BBCStargazing

Stargazing Live 2017 Recap

We recently had a very successful (and longer than usual) Stargazing Live. I wanted to talk a little about the work that our team did in the weeks leading up to this and also recap what actually happened behind the scenes during the two weeks of events.

If you’re not familiar with it, Stargazing Live is an annual astronomy TV show on BBC Two in the UK, which is broadcast live on three consecutive nights. Each year we launch a project in collaboration with the show, and this always proves to be the busiest time of our year. This year, for the first time there was a second week of shows for ABC Australia, so this time we launched two projects instead of one: Planet 9 and Exoplanet Explorers.

A lot of work went into making sure that our site stayed up for this year’s shows. In previous years we’ve had issues that have resulted in either a brief outage or reduced performance for at least some of the time during the show. This year everything worked perfectly and we actually found ourselves reducing our capacity (scaling down) much sooner than we anticipated. The prep work fell into three areas:

  • Optimisations to the frontend to reduce the number of API calls made by the site while people were using it. This involved a combination of refactoring, fixing bugs, and modifying the backend to return frequently requested data without it having to be requested separately (e.g. when checking if the user has favourited a subject).
  • Reducing the load on our databases. We reduced the number of requests that result in database queries through caching in the backend (with memcache), and we started using a new microservice (called Designator) to keep track of what each user has seen and serve them new subjects. We also separated some services onto a read replica rather than having them query the primary database.
  • Adding feature flags so that we could turn off anything non-essential, and so that we could shut down any features that were causing problems, using the Flipper Ruby gem.
The Oxford team gathers in the office to watch the show.

On the first night of the BBC show it was all hands on deck. Our teams in the US and the UK were in our offices, despite it being evening in the UK, and in Oxford we gathered around the TV expectantly awaiting the moment when Chris would announce the project’s URL on air. That moment is usually a bit frantic, as several thousand people all turn up on the site at once and start clicking around, registering, logging in, and submitting classifications. We’re always closely watching our monitoring systems, keeping an eye on various performance metrics, watching for any early signs of problems that might affect the performance of the site. This year when that moment came the number of visitors on site shot up to over 5,000, and then… everything just kept running smoothly.

The first night of the BBC show we peaked at about 0.9 million requests per hour, with 1.1 million per hour the second night.

Requests to Zooniverse.org during BBC Stargazing Live 2017.

We scaled our API service to 50 of EC2’s m3.medium instances the first night and the average CPU utilisation of these instances reached about 30% at peak traffic. The next two nights we reduced the number of instances to 40. In hindsight we could have gone even lower, but from past experience the amount of traffic we receive on the second and third nights can be difficult to predict, so we decided to play it safe.

API scaling and CPU utilisation during BBC Stargazing Live 2017.

Traffic during the ABC show was lower than during the BBC show (Australia has a smaller population than the UK, so this was as expected). That week we scaled the API to 40 instances the first night, and 20 instances for the second and third nights.

In the past we’ve had problems with running out of available connections in PostgreSQL. The connection limit depends on available memory, and we find this to be more of a problem than CPU or network constraints. During the shows we scaled the PostgreSQL instance for our main API to RDS’s m4.10xlarge and our Talk/microservices database to m4.2xlarge, primarily to give us enough leeway to avoid the connection limit. In the future we’d like to implement connection pooling to avoid this.

This was all a big improvement on previous years. While before we found ourselves extremely busy fighting fires and fixing bugs between shows, this time we had time to just relax and watch the show. We have more work to do on optimisations, because we did still have to scale up our capacity more than we’d like, but overall we’re very happy with how well things went this year.

 

Advertisements

Stargazing Live: The Results Are In

IMG_1172
The Lovell Telescope observing 9io9: a candidate lens spotted by volunteers on Space Warps.

BBC Stargazing Live 2014 has been asking people to visit the Zooniverse’s Space Warps site to identify gravitational lenses: extremely rare events caused by one galaxy passing in front of another (very distant) galaxy. Tens of thousands of you have taken part and classified more than 6.5 million images.

Your classifications have already led to the discovery of more than 50 potential gravitational lenses! Amongst them are several beautiful and interesting discoveries. You can see a few of our favourite candidates above. For Stargazing Live’s third and final show we have focussed on the spectacular red arc/ring shown below, it has been nicknamed 9io9 by the team right now, because of it’s Zooniverse ID. You can see more of what our volunteers are saying about it here on Talk.

Credit: Jim Geach / VICS82
Credit: Jim Geach / VICS82

The Space Warps team have produced a model of it and currently think the background (red) galaxy is at redshift of about 2, which means the light has taken more than 10 billion years to reach us! You can see the comparison of the model and the data below. There’s a chance it could be further away but we’ll keep you posted. The nearer object (white/yellow) is about 2 billion light years away and has a mass of 100 billion times that of our Sun – which makes it about the same size as our own galaxy.

Model_and_Data_Credit_A_More
Comparison of the model (left) and real (right) data.

We know all this because we have spent the last 24 hours calling in every favour we have worldwide. The Space Warps science team, and various Zooniverse scientists from other projects, have been literally asking favours from people using the world’s biggest telescopes. We were even able to get some time on the massive Keck telescope in Hawai’i, where astronomers were having to break ice off the dome to get data. Astronomers love a good challenge!

Of course Stargazing Live is filmed at Jodrell Bank, home to one of the world’s largest radio dishes: the Lovell Telescope. This candidate lens is perfect for a radio observation – which can tell us more about its mass and position in space – and I’m excited to say that the giant dish is observing the target as I write!

Space Warps has been a huge success over the past three days and the project continues! Every classification on Space Warps helps our computers understand the whole data set, and so in a way all the objects discovered on Space Warps are the result of everybody’s combined work. You can keep up to date with news from Space Warp via the project’s blog, Twitter and Facebook sites.

A huge thank you to the BBC crew, the Jodrell Bank team, the Space Warps scientists, developers and moderators, and to everyone that took part this week. Keep clicking!

Space Warps for BBC Stargazing Live

This week is the BBC’s Stargazing Live show: three now-annual nights of live stargazing and astronomy chatter, live from Jodrell Bank. In 2012 we asked the Stargazing Live viewers to help us discover a planet with Planet Hunters, in 2013 we explored the surface of Mars with Planet Four. This year we are inviting everybody to use our Space Warps project to discover some of most beautiful and rare objects in the universe: gravitational lenses.

Space Warps launched last year and originally the project asked everyone to spot gravitationally lensed features in optical images from the CFHT Legacy Survey. We’re still busy processing the data but you seem to have found lenses – including the three shown at the top of this post! For Stargazing Live we’re adding a whole new dataset of infrared images, which has not been deeply searched for lenses before. We’re also now working with ’targeted’ data. This means that we are only showing images containing objects in them that could either be lenses, or would be interesting if they were being lensed. So your odds of finding something amazing have really gone up!

Screenshot of Space Warps

Gravitational lenses occur when a massive galaxy – or cluster of galaxies – pass in front of more distant objects. The enormous mass of the (relatively) closer object literally bend light around them and distort the image of the distant source. Imagine holding up a magnifying glass and waving it around the night sky so that starlight is bent and warped by the lens. You can see more about this here on the ESO website.

We’re blogging right now from Jodrell Bank (the dish is looking lovely BTW) and the Stargazing Live set and everyone here is buzzing with the idea that we might find some gravitational lenses that have never been seen before! Good luck, and happy classifying. Even K9 is excited.