I’ve just released version 1.1 of the Panoptes Client for Python. The changelog has a full list of what’s new, but there are a few things I wanted to highlight, the first two of which will make it substantially faster to create new subjects:
- Multithreaded media uploads – the client will automatically use several threads to upload media when you first save a new subject. So, for example, if you create a subject which has three images they will all upload simultaneously (up to five simultaneous uploads, then it will queue them).
- Multithreaded subject creation – you can also simultaneously create the subjects themselves. That means if you’re creating, say, a thousand subjects, the client can queue them all and create up to five of them simultaneously. This works in conjunction with the media uploads, using one combined queue for the subject creation and the media uploads, to avoid overloading the network and to make sure the subject creation doesn’t get too far ahead of the uploads. This one isn’t automatic – you’ll need to create your subjects with the new SubjectSet.async_saves() context manager to take advantage of it.
- Retries for all GET requests – we’re quite proud of how reliable the Zooniverse platform is, but sometimes server-side errors do happen. The client will now automatically retry all GET requests (i.e. the ones that don’t modify any data) if an error occurs, improving reliability.
- Retries for batch linking operations – similar to above, the client will retry any add/remove operations via the new LinkCollection class, which handles linking groups of objects (i.e. subjects to a subject set, subjects to a collection, etc.). This means you should see far fewer failures when linking thousands of subjects to a subject set, for example.
- Context manager for multiple connections – the Panoptes class can now act as a context manager, providing a safe way to perform operations as multiple users (for example, in a web app).
You can install the update by running pip install -U panoptes-client. Any bugs or issues should be raised via GitHub.
Hot on the heels of last week’s update, I’ve just released version 1.0.3 of the Python Panoptes Client, which fixes a bug introduced in the previous release. If you encounter a TypeError when you try to create subjects, please update to this new version and that should fix it.
This release also updates the default client ID that is used to identify the client to the Panoptes API. This is to ensure that each of our API clients is using a unique ID.
As before, you can install the update by running pip install -U panoptes-client.
We’ve just released an update for the Panoptes Client Python module which fixes a couple of bugs with saving subjects:
- Fix saving subjects with updated metadata
- Fix calling Subject.save() when nothing has changed
You can install the updates by running pip install -U panoptes-client.
We’ve recently released updates for the Panoptes command-line interface and the Panoptes Client module for Python containing a few bug fixes.
From the changelog for Panoptes Client:
- Fix: Exports are not automatically decompressed on download
- Fix: Unable to
save a Workflow
- Fix: Fix typo in documentation for Classification
- Fix: Fix saving objects initialised from object links
And from the CLI:
- Fix: Modifying projects makes them private
You can install the updates by running pip install -U panoptescli and pip install -U panoptes-client.
Following on from the release of Panoptes Client 1.0 for Python, we’ve just released version 1.0 of the Panoptes CLI. This is a command-line client for managing your projects, because some things are just easier in a terminal! The CLI lets you do common project management tasks, such as activating workflows, linking subject sets, downloading data exports, and uploading subjects. Let’s jump in with a few examples.
First, downloading a classification export (obviously you’d insert your own project ID and a filename of your choice):
panoptes project download 764 Downloads/pulsar-hunters-classifications.csv
This command will optionally generate a new export and wait for it to be ready before downloading. No more waiting for the notification email!
New subjects can be uploaded to a new subject set like so (again, inserting your own IDs):
panoptes subject-set create 7 "November 2017 subjects"
panoptes subject-set upload-subjects 16401 manifest.csv
You can also pipe the output from the CLI into other standard commands to do more powerful things, such as linking every subject set in your project to a workflow using the xargs command (where 1234 and 5678 are your project ID and workflow ID respectively):
panoptes subject-set ls -q -p 1234 | xargs panoptes workflow add-suject-sets 5678
Visit GitHub to get started with the CLI today!
I’m happy to announce that the Panoptes Client package for Python has finally reached version 1.0, after nearly a year and a half of development. With this package, you can automate the management of your projects, including uploading subjects, managing subject sets, and downloading data exports.
There’s still more work to do – I have lots of additional features and improvements planned for version 1.1 – but with the release of version 1.0, the Client has a stable set of core features which are useful for managing projects (both large and small).
I know a lot of people have already been using the 0.x versions while we’ve been working on them, so thanks to everyone who submitted feature requests, bug reports, and pull requests on GitHub. Please do upgrade to the latest version to make sure you have the latest bug fixes, and keep the requests and bug reports coming!
You can find installation and upgrade instructions on GitHub, and full documentation on Read the Docs.