Feeds:
Posts
Comments

Archive for the ‘Data availability’ Category

The following is a guest post from science journalist John Bohannon. We asked him to give us some background on his recent dataset in Dryad and the analysis of that data in Science. What stories will you find in the data? – EH

_______

Scihub_raven

Sci-Hub is the world’s largest repository of pirated journal articles. We will probably look back and see it as inevitable. Soon after it became possible for people to share copyrighted music and movies on a massive scale, technologies like Napster and BitTorrent arrived to make the sharing as close to frictionless as possible. That hasn’t made the media industry collapse, as many people predicted, but it certainly brought transformation.

Unlike the media industry, journal publishers do not share their profits with the authors. So where will Sci-Hub push them? Will it be a platform like iTunes, with journals selling research papers for $0.99 each? Or will Sci-Hub finally propel the industry into the arms of the Open Access movement? Will nonprofit scientific societies and university publishers go extinct along the way, leaving just a few giant, for-profit corporations as the caretakers of scientific knowledge?

There are as many theories and predictions about the impact of Sci-Hub as there are commentators on the Internet. What is lacking is basic information about the site. Who is downloading all these Sci-Hub papers? Where in the world are they? What are they reading?

48 hours of Sci-Hub downloads. Each event is color-coded by the local time: orange for working hours (8am-6pm) and blue for the night owls working outside those hours.

Sometimes all you need to do is ask. So I reached out directly to Alexandra Elbakyan, who created Sci-Hub in 2011 as a 22 year-old neuroscience graduate student in Kazakhstan and has run it ever since. For someone denounced as a criminal by powerful corporations and scholarly societies, she was quite open and collaborative. I explained my goal: To let the world see how Sci-Hub is being used, mapping the global distribution of its users at the highest resolution possible while protecting their privacy. She agreed, not realizing how much data-wrangling it would ultimately take us.

Two months later, Science and Dryad are publicly releasing a data set of 28 million download request records from 1 September 2015 through 29 February 2016, timestamped down to the second. Each includes the DOI of the paper, allowing as rich a bibliographic exploration as you have CPU cycles to burn. The 3 million IP addresses have been converted into arbitrary codes. Elbakyan converted the IP addresses into geolocations using a database I purchased from the company Maxmind. She then clustered each geolocation to the coordinates of the nearest city using the Google Maps API. Sci-Hub users cluster to 24,000 unique locations.

The big take-home? Sci-Hub is everywhere. Most papers are being downloaded from the developing world: The top 3 countries are India, China, and Iran. But the rich industrialized countries use Sci-Hub, too. A quarter of the downloads came from OECD nations, and some of the most intense download hotspots correspond to the campuses of universities in the US and Europe, which supposedly have the most comprehensive journal access.

But these data have many more stories to tell. How do the reading habits of researchers differ by city? What are the hottest research topics in Indonesia, Italy, Brazil? Do the research topics shift when the Sci-Hub night owls take over? My analysis indicates a bimodal distribution over the course of the day, with most locations surging around lunchtime, and the rest peaking at 1am local time. The animated map above shows just 2 days of the data.

Something everyone would like to know: What proportion of downloaded articles are actually unavailable from nearby university libraries? Put another way: What is the size of the knowledge gap that Sci-Hub is bridging?

Download the data yourself and let the world know what you find.

The data:

http://dx.doi.org/10.5061/dryad.q447c

My analysis of the data in Science:

http://www.sciencemag.org/news/2016/04/whos-downloading-pirated-papers-everyone

 

 — John Bohannon

Read Full Post »

2015While gearing up for the Dryad member meeting (to be held virtually on 24 May – save the date!) and publication of our annual report, we’re taking a look at last year’s numbers.

2015 was a “big” year for Dryad in many respects. We added staff, and integrated several new journals and publishing partners. But perhaps most notably, the Dryad repository itself is growing very rapidly. We published 3,926 data packages this past year — a 44% increase over 2014 — and blew past the 10,000 mark for total data packages in the repository.

Data package size

Perhaps the “biggest” Dryad story from last year is the increase in the mean size of data packages published. In 2014, that figure was 212MB. In 2015, it more than doubled to 481MB, an increase of a whopping 127%.

This striking statistic is part of the reason we opted at the beginning of 2016 to double the maximum package size before overage fees kick in (to 20GB), and simplified and reduced our overage fees. We want researchers to continue to archive more (and larger) data files, and to do so sustainably. Meanwhile, we do continue to welcome many submissions on the smaller end of the scale.

boxplot_logscale_labels

Distribution of Dryad data package size by year. Boxplot shows median, 1st and 3rd quartiles, and 95% confidence interval of median. Note the log scale of the y-axis.

In 2015, the mean number of files in a data package was about 3.4, with 104 as the largest number of files in any data package. To see how times have changed, compare this to a post from 2011 (celebrating our 1,000th submission), where we noted:

Interestingly, most of the deposits are relatively small in size. Counting all files in a data package together, almost 80% of data packages are less than one megabyte. Furthermore, the majority of data packages contain only one data file and the mean is a little less than two and a half. As one might expect, many of the files are spreadsheets or in tabular text format. Thus, the files are rich in information but not so difficult to transfer or store.

We have yet to do a full analysis of file formats deposited in 2015, but we see among the largest files many images and videos, as would be expected, but also a notable increase in the diversity of DNA sequencing-related file formats.

So not only are there now more and bigger files in Dryad, there’s also greater complexity and variety. We think this shows that more people are learning about the benefits of archiving and reusing multiple file types, and that researchers (and publishers) are broadening their view of what qualifies as “data.”

Download counts

2015speciesSo who had the biggest download numbers in 2015? Interestingly, nearly all of last year’s most-downloaded data packages are from genetics/genomics. 3 of the top 5 are studies of specific wild populations and how they adapt to changing circumstances — Sailfin Mollies (fish), blue tits (birds), and bighorn sheep, specifically.

Another top package presents a model for dealing with an epidemic that had a deadly impact on humans in 2015. And rounding out the top 5 is an open source framework for reconstructing the relationships that unite all lineages — a “tree of life.”

In 5th place, with 367 downloads:

In 4th place, with 601 downloads:

In 3rd place, with 1,324 downloads:

In 2nd place, with 1,868 downloads:

And this year’s WINNER, with 2,678 downloads:

The above numbers are presented with the usual caveats about bots, which we aim to filter out, but cannot do with perfect accuracy. (Look for a blog post on this topic in the near future).

As always, we owe a huge debt to our submitters, partners, members and users for supporting Dryad and open data in 2015!

Read Full Post »

We are delighted to announce the launch of a new partnership with The Company of Biologists to support their authors in making the data underlying their research available to the community.

COBNewLogo300dpiThe Company of Biologists is a not-for-profit publishing organization dedicated to supporting and inspiring the biological community. The Company publishes five specialist peer-reviewed journals:

The Company of Biologists offers further support to the biological community by facilitating scientific meetings, providing travel grants for researchers and supporting research societies.

Manuscript submission for all COB journals is now integrated with data submission to Dryad, meaning COB authors can conveniently submit their data packages and manuscripts at the same time. Dryad then makes the data securely available to journal reviewers, and releases them to the public if/when the paper is published.

We congratulate The Company of Biologists on taking this important step to help facilitate open data. To learn more about how your organization or journal can partner with Dryad, please contact us.

Read Full Post »

Our latest featured data package is from Alexandra Swanson and colleagues at the Snapshot Serengeti project, and accompanies their peer-reviewed article in Scientific Data.  It provides a unique resource for studying one of the world’s most extraordinary mammal assemblages and also for studies of computer vision and machine learning. In addition, data from Snapshot Serengeti is already being used in biology and computer science classrooms to enable students to work on solving real problems with authentic research data.

 lion

Snapshot Serengeti, CC BY-NC-SA 3.0

The raw data (which are being made available from the University of Minnesota Supercomputing Institute) consist of 1.2 million sets of images collected between February 2011 and May 2013 from 225 heat and motion triggered cameras, operating day and night, distributed over 1,135 sq. km. in Serengeti National Park in Tanzania.  This staggering trove of images was classified by 28,040 registered and ~40,000 unregistered volunteers on Snapshot Serengeti (a Zooniverse project) according to the species present (if any), the number of individuals, the presence of young, and what behaviors were being displayed, such as standing, resting, moving, eating, or interacting.

Remarkably, this vast army of citizen scientists was classifying the images faster than they were being produced, and each image set was classified on average by nine different volunteers.  This led to consensus classifications with high accuracy, 96.6% for species identifications relative to an expert-classified gold set.  Of the more than 300,000 image sets that contain animals, 48 different species were seen, including rare mammals such as the aardwolf and the zorilla.

zorilla

zorilla (image from Snapshot Serengeti CC BY-NC-SA 3.0)

The Dryad data package includes the classifications from all the individual volunteers, the consensus classifications, information about when each camera was operational, and the expert classification of 4,149 image sets as a gold standard.

References:

  • Swanson et al. (2015) Snapshot Serengeti, high frequency annotated camera trap images of 40 mammalian species in an African savannah. Scientific Data.  http://dx.doi.org/10.1038/sdata.2015.26
  • Swanson et al. (2015) Data from: Snapshot Serengeti, high frequency annotated camera trap images of 40 mammalian species in an African savannah. Dryad Digital Repository http://doi.org/10.5061/dryad.5pt92

Read Full Post »

The reason why Dryad is in the business of archiving, preserving, and providing access to research data is so that it will be reused, whether for deeper reading of the publication, for post-publication review, for education, or for future research. While it’s not yet as easy as we would like to track data reuse, one metric that is straightforward to collect is the number of times a dataset has been downloaded, and this is one of two data reuse statistics reported by our friends at ImpactStory and Plum Analytics.

2014 with fireworks

The numbers are very encouraging. There are already over a quarter million downloads for the 8,897 data files released in 2014 (from 2,714 data packages). That’s over 28 downloads per data file. While there is always the caveat that some downloads may be due to activity from newly emerged bots that we have yet to recognize and filter out, we think it is safe to say that most of these downloads are from people.

To celebrate, we would like to pay special tribute to the top five data packages from 2014, as measured by the maximum number of downloads for any single file (since many data packages have more than one) at the time of writing. They cover a diversity of topics from livestock farming in the Paleolithic to phylogenetic relationships among insects. That said, we are struck by the impressively strong showing for plant science — 3 of the top 5 data packages.

In 5th place, with 453 downloads

In 4th place, with 581 downloads

In 3rd place, with 626 downloads

In 2nd place, with 4,672 downloads

And in 1st place, with a staggering 34,879 downloads

Remarkably, given the number of downloads, this last data package was only released in November.

We’d like to thank all of our users, whether you contribute data or reuse it (or both), for helping make science just a little more transparent, efficient, and robust this past year. And we are looking forward to finding out some more of what you did with all those downloads in 2015!

 

 

 

 

Read Full Post »

Dryad has been proud to support integrated data and manuscript submission for PLOS Biology since 2012, and for PLOS Genetics since 2013.  Yet there are over 400 data packages in Dryad from six difFeatured imageferent PLOS journals in addition to two research areas of PLOS Currents. Today, we are pleased to announce that we have expanded submission integration to cover all seven PLOS journals, including the two above plus PLOS Computational BiologyPLOS MedicinePLOS Neglected Tropical DiseasesPLOS ONE, and PLOS Pathogens.  

PLOS received a great deal of attention when they modified their Data Policy in March providing more guidance to authors on how and where to make their data available and introducing Data Availability Statements. Dryad’s integration process has been enhanced in a few ways to support this policy and also the needs of a megajournal like PLOS ONE.  We believe these modifications provide an attractive model for integration that other journals may wish to follow. The key difference for authors who wish to deposit data in Dryad is that you are now asked to deposit your data before submitting your manuscript.

  1. PLOS authors are now asked to provide a Data Availability Statement during initial manuscript submission, as shown in the screenshot below. There is evidence that introducing a Data Availability Statement greatly reinforces the effectiveness of a mandatory data archiving policy, and so we expect this change will substantially increase the availability of data for PLOS publications.  PLOS authors using Dryad are encouraged to provide the provisional Dryad DOI as part of the Data Availability Statement.
  2. PLOS authors are now also asked to provide a Data Review URL where reviewers can access the data, as shown in the second screenshot. While Dryad has offered secure, anonymous reviewer access for some time, the difference now is that PLOS authors using Dryad will be able to enter the Data Review URL  at the time of initial manuscript submission.
  3. In addition to these visible changes, we have also introduced an Application Programming Interface (API) to facilitate behind-the-scenes metadata exchange between the journal and the repository, making the process more reliable and scalable. This was critical for PLOS ONE, which published 31,500 articles in 2013.  Use of this API is now available as an integration option to all journals as an alternative to the existing email-based process, which we will continue to support.

PLOS Data Availability Statement interface

PLOS Data Review URL interface

The manuscript submission interface for PLOS now includes fields for a Data Availability Statement and a Data Review URL.

If you are planning to submit a manuscript but are unsure about the Dryad integration options or process for your journal, just consult this page. For all PLOS journals, the data are released by Dryad upon publication of the article.  Should the manuscript be rejected, the data files return to the author’s private workspace and the provisional DOI is not registered.  Authors are responsible for paying Data Publication Charges only if and when their manuscript is accepted.

Jennifer Lin from PLOS and Carly Strasser from the California Digital Library recently offered a set of community recommendations for ways that publishers could promote better access to research data:

  • Establish and enforce a mandatory data availability policy.
  • Contribute to establishing community standards for data management and sharing.
  • Contribute to establishing community standards for data preservation in trusted repositories.
  • Provide formal channels to share data.
  • Work with repositories to streamline data submission.
  • Require appropriate citation to all data associated with a publication—both produced and used.
  • Develop and report indicators that will support data as a first-class scholarly output.
  • Incentivize data sharing by promoting the value of data sharing.

Today’s expanded and enhanced integration with Dryad, which inaugurates the new Data Repository Integration Partner Program at PLOS, is an excellent illustration of how to put these recommendations into action.

Read Full Post »

The following is a guest post from Tom Jefferson of The Cochrane Collaboration, Peter Doshi of the University of Maryland and Carl Heneghan from the University of Oxford. We asked them to tell the story behind their recent Cochrane systematic review [1] and dataset in Dryad [2] which holds valuable lessons about the evidence-base on which major public health recommendations are decided.  -TJV1918 Influenza Poster

In the late 2000s, half the world was busy buying and stockpiling the neuraminidase inhibitors oseltamivir (Tamiflu, Roche) and zanamivir (Relenza, GSK) in fear of an influenza pandemic.

The advice to stockpile for a pandemic and also use the drugs in non-pandemic, seasonal influenza seasons came from such august bodies as the World Health Organization (WHO), the US Centers for Disease Control and Prevention (CDC) and its European counterpart, the ECDC. However, they were stockpiling on the basis of an unclear rationale, mixing the effect of the antiviral drugs on the complications of influenza (mainly pneumonia and hospitalizations) and their capacity to slow down viral spread giving time for vaccines to be crash produced and deployed.

It has since become clear that none of these parties had seen all the clinical trial evidence for these drugs. They had based their recommendations on reviews of “the literature” which sounds impressive, but in fact refers to short trial reports published in journal articles rather than the underlying detailed raw data. For example, key assumptions of antiviral performance found in the US national pandemic plan trace back to a six page long journal article written by Roche which reported on a pooled-analysis of 10 randomized trials of which only 2 have ever been published.

In contrast, each of the corresponding internal clinical study reports for these 10 trials runs thousands of pages (for background on what clinical study reports are, see here.) Despite the stockpiling, these reports have never been reviewed by CDC, ECDC, or WHO. The WHO and CDC both refused to answer our questions on the evidence base for their policies.

Our Cochrane systematic review of neuraminidase inhibitors, funded by the National Institute for Health Research in the UK, was based on analysis of the full clinical study reports for these drugs, not short journal publications. We obtained these reports from the European Medicines Agency, Roche, and GlaxoSmithKline.  It took us nearly four years to obtain the full set of reports. The story of how we got hold of the complete set of clinical trials with no access restrictions is told in our essay “Multisystem failure: the story of anti-influenza drugs”.

With the publication of our review, we are making all 107 full clinical study reports publicly available. If you disagree with our findings, if you want to carry out your own analysis or if you are just curious to see what around 150,000 pages of data look like, they are one click away. Now the discussion about how well these drugs work can happen with all parties able to independently analyze all the trial evidence. This is called open science.

Be aware that there are some minimal redactions carried out by GSK and Roche. They did this to protect investigator and participant identity. While protecting participant identity is understandable, the EMA carries a different view towards protecting investigator identity: “names of experts or designated personnel with legally defined responsibilities and roles with respect to aspects of the Marketing Authorisation dossier (e.g. QP, QPPV, Clinical expert, Investigator) are included in the dossier because they have a legally defined role or responsibility and it is in the public interest to release this data”.

References

  1. Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, Howick J, Heneghan CJ (2014) Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database of Systematic Reviews, online in advance of print. doi:10.1002/14651858.CD008965.pub4
  2. Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, Howick J, Heneghan CJ (2014) Data from: Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Dryad Digital Repository. doi:10.5061/dryad.77471

Read Full Post »

Older Posts »

Follow

Get every new post delivered to your Inbox.

Join 10,155 other followers