edsc2017 (3 out of 3)

Review of "Utilising Semantic Web Ontologies to publish Experimental Workflows"

In reply to: https://openscience.adaptcentre.ie/publications/2017/eswc/

I can see how the ultimate goal described by the article - to publish semantic representations of experimental workflows - contributes towards the vision of decentralised scholarly communication. Unfortunately I'm missing how the work described in the article contributes towards this ultimate goal. The abstract says you want to see how semantic workflow creation can "be combined with traditional forms of documentation and publication" but I don't see a result pertaining to this.

To begin with, the purpose of the tool is quite narrow. How do you know for sure that "advancing knowledge about the use of vocabularies in facilitating sharing and repeatability of experiments and replication of results" ultimately contributes towards reproducability of results, which is the high level goal of this work? How does knowledge of OPMW tie in with the authoring or experimentation process, for example? Are researchers expected to document their workflows as they go along, or retrospectively after the fact (I imagine this depends on the task at hand). For which stage is your tool intended? Or is it simply a teaching tool rather than designed for actually documenting workflows? This isn't clear.

I thought I might understand better by running the tool, but it seems to be broken.

I'm skeptical about the open world assumption and the "nature of Linked Data" being used as a reason to not provide any instruction for using the tool... I would have thought that whether instruction is needed is a UI concern.

There is no results or analysis section, and the tense makes it sound like the described experiment hasn't actually been carried out. Are you asking for feedback about the design of the experiment? If so, you should state this clearly in the introduction and abstract. if not then I'd like to read this again when your findings are ready.

The background work sections appear to be fairly comprehensive, but this is not my area of expertise, so if there is related work or other background information missing I am unable to point it out. It's not entirely clear how all of the related work describe relates to the problem at hand though so I'd like to see this be made explicit.

The discussion about nuances of licensing is interesting, for example licensing different parts of a workflow separately, and conveying this to people who want to replicate experiments and use the data produced. I think maybe the licensing topic deserves an article and experiment all of its own. Colour coding or different levels of alerts for different licensing to help people understand is an interesting UI challenge. I think ongoing work on Data Terms of Use might be interesting to you (this is about personal data rather than experimental data).

In summary, the stated goals make this worth further discussion, but it's not clear how the work you've done so far meets these goals. I'd like to know what are your next steps forward for this work, and technically how this could integrate with other projects related to exposing more semantically enriched academic research to the world.

🏷 academic publishing edsc2017 linked research reviews semantic web

Sat 27th of May 2017 at 6:35pm (+02:00)

A response to "Scholarly Social Machines"

In reply to: http://www.oerc.ox.ac.uk/sites/default/files/users/user384/scholarly-social-machines.html

This article looks at the scholarly communication ecosystem through the lens of social machines; something I have thought about a lot but only partially articulated through my contributions to Linked Research. In particular, the hypothesis that we are struggling to improve the state of scholarly communication despite advancements in technology because we are trapped in a machine we have ceased to see is interesting. I've had many conversations with people over the last couple of years who balk at any suggestion of a radical upheaval of scholarly communication, hiding behind notions of futility of incentivising researchers to do anything other than pubish or persih, the immovability of the for-profit publishing industry. They complain about the state of things, yet are transfixed by the status quo. I think that framing this problem as a 'trap' will perhaps nudge folks into looking for a way out, and social machines is a useful concept to work with, something very demonstrably real in other domains, and this might help with finding a perspective on scholarly communication.

The article mentions the 'ecosystem' and lists some of the stages. I think it's worth noting that in Linked Research: An Approach for Scholarly Communication we characterised the different stages, roles and actors in scholarly communication in detail (there's even a diagram!). It's important because this is a hard problem, too big a space, to talk about all at once, and breaking it down into smaller chunks that nonetheless interlink (and the interlinking is critical) can help.

Dave writes that we see the same articles about how to improve being published over and over again, and this is true. I assembled a collection of some of them here. People lament that it's too hard to change, and us Linked Research-y types release software and publish our work in a Webby decentralised way (just like Dave has done, for the version of his article that I'm reading) and shove it under peoples' noses and cry that here is the proof that it's not too hard, and if only a few more people would bother we could iron out the wrinkles and make it easier and easier for everyone to participate. It's catching on, but agonisingly slowly. Most people still smile sadly, with a look of concern about the non-future of our academic careers. Dave writes of the "Catch-22" of using traditional publishing to try to solve some of the problems; and it's true. Even csarven is pushing 'papers' to academic conferences about Linked Research and related tooling in the hopes that academics will pay attention (and so he can get his PhD within this broken system). We console ourselves that at least the PDFs were generated from HTML and CSS, and a link to an open access Web version remains in the abstract even from behind Springer's paywall. Yet somehow the dual effort of publishing 'properly' and demonstrating alternatives is still not ticking the right boxes! What's up with that? So we need a new tactic, and the questions raised in this article are pragmatic and challenging, and a step in the right direction.

Question 8 is important because I often find that when we provocatively push "extreme decentralisation" via Linked Research, people push back against that as too difficult, too flakey, underdeveloped. We're arguing from this end of the spectrum because we're fighting against the very other end right now. But it is a spectrum, and better - more robust, more powerful - would be something in between. One example: the response that an individual can't possibly be counted on to keep their research online for eternity ignores the roles libraries and institutions (and indeed traditional publishers) can play in archiving and indexing work over time.

Another note to reinforce the importance of stepping outside to look in at scholarly communication: people, mostly software engineers, with whom I work on decentralised Social Web technologies look on at academia with confoundment. People claiming to be Web Scientists are putting PDFs behind paywalls? Life-saving medical research is accessible to what percentage of human beings? It seems obvious to the outside that we're really screwing things up here. Dave implies that we need to push people towards revolution over evolution, but we need to open a few more eyes before this can happen.

So how do we get people to step outside and look back in? This article doesn't address this. Arguably the people who will show up to Dave's presentation of this are already eyes-open, so perhaps together we can scheme to break the spell that the status quo holds over others.

Framing this (improving the scholarly comms ecosystem) as an interdisciplinary research problem could be emphasised more. Certainly we could solve all of the technical problems, but the social and political ones remain. I hope that we can take some inspiration from SOCIAM in tackling this in the years to come.

🏷 academic publishing edsc2017 linked research reviews social machines https://linkedresearch.org/

Sat 27th of May 2017 at 3:47pm (+02:00)

Review of Instrumenting Continuous Knowledge Extraction, Sharing, and Benchmarking

In reply to: https://marco-brambilla.com/2017/05/11/instrumenting-continuous-knowledge-extraction-sharing-and-benchmarking/

I like how this article seeks to accommodate a broad view of viable data sources for research, and particularly encourages data sharing and reuse between researchers.

The authors provide three examples of existing tools which could do (or be adapted to do) parts of the suggested pipeline. I hope that publishing this encourages others who are developing tools along these lines to come forward and let the authors and others know so that the community can start a comprehensive directory, as I'm sure there are plenty more.

It would be helpful to also have a characterisation of what is definitely missing as far as the authors know, and what the authors think are good directions to priortise for near term research and development.

Obviously I agree with the authors' call to open source such tooling for community benefit. I'd be particularly interested to hear your thoughts on the "agreed-upon integration platform" and what you think the best forum for discussing such a platform would be. Hopefully we can come up with ideas for that during the EDSC workshop discussion sessions!

🏷 academia academic publishing edsc2017 linked research review semantic web https://linkedresearch.org/

Fri 19th of May 2017 at 1:01pm (+02:00)