Review of "Utilising Semantic Web Ontologies to publish Experimental Workflows"

In reply to: https://openscience.adaptcentre.ie/publications/2017/eswc/

I can see how the ultimate goal described by the article - to publish semantic representations of experimental workflows - contributes towards the vision of decentralised scholarly communication. Unfortunately I'm missing how the work described in the article contributes towards this ultimate goal. The abstract says you want to see how semantic workflow creation can "be combined with traditional forms of documentation and publication" but I don't see a result pertaining to this.

To begin with, the purpose of the tool is quite narrow. How do you know for sure that "advancing knowledge about the use of vocabularies in facilitating sharing and repeatability of experiments and replication of results" ultimately contributes towards reproducability of results, which is the high level goal of this work? How does knowledge of OPMW tie in with the authoring or experimentation process, for example? Are researchers expected to document their workflows as they go along, or retrospectively after the fact (I imagine this depends on the task at hand). For which stage is your tool intended? Or is it simply a teaching tool rather than designed for actually documenting workflows? This isn't clear.

I thought I might understand better by running the tool, but it seems to be broken.

I'm skeptical about the open world assumption and the "nature of Linked Data" being used as a reason to not provide any instruction for using the tool... I would have thought that whether instruction is needed is a UI concern.

There is no results or analysis section, and the tense makes it sound like the described experiment hasn't actually been carried out. Are you asking for feedback about the design of the experiment? If so, you should state this clearly in the introduction and abstract. if not then I'd like to read this again when your findings are ready.

The background work sections appear to be fairly comprehensive, but this is not my area of expertise, so if there is related work or other background information missing I am unable to point it out. It's not entirely clear how all of the related work describe relates to the problem at hand though so I'd like to see this be made explicit.

The discussion about nuances of licensing is interesting, for example licensing different parts of a workflow separately, and conveying this to people who want to replicate experiments and use the data produced. I think maybe the licensing topic deserves an article and experiment all of its own. Colour coding or different levels of alerts for different licensing to help people understand is an interesting UI challenge. I think ongoing work on Data Terms of Use might be interesting to you (this is about personal data rather than experimental data).

In summary, the stated goals make this worth further discussion, but it's not clear how the work you've done so far meets these goals. I'd like to know what are your next steps forward for this work, and technically how this could integrate with other projects related to exposing more semantically enriched academic research to the world.

🏷 academic publishing edsc2017 linked research reviews semantic web