+ Recogito annotation platform
Amy added http://recogito.pelagios.org/rhiaro to https://rhiaro.co.uk/bookmarks/
annotation (9 out of 9)
- Finding content
- Media types * mostly text at the moment, little integration of different types
- Specific tasks - not much connection of results with user tasks.
- Search - serendipitous or focussed around a theme (or both). Different search goals.
- Sharing, passing it on.
- to identify (parts of) media assets.
- associate metadata with a fragment.
- agree on meaning of metadata.
- enable meaningful structures to be composed, identified and annotated.
- Canonical processes of media production
- Reduced to the simplest form possible without loss of generality.
- Feature extraction.
- News production.
- New media art.
- An interactive exhibit that responded to audience present.
- Hyper-video.
- Linked video.
- Photo book production (CeWe).
- (Using this example for explaining processes).
- Ambient multimedia systems with complex sensory networks.
- Premediate, eg.
- remember to take your camera on holiday.
- write scripts, plan shots.
- place a security camera in the right location.
- Construct Message (not really in the chain, appears all over the place); what to conveny with media? Intention? eg.
- show people a great holiday.
- sell a product.
- inform/advise.
- Create (method of creation might be important, so record in metadata), eg.
- take photos.
- make video.
- Annotate, eg.
- automatic or manual. Stuff that is embedded by device vendors (but there's so much more...)
- domain annotations: landscapes/portraits, timestamps, face recognition.
- Publish, eg.
- compose images into photobook.
- Distribute, eg.
- print photo book and post.
- cyclic processes online.
- What can be expressed explicitly?
- Message (somewhere between a html page and poetry).
- Objects depicted.
- Domain information. <\\\\--- li="">
- Human communicaiton roles (discourse). <\\\\--- li="">
- Rhetorical statement; argumentation model (documentary techniques).
- Descriptive (which questions asked, interviewee, filmic).
- Filmic: continuity like camera movements, framing, direction of speaker, lighting, sound - rules that film directors know.
- Statement encoding (eg. summary what the interviewee said):
- subject - modifier - object statements.
- Thesauri for terms.
- Can make a statement graph, finding which statements contradict and which agree.
- (He encoded this stuff by hand - automated techniques aren't good enough).
- Argumentation model - claims, concessions, contradictions, support.
- Are we more forgiving watching video? (Than reading these statements as text). Peoples' own interpretations strongly affect understanding of the message.
- Expert interviews (governance, broadcast).
- User interviews - what people thought they need (215 ppts).
- User experiments - what people actually need.
- eg. people wanted factual information from independent sources; what the benefits are; community scale information.
- We can give useful annotations to media access, useful at different stages of interactive access (not just search).
- Clarify intended message. Explicity with annotations.
- Manual or automatic.
- Media content and annotations can be passed among systems.
- No community agreement in how to do this. <\\\\--- li="">
- How to store?
- Technical / objective / statistical data: eg. media type; shutterspeed; framerate; duration; resolution; date created; number of times viewed at a particular source; number of time shared ...
- Bibliographic: creators and contributors and their roles; methods/location of publication; methods/locations of creation ...
- Content*: fictional characters; locations; camera movements; scene transitions; colours ...
- BBC Programme Ontology - here, and here
- BBC Mythology Engine - here, and here
- Yahoo media RSS standards
- YouTube RSS standards
- ComicsML
- Comic Book Markup Language
- Comics Ontology hack - here, and here
- Media Value Chain Ontology
- I want a better name for this, since I'm referring to everything as 'content' anyway. So some better way of saying 'content of content'.
- Information about creative processes for digital artworks.
- Anything to do with open data or decentralized social networking movements.
- Potential case studies or people to work with.
- ... to be discovered?
- Annotations must be shared in a community.
- Annotations can be reviewed and edited (/audited) (by group)
- Collaborative
- Central repo.
- Annotations contain semantic metadata.
- Content of resource annotated, not URL.
- Communication layer that doesn't interrupt annotation (uses external services).
- Annotea [9] [13]
-
- _J. Kahan, M.R. Koivunen, E. Prud Hommeaux, R.R. Swick. Annotea: an open _RDF infrastructure for shared Web annotations. Computer Networks. 2002.
- __
- _M Koivunen. Annotea and Semantic Web Supported Collaboration. Proc. Of ___
- the ESWC2005 Conference, 2005.
- No communication layer (but has discussion threads, wat?).
- Can only be edited by author, but can be reviewed by others.
- Can be local, private or shared. RDF.
-
- Piggy Bank [10]
-
- _D Huynh, S Mazzocchi, D Karger. Piggy Bank: Experience the Semantic Web _Inside Your Web Browser. Springer-Verlag GmbH. 2005.
- RDF.
- Auto and manual. Bundled with scrapers; if they fail, manual. Only of one type.
- Share group or global, or save to local 'semantic bank' <\\\\-- find="" is="" out="" this="" what="">
- Reviewed by all, edited by author.
- Community of users, but no SNS integration.
-
- KIM [14]
-
- A Kiryakov, B Popov, D Ognyanoff, D Manov, A Kirilov, M Goranov. Semantic Annotation, Indexing and Retrieval. Journal of Web Semantics, Springer. 2004.
- Automatic named entity recognition.
- Links to knowledgebase with ontology.
- Creates new URIs for new entities or link swith entities it already knows about.
- Global sharing.
- Can be deleted but not edited.
- No social involvement.
-
- Magpie [11]
-
- _J Domingue, M Dzbor and E Motta. Semantic Layering with Magpie. _Handbook on Ontologies. 2004.
- Auto annotate webpage.
- Similar to KIM, but does not hyperlink to knowledgebase; instead each item gets context menu (right click) with services depending on entity.
- 'Multi-dimensional approach'. Uses ontology to trigger other services depending on concept.
- Plugin for IE.
- Simply looks for entities that are in ontology (Dzbor 2004).
-
🔁 https://twitter.com/csarven/status/775368043094573060
Amy shared https://twitter.com/csarven/status/775368043094573060
This is super important and exciting development for decentralising and owning your notes, comments and annotations!
You can use dokieli to publish your posts and articles, which provides a UI to let people annotate and reply. For people with their own personal storage, they can choose to store their notes themselves. But this isn't many people, so you can also provide an Annotation Service and host your respondents notes for them instead (lowering the barrier to annotating, without locking them in to your platform permanently).
"⚙ #UI to 💾 #WebAnnotations @ annotationService & #LDP #Solid personal storage https://github.com/linkeddata/dokieli 🌟 #LinkedData"
- @csarven
Week(s) in review: #SSSW2013, figuring stuff out and annotating YouTube
8th - 14th July
Semantic Web Summer School, much heat, much fun, much learning... Here's an index of my posts.
15th - 21st July
Friends visited. Progress included writing notes to myself to figure out just what my PhD outcomes really are, and why. Came up with:
1\\\\. Recommending how to usefully describe diverse amateur creative digital
content (ACDC) using an ontology.
a) What are the parts of ACDC that need to be represented? Identify and categorise properties. How do these differentiate it from other similar content?
b) What existing ontologies can be used to do this, and how do they need to be extended?
2\\\\. Building an initial set of linked data about ACDC, and providing means for
its growth and use.
a) Manual annotation of ACDC, and refinement (to test ontology).
b) Tools for automatic annotation of the parts of ACDC that it is possible to automatically annotate.
c) Tools for manual annotation by the community of content creators and consumers for the parts of ACDC that cannot be automatically annotated.
d) Tools to expose the linked data for use by third-party applications.
3\\\\. Create and test an example service which uses the linked data to benefit
content creators and/or consumers.
eg. Unobtrusive recommendations for collaborative partners (most likely); content recommendation; content consumption analysis (like tracking viral content); community building / knowledge sharing in this domain; ... .
22nd - 28th July
Brainstormed with Ewan about stage 3 (above), and came up with the idea of an interface that allows content creators to allocate varying degrees of credit for roles played by different people when collaborating on a project. This would serve to both gather collaborative bibliographic data, learn things about how different segments of the community allocate credit, and provide a potentially useful tool for content creators. With the future value that, if we can learn enough to estimate role inputs from different people, it could be used for things like automatic revenue sharing.
Then spent the rest of the week in London, frolicking amongst the YouTubers (including attending a meeting at Google about secret YouTube-y stuff), and annotated some ACDC. Write-up coming soon.
Last modified:
[Notes] Lynda Hardman at #SSSW2013
RELEVANT.
Users (consumers?):
More data than just what you seen in the media (cue my Venn diagram).
Plus, eg. paintings - lots of 'cultural baggage'.
Care more about the story than the media.
Interpretation by end users. Hopefully message that the author intended.
Meaning of combination of assets.
eg. Exhibition of artists work.
Interacting further with the media.
(SW and multimedia community need to work together).
-> Raphael Troncy on Friday - attaching semantics to multimedia on the Web.
Need mechanisms:
Workflow for multimedia applications
Heard of MPEG-7? Don't bother.. very much from a media algorithms perspective.
Applications:
Canonical processes overview...
There's a paper.
CeWe photobook - automatic selection, sorting and ordering of photos.
Context (timestamp, tags) analysis and content (colours, edges) analysis.
Things from these you want to represent your digital system (ie with LOD):
COMM - Core Ontology for Multimedia.
Premediate and construct message - human parts, she doesn't expect them to be digitised any time soon.
Using Semantics to create stories with media
Can we link media assets to existing linked data and use this to improve presentation?
How can annotations help?
Vox Populi (PhD project)
Traditionally video documentary is a set of shots decided by director/editor.
vs.
Annotating video material and showing what the user asks to see.
interviewwithamerica.com
Annotations for these documentary clips:
Automatically generated coherant story.
Vox Populi has (not for human consumption) GUI for querying annotated video content.
User can determine subject and bias of presentation.
Documentary maker can just add in new videos and new annotations to easily
generate new sequence options.
User informatio needs - Ana Carina Palumbo
Linked TV. Enhancing experience of watching TV. What users need to make decisions / inform opinions.
Experiment - oil worth the risk?
Published at EuroITV.
Conclusions
Questions
Hand annotations are error prone - how to validate?
Media stuff - there can be uncertainty, people don't always care.
Motivating researchers to annotate...
Make a game.
Store whole video or segements?
W3C fragment identification standards - timestamps via URLs.
Last modified:
Week in review: annotating multimedia content
6th - 12th May
Discovering lots of things to write about semantically annotating multimedia content. I decided there are three main ways to do this:
These categories overlap somewhat really, and when I get round to it I'll type my Venn diagram up.
Technical is easy, and a lot of that is automatically captured by hardware or software used to produce and edit works. It's also relatively easy to extract automatically. Standards like MPEG-7 and MPEG-21 take care of formalising it, and Jane Hunter turned these standards into semantic ontologies in 2002.
Bibliographic can largely - but not entirely - be covered by vocabularies that have been around forever like Dublin Core, FOAF and various library- originated things. Things that might be missing (or I just haven't found them yet) are associating roles with tasks involved in digital media production, since pieces are often a collaborative effort. has some idea of participants and roles, but the purpose of is digital rights management stuff, so it's more concerned with the distribution change, I think, than granular production of content. I haven't read much about it yet.
Content is more interesting, and potentially more useful for ordinary human beings. Imagine querying IMDB for "that film where John Goodman arrests an animated talking moose on a US highway" instead of scouring John Goodman's filmography or googling for pictures of animated meese until you see the right one. Annotating characters, objects and events, and stringing them onto a timeline is possible with OntoMedia. It's very focussed around narratives, which is great, but doesn't link back to technical so much. So if you did find the answer to that query, it wouldn't be able to serve up the timestamp of that particular scene.
On top of what I've looked at already, I still have this list to (re)investigate:
A thing I want to do is annotate some amateur content with OntoMedia and with ABC to see how they compare. Maybe I'll do asdfmovie, because it has associated comics, and multiple people participating in production. Then I'll do something live action as well, because I can't base all my research on non- sequitur lolrandom stick figure cartoons.
Now, back to work..
Last modified:
Week in review: foreverreading
11th - 17th March
Had a sensible conversation with Ewan about when I should stop reading and start actually producing something. The answer is sooner, rather than later. I'm aiming for a literature review that is "neither a first nor final" draft by the end of April. I started outlining topics that should be in it, and began faffing about with LaTeX (and Markdown and Pandoc).
Then started panicking and promptly quadrupled the amount of things on my to- read list. Oops. I blame Matthew Rowe.
I finished reading his first year review (2007) as well as his paper about Meervisage: Community-based Annotation for the Semantic Web. I compiled a massive list of more things to read from his lit. review, and then looked up what he's been doing since then. A lot. Mostly relevant. At the very least, interesting. Damn. Don't worry, I'll prioritise that list, and don't aim to finish it before I start writing.
Going to Serbia on Tuesday, for Resonate.io. What should I bring back?
Had a chat with Russell of Makur Media about video metadata and ontologies for describing film/TV production processes. We're doing some of the same research and keeping in touch is likely to be mutually beneficial. Note to self: Summerhall Cafe is nice and the food is cheap; go there for lunch one day.
Last modified:
Notes about Meervisage - A Community Based Annotation Tool (for the Semantic Web)
Rowe, M. (2007) Meervisage - A Community Based Annotation Tool. ‘Towards a Social Science of Web 2.0’ Conference at the University of York 5-6th September, 2007.
How SW can benefit from incorporation with existing 'Social Web'.
"...collaborative generation of metadata... using social networks as a user base..."
Uses fb groups created for sharing and organisation of research. Suggests posting links to useful resources is comparable to annotating the resource. Comments are more metadata.
Points out usual stuff of actually generating semantic data being a problem for SW.
System requirements:
Review of existing systems:
[1] Using existing information to derive semantics from folksonomies
(delicious):
_X Wu, L Zhang, Y Yu. Exploring social annotations for the Semantic Web.
_Proceedings of the 15th international conference on the World Wide Web,
2006.
[15] Social bookmarking tools and how semantic info aids resource discovery. Probabalistic model of how resources are annotated:
_A Plangprasopchok, K Lerman. Exploiting Social Annotation for Automatic _Resource Discovery. Eprint arXiv, 2007.
[16] Distributed nature of folksonomies. Improve search mechanisms. Tags not great:
S Choy, A. Lui. Web Information Retrieval in Collaborative Tagging Systems. Proceedings of International Conference on Web Intelligence, 2006.
(vs.)[17] Rigid taxonomies not great:
_C Shirky. Ontology is Overrated: Categories, Links, and Tags. Clay Shirky’s _Writings About the Internet, 2005.
[18] Methodology for easier browsing of large scale social annotations:
Z Xu, Y Fu, J Mao, D Su. Towards the semantic web: Collaborative tag suggestions. Collaborative Web Tagging Workshop at WWW2006, 2006.
All use one annotation per resource, not annotation of content within, so only one lot of metadata about a page.
Meervisage
"To aid the process of collaborative annotation of web documents"
Allows sharing of annotations between subset of SNS users (eg. fb group).
Management of users and groups offloaded to third party.
Stored in central annotation store.
Annotations contain author, SNS, folksonomies and date. Made from content within.
Meerkat is "responsible for generating semantic metadata by annotating external web resources." Meervisage for management via social network.
Meerkat allows a user to edit another user's annotations if they are members of the same group on facebook.
Popularity rating of resources rises with fb discussion.
Meerkat informs browser users if they come across a resource that has been heavily discussed on fb, and by which group etc.
Meervisage also provides RSS feed.
Evaluate by comparing precision and recall metrics of annotations by one user in an allotted time, and those by a group.
-> Don't know how this helps to assess quality of annotations; maybe I'm dumb? Find out.
Limited to private, says it's like that's a good think :s
Oh, because public access would be "laborious and resource intensive".
Annotations rated on usefulness and weighted.
[20] Attempt to describe folksonomies as part of formal ontology. Meervisage doesn't; limited to users' viewpoint:
_S Angeletou, M Sabou, L Specia, E Motta. Bridging the Gap Between Folksonomies and the Semantic Web: An Experience Report. Workshop: Bridging the Gap between Semantic Web and Web 2.0, European Semantic Web Conference, _2007.
[9] + [13] are most similar. Have groups, but groups aren't already established networks.
Future work
Annotating multimedia.
Matching assigned tags with ontology terms mined from Web.
[19] Desktop app for annotating text with ontology:
A Chakravarthy, F Ciravegna, V Lanfranchi. AKTiveMedia: Cross-media Document Annotation and Enrichment. Poster Proceedings of the Fifteenth International Semantic Web Conference, 2006.
Last modified:
Week in review: mostly social computing
4th - 10th March
I played with the Tell Me Scotland SPARQL endpoint to put Scottish public notices on an OpenStreetMap: https://rhiaro.co.uk/publicnotices/map.php. It works intermittently, as the TMS endpoint is a 'proof of concept'. Okay, that's not PhD related, but it's still interesting.
I went to the 5th OKFN Edinburgh meetup at Napier. Look, evidence, I'm in the picture!
Dave Robertson asked me hard questions about how I'll turn my vague research interests and intentions to make something useful into a PhD with genuine contributions to knowledge and I floundered a bit, but he agreed to be my second supervisor nonetheless. I hadn't properly thought about it in those terms, and I'm still figuring it out with like, words. (As opposed to a gut feeling). I'd always planned to plough ahead enthusiastically and hope for the best. That's how I approach everything, actually.
I started reading Community-Based Annotation for the Semantic Web by Matthew Rowe (2007); I think it was a first-year PhD review, and I was primarily reference-harvesting. More on that next week.
I attended a Social Informatics Cluster meeting, and heard about the Smart Society project; very much in its figuring out stages, but worth following as it appears to have many diverse goals, and various crowd-computation related outcomes might be relevant to what I'm doing.
I went to an ESALA lunchtime talk entitled 'Data Objects', by Ian Gwilt of Sheffield Hallam. I have to admit, I was expecting something more technical and internet-of-things-y. Actually data objects are primarily tangible visualisaitons; 3D printed, carved out of wood or sculpted from bronze. The research was about how people reacted to and understood data differently when it was presented in different forms. Interesting, and they got some very pretty artefacts out of it, but not directly relevant so I don't have room in my head to store it unfortunately.
I attended the new Social Computing interest group kick-off meeting, coordinated by Dave Murray-Rust. Everyone introduced themselves, and we arranged a time slot for future meetings. It looks like I'll be presenting there at some point in the probably not-so-distant future (it was decided that everyone should). There were a very diverse bunch of people there, and 'social computing' hasn't been defined officially for the group yet. Quite a few people are attending to essentially see what all the fuss is about. I'm hopeful that there will be a hardcore technical leaning, because that's where most of the gaps in my knowledge are. Well, I'm more gaps than knowledge about everything, but that's where I feel particularly vulnerable.
Last modified:
Computer Mediated Social Sense-Making
I was fortunate enough to attend the Computer Mediated Social Sense- Making workshop, conveniently situated on the ground floor of the building I work in, on the 14th of February.
Whilst more technical than the Digital Methods conference I went to in December, the talks and panel sessions served to build upon things I started to think about then. Namely, beginning to situate my research interests amongst many concepts from the currently quite alien fields of sociology and anthropology.
The talks were varied, and key themes that emerged were the collection/use of data for social improvement (health and wellbeing, teaching and learning, disaster recovery), and the importance of context in making collected data genuinely useful. A notable challenge is that one piece of data might have a thousand different contexts from the perspectives of a thousand different human beings. So how to communicate these variations to software that processes this data, and perhaps makes decisions using it?
Perhaps not to worry too much about that at all. Process things locally instead of globally, using local contexts and understandings, but make sure everything is annotated such that information can still be exchanged across the whole network, and differences in understanding can be accounted for or reasoned out if a need occurs.
For the record, I'm looking at how Semantic Web technologies could be used to better connect human and machine in the context of amateur digital content creation (movies, comics, music, art), including how semantically annotating creative (often collaborative) processes as well as the end products of these processes and the engagement of an audience with these products, could improve the overall experience of creating content (along a number of dimensions). A massive part of this will be creating tools that actually collect the necessary data from users. Ultimately, these tools will need to be invisible, ie. easily integrated into existing online routines, with no effort required to use them for the non-technically minded so that a network effect can take place.
Incentives for crowdsourcing came up during CMSSM, and someone pointed out that by gamifying data collection for research projects, incentives become the same as ones offered by gambling companies; something competitive and potentially addictive. I think things like global systems of reputation and trust are useful on a network where people are to share data about their own work (or opinions of the work of others) and may be nurturing a desire for popularity or exposure on the network (a network where the people are central, because the data could not exist without them, but where the users and the data are simultaneously co-dependant).
Anyway, I'm still brainstorming.
Last modified: