This post is my own opinion, and does not necessarily represent the opinion of the Social Web WG!
We met at MIT in Cambridge, MA, on the 17th and 18th of November. See also day 1 minutes and day 2 minutes.
Everything is progressing. Naming things is hard. We need your implementations please or features may be dropped from some specs. We hope to extend by 6 months, not so we all make more work to do, but just so that the newest spec has time to make it to rec given process timing requirements. We're transitioning any other work into a CG.
The Social Web Incubator Community Group, should be pronounced swi-kig, but by the end of the meeting everyone had taken to calling it swish. I think it was Evan's fault. Anyway it's live, and is where we'll continue to develop on top of the things we started in the WG, as well as think about how to tackle things we haven't got to yet. Aaron and Chris are chairing it, and plan for discussion to take place primarily on github and IRC, with mailing lists for broadcast only. You should join.
At the last face-to-face we renamed PubSubHubbub to PubSub. We subsequently realised this is too generic a term for quite a specific spec, and as a result is hard to search the Web for, and hard to find/name libraries and packages for. Renaming it again took the better part of a month. Heh. A few weeks ago we developed a fairly long shortlist on the wiki, listing pros and cons, and a few people voted and left their rationale. On day one of this face-to-face, we ruled out every single one of those suggestions, and came up with three new ones (WebSub, WebFollow and WebSubscribe).
We slept on it, and just before lunch of day 2, voted between these three. WebSub won. I like it for its closeness to PubSub; WebFollow is a good name for a user-facing feature that implements the WebSub protocol. Then we proceeded to brainstorm more names in the google doc, progressively making the font smaller and introducing columns so we could see them all at once.
In less important news, we added Aaron as a coeditor of the WebSub spec, resolved a bunch of issues, and there's an updated working draft up.
We decided to go ahead with a new CR for ActivityStreams 2.0. Though it's frustrating to increase the time to exit, it's also not infeasible that getting implementation reports which sufficiently cover all features will take another month anyway. Plus, this extra time ensures that the ActivityPub implementations will make it into AS2 implementation reports.
So we have a bunch of changes to AS2 since we entered CR, although none of them affect implementations or are technically normative changes, which is why we could get away without restarting CR if necessary. But we decided updating the spec with these changes (mostly editorial, clarifications, etc, which do not change the intent of the spec) is important enough not to save them all for the PR publication. Personally I think we should publish a version with the new wording around name and summary (a plaintext summary for all objects is required in the absence of name) as soon as possible.
Another useful clarification is explicitly stating that the value of the @context key may be a plaintext string, an array, or an object. We added examples of each of these, so it's clear for consumers what to look for. This is particularly important for making sure implementations which include extensions - for which the @context is necessarily an array or an object - are not completely dropped on the floor by consumers. Consumers can of course ignore extension properties they don't understand, but they should not ignore standard AS2 properties just because there are extensions alongside it.
This also means that it's possible to use the JSON-LD @language construct properly (inside the @context object) to set the base language for a whole AS2 object. As there are other ways to set the language, for individual objects or for specific values, setting the @language is not required. Further, you should not set a language if you don't actually know what it is. And we haven't dumped language tags in all of the examples in the spec, in order to avoid people copying and pasting the examples without updating the language tags we use. Apparently this phenomenon is seen all over the Web, with EN language markers alongisde text that is most certainly not EN.
We skimmed through a few issues for each of Micropub, LDN and ActivityPub, and checked in on how test suites and implementation reports are doing. The editors (Aaron, Sarven, and Chris respectively) are working exceptionally hard on building the test suites and chasing implementors. They are all at various stages of progress, and we know we have at least some implementations of some features of each.
The Working Group's charter expires at the end of this year. Due to minimum time constraints on various parts of the publication process, as WebSub was late to join the WG we need until at least April to take it through to a recommendation, and that's with absolutely nothing going wrong. We were aiming, obviously, for all of our other specs to be wrapped up before the various December holidays, but it'd be tight. Adding buffer time for unexpected issues, and editors-not-having-to-make-themselves-ill-with-allnighters time, we figured they'll be exiting CR in January or early February at the latest. So we expect to get an extension of 6 months, and reduce our telecon time to monthly after January. The extra time on top of April means we won't need to freak out if for any reason WebSub has to have a second CR. This also overlaps with the opening of the Community Group, so it should help with the transition.
An extra shoutout to anyone who is thinking of or starting to implement any part of any of our specs! Please let us know, either by filing implementation reports (even partial ones are helpful) or pinging us on IRC (#social) or the mailing list so we know to chase you at some point in the future. If you don't want a feature of a spec to be dropped, ie. because you want to use it, we have to prove it has been implemented. If possible, don't wait around for us to exit CR, because we need your implementations to make it that far.
Post created with https://rhiaro.co.uk/sloph
This post is my own opinion, and does not necessarily represent the opinion of the Social Web WG!
We met in Lisbon, Portugal, during the last two days of the W3C's annual TPAC conference on the 22nd and 23rd of September. See also day 1 minutes and day 2 minutes.
It's crunch time, but we're on track. We have a selection of specifications which essentially cover creating content and then pinging people about it in various different ways. Many people in the group are using one or more of these specs to power their own personal online social stuff. There are so many more Social Web problems to solve, and we're just trying to dig some foundations and plant some posts in the ground to get everyone else started.
The first time the Social WG gathered last week was for an hour on Wednesday, to demo the products of all of our specs to the broader audience of TPAC attendees. We had a full house. Everything went smoothly, and our demos were fairly coherent. Aaron showed Micropub and Webmention working together in the context of posting a reply to his site (with Micropub), which automatically sent a Webmention to post he was replying to, and after a few seconds his reply showed up there (on the site of someone who isn't a WG member, and had implemented Webmention receiving independently).
My site returns ActivityStreams 2.0 for everything when you request application/activity+json or application/ld+json, so I showed collections and individual objects at the commandline. Dumping a blob of JSON on the screen isn't a particularly compelling demo, but Chris was able to furnish me with a screenshot of his AS2 consumer, emacs-based, rendering one of my posts. Maybe you want to make an AS2 reader so you can design your own reading experience for my posts?
Chris showed how he is able to create notes on his site using an emacs-based posting client, which prompts his server to send the message on to the inbox of anyone addressed, all using ActivityPub.
Sarven showed dokieli; sharing an article with someone (automatically pulled from the contacts list in his profile) which sends an ActivityStreams Announce to their Inbox using LDN. Also annotating an article, whereby he could save the annotation in his own personal datastore (via LDP) and optionally send a copy to the AnnotationService provided by the publisher (using the Annotations protocol, plus send a notification to the publisher with LDN.
What followed was some discussion of how we're planning to tackle spam and abuse. ActivityPub includes a mechanism for blocking another user; LDN encourages imposing constraints on one's inbox to filter what ends up there; and self-hosted or known/trusted hosts for you content can give you more control over what shows up where, but there's way more to be done here, and we're not close to tackling that yet.
Guidance was recently issued that specifications should ask for horizontal review (review by other W3C groups) three months before going to Candidate Recommendation. We're obviously far too late for that, so we're doing the best we can with the time remaining. Horizontal review is part of 'wide review', which specifications must also demonstration before they can transition to CR. Wide review invovles having input and feedback from a variety of different sources outside of the Working Group, and even W3C. We can demonstrate this through issues raised on github, implementations reported to us, and feedback on public mailing lists. As such, if you're going to send any of us a message about any of our specs, please do so somewhere that is public! The public-socialweb list is archived by W3C, so that's a good one to CC if you don't want to raise an issue on github directly. Pointing out typos, requests for clarification, and pledges to implement are welcome too.
The Security & Privacy groups provide a questionnaire to guide through common issues that might be flagged during review by those groups. We collectively decided to carry out this questionnaire and add it to the 'Security and Privacy Considerations' section of all of our specs. The Internationalisation (i18n) WG also provide a checklist to run through which helps them perform their review more quickly.
ActivityStreams 2.0 (core and vocab), Webmention, and Micropub are currently in Candidate Recommendation (CR), which means we're actively soliciting implementations and accepting reports. They all have test suites in some stage of readiness, and the implementation report templates are provided in the respective repos, and designed largely around reporting which tests your implementation passes. Issues raised during CR which require substantive spec changes and may affect existing implementations require restarting the CR period (pushing back CR exit for a minimum of four weeks), so the earlier we hear about those the better. We went through issues raised on our CR specs to determine whether they result in normative changes, and for the most part they were editorial.
Webmention dropped returning a human-readble message in response to a sender's POST request, because anything involving human-readable text opens internationalisation challenges. This message was primarily for developer debugging as it's unlikely to ever been seen by an end user, and it was optional anyway, really only mentioned as a possibility, so this was always at the implementations' discretion. Servers are expected to 'do HTTP properly' and send and honour Accept headers for language if they are sending human-readable responses, which doesn't need explicitly calling out in the Webmention spec.
There was discussion of requiring a "backoff" strategy when trying to discover a Webmention endpoint. This also applies to LDN, and any other specs which requiring probing at webpages to determine if they accept whatever the discover-er is trying to send. The issue is that a sender may be triggered to attempt discovery multiple times on the same URL, or for different URLs on the same domain. There's a point at which they should stop trying if they fail, or risk the server being probed blocking them completely for an unreasonable amount of GET or HEAD requests. Webmention has added a suggestion to include a User-Agent header which mentions "webmention" with the initial discovery request, so that an unsuspecting target may at least have some idea of what all of these requests are about, and can act on more information than just assuming it's some kind of DOS attack. This isn't a requirement though, as implementations in JavaScript are unable to edit the User-Agent, and implementations that are part of some other system may already have a custom User-Agent that they can't or don't want to replace. Consequently I've added a section to Social Web Protocols to give additional general guidance about not making enemies when you're looking for endpoints, including honouring eg. Retry-After headers on the server.
The biggest Micropub issue concerned the media type of the (very specific) JSON structure returned in the event of an error. The conclusion was not to do anything special now, but if there are future versions of Micropub which modify the structure, they will use a profile to indicate which version they are, so implementations encountering that error know how to interpret it.
During our crossover meeting with the i18n WG we noticed that Micropub's dependency on Microformats2 means that it has no way to indicate content language. This is being worked on in mf2 though, and Micropub will get a note to indicate this, and any improvements to mf2 will be incorporated in Micropub by reference. This seems at first glance a bit fishy since we're only supposed to reference 'stable' external specs, but this was resolved earlier in the year by the various mf2 sub-specs documenting their change policy, and committing to clearly indicate which parts are 'stable' and how they are updated.
ActivityStreams 2.0 has acquired a bunch of editorial comments since going to CR, and these have been addressed. There's a bigger question of how to manage (append-only) extensions after the close of the WG, and we settled on Proposal 2 in issue 370. I'll be writing this up in the (pending) human-readable namespace document shortly.
Then there was an even bigger question of whether or not name is a required property for all objects and activities. At the time of the meeting it was required, but given some implementor experience - in particular with people adding dummy meaningless names to activities just to pass the validator - we revised this. The usefulness of a name for every entity is giving consumers something to display even if they don't understand any other properties. A drawback of developers adding auto-generated or dummy names is that a consumer can't tell the difference between this and an actually meaningful name, so it can't make a sensible decision on whether it can overwrite it with something more useful (eg. in a different language, or something particular to the application at the time). We decided to shift the burden of assigning names to unnamed entities from publishers to consumers, as consumers are more aware of the context in which the entity is being used or displayed. Publishers can of course assign a name, which should be consistently respected by consumers as they'll now have confidence the name was assigned deliberately. We still think there should be a name for Article type objects. So we resolved to switch to having two properties:
We haven't decided exactly what the names for these two properties are yet. Probably 2. is name and I like fallbackName for 1., but others up for discussion are title and label... but watch this space... (and issue 312).
We really need implementation reports for AS2 by the way... in case that was news. Let us know what you're working on!
The most substantive issue we discussed on ActivityPub was adding a source property, to indicate the original input the user made in case it is converted from something else to HTML for the content field. The obvious use case for this is letting users author their content in markdown. Storing the original markdown alongside the converted HTML lets the client present the markdown back again for the user to edit in future. Current pump.io implementations have problems with letting users create content at first with markdown, but presenting HTML back to them later, which can be confusing. Personally I think a client that offers markdown editing and can do the conversion to HTML to send to the server should also be able to take that HTML content and convert it back to markdown for the user in the case of editing an existing post. Apparently going back the other way is more challenging though. And Chris's use case was for having a representation of the content in emacs org mode, not markdown. I was worried that the source and content fields could get out of sync if editing the same content is done by multiple different clients. That's (probably?) fairly edge though (though I'm already using multiple APish clients on the same content). We compromised by saying that clients which find source and don't understand it must prompt the user to let them know the original source will be lost of they proceed with editing, as a feature "at risk". I'm a bit wary about specifying a UI thing in the spec, but we'll try it out and see how it goes. But maybe all it really is is a requirement that clients MUST understand this particular property; so far there aren't any properties a client can't ignore.
We got agreement to add ActivityPub terms which are extra to AS2 into the AS2 namespace and JSON-LD context, which will simplify things for implementors. This also stands as an initial experiment with how AS2 extensions can be done in the future.
Chris also proposed to define the media endpoint of AP as "at risk" as, even though they need it in MediaGoblin, it's not worth potentially holding the rest of AP up for, and they can easily develop it as an extension afterwards. There may also be a possibility of convergence between the Micropub and AP media endpoints.
We began our LDN discussion by learning that Kjetil, who had attended the first day of our meetings and then had to leave, implemented a portion of an LDN sender/consumer on his flight back to Norway.
LDN is addressing the "backoff" issue previously mentioned for Webmention by referring the new section in Social Web Protocols, and stressing that implementations should anticipate and respect relevant HTTP status codes. After all, it's only polite.
We briefly discussed LDN's need to add inbox to the LDP namespace, but we had a session on the Wednesday plenary day about extensibility of W3C namespaces in general, including a bunch of LDP people in the room, none of whom objected. We also had support in our github issue about this. The WG agreed to go ahead with whatever the relevant W3C and Semantic Web communities consent to. Hindsight addendum: I added inbox (at risk pending PR) to the LDP namespace a week or so later. Nobody has complained and the Web still seems to be up.
In LDN we had a section ("Security, Privacy and Content Considerations") which mixes normative and non-normative content, and I was getting confused about what kinds of things are supposed to be which. The experts in the room helped us to untangle this section, and we decided to move all normative content into the appropriate parts of the spec; to move some of the informative content into green 'note' boxes if they pertain to something specific, and anything that was left should apply to the spec in general and we could mark the entire section as entirely non-normative.
Both LDN and AP will have complete or partial test suites, or a detailed plan for a test suite, before they can enter CR.
Post Type Discovery right now defines a 'mapping' from combinations of properties a post might have (based on mf2 properties, though many of them are explicitly unstable so we can't actually reference them normatively) to a 'type' currently presented as an English-language string. We discussed at length the purpose of Post Type Discovery and came to the conclusion that it ultimately needs to be possible to generate ActivityStreams 2.0 types (rather than arbitrary strings, which are essentially yet another vocabulary). Many of us thought it was supposed to be doing this anyway, and were surprised that it hasn't yet. So this should see some significant updates in the near future towards this.
PubSubHubbub is raring to go as a FPWD, thanks to Julien's hard work in updating it to W3C spec format. We just have to jump some process hurdles, and that'll be published soon. Now is a great time to raise issues on it if you have or intended to implement it! We spent some time discussing a new name which is friendlier to non-English language speakers, and we're probably going to go with PubSub. We also decided to close the PuSH community group, as any further discussion around the spec should take place in the WG.
There's a general consensus that we've barely scratched the surface, but we formally resolved not to try to take on any more recommendation track documents henceforth. We need to focus on getting what we have to rec! However, to continue work after the group closes at the end of the year, including for managing errata and extensions, we'll open a Community Group. We'll do this towards the end of our charter, but if you're interested in being a part of it, drop an email to the public list.
We're planning another face-to-face sometime in on the 17 and 18 of November in either San Francisco or Boston. Whence we'll hopefully be wrapping things up!
There's another writeup here by Chris.
The demos at today's decentralised social web breakout session went suspiciously well. Some things weren't pretty, but everything basically worked, and none of us had done any special coordinating before hand. We just showed the stuff we worked on, and it fit together fairly coherantly.
Aaron demoed posting a reply with Micropub and behind-the-scenes Webmention sending.
I showed my site feeds and individual posts as ActivityStreams 2.0, plus a screenshot of Chris's emacs-based reader displaying it:
Chris...
Just live demo'ed ActivityPub client-to-server stuff and server-to-server stuff at TPAC and it worked. Yay for live demos not blowing up! - @dustyweb on twitter
demonstrated Pubstrate posting via ActivityPub to his local site, and peer to peer to another. - @t on twitter
Sarven showed LDN, Solid, Web Annotations in dokieli, notifications sent to me displayed in OnScreen, and the LDN spec self-dogfooding with its own annotations on the side.
We had some good questions/discussion about abuse prevention, and uptake.
Hopefully this energy can be sustained through the next two days of WG meetings!
Published a big update to Social Web Protocols. Read if you want to understand what the W3C Social Web WG is up to.
This post is my own opinion, and does not necessarily represent the opinion of the Social Web WG!
See also day 1 minutes and day 2 minutes.
We met in Portland on 6th and 7th June. What follows is more detail on my perspective of the main conversations we had over the two days. Clarifications and corrections welcome. This doesn't cover everything we talked about in detail; as well as the following, we resolved (or at least discussed) issues on all of the specs, and took a few to new Working Draft status.
I demoed my ActivityPub implementations; clients Burrow for checkins, Obtainium for consumption/purchase logging, Replicator for food logging and Seeulator for journeys and events. These all do create only by sending either appropriate activities (including some extensions) to my activitypub endpoint (aka outbox, but not discoverable as such yet).
Seeulator creates the right kind of activity based on what attributes are filled in or left blank, essentially doing post-type discovery - albeit my own algorithm rather than tantek's spec from the user input to generate the right as:Activity.
The newer thing I worked on was a client that only does updates of existing AS2 data. I wanted this so I could add captions to all my photos at img.amy.gy, so Morph does just that. This also means it has to be able to read/consume the AS2 data published at img.amy.gy about as:Collections of photos.
Aaron demoed the Webmention test suite at webmention.rocks and notes that there are Webmention Rocks stickers available for people submitting implementation reports..
Aaron also demoed a new feature in the Micropub spec which is the media endpoint. After some discussion recently it was established that all mainstream social APIs seem to post media (like images) that have been embedded in a post to a separate endpoint, then embed the returned URL in the post content, and MediaGoblin does this too. Aaron's implementation in Quill is really swish looking, uploading the file to the discovered media endpoint whilst you're typing the rest of the blog post, then embedding it back in the UI so you can see it straight away. I should probably implement something along these lines, and sync it up with what ActivityPub is doing (which is going to be basically the same); it's especially useful as I host my images on a completely different domain and stack from my blog posts and right now I have a by-hand process of uploading images to one server, then copying the URL into a blog post to embed.
Evan showed the Wordpress plugin implementation of AS2 by pfefferle, demonstrated on his fuzzy.ai blog. We all noticed a few bugs and AS1-isms in the implementation, but all correctable and all good flags for when we give advice for people switching existing implementations from AS1 to AS2.
For a while I've been pushing to break ActivityPub up into several separate specs for each part, modularised by functionality, reasoning that this will lower the barrier to both CR and conforming implementations. I feel strongly that distinct functionalities should not be dependent upon one another to conform to the spec; ie. if I only want to implement subscribing/reading in my application, I shouldn't be required to implement creating new content as well. I've been back and forth on this with Chris and Jessica for at least a year and we're all getting closer to understanding one another. The WG resolved at this meeting to split into ActivityPub (reading, creating, updating, deleting content) and ActivitySub (name pending; subscription and delivery of content). It took me a little longer than it should have to really grok how closely tied 'delivery' and 'notifications' are, but now I realise that regardless of what triggers 'delivery' of an activity, the process of 'delivery' to someone's inbox is the same. The triggering part can be a subscription (a special side effect of receiving a Follow activity) or a notification (an activity or object is created which is addressed to or links to a user or other activity/object). Thus I anticipate ActivitySub describing how the triggers work, then how delivery works upon a trigger. I'd still like to be able to conform to the 'delivery' part without worrying about the 'trigger' part (maybe I want to implement an entirely different subscription trigger mechanism) but this can be achieved with conformance classes if splitting the spec up further is too much.
The working group wraps up at the end of 2016. There's still time for us to work on new specs, but the ideal is that anything new being presented to the group will have been incubated (worked on, tested, implemented) outside of the group beforehand, either in a CG or other community or organisation. Coming soon to an editor's draft near you: PubSubHubbub!
We confirmed we'll meet on Thursday and Friday at TPAC in Lisbon in September. We'll also run a social web breakout session on the plenary day (Wednesday) like we did last year.
Updated working draft of Social Web Protocols published ahead of the Social Web WG face-to-face next week: https://www.w3.org/TR/social-web-protocols/ This describes and explains the various specs the WG is working on. Far from done, but we're making progress..
This post is my own opinion, and does not necessarily represent the opinion of the Social Web WG!
See also day 1 minutes and day 2 minutes.
We met in Boston on 16 and 17 March. What follows is more detail on my perspective of the main conversations we had over the two days. Clarifications and corrections welcome.
AS2 is inching closer to CR. Evan has made a validator at as2.rocks and done a lot of work on conformance criteria which we went through as a group and updated a little; mostly changing SHOULDs to MUSTs.
Discussed and not necessarily resolved a few new open issues, including: considering dropping the Relationship object and reviving it as an extension if necessary; a proposal for a new property to say when something was deleted; weakening the SHOULD requirement on name; clarifying scope and context.
Stay tuned for a new working draft sometime soon.
The most exciting thing I thought was agreeing on the potential for convergence between the create, update and delete parts of Activitypub and Micropub.
Micropub started life as a super small and simple way for clients and servers to agree how to create content on a website by POSTing form encoded parameters to an endpoint. As a result of this simplicity, there are dozens of client and server implementations, allowing people to use each others posting clients to add posts to their site, from simple text-only posts to photos, events, RSVPs, likes, bookmarks, reposts. When Micropub needed update and delete, it grew beyond what form-encoded parameters could sensibly handle, and added in a JSON syntax which I think to date only the editor has implemented.
Activitypub uses a JSON syntax (ActivityStreams2) from the outset for create, update and delete, and when you compare this with the Micropub JSON they look remarkably similar.
My posting endpoint implements create the AP way, and endpoint discovery the MP way. It also catches Micropub form-encoded requests and translates them to AS2 JSON before proceeding, so I can still use simple Micropub clients. My posting clients burrow (checkins), obtainium (purchases), replicator (food) and seeulator (events, RSVPs, travel plans) all post AS2 JSON... after discovering the endpoint via rel=micropub. Next on my list, and well overdue at this point, is adding update and delete to both server and clients.
So I proposed we write a document that unifies the common parts of AP and MP, iron out the smaller differences, and hope this coalesces into a small create/update/delete spec which both AP and MP can reference rather than duplicate. Because modularity is good, and common modules are better! I dubbed this temporarily (or is it?) SocialPub.
So what's left in Micropub? I hear you cry. The super simple form-encoded create which is what made Micropub do so well in the first place is really what makes Micropub micro, so I'd like to see this be the bulk of the Micropub spec, with just a pointer to SocialPub for people who want to level up to JSON.
There are still more than a few issues to be dealt with, though we handled a few during the meeting (such as media uploads). I'll be writing SocialPub up into the Social Web Protocols doc next week, stay tuned.
Jessica and Chris demo'd Media Goblin federating with pump.io! Which is cool. Which brings them a huge step closer to implementing things with AS2/AP and federating that way. They discussed how one of their main impediments had been database schema migration.
Aaron demo'd his Micropub editing UI, which allows partial edits on the post, only for data he is most likely to want to edit (tags, syndication URLs and date).
Aaron also demonstrated a new event posting interface in Quill which uses Micropub, and showed how RSVPs from Woodwind (a feed reader) work via Webmention. Tantek and Ben also demo'd RSVPs from their sites. And Ben demo'd how he can post reactjis as replies, exemplified with the poop emoticon, and there is no question that the future of the social web is in safe hands.
Frank demonstrated federation between OwnCloud servers, which uses WebDAV and CalDAV, and talked through their access control.
We also had a couple of admin/process related discussions. The first included agreeing to meet at TPAC in Lisbon in September as it already looks like there'll be critical mass to make it worthwhile.
Sandro has made a list of issue labels for github which we painstakeingly went through to make sure everyone understands them and editors are willing to use them on specs. This should help people to figure out at a glance what the current state of a spec is from the issues, as well as help passers-by to jump in if they want to get involved.
Today I broke my website completely and fixed it again. Rewrote lots of code in the process. I gained some functionality and lost some functionality. I may have lost more than I gained. Now I can do what I was planning to do tod.. oh. It's tomorrow.
+ https://apps.rhiaro.co.uk/burrow/
Amy added https://apps.rhiaro.co.uk/burrow/ to https://rhiaro.co.uk/bookmarks/
This post is my own opinion, and does not necessarily represent the opinion of the Social Web WG!
See also day 1 minutes and day 2 minutes.
What follows is more detail on my perspective of the main conversations we had over the two days. Clarifications and corrections welcome.
We steamrollared through AS2 issues, which included core simplifications, vocabulary changes and reductions and editorial clarifications. James made most edits during the meeting, so there's a much updated working draft available now. You should read it and post any issues you see to be CR-blocking by the 15th of December.
JSON-LD related stuff has been moved to its own section; since it is optional, having references sprinkled throughout was confusing for people who don't necessarily want to deal with it. Chris pointed out that even for extensions, if you know the extension you want to handle you can still do that in plain JSON.
Still some consternation about which alternative syntax examples should be in the spec. Since they're non-normative/editorial anyway they're not a CR-blocker so left alone for the time being.
We don't need a verison number in the URL because any future widely-deployed extensions that want to make it into the core will likely be incorporated as part of a full new version (ie. AS3). At some point we should start a registry for extensions; James is keeping track himself on github, we could move this to a wiki page and ultimately a CG when the WG wraps up.
Lots of discussion about testing frameworks and how to meaningfully test production and consumption of syntax and vocabulary. I refer you to the minutes, as this is beyond my ability to summarise well enough, but it seemed like the people working on this gained some clarity and a plan to move forwards.
There's some contention around what it means to accept a spec as an Editor's Draft in the WG. Our general consensus was that it means a spec isn't necessarily going to be rec-track, or even the direction the group is going to take, but it's in-scope and worth some portion of our attention, even if that is just to inform other things. Most specs we've picked up are work that the editors were doing anyway, and this just means the WG should explicitly not ignore them. It's expected that the specs will change significantly going forward, in response to input from the WG.
As such, it's okay that some of our now-ED specs currently cover overlapping territory. We hope that WG attention will serve to refine, cut, expand, merge, and otherwise sort this out. We may end up multiple small specs derived from our current set of EDs which cover pieces of the social puzzle, or several specs will demonstrate multiple viable ways of doing the same thing. My preference is for the former, but the latter is better than nothing. (I tentatively extrapolate the latter into either one gets wide adoption and the others quietly fade out, or they all get equal adoption and people build bridges between them, either way not a total loss).
I finally convinced people to stop calling this "Amy's SocialAPI Document" and renamed everything to Social Web Protocols. This document describes the individual componants we are trying to standardise (based on user stories), covering both the API and Federation. Given the potential for the WG to produce multiple small specs, work on this is to continue to describe and serve as a guide to each building block. This should highlight both points of convergence between separate specs, and gaps that no existing specs are filling adequately.
Issues filed should be to that end. Point out obvious points of commonality between specs that I haven't noted, or where it would be worth replacing the vague overview with more spec-like details.
I aim to take this to FPWD next week (which does not imply WG consensus on the contents yet) with the expectation that this is currently an overview document: a guide to the different areas being standardised by the WG. If we end up with a bunch of small (or overlapping) specs, this could end up as a Note, detailing how they relate to each other as a guide to implementors. If we end up with one spec that covers everything, either this becomes it following input from the other drafts, or this has done its job at converging things and is dropped completely. The rationale behind going to FPWD with this document is to better advertise and explain the different angles of work the WG is doing to other WGs and the public, and to seek wider feedback thereon. The issues become a place to discuss features based on functionality, where they are not specific to one of the other individual specs.
Both now EDs. Micropub covers a subsection of functionality of ActivityPump, but is uncoupled from any other pieces of functionality, whereas ActivityPump intertwines lots of things. There are distinct similarities - both POST JSON to a specific endpoint to create, update and delete content. The editors are keen to cooperate so there's value in working on them both in sync.
jf2 is a social syntax to complement (not compete with!) AS2, where jf2 is content-centric and AS2 is activity-centric. The editors of both agreed that it would be beneficial to work on these in conjunction. I've written more about the relationship between them here.
PTD is an algorithm to help consumers who find themselves with implicitly typed objects to derive explicit AS2 types, if that's what they prefer to work with. My concern is that it's biased towards the microformats2 vocabulary and currently useful for a niche - but a very small niche. However, James pointed out that types are actually optional in AS2 in general, so it could be expanded to help go from untyped AS2 objects to typed ones as well.
My other concern is that this space is such a moving target, and having a fixed algorithm to derive post types based on properties is going to get dated really fast. A constantly shifting algorithm doesn't fit into W3C workflow, and of course brings its own problems (who has implemented which version as it changes over time). I don't have an answer to this.
Evan forced us to slow down and really think about what we meant by federation. I don't know if anyone else did, but I had a bit of a lightbulb during this discussion. Bogged down in the particular bits of federation that are within reach to me (basically, notifications), I forgot there are a great many other things one might want to federate, like search, following topics, user discovery, recommendations... The WG is not required to tackle all of these, as a 'federation protocol' is the icing on our charter cake, but we absolutely don't want to de-prioritise federation completely as it's important and actually ties in well with a lot of the work we're already doing. But taking a step back to reassess - and clarify to the outside world - what we're actually doing is important, and there's some effort going into that now.
Webmention has been an ED for a short while, and there were some issues to work through. Some really interesting points have been raised around technical details, security concerns and functionality enhancements, and lots of different ways to refine this spec are emerging. We talked through open issues, and group resolutions were made for most of them.
The one I'm most interested in is addition of the property parameter for better disambiguation of the assertion being made by a webmention. It surfaces a need of people outside of the current core webmention implementors to send and verify claims more precisely, whilst adding minimal additional overhead for those who don't need it. Since bringing this spec to the WG is an effort to gain wider adoption and investigate broader use cases than what we have at present, taking these kinds of expansions seriously is good. The benefits were positively acknowledge overall, and I was hoping we'd see this added to the current version, even if marked 'at risk' pending future implementations. But the decision was taken to leave it out, in the same vein as AS2 currently relegating all addition suggestions to extensions. On the plus side, the webmention spec will link to all proposed extensions which are written up as specs, and if any extensions see enough adoption over the course of the work they can be integrated into the core.
Bridging between worlds has been an ongoing theme in this WG.
Chris has done some great work on Activipy, a Python library for handling AS2, and overnight between meeting days he added support for jf2 as well by simply passing in an alternative JSON-LD context. This means you can pass in AS2 and serialize out again as jf2, and vice versa! More or less, at least... this might even be improved by adding PTD in between for where mappings aren't already obvious.
In general, tensions between specs that could be overlapping have changed to editors supporting each other to drive all of the work forward, and, I'm hoping, to optimise where redundancies exist. We all have the same goals, after all. We've still got a long way to go, but we ended on the feeling that we can probably get there.
And we played almost no SocialWG bingo, so it must have been a good meeting.
If micropub used JSON instead of (/as well as) form encoding, what might submissions look like? Aaaron Parecki brainstormed about this on our Social APIs brainstorming doc. I'm going to repeat his examples for create/update/delete here, compare them with equivalent AS2.0 json, then explore other post 'types'.
Note: I've left out @context from all AS examples because we agreed when missing, this is implied to be "https://www.w3.org/ns/activitystreams".
{
"object": {
"content": "hello world",
"category": ["indieweb"],
"published": "2015-05-15T13:06:00+02:00",
"author": "https://rhiaro.co.uk/about#me"
}
}No URL is given so it's implied that a new post is being created. Most of this is Aaron's example but I added the explicit published date and author, but these could also be added automatically by the micropub endpoint as the current time and the authenticated user who is sending the request.
ISSUE: actor vs author may never get agreed upon. I think actor is too abstract and author is too specific. I'm a fan of agent to mean 'whatever caused this to happen' (be it human, group, process or bot) but I know that's not an easy sell either. I think one side is going to have to suck it up, and I don't know who this is going to be. AS also has attributedTo which may be a reasonable compromise but seems a bit clunky.
Response per micropub spec: 201 Created and the URL of the post in the Location header.
An AS2.0 Activity for creating a new post would look like:
{
"@type": "Post",
"published": "2015-05-15T13:06:00+02:00",
"actor": "https://rhiaro.co.uk/about#me",
"object": {
"content": "hello world",
"category": ["indieweb"]
}
}Response per ActivityPump spec: "This has the side effect that of the object that they are posting is being created." - either I missed something more specific in the spec or this remains to be clarified (I'm not criticising, AP is very much under-development!).
I prefer the first one. Key differences:
type on the activity (whereas Post or Create is implied in the micropub example).published and actor) are attached to the activity, not the object.There has been a bit of discussion about whether things like audience targeting (to) would be better off attached to the object rather than the activity; I'm inclined to think there's no metadata related to the creating of an object that can't/shouldn't be attached to the object itself, rendering the activity redundant. I keep asking people of examples that contradict this, but so far nobody has given me one.
Aaron's micropub example:
{
"url": "http://example.com/post/1",
"object": {
"content": "hello moon"
}
}ActivityStreams:
{
"@type": "Update",
"object": "http://example.org/post/1"
}But the latter doesn't contain the updates made to the object... I might be missing something, but there are no specifics in the ActivityPump spec about this; presumably the user updates the object directly and the Update activity is generated from that, which is a curiously object-centric happening in an activity-centric world. Neither contain the updated date; presumably for micropub the endpoint is expected to fill this gap automatically, and possibly in the AS example the activity is meant to have a published property which could also automatically be filled in by the server. I'd like to be explicit about the updated date, though I don't know if this is a must or a should or a may. An alternative I like is:
{
"object": {
"@id": "http://example.org/post/1",
"updated": "2015-05-23T15:00:00+02:00",
"content": "hello moon"
}
}id is on the object as in AS, rather than outside it as in micropub, so what you're sending is just the object.type) because an object with an id was supplied.content is submitted, which will replace what exists.updated date is explicit.ISSUE: I feel like persuading microformats afficondos to use @id instead of url might be a toughie.
Aaron's example:
{
"url": "http://example.com/post/1",
"action": "delete"
}ActivityStreams:
{
"@type": "Delete",
"object": "http://example.org/post/1"
}They look pleasantly similar :) ActivityPump specifies that "this must leave a shell of the object that will be displayed in activities which reference the deleted object. If the deleted object is requested the server should respond with the HTTP 410 Gone status code."
The empty 'shell' with the deleted date is useful if you want to preserve when you deleted things. Discussed this in #indiewebcamp IRC a bit, and tantek suggested that emptying a post of all metadata but the updated property would suffice for implying a delete. An alternative to the examples above could be:
{
"object": {
"@id": "http://example.org/post/1",
"updated": "2015-05-23T15:00:00+:02:00",
"...": "...."
}
}where "...": "..." means... well, we need to define update behaviour before we can define delete-via-update behaviour. If omitting properties during an update removes them, then we don't need to send empty values for a delete. But if partial updates work, and the properties you send are changed and all others remain the same (which I think is better) then we need to send empty values for everything to delete them. This also allows the possibility of replacing content with say "This post has been deleted" or "this post was removed due to its offensive nature" or "I spelt literally every word wrong and it seemed easier just to forget about it" and so on, which could be useful for smarter display of deleted posts (ie. as normal posts that have been updated to say they've been deleted). I'm sure someone will have something to say about the semantics of delete vs update; if so, please back it up with practical examples/use cases :)
We also need to consider that someone deleting a post may not want to leave a trace, even if just an updated date and simply return a 410 if someone tries to access. This should be allowed, with the caveat that it's not possible to advise other servers that may have consumed the post that it's gone if they don't actively try to retrieve it and see the 410.
Or posting a like. Depending on whether you have object- or activity-tinted glasses. New grounds, so I'll start with what exists in ActivityStreams:
{
"@type": "Like",
"published": "2015-05-15T13:06:00+02:00",
"actor": "https://rhiaro.co.uk/about#me",
"object": "https://theperplexingpariah.co.uk/2015/05/a-post",
}Remember in microformats we have the like-of property which would have the same value as object does above, this time, and the 'like' is a first-class post object in itself. So submitting one via micropub could look like:
{
"object": {
"published": "2015-05-15T13:06:00+02:00",
"author": "https://rhiaro.co.uk/about#me"
"like-of": "https://theperplexingpariah.co.uk/2015/05/a-post"
}
}These are preeetttty different with regards to how terms are being used. A more AS-friendly version where the like is still treated as a first-class object would be:
{
"@type": "Like",
"published": "2015-05-15T13:06:00+02:00",
"actor": "https://rhiaro.co.uk/about#me",
"object": "https://theperplexingpariah.co.uk/2015/05/a-post",
"result": {
"published": "2015-05-15T13:06:00+02:00",
"author": "https://rhiaro.co.uk/about#me"
}
}Even though a like post is implicitly created through the result you have to go via the activity that generated it to find out what it's a like of so it doesn't really stand on its own. In the same way as audience targeting and other metadata is useful attached to the object I think having the like relation directly between the objects is practical. For semantic pedants, one post is a like-of another post, so it's closer to having a like type than being a like verb. But it's an implicit type, rather than an explicit one. Just to reiterate: I'm not saying 'post A likes post B'. That would be silly. 'Post A like-of post B' pretty easily implies the author of post A likes post B. It's not worth trying to overthink that, and you don't need RDF inferencing to make sense of it.
In summary, even for things other than creating posts, I think just sending an object works. You can attach other things like "content": "I like Jessica's post" too if you wanted, for nicer display.
The closest example in ActivityStreams is an Arrive activity:
{
"@context": "https://www.w3.org/ns/activitystreams",
"@type": "Arrive",
"actor": {
"@type": "Person",
"displayName": "Sally"
},
"location": {
"@type": "Place",
"displayName": "Work"
},
"origin": {
"@type": "Place",
"displayName": "Home"
}
}I have never needed origin with a checkin so I'm going to ignore that for now. I think I want to send to my micropub endpoint something more like:
{
"object": {
"published": "2015-05-23T13:00:00+02:00",
"location": "https://rhiaro.co.uk/home",
"content": "I got home"
}
}or
{
"object": {
"published": "2015-05-23T13:00:00+02:00",
"location": "http://dbpedia.org/resource/Edinburgh",
"content": "Finally back in Auld Reekie!"
}
}These are just like regular posts, but with a location property. Which is magically compatible with ActivityStremas and Microformats!
Where location is always a URI, so that display name, latitude, longitude, etc, can be attached to that and don't need to be included in the checkin. I'm a big fan of not nesting. If you submit a checkin with a nested venue object, the endpoint has to try to create the venue before it can do the checkin, which means it needs to attempt to de-dup venues as part of the checkin process; and if the creation fails then there's more hassle... there's probably a smart way of designing the UI plus use of javascript that can make this pretty smooth, but it's definitely not something I want to have to deal with.
In AS we have Accept which is supposed to have an object of Invite which in turn has an object of Event (whether these are all nested in the Accept activity or just referenced by URI doesn't really matter, the structure is the same. I included the nesting here cos I just copied the example directly from the AS vocabulary spec):
{
"@context": "https://www.w3.org/ns/activitystreams",
"@type": "Accept",
"actor": {
"@type": "Person",
"displayName": "Sally"
},
"object": {
"@type": "Invite",
"actor": "http://john.example.org",
"object": {
"@type": "Event",
"displayName": "A Party!"
}
}
}While it's all semantically-pedantically very lovely, I reckon the microformats way of just having an RSVP be a regular post with a reply-to another regular post which happens to have a location, start and end (thus making it an event) is easier to follow and generally simpler (and effective as evidenced by all the people posting and RSVPing to indie events). In JSON, I think that'd look like:
{
"object": {
"published: "2015-05-23T13:00:00+02:00",
"reply-to": "http://indiewebcamp.com/2015/Edinburgh",
"rsvp": "Yes",
"content": "Really looking forward to IWC Edinburgh!"
}
}Microformats does have invitations too though - what's the fun of RSVPing to an event if you can't invite your friends! Invite posts can be part of an event post, part of an RSVP post, or a post by themselves. That's because all we're really doing to create them is adding invitee properties for everyone you want to invite, and then sending them webmentions. If your invitation is not also the original event, you can send the original event post a webmention too (since you've mentioned it, after all) which could allow it to update its own list of who has been invited. Your invitee should reply with their RSVP both to your invitation and the event (which they can do with one post). Brainstorming about this, which is by no means sent in stone, on IWC wiki.
Here is an event with a list of invitees:
{
"object": {
"published: "2015-05-23T13:00:00+02:00",
"name": "IndieWebCamp Edinburgh",
"start": "2015-07-25T10:00:00+01:00"
"end": "2015-07-26T18:00:00+01:00",
"location": "http://dbpedia.org/resource/Edinburgh",
"content": "Bring your friends to IWC Edinburgh!",
"invitee": ["http://tantek.com", "http://theperplexingpariah.co.uk", "http://aaronpk.com"]
}
}Neither ActivityStreams nor Micorformats have things to do with eating and drinking, but this is something I log on my site. My micropub endpoint currently recognises Aaron's p3k-food property so I'll use that here, but I'd prefer it to be something more generic..
{
"object": {
"published": "2015-05-23T17:00:00+02:00",
"p3k-food": "A burrito",
"category": ["mexican","burrito","yum"]
}
}A potentially useful fall back for something that wants to display this post but doesn't understand p3k-food could just to be to treat it as content.
Yes, I do desire to tag my food posts.
The value of p3k-food could also be a handy URI for the food, if one existed, which when dereferenced could give you things like different language display names, calorie count, cuisine affiliations, ingredients, etc.
My micropub endpoint will accept JSON and expect an object with a bunch of properties. From these properties my server can infer how to treat them with regards to who to send webmentions to (ie. who to explicitly notify that this object was created; if I was implementing ActivityPump I'd send the whole object to these people's inboxes) and how to display them. It dosen't need explicit types, or properties spread over activities and objects to do this.
In anther post I'll run through this for (un)follows, adding to and removing from collections, exercise posts, music listens, code commits, travel plans, offers and requests, and see if they hold up the same.
If you have examples of online social stuff you do that you don't think you could publish and propagate this way, please tell me all about it. I really want to know.
Found a micropub bug, turned my back for 5 minutes and they'd taken the coffee away and started the next conference session. Indieweb problems.
Finally you can get my blog post content in a number of different formats. You'll notice alternative URLs (noted currently as 'Permalink') in the metadata section of a normal view of a post. Hit this up with no Accept header, and you'll get neat markdown (which is what I authored it in; the 'source', as it were).
Browsers automatically send Accept: text/html, in which case they are redirected to the rendered HTML version of the post you're probably reading now. But here are some alternatives you can try:
curl https://rhiaro.co.uk/llog/micropub-test-> plain markdown.
curl -H "Accept: application/json" https://rhiaro.co.uk/llog/micropub-testbecause there's obviously not enough JSON in the world. Note this isn't correct JSON-LD yet. I'll sort that out another time.
curl -H "Accept: text/turtle" https://rhiaro.co.uk/llog/micropub-testOhemgee! RDF! It's what you've all been waiting for, I know.
For those of you who like angular brackets, you could try:
curl -H "Accept: rdf/xml" https://rhiaro.co.uk/llog/micropub-testBriefly discussed with Tantek, Aaron and Bret at IWC Cambridge and on IRC about how a micropub client could fetch post content for editing. Many people aren't editing raw HTML as it's presented on the page, but maybe markdown or some other syntax, so a client needs to be able to discover this (the 'source') to present it to be edited. Now if someone makes their client check a post for a rel="source" or shoots off a request with Accept: text/plain (less likely, since static sites can't do conneg) then they'll get my markdown directly (uh, when I actually put rel="source" in my HTML, which I haven't yet).
As a related aside, I also return source="markdown" if a micropub client asks my endpoint q=source.
"Micropub is an open API standard that is used to create posts on one's own domain using third-party clients. Web apps and native apps (e.g. iPhone, Android) can use Micropub to post short notes, photos, events or other posts to your own site, similar to a Twitter client posting to Twitter.com." (Micropub on Indiewebcamp)
This means I can now publish blog posts to my site using other peoples' posting clients, of which there are several. Previously, I was ftping markdown files to my server, then running a slightly unreliable PHP script to process the queue based on when the files were last updated. Which mostly worked, but when it didn't it was a pain to untangle.
I wasn't expecting to have a micropub endpoint any time soon because it needs you to be able to sign into your site with IndieAuth. Fortunately, thanks to amazing work of aaronpk and others, most of the legwork has been done here, and you can delegate your IndieAuth; in your site <head>:
<link rel="authorization_endpoint" href="https://indieauth.com/auth" />
<link rel="token_endpoint" href="https://tokens.indieauth.com/token" />Then create a micropub script (eg. micropub.php) and make it discoverable in the same way:
<link rel="micropub" href="https://rhiaro.co.uk/micropub.php" />Initially I wanted my endpoint at https://rhiaro.co.uk/micropub and set an .htaccess rule to redirect to the PHP script. This doesn't work; POST requests can't follow redirects.
Next, here's a bare minimum PHP script to store things posted to your site with a micropub client. This just dumps the POST request to a text file, which is where I started to see if it would work. What I actually do now is turn posts and metadata into triples and insert them into my triplestore, but you probably don't want to hear about that :)
<?
// Tell client where I can syndicate to
if(isset($_GET['q']) && $_GET['q'] == "syndicate-to"){
header('Content-type: application/x-www-form-urlencoded');
echo "syndicate-to[]=twitter.com%2Frhiaro";
exit;
}
// Check for post
if(!empty($_POST)){
$headers = apache_request_headers();
// Check token is valid
$token = $headers['Authorization'];
$ch = curl_init("https://tokens.indieauth.com/token");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, Array(
"Content-Type: application/x-www-form-urlencoded"
,"Authorization: $token"
));
$response = Array();
parse_str(curl_exec($ch), $response);
curl_close($ch);
// Check for scope=post
// Check for me=https://rhiaro.co.uk
$me = $response['me'];
$iss = $response['issued_by'];
$client = $response['client_id'];
$scope = $response['scope'];
if(empty($response)){
header("HTTP/1.1 401 Unauthorized");
exit;
}elseif($me != "https://rhiaro.co.uk" || $scope != "post"){
header("HTTP/1.1 403 Forbidden");
exit;
// Check that something was posted
}elseif(empty($_POST['content']){
header("HTTP/1.1 400 Bad Request");
echo "Missing content";
}else{
// DO YOUR THING HERE
// ie. insert post content and metadata into your store, write it to a file, whatever you do to add it to your site.
// For demonstration purposes, let's dump the POST request into a file and return the URL of the file.
$fn = "posts/".time().".txt";
$h = fopen($fn, 'w');
foreach($_POST as $k => $v){
$data .= "[$k] => $v<br/>";
}
fwrite($h, $data);
fclose($h);
// Set headers, return location
header("HTTP/1.1 201 Created");
header("Location: ".$fn);
}
}
?>I got this far by working through the instructions as part of setting up Quill, so that's a good place to start!