How I've implemented webmentions for twtxt

26-04-2024

I have implemented a variant of webmention as part of my timeline single user twtxt/yarn pod. It works, but for now it only works for me talking to myself, which is not that fun or useful.

tl;dr

  1. Declare webmentions endpoint direcly in the twtxt.txt (# webmention = https://darch.dk/timeline/webmention)
  2. Use timestamp (2021-01-19T18:20:11Z) as reference, and not the twt-hash (#abcd123)
  3. Use the "Text fragments" syntax to reference the twt post (http://example.com/twtxt.txt#:~:text=2024-03-04T17:11:38Z)
  4. Notify the target user about new mentions in a private feed (twt-mentions.txt)

Vision: twtxt for (all?) your social media needs - Social media in the way that each person or persona have a feed, or a set of feeds, that others can follow to see what they are up to.

Context: (TODO: About twtxt and plain text somewhat separate from "the web" of html etc.)

The issue

The issue with webmentions is that it depends on a canonical webpage to verify the validity of the mention, but not all twtxt feeds have such a webpage (e.g. https://lyse.isobeef.org/twtxt.txt). And in the indieweb world they like to parse HTML which adds more complexity to the implementation compare to, if we only use txt files. Keeping it simple and plain text based in the backend makes sure we can also support other protocols like Gemini and not just HTTP.

Should we support the existing Webmention specification or make a specific mentions format for twtxt.txt?

I'm in favor of a specific twt mention to keep the base layer plain text and letting the clients deal with the UI stuff.

Exposing and detecting the endpoint

For webmentions to work, you need to know where to send them, i.e. we need some kind of "inbox", that has an address.

twtxt.txt canonical way of doing this is to relay(?) on server logs and the user-agents left herein (TODO: REF). These logs can be fairly noisy and not all user will have access to them.

According to the webmentions specs (TODO: REF), the endpoint declare like this in a HTML document:

<link rel="webmention" href="https://darch.dk/timeline/webmention" />

There are a couple of reason why I don't think we should to this for twtxt.txt.

To keep thing simple I've opted for declaring the webmention endpoint as a field to the twtxt.txt file using this syntax:

# webmention = https://darch.dk/timeline/webmention

but then your client also need to be able to parse html documents, which adds complexity, All you should need for twtxt to work is just a twxt.txt file, so let's not complicate or fragment our implementation more than needed.

Another reason for this decision is to avoid confusion between how to confirm

Sending a webmention from a client

When a client detects a mention (@<nick url>) in a new post, it sends this as a webmention to the url if it can detect a valid endpoint.

The mention should include the twt timestamp (2021-01-19T18:20:11Z), since this is more universal than the twt hash (#abcd123). This makes it easier to implement both endpoints and clients since you don't need cryptography to make it work. You can still send a webmention all by hand, if needed.

The POST Request

This is how a normal webmention request looks (from: https://www.w3.org/TR/webmention/#sender-notifies-receiver)

POST /webmention-endpoint HTTP/1.1
Host: aaronpk.example
Content-Type: application/x-www-form-urlencoded

source=https://waterpigs.example/post-by-barnaby&
target=https://aaronpk.example/post-by-aaron

The Text fragments, where you can refer to a string on a page or in a document, seems like the best option for referencing a specific line in a plain text file, so that is what I've used. (Text fragments | MDN, Boldly link where no one has linked before: Text Fragments, URL Fragment Text Directives)

Using the Text fragments syntax pointing to a specific twt post would look like this:

http://example.com/twtxt.txt#:~:text=2024-03-04T17:11:38Z

It works for plain text files out of the box in Safari, but not in Chrome or Firefox, which stills needs a plugin (Firefox Text Fragment Extension)

So using curl, sending a webmention request would look like this:

curl -i -d "source=http://algorave.dk/twtxt.txt#:~:text=2024-03-04T17:11:38Z&target=http://darch.dk/twtxt.txt#:~:text=2024-03-05T20:33:55Z" https://darch.dk/timeline/mention

Receiving a webmention (Server)

  • The endpoints receives the webmention and responds with 200 OK, if it can find a twt in the sender's twtxt.txt with the included timestamp.

  • Then the endpoint needs to inform the user of the received webmentions in some way.

    1. Make a special private feed like a mentions.txt, where mentions are written to.
    2. Send the user an email with the mentions.
    3. Some custom UI element in the receiver's application.

I've decided to start with option 1, since that allowed be to reuse the rendering of a timeline that my applications already have. It could either include the full message as a block quote or just a link to a web-based UI with the calculated twt hash.

Prologic mentioned you:
> @<sorenpeter https://darch.dk/twtxt.txt> what do you think about AI?

That's all folk! Now go and setup some mentions feature yourself, so we can keep the conversation going.

Things to consider and discuss

  • Does it still make sense to call it WEBmentions? or should it be called something like twt-mentions or txt-mentions?
  • Should the client also send a mention when replying to a twt using (#abcd123) that does not include an explicit @mention?
  • And should it also be possible to send a mention when you start following a feed?

Tags: blog