Rss on Hyteck

I did something naughty: Circumventing Authorized-Fetch as implemented by GoToSocial

2024-12-11T06:10:10+02:00

Yes the title is correct, but I had nothing malicious in mind!

What this is about

For @qzt@queereszentrumtuebingen.de we include the public feed in a sidbar on the homepage. Initially this was done using the standard API to fetch statuses /api/v1/accounts/{account_id}/statuses and worked like a charm. The problem started when GoToSocial (the fediverse server we use, similar to mastodon) implemented authorized fetch. This is a a good thing! Authorized fetch means, that every call to a endpoint needs to be authorized by an access_token. You get an access token from a fedi account. It’s what fediverse clients like Tusky or Phanpy do on your behalf to get the posts that make up you timeline.

Authorized fetch has major advantages as

data scraping can only be done by other fediaccounts
blocking can not be circumvented by using the public API

and much more. Sadly it also broke our website integration.

Possible Solutions

So what now? I initially wanted to turn of authorized fetch for @qzt@queereszentrumtuebingen.de by messing with the GoToSocial code and turning it off for the whole server. This would have been possible as this is the only user on the server. The GoToSocial devs helped me manage to find where to do that. But it’s not ideal and would make me build a custom docker image fore each update.

Next idea: The whole point of authorized fetch is, that only fedi-accounts (and apps they authorized) can access the API. So lets do that! Set up a new account, add app and authorize it as described in the GoToSocial documentation. I used #Bruno for that, that was much more comfortable than using curl for me. With that authorization code you can now get an access token for your app. Put that in the Javascript that loads posts and we are good right? Sadly no. It would totally work. But it would also allow anyone to read and post on behalf of the account. That calls for malicious actors using this for scraping or spamming.

So instead, we need a proxy that stores the access token securely and restricts the actions.

The proxy

Such a proxie must

offer the endpoint that provides the same data as the FediverseAPI
authorize itself to the FediverseAPI via access_token
restrict to read access of consenting accounts

The last point is really important, as we don’t want to allow others to use this endpoint to scrape data unauthorized.

I wrote a short FastAPI server that offers this. It only implements one method

@app.get("/api/v1/accounts/{account_id}/statuses")
async def fetch_data(account_id):
if account_id not in ALLOWED_ACCOUNTS:
raise HTTPException(status_code=401, detail="You can only use this proxy to access configured accounts")
headers = {"Authorization": f"Bearer {ACCESS_TOKEN}"}
response = requests.get(f"{EXTERNAL_API_BASE_URL}/api/v1/accounts/{account_id}/statuses", headers=headers)
return response.json()

Basically this is the whole API code, I only trimmed a few checks and error handling.

Deployment

To deploy, I put it in a docker container and started it via docker-compose. Reverse proxing is handled by Traefik, I won’t go into detail here.

services:
fediproxy.example.org:
image: docker.io/moanos/fediproxy
container_name: "fediproxy.example.org"
restart: unless-stopped
environment:
EXTERNAL_API_BASE_URL: ${EXTERNAL_API_BASE_URL}
ACCESS_TOKEN: ${ACCESS_TOKEN}
ALLOWED_ACCOUNTS: ${ALLOWED_ACCOUNTS}
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
- "traefik.http.routers.fediproxy.rule=Host(`fediproxy.example.org`)"
- "traefik.http.routers.fediproxy.service=fediproxy-service"
- "traefik.http.routers.fediproxy.entrypoints=web-secure"
- "traefik.http.routers.fediproxy.tls=true"
- "traefik.http.routers.fediproxy.tls.certResolver=default"
- "traefik.http.services.fediproxy-service.loadbalancer.server.port=8000"
networks:
- traefik
networks:
traefik:
name: "traefik"
external: true

I added a short .env to configure:

ACCESS_TOKEN=VERYSECRETTOKENTHATISDEFINETLYREAL
EXTERNAL_API_BASE_URL=https://gay-pirate-assassins.de
ALLOWED_ACCOUNTS=ZGGZF4G8NNOTREAL81Z8G7RTC

Results

Now I can again use something like the wordpress plugin Include Mastodon Feed just by pointing to the proxy: [include-mastodon-feed instance="fediproxy.example.org.de" account="ZGGZF4G8NNOTREAL81Z8G7RTC"]

Hope you enjoyed the read. Source code for the proxy can be found here: https://git.hyteck.de/moanos/FediProxy If you want to play around a bit you can use https://git.hyteck.de/moanos/include-fedi

Sloth logo of GTS by Anna Abramek, Creative Commons BY-SA license.

Styling an Django RSS Feed

2024-04-16T12:10:10+02:00

Introduction

RSS is amazing! While not everyone thinks that, most people that understand RSS, like it. This presents a problem, as most people don’t have chance to learn about it. Unless there is a person in the community that doesn’t shut up about how great RSS is (maybe that person is you), they might not even know what it is, let alone use it.

One big reason for this is, that when you click an link to an RSS feed you download a strange file that most people don’t know how to deal with. Maybe your browser is nice and renders some XML which is also not meant for human consumption. Wouldn’t it be better if people clicked on the RSS link and were greeted by a text explaining RSS and how to use it? And if the site would still be a valid RSS feed?

Luckily you don’t have to imagine that - it’s possible! You can even try it on this blog by clicking the RSS link in the menu (direct link).

Doing this has not been my idea. Darek Kay described this in the blog post Style your RSS feed and I just copied most of their work! This was fairly easy for this Hugo blog and is available in my fork of the hugo-nederburg-theme. However, in a Django project it get’s a bit more complicated. Let me explain.

The Problem

Django has the great Syndication feed framework, a high level framework to create RSS and Atom Feeds. This is great as we only need a few lines of code to create a feed. Here is an example from notfellchen.org that list animals that are in search for a new home. People should be able to follow the RSS feed to see new adoption notices. So lets do it

# in src/fellchennasen/feeds.py
from django.contrib.syndication.views import Feed

from .models import AdoptionNotice

class LatestAdoptionNoticesFeed(Feed):
 title = "Notfellchen"
 link = "/rss/"
 description = "Updates zu neuen Vermittlungen."

 def items(self):
 return AdoptionNotice.objects.order_by("-created_at")[:5]

 def item_title(self, item):
 return item.name

 def item_description(self, item):
 return item.description

# in src/fellchennasen/urls.py
urlpatterns = [
 path("", views.index, name="index"),
 path("rss/", LatestAdoptionNoticesFeed(), name="rss"), # <--- Added
 ...

Wait that’s it? Yeah! We have a working RSS feed. And it was very convenient, Django allows us to create by just pointing it to the right model and fields we want to display.

But here is the problem: How do we style this? We can’t just add a link to a stylesheet here.

The solution

First we need to add our styling files. I’ll not go into detail how they work her, just refer Darek’s blog post for that. In Django we add them to our static files

static/rss.xsl will be adjusted based on this file. It is responsible for creating a html rendering of your XML file
for static/css/rss-styles you can drop in this file, which is a basic CSS file you can edit to your liking.

After that comes the hard part. How do tweak this wonderfully simple Feed class to include a link to our style sheet? I first thought “that must be easy, just follow the docs on custom feed generators and add a root element. Something like this:

class FormattedFeed(Rss201rev2Feed):

 def add_root_elements(self, handler):
 super().add_root_elements(handler)
 # We want <?xml-stylesheet href="/static/rss.xsl" type="text/xsl"?>
 handler.addQuickElement("?xml-stylesheet", f'href="{static("rss.xsl")}"')

class LatestAdoptionNoticesFeed(Feed):
 feed_type = FormattedFeed
 title = "Notfellchen"
 ...

Looks good. Let’s try. Oh no what is this?

<?xml-stylesheet href="/static/rss.xsl"/>

Yes, we can’t correctly close this tag. There is (to my knowledge) no easy way to do this. So let’s take the hard road an implement a custom write function. In the following the write function will be copied from django.utils.feedgenerator.RssFeed. We make two important changes to the class:

Changing the content type from content_type = "application/rss+xml; charset=utf-8" to content_type = "text/rss+xml; charset=utf-8. This will make a browser display the content rather than opening it in a app.
Adding our xml-stylsheet information. This is done in the write() function with this line

handler._write(f'<?xml-stylesheet href="{static("rss.xsl")}" type="text/xsl"?>')

Putting it all together we still have a relativly simple solution with only the necessary adjustments. Here is the full feeds.py:

from django.contrib.syndication.views import Feed
from django.utils.feedgenerator import Rss201rev2Feed
from django.templatetags.static import static
from django.utils.xmlutils import SimplerXMLGenerator

from .models import AdoptionNotice


class FormattedFeed(Rss201rev2Feed):
 content_type = "text/xml; charset=utf-8"
 def write(self, outfile, encoding):
 handler = SimplerXMLGenerator(outfile, encoding, short_empty_elements=True)
 handler.startDocument()
 handler._write(f'<?xml-stylesheet href="{static("rss.xsl")}" type="text/xsl"?>')
 handler.startElement("rss", self.rss_attributes())
 handler.startElement("channel", self.root_attributes())
 self.add_root_elements(handler)
 self.write_items(handler)
 self.endChannelElement(handler)
 handler.endElement("rss")


class LatestAdoptionNoticesFeed(Feed):
 feed_type = FormattedFeed
 title = "Notfellchen"
 link = "/rss/"
 description = "Updates zu neuen Vermittlungen."

 def items(self):
 return AdoptionNotice.objects.order_by("-created_at")[:5]

 def item_title(self, item):
 return item.name

 def item_description(self, item):
 return item.description

And finally we have what we want! A RSS feed displayed in the browser, with beginner-friendly explanation and still completely spec-compliant.

Outlook

Now you may recognize I’m not a frontend person. The style could be prettier and provide a better overview. But I’d argue the improvement is immense and might help a user to get started with RSS.

There are still a couple things to improve:

Translation: The current text is only displayed in english
The rss.xsl file has a hard-coded link to the css stylesheet in it

<link rel="stylesheet" type="text/css" href="/static/fellchensammlung/css/rss-styles.css"/>

Both can be solved by templating the rss.xsl instead of serving it as static file.

So have fun playing around! If you have created or found a nice-looking RSS feed let me know. Let’s keep RSS alive and thriving!