Hacking Django Channels for Fun and Profit

Django is growing some great async features and Channels has been great for handling websockets connecting from your visitor's browser. But what happens when you need to do more? What if you want to keep long-running connections from Django to other websockets, such as Discord servers? How do you do this and still leverage all the batteries included with Django? We will show an approach that makes this all possible and easy as a developer.

Transcript

1. HACKING DJANGO CHANNELS FOR FUN AND PROFIT DJANGOCON EUROPE, JUNE 2021 CALVIN HENDRYX-PARKER
2. BACKSTORY » The web was mostly synchronous up until a few years ago » Async is becoming more a part of Django with each 3.x release
3. REQUESTS AND RESPONSES CommonMiddleware SessionMiddleware CsrfViewMiddleware AuthenticationMiddleware MessageMiddleware HttpRequest HttpResponse process_request process_view process_template_response process_response process_exception view function
4. Django Request Response Loop See https://www.youtube.com/watch?v=RLo9RJhbOrQ
5. WSGI VS ASGI LET’S MAKE IT WORK BI-DIRECTIONAL! Ok… so just swap wsgi.py with asgi.py? See See more on ASGI here: https://arunrocks.com/a-guide-to-asgi-in-django-30-and-its- performance/ https://youtu.be/uRcnaI8Hnzg
6. YOU NEED TO KNOW THAT A WEBSOCKET IS… » Let’s see it in action!
7. ENTER CHANNELS » Consumers 👈 What we think when we talk about Channels and WebSockets » Channel Layers 👈 How we talk to and between our code » Background Workers 👈 Allows for background tasks like Celery
8. CONSUMER EXAMPLE from channels.generic.websocket import JsonWebsocketConsumer class MyConsumer(JsonWebsocketConsumer): def connect(self): self.accept() self.send_json({"connected": "true"}) def disconnect(self, close_code): pass def receive_json(self, content, **kwargs)): type_code = content["type"] data = content["message"] if type_code == "greeting": if (name := data.get("name")): self.send_json({ "type": "reply_to_greeting", "message": f"hello there, {name}!" })
10. ASGI ROUTING application = SentryAsgiMiddleware( ProtocolTypeRouter( { # (http->django views is added by default) "websocket": AuthMiddlewareStack( URLRouter(loudswarm.chat.routing.websocket_urlpatterns) ), "channel": ChannelNameRouter( { "discord_send": DiscordSender, } ), }, ) )
11. LET’S TALK ABOUT BACKGROUND WORKERS THAT LAST BIT IS WHERE WE WILL FOCUS
12. WHAT IF WE WANT THE REVERSE?
13. A WORKER IN CHANNELS? What can you do with it?
14. LISTEN FOR MESSAGES ON THE CHANNEL LAYER AND THEN DO SOME WORK. » Fast » Easy » Lightweight Beware: at-most-once operation
15. REAL WORLD USE CASE DISCORD CHAT BOT BUILT INSIDE OF DJANGO
16. LET’S HACK OUR OWN WORKER We want something that Channels can do, but doesn’t out of the box
17. We will do like the Channels runworker and make our own from the asgiref.server.StatelessServer # Only the one method is different from the default worker async def handle(self): """ Listens on all the provided channels and handles the messages. """ # For each channel, launch its own listening coroutine lackeys = [] for channel in self.channels: lackeys.append(asyncio.ensure_future(self.listener(channel))) # Add coroutine for outgoing websocket connection to Discord API lackeys.append(asyncio.ensure_future(self.outgoing_connection())) # Most of our time is spent here, waiting until all the lackeys exit await asyncio.wait(lackeys) # See if any of the listeners had an error (e.g. channel layer error) [lackey.result() for lackey in lackeys]
18. WHY?
19. TWO EXAMPLES We receive messages from Discord and we send them to our clients class ChatClient(discord.Client): async def on_message(self, message): data = { "channel": str(message.channel.id), "event_ts": str(message.created_at.timestamp()), "text": message.content, "ts": str(message.created_at.timestamp()), "user": message.author.nick, "event_id": str(message.id), "event_time": int(message.created_at.timestamp()), "team_id": str(message.guild.id), } await make_webhook_transaction(data)
20. We generate a notification on a scheduled celery task and we want to send it to Discord so we write a consumer that listens for our message from discordworker import discord_client class DiscordSender(AsyncConsumer): async def send_to_discord(self, message): text_msg = message["message"] chat_chan = int(message["discord_channel"]) channel = discord_client.client.get_channel(chat_chan) await channel.send(text_msg) # this is Discord's "channel", not ours
21. and we send it like this await channel_layer.send( "discord_send", # name from ChannelNameRouter config dict( type="send_to_discord", # maps to method name on consumer discord_channel=self.chat_chan, message=message, ), )
22. Use this for any old long running task…
23. WISH LIST » Get this added to Channels codebase » Show examples of long running and single shot coroutines » Add options for two classifications of coroutines » Ones that start right away » Optionally ones the run after those stop
24. TIPS AND TRICKS (AKA NOT IN THE DOCS) » Channel Layer Capacity defaults to 100 If you push more to a channel group, they drop silently
25. SO LONG AND THANKS FOR ALL THE FISH…