Discord Binding

Source Code -> dentropy/discord-binding

Roadmap -> Backlog - Discord Binding

Why am I working on the the discord binding project?

TODO

Who is messaging who
Who are the admins
High Value Memer
Anime pfp
ML Experts
- Anime pfp
- Type in all in lowercase
- Venting about trivial shit
- esoteric papers

Description

Take data exported from discord guilds via DiscordChatExporter then transform it into a Postgres database, with separate tables for guilds, channels, messages, authors, reactions, attachments, roles, and mentions in order to make analysis of data easier. I am going to develop tooling to generate ad curate context for Discord Guilds.

Motivation

I personally find it difficult to navigate all discord guilds given how much notice there is. I am just thrown in channels.

If I am going to look through the archive of a Discord Guild I want to know who can be my guide, there is a query for that. If I am searching for an answer for a specific topics within a discord guild I want powerful search with embeddings in a vector database, named entity recognition, and semantic search.

Project Management Links

ETL Diagram

Discord ETL

ETL Steps

**Discord to Filesystem or S3 using DiscordChatExporter
- Make a discord account and join a couple guild
  - Where to find Discord Guilds to Join?
- Get Discord Authentication Token
  - How to get Discord User Token?
- Scrape a single Discord Guild
  - How to scrape a Single Discord Guild?
- Scrape all data on a single Discord Account
  - How to scrape a data from a Single Discord Account using DiscordChatExporter?
- Copy files from Guild to S3 using RClone
  - What do I do about those RClone Errors from copying Discord Data to S3?
S3 or File system to Postgres or Neo4J
Export Postgres to desired Formats

Application Dependency Diagram

Discord Binding Data Flow Diagram.png