Skip to content

Roadmap - Dentropy Daemon 0.0.1

Question Engine is my attempt at synthesizing a series of ETL projects in order to produce intelligence.

TLDR

  1. Discord Analytics Reports and Dashboard
  2. Graph Based Annotation on Top of Discord Data
  3. Allow for Generalized Questioning and add Additional Data Sources
  4. Proof of Meme Micro Bounty Platform

For my Udacity Data Engineering Capstone Project I synthesized data from Github and Reddit and matched references of the same domain name across data sets. That project did not produce any meaningful insights because I did not ask a SMART enough question. The question I want to try and answer with Question Engine is, "How do you identify the prime movers within internet communities?". In this document I outline a 4 step plan for a proof of concept that not only transforms data from internet communities into an indexed queryable format but also allows for the annotation, tagging, linking, and questioning any any segment of said data when conducting investigations into meaningful questions.

1. Discord Analytics Reports and Dashboard

To answer How do you identify the prime movers within internet communities? I must first break down the question into simpler questions. What online communities do I care about? and Where can I get the data for online communities I care about?

For the first question, What online communities do I care about? I decided on the answer DAOs. DAO's are digitally native communities that also have an economic nature to them via Blockchain Tokens. It is also easy to rank these communities via lists like this which also list the community links.

For the second question, Where can I get the data for online communities I care about?. The DAO Maketcap Marketlization list from coinmarketcap.com contains social links to each DAO. Most DAO's have a Discord Guild so I can start there. As for how I can get the data out of Discord I can simply use DiscordChatExporter.

For more info check out, Discord Binding

2. Graph Based Annotation on Top of Discord Data

  • Label where I want to exchange value
  • Highlight high agency individuals explicitly with custom human and LLM descriptions
  • Topic Modelling with results stored in Graph Database

3. Allow for Generalized Questioning and add Additional Data Sources

  • Keybase Data
  • Raindrop.io annotation
  • YouTube Playlists Annotation
  • Reddit Saved + Subreddits Annotation
  • Activity Watcher + Browsing History
  • Linkedin Connections

4. Proof of Meme Micro Bounty Platform

  • Turn the namespace of all questions, grammatical or not, into a mineable resource claimable by cryptographic identities