32 Commits

Author SHA1 Message Date
lars-winzer 01e3138f29 Change runner type from self-hosted to ubuntu-latest for overlooked file
Extends a41a483041
2025-12-08 14:45:51 +01:00
lars-winzer a41a483041 Change runner type from self-hosted to ubuntu-latest 2025-12-08 14:41:34 +01:00
lars-winzer 917e90b48b Remove renovate.json 2025-10-09 10:54:13 +02:00
lars-winzer 467aebc3af Add bandit task to run on push 2025-10-08 16:02:41 +02:00
lars-winzer 818f4dfdae Remove renovate workflow 2025-10-08 15:51:45 +02:00
lars-winzer 0db5d4d31c Add ruff task to run on push and pull_request
Source: https://docs.astral.sh/ruff/integrations/#github-actions
2025-10-08 15:17:04 +02:00
lars-winzer 6967fa5358 Add task to run and attach to docker container 2025-10-01 15:03:10 +02:00
lars-winzer 70363033c1 Add docker-build task and update .gitignore 2025-09-28 18:26:48 +02:00
lars-winzer baf774f4b0 Fix "No repositories found" error for renovate action 2025-09-21 17:35:50 +02:00
lars-winzer 64215ffda5 Ditch docker approach and run renovate directly 2025-09-21 17:30:57 +02:00
lars-winzer 13a9075376 Modify renovate workflow to use docker-mount 2025-09-21 17:26:08 +02:00
Cromatin 05e72bf0dc feat: Integrated Renovate Bot as gh action (#9) 2025-09-21 17:08:16 +02:00
lars-winzer 654274f632 Enable renovate workflow schedule 2025-09-21 17:04:47 +02:00
lars-winzer 56d7a8155c Add required files for renovate bot workflow
As outlined in: https://github.com/renovatebot/github-action
2025-09-21 16:57:24 +02:00
Cromatin b82d6fa518 Merge pull request #8 from official-Cromatin:refractor/remove-portal-class
Refractor: Remove portal class
2025-09-21 16:38:37 +02:00
lars-winzer c4cb477637 Move portal attributes to Bot class 2025-09-21 15:23:14 +02:00
lars-winzer 2f33d840f3 Move version to independent VERSION file 2025-09-21 14:59:12 +02:00
lars-winzer f01b942818 Merge 'bug/issue-6' into 'main' 2025-09-18 15:34:39 +02:00
lars-winzer 8e1b4269d1 Fix .github folder being included in the docker image 2025-09-18 15:23:23 +02:00
Cromatin e1455de14b Merge pull request #5 from official-Cromatin:official-Cromatin/issue4
Official-Cromatin/issue4
2025-09-17 14:43:07 +02:00
lars-winzer e6195d9a88 Add required permissions 2025-09-17 14:42:09 +02:00
lars-winzer 1221035369 Explicitly use bash shell 2025-09-17 14:41:46 +02:00
lars-winzer 796f43d629 Fix docker workspace error #4 2025-09-17 14:34:12 +02:00
Cromatin 0b031d20f9 Merge pull request #3 from official-Cromatin/release-v0.5
Merge branch 'release-v0.5' into 'main'
2025-09-16 18:49:29 +02:00
lars-winzer bfd497898e Update copyright year 2025-09-16 18:30:07 +02:00
lars-winzer 077f845b94 Merge commit '42bd226a6007780debe6ee1ca6357185f409f70e' into release-v0.5 2025-09-16 18:28:05 +02:00
lars-winzer 26e3a4bb8e Update LICENSE 2025-09-16 18:13:56 +02:00
lars-winzer f56636c1ff Create workflow to build and release docker image on ghcr 2025-09-16 17:30:08 +02:00
lars-winzer f427d06e61 Create Dockerfile and .dockerignore for upcoming official docker support
- Update .gitignore
2025-09-16 16:07:07 +02:00
lars-winzer 42bd226a60 Update LICENSE 2025-09-16 16:03:13 +02:00
lars-winzer 4dd578c9a8 Refine '/post'-command error handling when file size exceeds upload limit 2025-09-16 14:13:02 +02:00
lars-winzer 01d90e6625 Move imports to the top of the document
- Remove unnecessary prints
- Remove unnecessary f-strings
2025-09-16 14:10:04 +02:00
23 changed files with 957 additions and 253 deletions
+28
View File
@@ -0,0 +1,28 @@
**/__pycache__
**/.venv
**/.classpath
**/.dockerignore
**/.env
**/.git
**/.github
**/.gitignore
**/.project
**/.settings
**/.toolstarget
**/.vs
**/.vscode
**/*.*proj.user
**/*.dbmdl
**/*.jfm
**/bin
**/charts
**/docker-compose*
**/compose*
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/obj
**/secrets.dev.yaml
**/values.dev.yaml
LICENSE
README.md
+15
View File
@@ -0,0 +1,15 @@
name: Code Analysis (Bandit)
on: push
jobs:
bandit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: mdegis/bandit-action@v1.0
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
path: "src/"
level: high
confidence: high
exit_zero: true
+44
View File
@@ -0,0 +1,44 @@
name: Release Multi-Arch Docker Image
on:
release:
types: [published]
permissions:
contents: read
packages: write
jobs:
build-and-push-docker:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Transform 'github.repository' to lowercase
id: lowercase_repo
shell: bash
run: echo "repository=${GITHUB_REPOSITORY,,}" >> $GITHUB_OUTPUT
- name: Build and push multi-arch image
uses: docker/build-push-action@v5
with:
context: .
push: true
platforms: linux/amd64,linux/arm64
tags: |
ghcr.io/${{ steps.lowercase_repo.outputs.repository }}:${{ github.event.release.tag_name }}
ghcr.io/${{ steps.lowercase_repo.outputs.repository }}:latest
+9
View File
@@ -0,0 +1,9 @@
name: Linting (Ruff)
on: [push, pull_request]
jobs:
ruff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/ruff-action@v3
+1 -2
View File
@@ -4,9 +4,8 @@ config/*.ini
pastebin.txt
.DS_Store
test*.py
.dockerignore
docker-*
.vscode
.vscode/settings.json
# Byte-compiled / optimized / DLL files
+36
View File
@@ -0,0 +1,36 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Docker: Build Image",
"type": "docker-build",
"platform": "python",
"dockerBuild": {
"context": "${workspaceFolder}",
"dockerfile": "${workspaceFolder}/Dockerfile",
"tag": "post-it:dev"
}
},
{
"label": "Run post-it (interactive, show logs)",
"type": "shell",
"dependsOn": ["Docker: Build Image"],
"command": "docker",
"args": [
"run",
"--rm",
"-it",
"--name",
"post-it-dev",
"-v",
"${workspaceFolder}/config:/app/config",
"post-it:dev"
],
"presentation": {
"reveal": "always",
"panel": "shared"
},
"problemMatcher": []
}
]
}
+22
View File
@@ -0,0 +1,22 @@
# Base image (multi-arch via buildx)
FROM python:3.10-slim
# Prevents pyc files + enables unbuffered logging
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
# Install dependencies
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy only necessary files (via .dockerignore)
COPY . .
# Create non-root user
RUN adduser --disabled-password --no-create-home --gecos "" appuser \
&& chown -R appuser /app
USER appuser
# Default command
ENTRYPOINT ["python", "src/main.py"]
+3 -1
View File
@@ -1,3 +1,5 @@
discord.py==2.4.0
asyncpraw==7.8.1
colorama==0.4.6
python-dotenv==1.1.1
aiohttp==3.11.11
pillow==11.1.0
+1
View File
@@ -0,0 +1 @@
0.5.2
-25
View File
@@ -1,25 +0,0 @@
from discord.ext import commands
import discord
from utils.datetime_tools import get_elapsed_time_small
from logging import Logger
from datetime import datetime
class PostIt_Bot(commands.Bot):
"""Class to extend the built-in bot class"""
def __init__(self, startup_time:float, program_version:str, app_logger:Logger):
intents = discord.Intents.default()
intents.messages = True
super().__init__(command_prefix=None, help_command=None, intents=intents)
self.startup_time = startup_time
self.PROGRAM_VERSION = program_version
self.__logger = app_logger
async def setup_hook(self):
# Register cogs to handle commands
for cog_name in ["post", "reload"]:
await self.load_extension(f"cogs.{cog_name}")
await self.tree.sync()
async def on_ready(self):
self.__logger.info(f"Successfully logged in (after {get_elapsed_time_small(datetime.now().timestamp() - self.startup_time)}) as {self.user}")
View File
+4 -5
View File
@@ -1,11 +1,10 @@
"""Contains the BaseCog"""
import discord
from discord import app_commands
from discord.ext import commands
import logging
class BaseCog(commands.Cog):
def __init__(self, bot:commands.Bot, logger:logging.Logger):
self._bot = bot
class Base_Cog(commands.Cog):
def __init__(self, logger:logging.Logger):
self._logger = logger
async def cog_load(self):
+45
View File
@@ -0,0 +1,45 @@
import discord
from discord import app_commands
from discord.ext import commands
from cogs.base_cog import Base_Cog
import logging
from datetime import datetime
from utils.datetime_tools import get_elapsed_time_big
from utils.logger.decorator import log_command_execution
class Debug_Command(Base_Cog):
def __init__(self, bot:commands.Bot):
self.__bot = bot
super().__init__(logging.getLogger("cmds.debug"))
@app_commands.command(name = "debug", description = "Provides debug informations about the bot, useful for troubleshooting")
@log_command_execution
async def debug(self, ctx:discord.Interaction):
embed = discord.Embed(title="Debug Information")
embed.add_field(name="Version",
value=ctx.client.VERSION,
inline=True)
embed.add_field(name="Uptime",
value=f"{get_elapsed_time_big(datetime.now().timestamp() - ctx.client.STARTUP_TIMESTAMP)}",
inline=True)
embed.add_field(name="Bot Owner",
value=f"<@{ctx.client.bot_config['DISCORD']['OWNER_ID']}>",
inline=True)
embed.add_field(name="Latency to Gateway",
value=f"{round(self.__bot.latency * 1000, 2)}ms",
inline=True)
embed.add_field(name="Application ID",
value=f"{self.__bot.application_id}",
inline=True)
embed.add_field(name="Number of Guilds",
value=f"{len(self.__bot.guilds)}",
inline=True)
embed.add_field(name = "Number of executed commands", value=f"Total: {ctx.client.no_executed_commands}\nSucceeded: {ctx.client.no_succeeded_commands}\nFailed: {ctx.client.no_failed_commands}")
await ctx.response.send_message(embed=embed)
async def setup(bot:commands.Bot):
await bot.add_cog(Debug_Command(bot))
+160 -12
View File
@@ -1,14 +1,24 @@
"""Contains the implementation for the /post command"""
from .base import BaseCog
import logging
from discord import app_commands, Interaction
import discord
from discord import app_commands
from discord.ext import commands
from cogs.base_cog import Base_Cog
class PostCommand(BaseCog):
def __init__(self, bot):
logger = logging.getLogger("cmds.post")
super().__init__(bot, logger)
import logging
from urllib.parse import urlparse
from asyncpraw.models import Submission
import aiohttp
from PIL import Image
from io import BytesIO
from datetime import datetime
from utils.datetime_tools import get_elapsed_time_milliseconds
class NoMediaFound(Exception):
pass
class Post_Command(Base_Cog):
def __init__(self, bot:commands.Bot):
self.__bot = bot
super().__init__(logging.getLogger("cmds.post"))
@app_commands.command(name = "post", description = "Post an embed in the Current Channel with a link to the content")
@app_commands.describe(url = "URL to the post", custom_note = "Describe the post with your own note", use_title = "Display the title of the post", quality = "Specifies the quality of the converted image, closer to 100 is better")
@@ -21,8 +31,146 @@ class PostCommand(BaseCog):
app_commands.Choice(name = "Superior (95)", value = 95),
app_commands.Choice(name = "Perfect (100)", value = 100)
])
async def post(self, ctx:Interaction, url:str, custom_note:str = None, use_title:bool = True, quality:app_commands.Choice[int] = 95):
print("Hallo")
async def post(self, ctx:discord.Interaction, url:str, custom_note:str = None, use_title:bool = True, quality:app_commands.Choice[int] = 95):
try:
domain_info = urlparse(url)
toplevel_domain = '.'.join(domain_info.netloc.split('.')[-2:])
begin_process = datetime.now().timestamp()
match toplevel_domain:
case "reddit.com":
self._logger.debug(f"Recieved command by {ctx.user} ({ctx.user.id}) for reddit ({url})")
subm:Submission = await ctx.client.reddit_adapter.fetch(url)
image_urls = []
# Check if submission has a gallery
if hasattr(subm, "media_metadata"):
for media_id, media in subm.media_metadata.items():
file_extension = media["m"].split("/")[1]
if file_extension not in ("jpg", "jpeg", "png", "webp", "heic", "heif"):
continue
image_urls.append(f"https://i.redd.it/{media_id}.{file_extension}")
else:
image_urls.append(subm.url)
image_count = len(image_urls)
if image_count == 0:
raise NoMediaFound
self._logger.debug(f"Found {image_count} image urls for the post")
progress_title = f"`{image_count}` images are going to be converted, it may take a while."
progress_temp = progress_title + f"\n`0` of `{image_count}` have already been loaded"
await ctx.response.send_message(progress_temp, ephemeral = True)
if isinstance(quality, app_commands.Choice):
quality_value = quality.value
else:
quality_value = quality
# Download and convert each image
image_files:list[discord.File] = []
begin_conversion = datetime.now().timestamp()
async with aiohttp.ClientSession() as session:
index = 0
for image_url in image_urls:
async with session.get(image_url) as response:
response.raise_for_status()
image_data = await response.read()
original_image = Image.open(BytesIO(image_data))
webp_buffer = BytesIO()
original_image.save(webp_buffer, format = "WEBP", quality = quality_value)
webp_buffer.seek(0)
image_file = discord.File(webp_buffer, filename = f"image_{index}.webp")
image_files.append(image_file)
index += 1
progress_temp = progress_title + f"\n`{index}` of `{image_count}` have already been loaded"
await ctx.edit_original_response(content = progress_temp)
self._logger.debug(f"Downloaded and converted {len(image_files)} images in {get_elapsed_time_milliseconds(datetime.now().timestamp() - begin_conversion)}")
author = subm.author.name if subm.author else "Author not found"
content = f":copyright: [{author}]({url})"
if use_title:
content += f"\n# {subm.title}"
if custom_note:
content += f"\n> {custom_note}"
await ctx.delete_original_response()
message = await ctx.followup.send(
content = content,
suppress_embeds = True,
files = image_files
)
self._logger.info(f"Successfully processed the command executed by {ctx.user.name} ({ctx.user.id}) after {get_elapsed_time_milliseconds(datetime.now().timestamp() - begin_process)} (ID of message: {message.id})")
# No domain for seperation found
case _:
if toplevel_domain == "":
toplevel_domain = "not_found"
embed = discord.Embed(
title = "Domain not found",
description = f"The requested domain `{toplevel_domain}` is currently not supported\nOpen [an issue](https://github.com/official-Cromatin/Post-It/issues/new?assignees=&labels=feature-request&projects=&template=feature_request.yml) to request support for it.\n\nCurrently supported plattforms:\n- Reddit",
color = 0xED4337)
await ctx.response.send_message(embed = embed, ephemeral = True)
except NoMediaFound:
self._logger.error(f"Aborted issued command by {ctx.user.name} ({ctx.user.id}). Post had no media attatched")
# Delete the original response, if existing
try:
await ctx.delete_original_response()
except discord.NotFound:
pass
embed = discord.Embed(
title = "No media found",
description = "The post had no media attatched, or was in an unsupported format\nVideos are not supported! (yet)",
color = 0xED4337
)
embed.set_footer(text = "Supported image formats: jpg, jpeg, png, webp, heic, heif")
except discord.errors.HTTPException as error:
self._logger.error(f"Could not complete command by {ctx.user.name} ({ctx.user.id})")
self._logger.exception(error, stack_info = True)
match error.code:
case 40005:
self._logger.error("Failed to upload images, payload too large")
embed = discord.Embed(
title = "Error while processing",
description = "The attachments exceed the upload limit of Discord,\nchoose a different quality level via the argument `quality`",
color = 0xED4337
)
if ctx.response.is_done():
await ctx.followup.send(embed = embed, ephemeral = True)
else:
await ctx.response.send_message(embed = embed)
except Exception as error:
self._logger.error(f"Could not complete command by {ctx.user.name} ({ctx.user.id})")
self._logger.exception(error, stack_info = True)
# Delete the original response, if existing
try:
await ctx.delete_original_response()
except discord.NotFound:
pass
# Explain to the user what the error was
embed = discord.Embed(
title = "Error while processing",
description = f"While we processed your request, the following exception occured: `{error}`",
color = 0xED4337
)
if ctx.response.is_done():
await ctx.followup.send(embed = embed, ephemeral = True)
else:
await ctx.response.send_message(embed = embed)
async def setup(bot:commands.Bot):
await bot.add_cog(PostCommand(bot))
await bot.add_cog(Post_Command(bot))
-113
View File
@@ -1,113 +0,0 @@
import discord
from discord import app_commands
from discord.ext import commands
from .base import BaseCog
import logging
from datetime import datetime
from utils.datetime_tools import get_elapsed_time_milliseconds
class ReloadCommand(BaseCog):
def __init__(self, bot:commands.Bot):
self.__bot = bot
self.__cached_cog_names:list[str] = None
self.__reload_running = False
super().__init__(bot, logging.getLogger("cmds.reload"))
async def autocomplete_cog(self, ctx: discord.Interaction, cog_name_stw:str):
"""Helpermethod to recommend the right cog name to the user"""
# Cache the cog names if not cached yet
if not self.__cached_cog_names:
self.__cached_cog_names = list(self.__bot.extensions.keys())
if cog_name_stw == "":
# Return first 25 cog names, if provided name is empty
matching_cogs = sorted(self.__cached_cog_names)
else:
matching_cogs = []
for cog_name in self.__cached_cog_names:
if cog_name.startswith(cog_name_stw.lower()):
matching_cogs.append(cog_name)
choices = []
for matching_cog in matching_cogs[:25]:
choices.append(app_commands.Choice(name = matching_cog.removeprefix("cogs.").capitalize(), value = matching_cog))
return choices
@app_commands.command(name = "reload_all", description = "Reloads all cogs")
async def reload_all(self, ctx: discord.Interaction):
if self.__reload_running:
await ctx.response.send_message("Reload is allready being executed", ephemeral = True)
else:
self.__reload_running = True
task_start = datetime.now().timestamp()
cog_names = list(self.__bot.extensions.keys())
stepped_over = 0
self._logger.debug(f"Reloading all {len(cog_names)} cogs / extensions ...")
try:
embed_cog_stats = "Reloaded the following cogs:\n"
for extension_name in cog_names:
if "impls" in extension_name:
stepped_over += 1
else:
start_reload = datetime.now().timestamp()
await self.__bot.reload_extension(extension_name)
embed_cog_stats += f"- {extension_name.removeprefix('cogs.').capitalize()} (`{get_elapsed_time_milliseconds(datetime.now().timestamp() - start_reload)}`)\n"
except Exception as error:
embed_cog_stats += f"- {extension_name.removeprefix('cogs.').capitalize()} <--"
embed = discord.Embed(
title = "Reloading All Cogs",
description = f"{embed_cog_stats} \n\nDuring the reload an exception occured",
color = 0xED4337)
raise error
else:
elapsed_time = get_elapsed_time_milliseconds(datetime.now().timestamp() - task_start)
embed = discord.Embed(
title = "Reloading All Cogs",
description = f"{embed_cog_stats}Total time spend: `{elapsed_time}`",
color = 0x4BB543)
self._logger.info(f"Cogs successfully reloaded after {elapsed_time} (Ignored )")
finally:
await ctx.response.send_message(embed = embed, ephemeral = True)
self.__reload_running = False
@app_commands.command(name = "reload", description = "Reload specific cog")
@app_commands.autocomplete(cog_name = autocomplete_cog)
async def reload(self, ctx: discord.Interaction, cog_name:str):
if self.__reload_running:
await ctx.response.send_message("Reload is allready being executed", ephemeral = True)
else:
self.__reload_running = True
task_start = datetime.now().timestamp()
self._logger.debug(f"Reloading {cog_name} cog ...")
try:
await self.__bot.reload_extension(cog_name)
except Exception as error:
embed = discord.Embed(
title = f"Reloading `{cog_name.removeprefix('cogs.').capitalize()}` cog",
description = "Reload failed with an exception",
color = 0xED4337)
raise error
else:
embed = discord.Embed(
title = f"Reloading `{cog_name.removeprefix('cogs.').capitalize()}` cog",
description = f"Total time spend: `{get_elapsed_time_milliseconds(datetime.now().timestamp() - task_start)}`",
color = 0x4BB543)
self._logger.info(f"Cog {cog_name} successfully reloaded after {get_elapsed_time_milliseconds(datetime.now().timestamp() - task_start)}")
finally:
await ctx.response.send_message(embed = embed, ephemeral = True)
self.__reload_running = False
# Clear the cache
self.__cached_cog_names = None
async def setup(bot:commands.Bot):
await bot.add_cog(ReloadCommand(bot))
+8
View File
@@ -0,0 +1,8 @@
from pathlib import Path
# Read VERSION file
with open(Path(__file__).resolve().parent / "VERSION") as f:
VERSION = f.read().strip()
__all__ = VERSION
+117 -94
View File
@@ -1,115 +1,138 @@
import asyncio
from datetime import datetime
from utils.logger.custom_logging import Custom_Logger
import logging
from pathlib import Path
import sys, signal
import os
import dotenv
from bot import PostIt_Bot
from utils.datetime_tools import get_elapsed_time_milliseconds, get_elapsed_time_big
from utils.adv_configparser import Advanced_ConfigParser
from utils.datetime_tools import get_elapsed_time_smal, get_elapsed_time_big, get_elapsed_time_milliseconds
import discord
from discord.ext import commands
from pathlib import Path
import re
import sys
import traceback
import asyncio
from typing import Union
from platforms.reddit import Reddit_Adapter
from const import VERSION
async def main():
PROGRAM_VERSION = "1.0"
print(" ____ ____ ___________ __________")
print(" / __ \/ __ \/ ___/_ __/ / _/_ __/")
print(" / /_/ / / / /\__ \ / /_____ / / / / ")
print(" / ____/ /_/ /___/ // /_____// / / / ")
print(f" /_/ \____//____//_/ /___/ /_/ v{PROGRAM_VERSION}")
print(" Copyright (c) 2024-2025 Lars Winzer")
print()
print(" Source: https://github.com/official-Cromatin/Post-It")
print(" Report an Issue: https://github.com/official-Cromatin/Post-It/issues/new?assignees=&labels=bug&projects=&template=issue_report.yml")
print("\n")
print(" ____ ____ ___________ __________")
print(" / __ \/ __ \/ ___/_ __/ / _/_ __/")
print(" / /_/ / / / /\__ \ / /_____ / / / / ")
print(" / ____/ /_/ /___/ // /_____// / / / ")
print(f" /_/ \____//____//_/ /___/ /_/ v{VERSION}")
print(" Copyright (c) 2024-2025 Lars Winzer")
print()
print(" Source: https://github.com/official-Cromatin/Post-It")
print(" Report an Issue: https://github.com/official-Cromatin/Post-It/issues/new?assignees=&labels=bug&projects=&template=issue_report.yml")
print("\n")
startup = datetime.now().timestamp()
# Get timestamp for begin of actual execution
startup_time = datetime.now().timestamp()
# Initialize the logger
Custom_Logger.initialize()
app_logger = logging.getLogger("app")
startup_logger = logging.getLogger("app.startup")
# Initialize the logger
Custom_Logger.initialize()
app_logger = logging.getLogger("app")
app_startup_logger = logging.getLogger("app.startup")
app_startup_logger.info(f"Starting Post-It v{PROGRAM_VERSION} ...")
source_path = Path(__file__).resolve()
base_path = source_path.parents[1]
app_logger.info(f"Using the following path as entrypoint: '{base_path}'")
# Detect entrypoint
source_path = Path(__file__).resolve()
base_path = source_path.parents[1]
app_startup_logger.info(f"Using the following path as entrypoint: '{base_path}'")
intents = discord.Intents.default()
intents.messages = True
# Detect operating system and attach
shutdown_event = asyncio.Event()
def signal_handler(*args):
app_logger.info("Shutdown signal recieved, shutting down")
shutdown_event.set()
class MyBot(commands.Bot):
def __init__(self):
super().__init__(command_prefix=None, help_command=None, intents=intents)
self.__first_on_ready = False
match sys.platform:
case "linux":
app_startup_logger.info("Detected platform: Linux")
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
case "darwin":
app_startup_logger.info("Detected platform: MacOS (Darwin)")
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
case "win32":
app_startup_logger.info("Detected platform: Windows (Win32)")
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
self.VERSION = VERSION
self.STARTUP_TIMESTAMP: float = None
self.platforms_config: Advanced_ConfigParser = None
self.bot_config: Advanced_ConfigParser = None
self.reddit_adapter: Reddit_Adapter = None
case _:
app_startup_logger.fatal(f"Detected unsupported platform: {sys.platform}")
quit(1)
self.no_executed_commands:int = 0
self.no_succeeded_commands:int = 0
self.no_failed_commands:int = 0
# Load environment variables
if (os.getenv("skip_dotenv", False)):
app_startup_logger.warning("Skipped import of dotenv file")
else:
dotenv.load_dotenv(base_path / ".env")
app_startup_logger.warning("Imported dotenv file")
async def setup_hook(self):
# Register cogs to handle commands
for cog_name in ["debug", "post"]:
await self.load_extension(f"cogs.{cog_name}")
await self.tree.sync()
# Check if required environment variables are present
check_ok = True
required_variables = [
"DISCORD_TOKEN",
]
async def on_app_command_completion(self, interaction: discord.Interaction, command: Union[discord.app_commands.Command, discord.app_commands.ContextMenu]):
"""Called when a `app_commands.Command` or `app_commands.ContextMenu` has successfully completed without error"""
self.no_succeeded_commands += 1
for variable_name in required_variables:
if os.getenv(variable_name) is None:
app_startup_logger.error(f"Environment variable '{variable_name}' is missing")
check_ok = False
async def on_interaction(self, interaction: discord.Interaction):
"""Called when an interaction happened"""
match interaction.type.name:
case discord.InteractionType.application_command.name:
self.no_executed_commands += 1
case discord.InteractionType.ping.name:
print("App got pinged by discord")
case discord.InteractionType.autocomplete.name:
print("Interaction with autocomplete")
case discord.InteractionType.modal_submit.name:
print("Modal interaction submitted")
case discord.InteractionType.component.name:
print("Component interaction")
if not check_ok:
app_startup_logger.critical("Multiple required variables are missing. Aborting startup")
quit(1)
async def on_connect(self):
"""A coroutine to be called to setup the bot, after the bot is logged in but before it has connected to the Websocket"""
if not self.__first_on_ready:
startup_logger.info("Beginning startup routine ...")
routine_begin = datetime.now().timestamp()
await self.change_presence(status = discord.Status.dnd, activity = discord.CustomActivity("Executing pre startup routine"))
# Create bot instance and start bot
app_startup_logger.info(f"Preperations complete after {get_elapsed_time_milliseconds(datetime.now().timestamp() - startup_time)}, launching bot")
try:
bot_instance = PostIt_Bot(startup_time, PROGRAM_VERSION, app_logger)
# Create the adapters for the platforms
task_start = datetime.now().timestamp()
startup_logger.debug("Loading platforms config ...")
platforms_config = Advanced_ConfigParser(Path.joinpath(base_path, "config", "platforms.ini"))
self.platforms_config = platforms_config
startup_logger.info(f"Loaded platforms config after {get_elapsed_time_milliseconds(datetime.now().timestamp() - task_start)}")
bot_task = asyncio.create_task(bot_instance.start(os.getenv("DISCORD_TOKEN")))
shutdown_task = asyncio.create_task(shutdown_event.wait())
# Create platforms adapter
task_start = datetime.now().timestamp()
startup_logger.debug("Creating reddit adapter ...")
self.reddit_adapter = Reddit_Adapter(platforms_config["REDDIT"]["CLIENT_ID"], platforms_config["REDDIT"]["CLIENT_SECRET"])
startup_logger.info(f"Created reddit adapter after {get_elapsed_time_milliseconds(datetime.now().timestamp() - task_start)}")
# Wait if either the bot disconnects or the shutdown event is detected
finished_task, _ = await asyncio.wait(
[shutdown_task, bot_task],
return_when = asyncio.FIRST_COMPLETED
)
await self.change_presence(status = discord.Status.online, activity = None)
startup_logger.info(f"Startup routine finished after {get_elapsed_time_milliseconds(datetime.now().timestamp() - routine_begin)}")
self.__first_on_ready = True
else:
startup_logger.info("Startup routine allready executed, omitting this execution")
if shutdown_task in finished_task:
await bot_instance.close()
app_logger.info("Bot closed connection successfully")
except discord.errors.LoginFailure:
app_logger.critical("Improper token has been passed. Aborting startup")
quit(1)
async def on_ready(self):
app_logger.info(f"Successfully logged in (after {get_elapsed_time_smal(datetime.now().timestamp() - startup)}) as {self.user}")
finally:
app_logger.info(f"Exiting. Application ran for {get_elapsed_time_big(datetime.now().timestamp() - startup_time)}")
bot = MyBot()
bot.STARTUP_TIMESTAMP = startup
bot.bot_config = Advanced_ConfigParser(Path.joinpath(base_path, "config", "bot.ini"))
if re.match(r'[A-Za-z\d]{24}\.[\w-]{6}\.[\w-]{27}', bot.bot_config["DISCORD"]["TOKEN"]):
app_logger.critical("Bot (config/bot.ini) configuration invalid, please set a valid token")
quit(1)
elif bot.bot_config.compare_to_template() not in ("equal", "config_minus"):
app_logger.critical("Bot (config/bot.ini) configuration is missing some parts. Make sure it at least has all the same keys as the template")
quit(1)
else:
app_logger.info("Bot configuration valid, continuing with startup")
# Entry point for execution
if __name__ == "__main__":
asyncio.run(main())
# Setup handlers to handle states of command execution
@bot.tree.error
async def on_app_command_error(ctx:discord.Interaction, error):
"""Executed when exception during command execution occurs"""
print('Ignoring exception in command {}:'.format(ctx.command), file=sys.stderr)
traceback.print_exception(type(error), error, error.__traceback__)
bot.no_failed_commands += 1
try:
bot.run(bot.bot_config["DISCORD"]["TOKEN"], log_handler = None)
except discord.errors.LoginFailure:
app_logger.critical("Improper token has been passed. Aborting startup")
quit(1)
app_logger.info("Quitting application ...")
asyncio.run(bot.close())
app_logger.info(f"Exiting. Application ran for {get_elapsed_time_big(datetime.now().timestamp() - startup)}")
+45
View File
@@ -0,0 +1,45 @@
import asyncpraw
import asyncpraw.models
import logging
from utils.event_counter import Event_Counter
from utils.datetime_tools import get_elapsed_time_milliseconds
from datetime import datetime
class Reddit_Adapter(asyncpraw.Reddit):
"""A class that extends and abstracts the functionality of the `asyncpraw.Reddit` class by adding
logging and request tracking capabilities."""
VERSION = "1.0"
number_of_instances = 0
def __init__(self, client_id:str, client_secret:str):
"""Initializes the Reddit Adapter, while stating credentials for the login to the reddit api"""
self.__instance_number = self.__class__.number_of_instances
self.__class__.number_of_instances += 1
super().__init__(
client_id = client_id,
client_secret = client_secret,
user_agent="Small discord bot to embed posts (given by url) into an standardized format"
)
self.__events = Event_Counter(1000)
self.__logger = logging.getLogger(f"pltfm.reddit.{self.__instance_number}")
async def fetch(self, post_url:str) -> asyncpraw.models.Submission:
"""Fetches specified submission (post) and returns it"""
start_time = datetime.now().timestamp()
self.__events.increment()
subm = await self.submission(url = post_url)
self.__logger.debug(f"Submission for post (URL: {post_url}), successfully fetched after {get_elapsed_time_milliseconds(datetime.now().timestamp() - start_time)}")
return subm
def get_total_requests(self) -> int:
"""Returns the total number of requests made since the creation of the adapter"""
return self.__events.get_total_events()
def get_events_last_5m(self) -> int:
"""Returns the number of requests made in the last 5 minutes"""
return self.__events.get_count("5m")
def get_events_last_5m_10m_15m(self) -> tuple[int]:
"""Returns an tuple containing the number of requests made in the last 5, 10 and 15 minutes"""
return (self.__events.get_count("5m"), self.__events.get_count("10m"), self.__events.get_count("15m"))
+249
View File
@@ -0,0 +1,249 @@
import configparser
import logging
from os.path import isfile, split
from pathlib import Path
from shutil import copy
class Advanced_ConfigParser(configparser.ConfigParser):
"""Utility class to ease the use of the configparser libary"""
VERSION = "2.5"
number_of_instances = 0
def __init__(self, path:str, allow_template:bool = True, allow_update:bool = True) -> None:
"""
Initializes the Advanced_ConfigParser instance.
Args:
path (str): The path to the configuration file.
allow_template (bool, optional): Determines if an existing template can be used when the config file is missing. Default is True.
allow_update (bool, optional): Specifies if the config file should be updated based on a template. Default is True.
This constructor loads the configuration file if it exists, optionally creates one from a template if allowed,
and can update the configuration based on the template. It also initializes logging and tracks pending changes.
"""
super().__init__()
self.__instance_number = self.__class__.number_of_instances
self.__class__.number_of_instances += 1
self.__from_template = False
self.__logger = logging.getLogger(f"utils.config.{self.__instance_number}")
self.__logger.debug(f"New instance {self.__instance_number} of class created")
# Check existence of provided file
self.__template_path = self.__get_template_path(path)
if isfile(path):
self.__logger.debug(f"Opening existing file at path: '{path}'")
else:
# Check if it is allowed to use an existing template
if allow_template:
if isfile(self.__template_path):
self.__logger.info(f"Using existing template file {self.__template_path}")
copy(self.__template_path, path)
self.__from_template = True
else:
self.__logger.warning("No potential template found, starting with empty file")
# Load the configfile
self.__file_path = path
self.__pending_changes = 0
try:
self.read_file(open(path))
except:
pass
else:
self.__logger.info(f"Successfully opened config file at path {path}")
# Check for update if allowed
if allow_update:
self.check_for_update()
def __get_template_path(self, path:str) -> str | None:
"""Looks for a existing template and returns the path to it
If no config file was found, None is returned"""
path_to_file, file_name = split(path)
file_name = file_name.split(".")[0]
path_to_temp = Path(path_to_file) / f".{file_name}.template"
return path_to_temp
def __has_all_template_options(self, template:configparser.ConfigParser):
"""Check if config contains all sections and options from the template"""
for section in template.sections():
if section not in self:
return False
template_options = set(template.options(section))
config_options = set(config.options(section))
if not template_options.issubset(config_options):
return False
return True
def set(self, section: str, option: str, value: str | None = None) -> None:
old_value = self.get(section, option, fallback=None)
if old_value:
self.__logger.debug(f"New value in section '{section}' at option '{option}' added: '{value}'")
self.__pending_changes += 1
else:
if self.has_option(section, option):
self.__logger.debug(f"Value in section '{section}' at option '{option}' changed: '{old_value}' -> '{value}'")
if old_value != value:
self.__pending_changes += 1
return super().set(section, option, value)
def remove_option(self, section, option):
if self.has_option(section, option):
value = self.get(section, option)
self.__logger.debug(f"Option removed in section '{section}' for '{option}': '{value}'")
self.__pending_changes += 1
return super().remove_option(section, option)
def remove_section(self, section):
if self.has_section(section):
self.__logger.info(f"Section removed: '{section}'")
self.__pending_changes += 1
return super().remove_section(section)
def save(self):
"""Saves changes to disk"""
with open(self.__file_path, 'w') as configfile:
self.write(configfile)
self.__logger.debug(f"Contents ({self.__pending_changes} changes) have been saved to disk")
def get_config_file_path(self) -> str:
"""Returns the path to the config file"""
return self.__file_path
def get_template_file_path(self) -> str:
"""Returns the path to the template file, a config could be created from
Returns None if the is no template file"""
return self.__template_path
def compare_to_template(self) -> str:
"""Returns one of multiple possible keywords depending on the relation of both files
+-----------------+--------------------------------------------------------------------------------------------------------------------------+
| Return Keyword | Description |
+-----------------+--------------------------------------------------------------------------------------------------------------------------+
| not_found | No template file found |
| equal | Both files contain the same sections and options |
| config_plus | Config contains additional sections and/or options compared to template |
| config_options | Config contains additional options but same sections |
| config_base | Same as "config_plus" but additionally config contains all sections and options from the template |
| config_minus | Config contains fewer sections and/or options as template |
+-----------------+--------------------------------------------------------------------------------------------------------------------------+"""
# Check existence of template
if self.__template_path == None:
return "not_found"
template_config = configparser.ConfigParser()
template_config.read_file(open(self.__template_path))
config_sections = set(self.sections())
template_sections = set(template_config.sections())
# Check if config contains all the same (or more) sections than template
same_or_more_sections = config_sections >= template_sections
# Check if both files have the same amount of sections
same_amount_of_sections = len(config_sections) == len(template_sections)
# Consolidate the options of both files
all_config_options = set()
all_template_options = set()
all_sections = config_sections.union(template_sections)
config_all_present = True
template_all_present = True
for section in all_sections:
try:
for option in self.options(section):
all_config_options.add(f"{section}.{option}")
except configparser.NoSectionError:
config_all_present = False
try:
for option in template_config.options(section):
all_template_options.add(f"{section}.{option}")
except configparser.NoSectionError:
template_all_present = False
# Check if both have the same options
same_or_more_options = all_config_options >= all_template_options
# Check if both have the same amount of options
same_amount_of_options = len(all_config_options) == len(all_template_options)
# Check for different cases
# Since both files have the same (or more) options (and the options contain the section name) and contain the same amount, they must be equal
if same_or_more_options and same_amount_of_options:
return "equal"
# Since the config contains not even the same sections as present in the template, it doesn't contain the same base options
if not same_or_more_sections:
return "config_minus"
# If the config contains more sections or options than the template
if same_or_more_sections:
if same_or_more_options:
# If config contains all sections and options from template but also has extras
return "config_base"
else:
# Config has extra sections and/or options, but not necessarily all from template
return "config_plus"
# If the config contains the same sections but additional options within those sections
if same_amount_of_sections and not same_amount_of_options:
return "config_options"
return "config_plus"
def created_from_template(self) -> bool:
"""Will ONLY return `True` if the config file has been created from template at the first run"""
return self.__from_template
def check_for_update(self, omit_save:bool = False):
"""Updates the already created config to the same level as the template,
no options are deleted or their values changed."""
if self.__template_path == None:
raise FileNotFoundError
template_config = configparser.ConfigParser()
template_config.read_file(open(self.__template_path))
updated_options = 0
for section in template_config.sections():
options = template_config.options(section)
for option in options:
if not self.has_option(section, option):
self.set(section, option, template_config.get(section, option))
updated_options += 1
if updated_options:
_, file_name = split(self.__file_path)
self.__logger.error(f"The confg updated at {updated_options} locations, you might want check the {file_name} for unchanged placeholders")
if not omit_save:
self.save()
if __name__ == "__main__":
# Dummy logger
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
logger = logging.getLogger("utils")
logger.addHandler(console)
logger.setLevel(logging.DEBUG)
file_path = Path(__file__).resolve()
base_path = file_path.parents[2]
print(f"Base path of the program: {base_path}")
config = Advanced_ConfigParser(Path.joinpath(base_path, "config", "bot.ini"))
print(config.compare_to_template())
# config["DEFAULT"]["Compression"] = "no"
# config.save()
+1 -1
View File
@@ -10,7 +10,7 @@ def get_elapsed_time_ms(timestamp:float) -> str:
time_elapsed = datetime.fromtimestamp(timestamp)
return f"{time_elapsed.minute // 60}min {time_elapsed.second:02}sec {time_elapsed.microsecond // 1000}ms"
def get_elapsed_time_small(timestamp:float) -> str:
def get_elapsed_time_smal(timestamp:float) -> str:
"""Convert an timestamp into an predefined elapsed time format (00sec 000ms)"""
time_elapsed = datetime.fromtimestamp(timestamp)
return f"{time_elapsed.second:02}sec {time_elapsed.microsecond // 1000:03}ms"
+110
View File
@@ -0,0 +1,110 @@
from datetime import datetime
import re
class Event_Counter:
"""A class to track and count events, storing the event timestamps and allowing queries
for the number of events in specific time windows (e.g., last 5 minutes, 10 minutes)."""
REGEX_PATTERN = r'(\d+)\s*(sec|second|seconds|s|min|minute|minutes|m|h|hour|hours|d|day|days)'
TIME_MULTIPLIERS = {
'sec': 1, 'second': 1, 'seconds': 1, 's': 1,
'min': 60, 'minute': 60, 'minutes': 60, 'm': 60,
'h': 3600, 'hour': 3600, 'hours': 3600,
'd': 86400, 'day': 86400, 'days': 86400
}
def __init__(self, retention_duration:int = 900) -> None:
"""
Initializes the Event_Counter object with a specified retention duration.
Args:
retention_duration (int, optional):
The time period (in seconds) for which event data should be retained. Defaults to 900 (15 minutes).
uwu
"""
self.__events:list[float] = []
self.__total_events = 0
self.__retention_duration = retention_duration
def cleanup(self):
"""Cleans up events that are older than the retention duration.
Removes all events that occurred outside the defined retention window."""
offset_time = datetime.now().timestamp() - self.__retention_duration
while len(self.__events) > 0:
event_time = self.__events[0]
if event_time < offset_time:
self.__events.pop(0)
else:
break
@classmethod
def duration_to_seconds(cls, duration_str:str) -> int:
"""Converts a human-readable duration string (e.g., '5min 2sec') into its equivalent in seconds.
Args:
duration_str (str): The string representing the duration (e.g., "5min 2sec").
Returns:
int: The total duration in seconds derived from the input string."""
matches = re.findall(cls.REGEX_PATTERN, duration_str, flags=re.IGNORECASE)
total_seconds = 0
for value, unit in matches:
unit = unit.lower()
total_seconds += int(value) * cls.TIME_MULTIPLIERS[unit]
return total_seconds
def increment(self, count:int = 1, skip_cleanup:bool = False):
"""Increments the event count by a specified number of events.
Args:
count (int, optional): The number of events to add. Defaults to 1.
skip_cleanup (bool, optional): If True, skips the cleanup process. Defaults to False."""
timestamp = datetime.now().timestamp()
self.__total_events += count
self.__events.extend([timestamp] * count)
if skip_cleanup:
return
self.cleanup()
def get_count(self, time_window:str = "5M") -> int:
"""Returns the number of events that occurred within a specified time window.
Args:
time_window (str, optional):
The time window to query, e.g., "5M" for 5 minutes. Defaults to "5M".
Returns:
int: The count of events that occurred within the specified time window."""
self.cleanup()
offset_time = datetime.now().timestamp() - self.duration_to_seconds(time_window)
counter = 0
for event_time in self.__events:
if event_time > offset_time:
counter += 1
else:
break
return counter
def get_total_events(self) -> int:
"""Returns the total number of events tracked by the Event_Counter.
Returns:
int: The total number of events that have been recorded."""
return self.__total_events
if __name__ == "__main__":
from time import sleep
counter = Event_Counter(1)
counter.increment()
sleep(1)
counter.increment()
print(counter.get_count("5sEc"))
sleep(0)
print(counter.get_count("5sEc"))
+38
View File
@@ -0,0 +1,38 @@
class Singleton:
"""
A non-thread-safe helper class to ease implementing singletons.
This should be used as a decorator -- not a metaclass -- to the
class that should be a singleton.
The decorated class can define one `__init__` function that
takes only the `self` argument. Also, the decorated class cannot be
inherited from. Other than that, there are no restrictions that apply
to the decorated class.
To get the singleton instance, use the `instance` method. Trying
to use `__call__` will result in a `TypeError` being raised.
"""
# Source: https://stackoverflow.com/questions/31875/is-there-a-simple-elegant-way-to-define-singletons
def __init__(self, decorated):
self._decorated = decorated
def instance(self):
"""
Returns the singleton instance. Upon its first call, it creates a
new instance of the decorated class and calls its `__init__` method.
On all subsequent calls, the already created instance is returned.
"""
try:
return self._instance
except AttributeError:
self._instance = self._decorated()
return self._instance
def __call__(self):
raise TypeError('Singletons must be accessed through `instance()`.')
def __instancecheck__(self, inst):
return isinstance(inst, self._decorated)
+21
View File
@@ -0,0 +1,21 @@
def truncate_message_with_notice(output:str, max_length:int = 1000, suffix:str = "Output was truncated") -> str:
"""Trim a long output string to a specified length and append a suffix message if truncation occurs"""
total_max_length = max_length - len(suffix) # Adjust for the suffix length
# If the output is already shorter than the max length, return it as is
if len(output) <= max_length:
return output
# Split the output into lines and add lines until the length exceeds the allowed maximum
result = ""
lines = output.splitlines()
for line in lines:
if len(result) + len(line) + 1 > total_max_length: # +1 accounts for the line break
break
result += line + "\n"
# Append the suffix message
result += suffix
return result