LogalDeveloper/OwncastSentry
0
Fork 0
Code Issues 1 Actions Releases 7 Activity

Updated package README.
CI / Formatting (push) Failing after 6s
Details
CI / Linting (push) Successful in 6s
Details
CI / Tests (push) Successful in 36s
Details
CI / Type Checking (push) Successful in 11s
Details
CI / Spelling (push) Successful in 9s
Details

Browse Source
This commit is contained in:
LogalDeveloper
2026-05-24 14:12:05 -04:00
parent 58ad476a94
commit d02202295c
2 changed files with 156 additions and 229 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitea/workflows/cd.yml
+1 -1
View File
@@ -37,7 +37,7 @@ jobs:
grep "^version:" maubot.yaml
- name: Build plugin
run: 7z a -tzip -mx=9 -xr!__pycache__ owncastsentry-v${{ steps.metadata.outputs.version }}.mbp owncastsentry/ maubot.yaml base-config.yaml LICENSE.txt
run: 7z a -tzip -mx=9 -xr!__pycache__ -xr!README.md owncastsentry-v${{ steps.metadata.outputs.version }}.mbp owncastsentry/ maubot.yaml base-config.yaml LICENSE.txt
- name: Print artifact checksums
run: |
owncastsentry/README.md
+155 -228
View File
@@ -13,64 +13,46 @@ or tests.
- `__init__.py`: Maubot plugin entry point, service wiring, command
decorators, scheduler loop, shutdown, and metrics endpoint.
- `commands.py`: Maubot command handlers for subscribe, unsubscribe,
subscriptions, and live listings.
- `config.py`: Maubot configuration proxy for the Prometheus metrics endpoint
setting.
- `types.py`: immutable value objects, Owncast API response validation, derived
stream status, update results, and subscription domain errors.
- `repository.py`: database schema migrations plus stream and subscription
repositories.
- `metrics.py`: Prometheus counters, gauges, registry setup, and response timing
helpers.
- `notification_service.py`: Matrix notification formatting, broadcast,
per-domain notification cooldowns, cleanup notices, and delivery metrics.
- `owncast_client.py`: bounded aiohttp client for Owncast `/api/status` and
`/api/config` requests.
- `subscription_manager.py`: subscription use cases, domain normalization, and
first-subscription Owncast validation.
- `repository.py`: database schema migrations plus stream and subscription
repositories.
- `stream_monitor.py`: stream update orchestration, state transition detection,
progressive failure backoff, cleanup thresholds, notification decisions, and
stream metrics.
- `notification_service.py`: Matrix notification formatting, broadcast,
per-domain notification cooldowns, cleanup notices, and delivery metrics.
- `commands.py`: Maubot command handlers for subscribe, unsubscribe,
subscriptions, and live listings.
- `metrics.py`: isolated Prometheus registry, counters, gauges, and response
timing helpers.
- `subscription_manager.py`: subscription use cases, domain normalization, and
first-subscription Owncast validation.
- `types.py`: immutable value objects, Owncast API response validation, derived
stream status, update results, and subscription domain errors.
The package has a service-layer shape: `OwncastSentry.start()` builds the
services, commands call `CommandHandler`, command handlers call
`SubscriptionManager`, scheduled updates call `StreamMonitor`, and
`StreamMonitor` coordinates repositories, the Owncast client, notifications,
and metrics.
```mermaid
flowchart LR
Maubot["Maubot loads OwncastSentry"] --> Start["OwncastSentry.start()"]
Start --> Repos["StreamRepository and SubscriptionRepository"]
Start --> Owncast["OwncastClient"]
Start --> Metrics["MetricsService"]
Start --> Notify["NotificationService"]
Start --> Monitor["StreamMonitor"]
Start --> Manager["SubscriptionManager"]
Start --> Commands["CommandHandler"]
Commands --> Manager
Manager --> Repos
Manager --> Owncast
Monitor --> Repos
Monitor --> Owncast
Monitor --> Notify
Monitor --> Metrics
Notify --> Repos
Notify --> Matrix["Matrix client"]
```
`SubscriptionManager`, and scheduled updates call `StreamMonitor`.
`StreamMonitor` runs the main polling workflow: it checks Owncast streams,
applies backoff and cleanup rules, classifies state transitions, triggers any
live/title notifications, persists changed stream state, and records stream,
failure, and subscription metrics.
## Core Concepts
A few terms are used throughout the package:
- A stream is a normalized Owncast domain tracked by the plugin. The database
stores one stream row per domain, even if multiple Matrix rooms subscribe to
it.
- A stream is a tracked [Owncast](https://owncast.online/) instance, identified
by its normalized bare domain. The stored entry for that domain is the stream
record. There is one stream record per domain, even if multiple Matrix rooms
subscribe to it.
- A subscription is a Matrix room's request to receive notifications for one
stream domain.
- `StreamStateObservation` is one successful Owncast status API sample. The
monitor decides whether the observation starts a new persisted status period.
- `StreamStateObservation` is one successful Owncast status API sample used by
the monitor when updating stored stream state.
- `StreamConfigObservation` is metadata fetched from Owncast's config API,
currently the instance name and tags used in notifications and periodic
display-name refreshes.
@@ -83,9 +65,11 @@ Durable state lives in the database. The `streams` table stores the latest
known display and state fields for each Owncast domain, and the `subscriptions`
table stores which Matrix rooms follow each domain.
Subscriptions are room-scoped, while stream state is shared per domain.
Notification delivery fans out from one stored stream state to the rooms
currently subscribed to that domain.
Subscriptions belong to individual Matrix rooms, while stream state is shared
per domain. Notification delivery fans out by domain to the rooms currently
subscribed to that domain. Live/title notification text is built from the fresh
status observation and, when available, fresh config metadata for the update
being processed.
Stream identity is the normalized bare domain. Repositories, metrics,
notification state, and Owncast API calls all use that domain rather than the
@@ -100,41 +84,43 @@ commands, stream updates, or notifications.
are represented by new values that are persisted through repositories instead of
mutating existing objects.
## Service Wiring
## Plugin Boundary
Maubot interacts with the package through `OwncastSentry`. That plugin class is
the boundary between Maubot's runtime and the internal services that implement
OwncastSentry behavior.
`OwncastSentry.start()` is the composition root for one plugin instance. It
loads config, creates the metrics service, Owncast API client, repositories,
notification service, stream monitor, subscription manager, and command handler,
then passes dependencies into each service explicitly.
loads config, creates the Owncast client, repositories, metrics, notification
service, stream monitor, subscription manager, and command handler, then passes
dependencies into each service explicitly.
This keeps lower-level modules independent of Maubot globals. Commands do not
construct repositories, `StreamMonitor` does not know about the Maubot
scheduler, and notification delivery does not fetch its own subscription state
from the plugin instance.
The plugin boundary owns Maubot integration: command registration, scheduled
polling, and lifecycle cleanup. Scheduled ticks load subscribed domains and
call `StreamMonitor.update_all_streams()`. `StreamMonitor` owns the per-stream
polling workflow, while notification delivery and delivery cooldowns stay in
`NotificationService`.
The plugin boundary owns scheduling. The minute loop loads subscribed domains
from `SubscriptionRepository` and calls `StreamMonitor.update_all_streams()`.
The same pattern applies to new background work: schedule it from the plugin
boundary, keep the worker service callable directly, and close owned external
resources from `OwncastSentry.stop()`.
Internal services do not depend on Maubot globals, which keeps command handling,
scheduled polling, and tests wired through explicit dependencies.
## Subscription Management
`SubscriptionManager` coordinates the room-to-domain relationship. Command
handlers pass it user-supplied stream targets, and it turns those targets into
normalized domains before touching repository state.
normalized domains before creating or removing stored data.
Subscribing a room creates a subscription for the normalized domain and creates
the shared stream row when the domain is first seen. First-time domains are
the shared stream record when needed. Domains with no current subscribers are
validated through `OwncastClient`; domains that already have subscribers reuse
the existing stream record instead of revalidating.
Unsubscribing removes one room's subscription to a domain. It does not delete
the shared stream row or make remote Owncast requests. Long-term dead stream
cleanup is owned by `StreamMonitor`.
the shared stream record or make remote Owncast requests.
Listing methods stay room-scoped. They return the subscriptions relevant to one
Matrix room, with the shared stream state attached for display.
Listing methods return the subscriptions for one Matrix room with shared stream
state attached, so commands can display each instance's name, title, link,
status, and how long it has been online or offline.
## Owncast API Client
@@ -144,13 +130,12 @@ accepted responses into the value objects used by the rest of the package.
The session uses a plugin-specific User-Agent, a dummy cookie jar, no DNS cache,
a global connection limit, a per-host connection limit of one, and connect/read
socket timeouts. aiohttp's own DNS cache is disabled so DNS caching and policy
stay with the resolver configured for the deployment.
socket timeouts. aiohttp's own DNS cache is disabled; resolver selection is left
to aiohttp and the runtime environment.
The per-host connection limit and keepalive settings are intentional. Each
Owncast instance gets at most one reusable connection, so minute-by-minute
status checks can reuse the existing TLS session instead of opening a fresh
connection for every poll.
Owncast instance is intended to keep at most one long-lived connection warm, so
minute-by-minute status checks avoid unnecessary TLS renegotiation.
Responses are accepted only when they are HTTP 200 JSON objects under the
configured size limit. Invalid JSON, non-object JSON, oversized bodies,
@@ -163,147 +148,118 @@ for that domain.
## Stream Monitoring
`StreamMonitor` owns the core per-domain state machine. `update_stream()`
compares the stored stream state with the latest Owncast status response,
decides whether to send a notification, updates persistent state when needed,
and records stream metrics. Each poll first handles missing rows, polling
backoff, and failed Owncast requests. Only a valid status response enters the
stream transition policy. `update_all_streams()` wraps this flow for many
domains, isolates per-domain exceptions, and records subscription-count metrics.
`StreamMonitor` runs the package's main polling workflow for tracked Owncast
instances. `update_stream()` owns the per-domain state machine: it compares the
stored stream state with the latest Owncast status response, decides whether to
send a notification, updates persistent state when needed, and records stream
metrics. Each poll first handles stored stream lookup, polling backoff, and
failed Owncast requests. Only a valid status response enters the stream transition
policy. `update_all_streams()` wraps this flow for many domains, isolates
per-domain exceptions, and records subscription-count metrics.
```mermaid
flowchart TD
Start["update_stream(domain)"] --> Old["load stored StreamState"]
Old --> Missing{"row exists?"}
Missing -->|no| Noop["return success"]
Missing -->|yes| Backoff{"backoff allows poll?"}
Backoff -->|no| Skip["skip HTTP call<br/>increment failure counter"]
Skip --> CleanupSkip["run warning/deletion cleanup checks"]
CleanupSkip --> MetricsSkip["if row remains, update<br/>status and failure metrics"]
MetricsSkip --> SuccessSkip["return success"]
Backoff -->|yes| Fetch["fetch and validate /api/status"]
Fetch --> Fetched{"got valid state?"}
Fetched -->|no| Failure["increment failure counter"]
Failure --> CleanupFail["run warning/deletion cleanup checks"]
CleanupFail --> MetricsFail["if row remains, update<br/>status and failure metrics"]
MetricsFail --> Failed["return failure"]
Fetched -->|yes| Reset["reset failure counter and metric"]
Reset --> Timer["ensure offline timer<br/>cache entry exists"]
Timer --> Transition["classify stream transition"]
Transition --> First{"first observation?"}
First -->|yes| FirstLog["log suppressed notification"]
Start["update_stream(domain)"] --> Stored["load stored stream state"]
Stored --> Query{"backoff allows query?"}
Query -->|no| Defer["skip query for backoff"]
Defer --> Cleanup["apply cleanup policy"]
Cleanup --> Done["done"]
Query -->|yes| Status["fetch stream status from Owncast"]
Status --> Observation{"valid observation?"}
Observation -->|no| Failure["record failed check"]
Failure --> Cleanup
Observation -->|yes| Compare["compare observation with stored state"]
Compare --> First{"first observation?"}
First -->|yes| FirstNoNotify["no notification"]
First -->|no| LiveCheck{"went live?"}
FirstLog --> ConfigDecision
LiveCheck -->|yes| Brief{"observed offline less than<br/>temporary cooldown?"}
LiveCheck -->|no| TitleChanged{"title changed<br/>while online?"}
Brief -->|no| LiveNotify["go-live notification needed"]
LiveCheck -->|yes| Brief{"offline less than 7 minutes?"}
Brief -->|no| LiveNotify["select go-live notification"]
Brief -->|yes| BriefTitle{"title changed?"}
BriefTitle -->|yes| BriefNotify["title-change notification needed"]
BriefTitle -->|no| NoNotify["no notification"]
TitleChanged -->|yes| OfflineMarker{"offline marker newer than<br/>last notification?"}
BriefTitle -->|no| BriefNoNotify["no notification"]
BriefTitle -->|yes| TitleNotify["select title-change notification"]
LiveCheck -->|no| TitleChanged{"title changed while online?"}
TitleChanged -->|yes| TitleNotify
TitleChanged -->|no| WentOffline{"went offline?"}
WentOffline -->|yes| Offline["record offline timer"]
WentOffline -->|no| NoNotify
OfflineMarker -->|yes| ResumeNotify["go-live notification needed"]
OfflineMarker -->|no| TitleNotify["title-change notification needed"]
NoNotify --> ConfigDecision
Offline --> ConfigDecision
LiveNotify --> ConfigDecision
BriefNotify --> ConfigDecision
ResumeNotify --> ConfigDecision
TitleNotify --> ConfigDecision
ConfigDecision{"notification, first observation,<br/>or hourly refresh?"}
ConfigDecision -->|yes| Config["fetch /api/config"]
ConfigDecision -->|no| NotifyGate{"notification needed?"}
Config --> NotifyGate
NotifyGate -->|yes| Send["attempt live/title notification"]
NotifyGate -->|no| Save["save stream row<br/>repository ignores unchanged values"]
Send --> Save
Save --> FinalMetrics["set current status metric"]
FinalMetrics --> Done["return success"]
WentOffline -->|yes| Offline["record when stream went offline"]
WentOffline -->|no| NoNotify["no notification"]
FirstNoNotify --> Metadata
BriefNoNotify --> Metadata
NoNotify --> Metadata
Offline --> Metadata
LiveNotify --> Metadata
TitleNotify --> Metadata
Metadata["fetch instance config from Owncast if display metadata is needed"]
Metadata --> Notify["attempt sending notification if selected"]
Notify --> Save["save stream update"]
Save --> Done
```
Failure counters drive both status and polling behavior. A stream is displayed
as unknown when its failure counter is above `UNKNOWN_STATUS_THRESHOLD`.
Counters 0 through 4 poll every minute; later tiers poll only on selected
counter values and eventually only every fifteenth minute. Skipped cycles still
increment the counter so cleanup can progress.
Each stream has a failure counter that tracks consecutive failed checks and
queries skipped for backoff. The counter drives both status and polling behavior:
a stream is displayed as unknown after more than 15 minutes of failed or
skipped checks, and later failures progressively reduce query frequency until
the monitor checks only every 15 minutes. Skipped queries still increment
the counter so cleanup can progress.
Transition decisions use Owncast's `online` field. The stream status timestamp is
the bot's local UTC observation time for the first successful poll, a missing
timestamp repair, or an online/offline transition. Owncast's remote connect,
disconnect, and server timestamps are not required for status parsing.
After each successful fetch from Owncast's
[`/api/status`](https://owncast.online/api/latest/#tag/Internal/operation/GetStatus)
endpoint, the monitor compares the stored stream state with the latest
observation to identify transitions, such as going online, going offline, or
changing title. Online/offline transitions are based on Owncast's `online` field.
The stream status timestamp is the bot's UTC observation time for the first
successful poll or an online/offline transition.
First observation is special. If a new stream row has no status timestamp, the
first successful update fetches config and stores state but suppresses
notification attempts, even if the stream is already live.
The first successful observation stores state and fetches config but suppresses
notifications, even if the stream is already live. After that, only
offline-to-online transitions can select go-live notifications; mid-session
title changes select title-change notifications.
Go-live notification attempts happen when a stream moves from offline to online
and the monitor's in-memory offline timer shows it was not merely a brief
outage. A stream that returns within the temporary offline cooldown only attempts
a notification when the title changed, and then it uses a title change
notification.
Brief outages are treated as transient to avoid spamming Matrix rooms when an
instance is spotty. They do not select go-live notifications. If a stream returns
in under seven minutes with the same title, the monitor stores the new online
state without notifying rooms. If the title changed during that brief outage, the
monitor selects a title-change notification instead.
Mid-session title changes attempt title change notifications. If the monitor has
an offline marker that is newer than the last notification, it attempts a normal
live notification instead so rooms do not see a title change for what is
effectively a new session.
Instance config is fetched from Owncast's
[`/api/config`](https://owncast.online/api/latest/#tag/Internal/operation/GetWebConfig)
endpoint when a live/title notification is selected, on a stream's first
successful observation, and during the hourly refresh window for successfully
polled streams. The config response supplies the display name and tags for
notifications. If config is fetched successfully during an update, the persisted
stream name is refreshed from it; otherwise the stored name is left unchanged.
Offline transitions do not notify rooms. They update the database and record a
monotonic offline timestamp used by later live/title notification policy.
The live/title notification send step is an attempt, not a guaranteed Matrix message.
`NotificationService` still applies the shared live/title cooldown and records
the cooldown only when at least one room receives the message.
`/api/config` is fetched when the monitor is about to attempt a live or
title-change notification, on a stream's first successful observation, and
during the hourly refresh window for successfully polled streams. The config
response supplies the display name and tags for notifications. If config is
fetched successfully during an update, the persisted stream name is refreshed
from it; otherwise the stored name is left unchanged.
Selecting a notification does not guarantee a Matrix message. The notification
service can still suppress delivery when its per-domain live/title cooldown is
active, which prevents repeated messages from noisy stream state changes. That
delivery behavior is covered below.
## Cleanup Policy
A domain that remains unreachable is eventually cleaned up. The failure counter
represents minute ticks, including skipped backoff ticks.
At the 83-day threshold, `StreamMonitor` sends a cleanup warning to all rooms
subscribed to the domain. At the 90-day threshold or beyond, it sends a deletion
notice, deletes all subscriptions for the domain, deletes the stream row, clears
the monitor and notification caches for that domain, and removes per-domain
metrics.
```mermaid
flowchart TD
Counter["failure counter after failed or skipped check"] --> Warn{"counter == 83 days?"}
Warn -->|yes| Warning["send cleanup warning"]
Warn -->|no| Delete
Warning --> Delete{"counter >= 90 days?"}
Delete -->|no| Done["done"]
Delete -->|yes| Notice["send cleanup deletion notice"]
Notice --> Subs["delete subscriptions"]
Subs --> Stream["delete stream row"]
Stream --> Caches["clear local caches"]
Caches --> Metrics["remove per-domain metrics"]
Metrics --> DoneCleanup["done"]
```
After 83 days of continued failed or skipped checks, `StreamMonitor` sends a
cleanup warning to all rooms subscribed to the domain. After 90 days or more of
continued failed or skipped checks, it sends a deletion notice, deletes all
subscriptions for the domain, deletes the stream record, clears the monitor and
notification caches for that domain, and asks `MetricsService` to remove
existing per-domain metric labels. The enclosing update cycle may later recreate
the subscription-count gauge for that domain at zero.
## Notification Delivery
`NotificationService` sends plain text Matrix messages to every room subscribed
to a domain. Broadcasts run concurrently and use `asyncio.gather(...,
return_exceptions=True)` so one failed room does not block delivery to other
rooms.
`NotificationService` sends notifications to every room subscribed to a domain.
Delivery failures are isolated per room, so one failed Matrix send does not
block notifications to other subscribed rooms.
Live and title-change notifications share a per-domain cooldown. The cooldown
is recorded only when at least one room receives the message. If there are no
subscribed rooms, or every delivery fails, no cooldown is recorded.
Notification text is whitespace-normalized before sending. Stream names fall
back to the domain when config lookup fails or the instance name is empty. Tags
are appended as hash tags after filtering out empty tags and tags that start
with a dot.
Live/title notification text normalizes remote stream metadata before composing
the message, so instance-provided names, titles, and tags cannot add unexpected
line breaks or otherwise distort the notification. Messages fall back to the
domain when a display name is unavailable.
Cleanup warning and deletion notices bypass the live/title cooldown, but still
record delivery metrics.
@@ -316,33 +272,30 @@ that catch unexpected exceptions and record command error metrics.
`CommandHandler.subscribe()` and `CommandHandler.unsubscribe()` translate
domain errors into user-facing Matrix replies. `subscriptions()` and `live()`
read resolved room subscriptions and render Markdown output with escaped
untrusted text.
read room subscription listings with stream state attached and render Markdown
output with escaped untrusted text.
Command listings are room-scoped and ordered by domain through repository
queries. `subscriptions()` includes online, offline, and unknown streams.
`live()` includes only rows marked online with a failure count at or below the
unknown threshold.
Command listings are limited to the current room and ordered by domain through
repository queries. `subscriptions()` includes online, offline, and unknown
streams. `live()` includes only streams marked online with a failure count at or
below the unknown threshold.
## Persistence
The database schema is managed by `repository.py` through Maubot's upgrade
table.
Maubot owns the database connection and runs the schema upgrades registered by
`repository.py`. The repository classes wrap that database handle and expose the
package's stream and subscription persistence operations.
- Revision 1 creates `streams` and `subscriptions`.
- Revision 2 fixes `subscriptions.stream_domain` from `INTEGER` to `TEXT`.
- Revision 3 adds `streams.failure_counter`.
- Revision 4 replaces separate connect/disconnect timestamps with
`streams.online` and `streams.status_since`.
Repositories own SQL access. `StreamRepository.update()` writes display
and state fields; failure counters use dedicated methods.
The persistence model stores one stream record per normalized domain and one
subscription row per room/domain pair. `SubscriptionManager` normalizes user
input before repository calls. `StreamRepository` writes display and state
fields, while failure counters use dedicated methods.
`SubscriptionRepository` raises domain-specific errors for duplicate adds and
missing removes.
Resolved subscription listings join `subscriptions` to `streams`, which means
orphaned subscription rows without a stream row are skipped in room display
queries.
Room subscription listings join `subscriptions` to `streams`, which means
orphaned subscription entries without a matching stream record are skipped in
room display queries.
## Metrics
@@ -355,36 +308,10 @@ The registered metrics cover:
- current stream status per domain (`1` online, `0` offline, `-1` unknown);
- subscription count per domain;
- consecutive check failures per domain;
- last successful Owncast API response duration per domain;
- successful Owncast API response duration per domain;
- build/version information;
- open aiohttp connection count;
- internal scheduler and command errors.
Known counter label combinations are initialized to zero. This keeps the
Prometheus output stable before any events have occurred.
## Test Map
Start with these tests when changing package behavior:
- `tests/test_types.py`: immutable value objects, API response shape checks,
truncation, stream status derivation, and subscription errors.
- `tests/test_repository.py`: schema-backed stream and subscription repository
behavior.
- `tests/test_subscription_manager.py`: domain normalization, subscribe and
unsubscribe workflows, validation skipping, and room listing delegation.
- `tests/test_owncast_client.py`: Owncast API request handling, response body
limits, validation failures, User-Agent, response timing, and connection
counts.
- `tests/test_stream_monitor.py`: polling backoff, first-update suppression,
live/offline/title transitions, cleanup thresholds, exception isolation, and
monitor metrics.
- `tests/test_notification_service.py`: message formatting, sanitization,
cooldown behavior, broadcast failure accounting, cleanup notices, and
notification metrics.
- `tests/test_commands.py`: end-to-end Maubot command behavior, Markdown
escaping, duration formatting, room-scoped subscription listings, and live
listings.
- `tests/test_metrics.py`: Prometheus counters, gauges, response timer,
per-domain label removal, build info, open connections, and isolated registry
output.