Social features / ActivityPub federation (#1629)

* Support webfinger requests for the live account. Closes https://github.com/owncast/owncast/issues/1193

* Support for actor requests. Returns response for live actor. Closes https://github.com/owncast/owncast/issues/1203

* Handle follow and unfollow requests. Closes
https://github.com/owncast/owncast/issues/1191 and https://github.com/owncast/owncast/issues/1205 and https://github.com/owncast/owncast/issues/1206 and https://github.com/owncast/owncast/issues/1194

* Add basic support for sending out text activities. For https://github.com/owncast/owncast/issues/1192

* Some error handling and passing of dynamic local account names.

* Add hardcoded example image attachment to test post

* Centralize the map of accounts and inboxes

* No longer disable the preview generator based on YP toggle

* Send a federated message to followers when stream starts. For https://github.com/owncast/owncast/issues/1192

* Placeholder for attaching tags

* Add image description

* Save and get to outbox persistence. Return using outbox endpoint for actor

* Pass payloads to be handled through the gochan

* Handle undo follow requests explitly, not all undo requests

* Add API for manually sending simple federated messages. Closes #1215

* Verify inbox requests. Closes #1321

* Add route to fetch a single AP object by ID. For #1329

* Add responses to fediverse nodeinfo requests

* Set and get federation config values for admin

* Handle host-meta requests

* Do not send out message if disabled. Use saved go live message.

* Require AP-compatible content types for AP-related requests

* Rename ap models to apmodels for clarity

* Change how content type matching takes place.

* io -> ioutil

* Add stub delete activity callback

* Handle likes and announces to surface engagement in chat. Part of #1229

* Append url to go live posts

* Do not require specific content types for nodeinfo requests

* Add follow engagement chat message via AP

* add owncast user-agent to requests

* Set note visibility to public (for now)

* Fix saving/fetching a single object

* Add support for x-nodeinfo2 responses

* Point to the dev admin branch for ap

* Bundle in dev admin for testing

* Add error logging

* Add AP middleware back

* Point to the new external compatible logo endpoint

* Clean up more AP logging to help testing

* Tweak go live text and link hashtags

* Fix bug in fetching init time

* Send update actor activities when server details/profile is updated

* Add federation config overview to web client config

* Add additional actor properties

* Make the AP middleware checking more flexible when looking at types

* First pass at remote fediverse follow flow. For #1371

* Added a basic AP actor followers endpoint

* WIP client followers API

* Add profile-page reference to webfinger response

* Add aliases to webfinger response

* Fix content-type returned to be expected activitypub+json

* First pass at followers api

* Point at local dev copy of go-fed/activity

* Add custom toot Hashtag objects to posts

* Store additional user details to followers table

* Fix AP followers endpoint. Closes #1204

* Add owncast hashtag as an invisible tag to go live posts

* Reject AP requests when it is disabled

* Add actor util for generating full account user from person object

* Verify inbox requests before performing any other work

* Accept actor update requests

* Fix linter errors in federation branch

* Migrate AP SQL to sqlc for type safe queries

* Use the @unclearParadigm REST parameter helper

* Fix verifying post ID on AP engagement

* WIP privacy/request approval

* Style the remote follow modal

* First pass at a followers list component w/ mock data. #1370

* Revert "Use the @unclearParadigm REST parameter helper"

This reverts commit c8af8a413f6f53e7d1a15a7d823ff28be2db3c23.

* Fix get followers API

* Add support for requiring approval. Closes https://github.com/owncast/owncast/issues/1208

* Handle Applications as Actors partly for PeerTube support

* add temp todo list

* check route on load, this might change later

* style followers

* account for just 1 tab case

* Remove mock data. Allow showing follow button even when there are no external actions defined

* Point to actual followers API

* Support fallback img for follower views

* Remove duplicate verification. Add some additional verbose logging

* Bundle dev admin

* Add type to host-meta webfinger template response

* Tweak remote follow modal content

* WIP federation followers refactor

* Do not send pointer to middleware

* Update admin

* Add setting for toggling displaying fediverse engagement. Closes #1404

* Add in-development admin

* Do not enable cors on admin followers api

* Add db migration for updating messages table

* Enable empty string go live messages to disable

* Remove debug messages

* Rework some ActivityPub handling.

Create new Actor->Person handling.
Create new Actor->Service handling.
Add engagement handlers to send chat events and store event objects.
Store inbound activities to new ap_inbound_activities table.

* Support federated engagement events.

Store them in the messages table and surface them via chat events.

* Support federated event engatement in the chat

* Tweak web UI followers handling

* Point go.mod at remote fork instead of local

* Update admin

* Merged in develop. Couple fixes

* Update dev admin

* Update fedi engagement posts.

- Fix incorrect action text.
- Add action icons.

* Set public as to instead of cc for ap msg

* Updated styling for federated actions in chat

* Add support for blocking federated domains. Closes #1209

* Force checking of https in verify step

* Update dev admin

* Return user scopes in chat history api. Closes #1586

* Update dev admin

* Add AP outbound request worker pool. Closes #1571

* Disable (temporarily?) owncast tag on AP posts

* Consolidate creating activity+notes in outbound AP messages

* Add inbox worker pool. Closes #1570

* Update dev admin bundle

* Clean up some logs

* Re-enable inbound verfication

* Save full IRI to outbox instead of path

* Reject if full IRI is not found in outbox

* Use full ActivityPub user account in chat event

* Fix and expand follower APIs

- Add missing IDs to AP follower endpoints
- Split AP follower endpoints into initial request and pages.
- Support pagination in AP requests.

* Include IRI in error message

* Hide chat toggle when chat is hidden. Closes #1606

* Updates to followers pagination

* Set default go live message

* Remove log

* indirect -> direct import

* Updates for inbound federated event handling.

- Keep track of existing events and reject duplicates.
- Change what is sent to chat for surfing federated engagement.
- Keep track if outbound events are automated "go live" events or not.

* Update chat federated engagement.

* Update dev admin.

* Move from being a person to a bot (service). Closes #1619

* Only set server init date if not already set

* Only save notes to outbox able

* Rework private-mode followers/approvals

* API for returning a list of federated actions for #1573

* Fix too-small follower cells and jumpy tabs. Closes #1616 and closes #1516

* Fix shortcuts getting fired on inputs. Fixes #1489 and #1201

* Add spinner, autoclose + other fixes to follow modal. Fixes #1593

* Fix fetching a single object by IRI

* SendFederationMessage -> SendFederatedMessage

* Autolink and create tag objects from manual posts. Closes #1620

* Update dev admin bundle

* Handle engagement from non-automated/live posts

* Reject federated engagement actions if they do not match a local post

* Update dev admin bundle

* A bunch of cleanup

* Fix unused assignments and logic

* Remove unused function

* Add content warning and sentive content flag if stream is NSFW. Closes #1624

* Disable fetching objects by IRI when in private mode. Closes #1623

* Update the error message of the remote follow dialog. closes #1622

* Update dev admin

* Fix NREs throwing in test content

* Fix query that wasn't properly filtering out hidden messages

* Test against user being disabled instead of message visibility

* Fix automated test NRE

* Update comment

* Adjust federated engagement chat views. Closes #1617

* Add additional index to users table

* Add support for removing followers/requests. Closes #1630

* Reject federated actions from blocked actors. #1631

* Use fallback avatar if it fails to load. Closes #1635

* Fix styling of follower list. Closes #1636

* Add basic blurb stating they should follow the server. Closes #1641

* Update dev admin

* Set default go live message in migration. Closes #1642

* Reset the messages table on 0.0.11 schema migration

* Fix js error with moderation actions. Closes #1621

* Add a bit more clarification on follow modal. Closes #1599

* Remove todos

* Split out actor and domain blocking checks

* Check for errors on default values being set

* Clean up actor rejection due to being blocked

* Update dev admin

* Add colon to error to make it easier to read

* Remove markdown rendering of go live message. Reorganize text. Remove content warning. Closes #1645

* Break out the sort+render messages logic so it can be fired on visibility change. Closes #1643

* Do not send profile updates if federation is disabled

* Save follow references to inbound activities table

* Update dev admin

* Add blocked actor test

* Remove the overloaded term of Follow from social links

* Fix test running in memory only

* Remove "just" in engagement messags

* Replace star with heart for like action.

* Update dev admin

* Explicitly set cc as public

* Remove overly using the stream name in fediverse engagement messages

* Some federated/follow UI tweaks

* Remove explicit cc and bcc as they are not required

* Explicitly set the audience

* Remove extra margin

* Add Join Fediverse button to follow modal. Closes #1651

* Do not allow multiple follows to send multiple events. Closes #1650

* Give events a min height

* Do not allow old posts to be liked/shared. Closes #1652

* Remove value from log message

* Alert followers on private mode toggle

* Ignore clicks to follow button if disabled

* Remove underline from action buttons

* Add moderator icon to join message

* Update admin

* Post-merge remove unused var

* Remove pointing at feature branch

Co-authored-by: Ginger Wong <omqmail@gmail.com>

core/chat/chat.go

@@ -78,12 +78,39 @@ func SendSystemMessage(text string, ephemeral bool) error {
 	}
 
 	if !ephemeral {
-		saveEvent(message.ID, "system", message.Body, message.GetMessageType(), nil, message.Timestamp)
+		saveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
 	}
 
 	return nil
 }
 
+// SendFediverseAction will send a message indicating some Fediverse engagement took place.
+func SendFediverseAction(eventType string, userAccountName string, image *string, body string, link string) error {
+	message := events.FediverseEngagementEvent{
+		Event: events.Event{
+			Type: eventType,
+		},
+		MessageEvent: events.MessageEvent{
+			Body: body,
+		},
+		UserAccountName: userAccountName,
+		Image:           image,
+		Link:            link,
+	}
+
+	message.SetDefaults()
+	message.RenderBody()
+
+	if err := Broadcast(&message); err != nil {
+		log.Errorln("error sending system message", err)
+		return err
+	}
+
+	saveFederatedAction(message)
+
+	return nil
+}
+
 // SendSystemAction will send a system action string as an action event to all clients.
 func SendSystemAction(text string, ephemeral bool) error {
 	message := events.ActionEvent{
@@ -100,7 +127,7 @@ func SendSystemAction(text string, ephemeral bool) error {
 	}
 
 	if !ephemeral {
-		saveEvent(message.ID, "action", message.Body, message.GetMessageType(), nil, message.Timestamp)
+		saveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
 	}
 
 	return nil

core/chat/events/eventtype.go

@@ -34,4 +34,10 @@ const (
 	ErrorMaxConnectionsExceeded EventType = "ERROR_MAX_CONNECTIONS_EXCEEDED"
 	// ErrorUserDisabled is an error returned when the connecting user has been previously banned/disabled.
 	ErrorUserDisabled EventType = "ERROR_USER_DISABLED"
+	// FediverseEngagementFollow is an event representing a follow action that took place on the fediverse.
+	FediverseEngagementFollow EventType = "FEDIVERSE_ENGAGEMENT_FOLLOW"
+	// FediverseEngagementLike is an event representing a like action that took place on the fediverse.
+	FediverseEngagementLike EventType = "FEDIVERSE_ENGAGEMENT_LIKE"
+	// FediverseEngagementRepost is an event representing a re-post action that took place on the fediverse.
+	FediverseEngagementRepost EventType = "FEDIVERSE_ENGAGEMENT_REPOST"
 )

core/chat/events/fediverseEngagementEvent.go

@@ -0,0 +1,32 @@
+package events
+
+import "github.com/owncast/owncast/core/data"
+
+// FediverseEngagementEvent is a message displayed in chat on representing an action on the Fediverse.
+type FediverseEngagementEvent struct {
+	Event
+	MessageEvent
+	Image           *string `json:"image"`
+	Link            string  `json:"link"`
+	UserAccountName string  `json:"title"`
+}
+
+// GetBroadcastPayload will return the object to send to all chat users.
+func (e *FediverseEngagementEvent) GetBroadcastPayload() EventPayload {
+	return EventPayload{
+		"id":        e.ID,
+		"timestamp": e.Timestamp,
+		"body":      e.Body,
+		"image":     e.Image,
+		"type":      e.Event.Type,
+		"title":     e.UserAccountName,
+		"user": EventPayload{
+			"displayName": data.GetServerName(),
+		},
+	}
+}
+
+// GetMessageType will return the event type for this message.
+func (e *FediverseEngagementEvent) GetMessageType() EventType {
+	return e.Event.Type
+}

core/chat/persistence.go

@@ -34,17 +34,19 @@ func setupPersistence() {
 
 // SaveUserMessage will save a single chat event to the messages database.
 func SaveUserMessage(event events.UserMessageEvent) {
-	saveEvent(event.ID, event.User.ID, event.Body, event.Type, event.HiddenAt, event.Timestamp)
+	saveEvent(event.ID, &event.User.ID, event.Body, event.Type, event.HiddenAt, event.Timestamp, nil, nil, nil, nil)
 }
 
-func saveEvent(id string, userID string, body string, eventType string, hidden *time.Time, timestamp time.Time) {
+func saveFederatedAction(event events.FediverseEngagementEvent) {
+	saveEvent(event.ID, nil, event.Body, event.Type, nil, event.Timestamp, event.Image, &event.Link, &event.UserAccountName, nil)
+}
+
+// nolint: unparam
+func saveEvent(id string, userID *string, body string, eventType string, hidden *time.Time, timestamp time.Time, image *string, link *string, title *string, subtitle *string) {
 	defer func() {
 		_historyCache = nil
 	}()
 
-	_datastore.DbLock.Lock()
-	defer _datastore.DbLock.Unlock()
-
 	tx, err := _datastore.DB.Begin()
 	if err != nil {
 		log.Errorln("error saving", eventType, err)
@@ -53,7 +55,7 @@ func saveEvent(id string, userID string, body string, eventType string, hidden *
 
 	defer tx.Rollback() // nolint
 
-	stmt, err := tx.Prepare("INSERT INTO messages(id, user_id, body, eventType, hidden_at, timestamp) values(?, ?, ?, ?, ?, ?)")
+	stmt, err := tx.Prepare("INSERT INTO messages(id, user_id, body, eventType, hidden_at, timestamp, image, link, title, subtitle) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")
 	if err != nil {
 		log.Errorln("error saving", eventType, err)
 		return
@@ -61,7 +63,7 @@ func saveEvent(id string, userID string, body string, eventType string, hidden *
 
 	defer stmt.Close()
 
-	if _, err = stmt.Exec(id, userID, body, eventType, hidden, timestamp); err != nil {
+	if _, err = stmt.Exec(id, userID, body, eventType, hidden, timestamp, image, link, title, subtitle); err != nil {
 		log.Errorln("error saving", eventType, err)
 		return
 	}
@@ -71,8 +73,134 @@ func saveEvent(id string, userID string, body string, eventType string, hidden *
 	}
 }
 
-func getChat(query string) []events.UserMessageEvent {
-	history := make([]events.UserMessageEvent, 0)
+func makeUserMessageEventFromRowData(row rowData) events.UserMessageEvent {
+	scopes := ""
+	if row.userScopes != nil {
+		scopes = *row.userScopes
+	}
+
+	previousUsernames := []string{}
+	if row.previousUsernames != nil {
+		previousUsernames = strings.Split(*row.previousUsernames, ",")
+	}
+
+	displayName := ""
+	if row.userDisplayName != nil {
+		displayName = *row.userDisplayName
+	}
+
+	displayColor := 0
+	if row.userDisplayColor != nil {
+		displayColor = *row.userDisplayColor
+	}
+
+	createdAt := time.Time{}
+	if row.userCreatedAt != nil {
+		createdAt = *row.userCreatedAt
+	}
+
+	u := user.User{
+		ID:            *row.userID,
+		AccessToken:   "",
+		DisplayName:   displayName,
+		DisplayColor:  displayColor,
+		CreatedAt:     createdAt,
+		DisabledAt:    row.userDisabledAt,
+		NameChangedAt: row.userNameChangedAt,
+		PreviousNames: previousUsernames,
+		Scopes:        strings.Split(scopes, ","),
+	}
+
+	message := events.UserMessageEvent{
+		Event: events.Event{
+			Type:      row.eventType,
+			ID:        row.id,
+			Timestamp: row.timestamp,
+		},
+		UserEvent: events.UserEvent{
+			User:     &u,
+			HiddenAt: row.hiddenAt,
+		},
+		MessageEvent: events.MessageEvent{
+			Body:    row.body,
+			RawBody: row.body,
+		},
+	}
+
+	return message
+}
+
+func makeSystemMessageChatEventFromRowData(row rowData) events.SystemMessageEvent {
+	message := events.SystemMessageEvent{
+		Event: events.Event{
+			Type:      row.eventType,
+			ID:        row.id,
+			Timestamp: row.timestamp,
+		},
+		MessageEvent: events.MessageEvent{
+			Body:    row.body,
+			RawBody: row.body,
+		},
+	}
+	return message
+}
+
+func makeActionMessageChatEventFromRowData(row rowData) events.ActionEvent {
+	message := events.ActionEvent{
+		Event: events.Event{
+			Type:      row.eventType,
+			ID:        row.id,
+			Timestamp: row.timestamp,
+		},
+		MessageEvent: events.MessageEvent{
+			Body:    row.body,
+			RawBody: row.body,
+		},
+	}
+	return message
+}
+
+func makeFederatedActionChatEventFromRowData(row rowData) events.FediverseEngagementEvent {
+	message := events.FediverseEngagementEvent{
+		Event: events.Event{
+			Type:      row.eventType,
+			ID:        row.id,
+			Timestamp: row.timestamp,
+		},
+		MessageEvent: events.MessageEvent{
+			Body:    row.body,
+			RawBody: row.body,
+		},
+		Image:           row.image,
+		Link:            *row.link,
+		UserAccountName: *row.title,
+	}
+	return message
+}
+
+type rowData struct {
+	id        string
+	userID    *string
+	body      string
+	eventType models.EventType
+	hiddenAt  *time.Time
+	timestamp time.Time
+	title     *string
+	subtitle  *string
+	image     *string
+	link      *string
+
+	userDisplayName   *string
+	userDisplayColor  *int
+	userCreatedAt     *time.Time
+	userDisabledAt    *time.Time
+	previousUsernames *string
+	userNameChangedAt *time.Time
+	userScopes        *string
+}
+
+func getChat(query string) []interface{} {
+	history := make([]interface{}, 0)
 	rows, err := _datastore.DB.Query(query)
 	if err != nil || rows.Err() != nil {
 		log.Errorln("error fetching chat history", err)
@@ -81,69 +209,47 @@ func getChat(query string) []events.UserMessageEvent {
 	defer rows.Close()
 
 	for rows.Next() {
-		var id string
-		var userID string
-		var body string
-		var messageType models.EventType
-		var hiddenAt *time.Time
-		var timestamp time.Time
-
-		var userDisplayName *string
-		var userDisplayColor *int
-		var userCreatedAt *time.Time
-		var userDisabledAt *time.Time
-		var previousUsernames *string
-		var userNameChangedAt *time.Time
+		row := rowData{}
 
 		// Convert a database row into a chat event
-		err = rows.Scan(&id, &userID, &body, &messageType, &hiddenAt, &timestamp, &userDisplayName, &userDisplayColor, &userCreatedAt, &userDisabledAt, &previousUsernames, &userNameChangedAt)
-		if err != nil {
+		if err = rows.Scan(
+			&row.id,
+			&row.userID,
+			&row.body,
+			&row.title,
+			&row.subtitle,
+			&row.image,
+			&row.link,
+			&row.eventType,
+			&row.hiddenAt,
+			&row.timestamp,
+			&row.userDisplayName,
+			&row.userDisplayColor,
+			&row.userCreatedAt,
+			&row.userDisabledAt,
+			&row.previousUsernames,
+			&row.userNameChangedAt,
+			&row.userScopes,
+		); err != nil {
 			log.Errorln("There is a problem converting query to chat objects. Please report this:", query)
 			break
 		}
 
-		// System messages and chat actions are special and are not from real users
-		if messageType == events.SystemMessageSent || messageType == events.ChatActionSent {
-			name := "Owncast"
-			userDisplayName = &name
-			color := 200
-			userDisplayColor = &color
-		}
+		var message interface{}
 
-		if previousUsernames == nil {
-			previousUsernames = userDisplayName
-		}
-
-		if userCreatedAt == nil {
-			now := time.Now()
-			userCreatedAt = &now
-		}
-
-		user := user.User{
-			ID:            userID,
-			AccessToken:   "",
-			DisplayName:   *userDisplayName,
-			DisplayColor:  *userDisplayColor,
-			CreatedAt:     *userCreatedAt,
-			DisabledAt:    userDisabledAt,
-			NameChangedAt: userNameChangedAt,
-			PreviousNames: strings.Split(*previousUsernames, ","),
-		}
-
-		message := events.UserMessageEvent{
-			Event: events.Event{
-				Type:      messageType,
-				ID:        id,
-				Timestamp: timestamp,
-			},
-			UserEvent: events.UserEvent{
-				User:     &user,
-				HiddenAt: hiddenAt,
-			},
-			MessageEvent: events.MessageEvent{
-				Body:    body,
-				RawBody: body,
-			},
+		switch row.eventType {
+		case events.MessageSent:
+			message = makeUserMessageEventFromRowData(row)
+		case events.SystemMessageSent:
+			message = makeSystemMessageChatEventFromRowData(row)
+		case events.ChatActionSent:
+			message = makeActionMessageChatEventFromRowData(row)
+		case events.FediverseEngagementFollow:
+			message = makeFederatedActionChatEventFromRowData(row)
+		case events.FediverseEngagementLike:
+			message = makeFederatedActionChatEventFromRowData(row)
+		case events.FediverseEngagementRepost:
+			message = makeFederatedActionChatEventFromRowData(row)
 		}
 
 		history = append(history, message)
@@ -152,16 +258,16 @@ func getChat(query string) []events.UserMessageEvent {
 	return history
 }
 
-var _historyCache *[]events.UserMessageEvent
+var _historyCache *[]interface{}
 
 // GetChatModerationHistory will return all the chat messages suitable for moderation purposes.
-func GetChatModerationHistory() []events.UserMessageEvent {
+func GetChatModerationHistory() []interface{} {
 	if _historyCache != nil {
 		return *_historyCache
 	}
 
 	// Get all messages regardless of visibility
-	query := "SELECT messages.id, user_id, body, eventType, hidden_at, timestamp, display_name, display_color, created_at, disabled_at, previous_names, namechanged_at FROM messages INNER JOIN users ON messages.user_id = users.id ORDER BY timestamp DESC"
+	query := "SELECT messages.id, user_id, body, title, subtitle, image, link, eventType, hidden_at, timestamp, display_name, display_color, created_at, disabled_at, previous_names, namechanged_at, scopes FROM messages INNER JOIN users ON messages.user_id = users.id ORDER BY timestamp DESC"
 	result := getChat(query)
 
 	_historyCache = &result
@@ -170,9 +276,9 @@ func GetChatModerationHistory() []events.UserMessageEvent {
 }
 
 // GetChatHistory will return all the chat messages suitable for returning as user-facing chat history.
-func GetChatHistory() []events.UserMessageEvent {
+func GetChatHistory() []interface{} {
 	// Get all visible messages
-	query := fmt.Sprintf("SELECT messages.id, user_id, body, eventType, hidden_at, timestamp, display_name, display_color, created_at, disabled_at, previous_names, namechanged_at FROM messages, users WHERE messages.user_id = users.id AND hidden_at IS NULL AND disabled_at IS NULL ORDER BY timestamp DESC LIMIT %d", maxBacklogNumber)
+	query := fmt.Sprintf("SELECT messages.id,messages.user_id, messages.body, messages.title, messages.subtitle, messages.image, messages.link, messages.eventType, messages.hidden_at, messages.timestamp, users.display_name, users.display_color, users.created_at, users.disabled_at, users.previous_names, users.namechanged_at, users.scopes FROM messages LEFT JOIN users ON messages.user_id = users.id WHERE hidden_at IS NULL AND disabled_at IS NULL ORDER BY timestamp DESC LIMIT %d", maxBacklogNumber)
 	m := getChat(query)
 
 	// Invert order of messages
@@ -200,7 +306,7 @@ func SetMessageVisibilityForUserID(userID string, visible bool) error {
 	}
 
 	for _, message := range messages {
-		ids = append(ids, message.ID)
+		ids = append(ids, message.(events.Event).ID)
 	}
 
 	// Tell the clients to hide/show these messages.
@@ -250,35 +356,3 @@ func saveMessageVisibility(messageIDs []string, visible bool) error {
 
 	return nil
 }
-
-// Only keep recent messages so we don't keep more chat data than needed
-// for privacy and efficiency reasons.
-func runPruner() {
-	_datastore.DbLock.Lock()
-	defer _datastore.DbLock.Unlock()
-
-	log.Traceln("Removing chat messages older than", maxBacklogHours, "hours")
-
-	deleteStatement := `DELETE FROM messages WHERE timestamp <= datetime('now', 'localtime', ?)`
-	tx, err := _datastore.DB.Begin()
-	if err != nil {
-		log.Debugln(err)
-		return
-	}
-
-	stmt, err := tx.Prepare(deleteStatement)
-	if err != nil {
-		log.Debugln(err)
-		return
-	}
-	defer stmt.Close()
-
-	if _, err = stmt.Exec(fmt.Sprintf("-%d hours", maxBacklogHours)); err != nil {
-		log.Debugln(err)
-		return
-	}
-	if err = tx.Commit(); err != nil {
-		log.Debugln(err)
-		return
-	}
-}

core/chat/pruner.go

@@ -0,0 +1,39 @@
+package chat
+
+import (
+	"fmt"
+
+	log "github.com/sirupsen/logrus"
+)
+
+// Only keep recent messages so we don't keep more chat data than needed
+// for privacy and efficiency reasons.
+func runPruner() {
+	_datastore.DbLock.Lock()
+	defer _datastore.DbLock.Unlock()
+
+	log.Traceln("Removing chat messages older than", maxBacklogHours, "hours")
+
+	deleteStatement := `DELETE FROM messages WHERE timestamp <= datetime('now', 'localtime', ?)`
+	tx, err := _datastore.DB.Begin()
+	if err != nil {
+		log.Debugln(err)
+		return
+	}
+
+	stmt, err := tx.Prepare(deleteStatement)
+	if err != nil {
+		log.Debugln(err)
+		return
+	}
+	defer stmt.Close()
+
+	if _, err = stmt.Exec(fmt.Sprintf("-%d hours", maxBacklogHours)); err != nil {
+		log.Debugln(err)
+		return
+	}
+	if err = tx.Commit(); err != nil {
+		log.Debugln(err)
+		return
+	}
+}