Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation * Disable chat if the user is disabled/blocked or the server hits max connections * Handle dropping sockets if chat is disabled * Fix origin in automated chat test * Work for updated chat moderation * Chat message markdown rendering and fix tests * Put /api/chat behind a chat user access token. Closes #1085 * Reject blocked username changes * More WIP moderation * Defer configuring chat until we know if it is enabled. Closes #1135 * chat user blocking. Closes #1096 * Add tests around user access for #1096 * Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092 * Delete old chat messages from db as to not hold on to excessive data. Closes #1152 * Add schema migration for messages. Closes #1155 * Commit updated API documentation * Add chat load test * Shared db mutex and db optimizations * Simplify past display name handling * Use a new test db for each test run * Wire up the external messages actions + add tests for them * Move access tokens to be actual users * Run message pruning at launch + fix comparison * Do not return API users in disabled users response * Fix incorrect highlighting. Closes #1160 * Consolidate user table statements * Set the max process connection limit to 70% of maximum * Fix wrong old display name being returned in name change event * Delete the old chat server files * Wire back up the webhooks * Remove unused * Invalidate user cache on changes * Do not send rendered body as RawBody * Some cleanup * Standardize names for external API users to ExternalAPIUser * Do not log token * Checkout branch when building admin for testing * Bundle in dev admin for testing * Some cleanup * Cleanup js logs * Cleanup and standardize event names * Clean up some logging * Update API spec. Closes #1133 * Commit updated API documentation * Change paths to be better named * Commit updated API documentation * Update admin bundle * Fix duplicate event name * Rename scope var * Update admin bundle * Move connected clients controller into admin package * Fix collecting usernames for autocomplete purposes * No longer generate username when it is empty * Sort clients and users by timestamp * Move file to admin controller package * Swap, so the comments stay correct Co-authored-by: Jannik <jannik@outlook.com> * Use explicit type alias Co-authored-by: Jannik <jannik@outlook.com> * Remove commented code. Co-authored-by: Jannik <jannik@outlook.com> * Cleanup test * Remove some extra logging * Add some clarity * Update dev instance of admin for testing * Consolidate lines Co-authored-by: Jannik <jannik@outlook.com> * Remove commented unused vars Co-authored-by: Jannik <jannik@outlook.com> * Until needed do not return IP address with client list * Fix typo of wrong var * Typo led to a bad test. Fix typo and fix test. * Guard against the socket reconnecting on error if previously set to shutdown * Do not log access tokens * Return success message on enable/disable user * Clean up some inactionable error messages. Sent ban message. Sort banned users. * fix styling for when chat is completely disabled * Unused * guard against nil clients * Update dev admin bundle * Do not unhide messages when unblocking user just to be safe. Send removal action from the controller * Add convinience function for getting active connections for a single user * Lock db on these mutations * Cleanup force disconnect using GetClientsForUser and capture client reference explicitly * No longer re-showing banned user messages for safety. Removing this test. * Remove no longer needed comment * Tweaks to forbidden username handling. - Standardize naming to not use "block" but "forbidden" instead. - Pass array over the wire instead of string. - Add API test - Fix default list incorrectly being appended to custom list. * Logging cleanup * Update dev admin bundle * Add an artificial delay in order to visually see message being hidden when testing * Remove the user cache as it is a premature optimization * When connected to chat let the user know their current user details to sync the username in the UI * On connected send current display name back to client. - Move name change out of chat component. - Add additional event type constants. * Fix broken workflow due to typo * Troubleshoot workflow * Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181) * Bump htm from 3.0.4 to 3.1.0 in /build/javascript Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0. - [Release notes](https://github.com/developit/htm/releases) - [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0) --- updated-dependencies: - dependency-name: htm dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Run npm run build and update libraries Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gabe Kangas <gabek@real-ity.com> * Commit updated Javascript packages * Re-send current user info when a rejected name change takes place * All socket writes should be through the send chan and not directly * Seed the random generator * Add keys and indexes to users table * a util to generate consistent emoji markup * console clean up * mod tidy * Commit updated API documentation * Handle the max payload size of a socket message. - Only close socket if x2 greater than the max size. - Send the user a message if a message is too large. - Surface the max size in bytes in the config. * Update admin bundle * Force all events to be sent in their own socket message and do not concatinate in a single message * Update chat embed to register for access token * Use a different access token for embed chat * Update the chat message bubble background color to be bolder * add base tag to open links in new window, closes #1220 * Support text input of :emoji: in chat (#1190) * Initial implementation of emoji injection * fix bookkeeping with multiple emoji * make the emoji lookup case-insensitive * try another solution for Caretposition * add title to emojis minor refactoring * bind moji injection to InputKeyUp * simplify the code replace all found emojis * inject emoji if the modifer is released earlier * more efficient emoji tag search * use json emoji.emoji as url * use createEmojiMarkup() * move emojify() to chat.js * emojify on paste * cleanup emoji titles in paste * update inputText in InputKeyup * mark emoji titles with 2*zwnj this way paste cleanup will not interfere with text which include zwnj * emoji should not change the inputText * Do not show join messages when chat is offline. Closes #1224 - Show stream starting/ending messages in chat. - When stream starts show everyone the welcome message. * Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222 * use maxSocketPayloadSize to calculate total bytes of message payload (#1221) * utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting * add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count Co-authored-by: Gabe Kangas <gabek@real-ity.com> Co-authored-by: Owncast <owncast@owncast.online> Co-authored-by: Jannik <jannik@outlook.com> Co-authored-by: Ginger Wong <omqmail@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
This commit is contained in:
Gabe Kangas
@@ -1,53 +0,0 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
// ScopeCanSendUserMessages will allow sending chat messages as users.
|
||||
ScopeCanSendUserMessages = "CAN_SEND_MESSAGES"
|
||||
// ScopeCanSendSystemMessages will allow sending chat messages as the system.
|
||||
ScopeCanSendSystemMessages = "CAN_SEND_SYSTEM_MESSAGES"
|
||||
// ScopeHasAdminAccess will allow performing administrative actions on the server.
|
||||
ScopeHasAdminAccess = "HAS_ADMIN_ACCESS"
|
||||
)
|
||||
|
||||
// For a scope to be seen as "valid" it must live in this slice.
|
||||
var validAccessTokenScopes = []string{
|
||||
ScopeCanSendUserMessages,
|
||||
ScopeCanSendSystemMessages,
|
||||
ScopeHasAdminAccess,
|
||||
}
|
||||
|
||||
// AccessToken gives access to 3rd party code to access specific Owncast APIs.
|
||||
type AccessToken struct {
|
||||
Token string `json:"token"`
|
||||
Name string `json:"name"`
|
||||
Scopes []string `json:"scopes"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
LastUsed *time.Time `json:"lastUsed"`
|
||||
}
|
||||
|
||||
// HasValidScopes will verify that all the scopes provided are valid.
|
||||
// This is not a efficient method.
|
||||
func HasValidScopes(scopes []string) bool {
|
||||
for _, scope := range scopes {
|
||||
if !findItemInSlice(validAccessTokenScopes, scope) {
|
||||
log.Errorln("Invalid scope", scope)
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func findItemInSlice(slice []string, value string) bool {
|
||||
for _, item := range slice {
|
||||
if item == value {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
package models
|
||||
|
||||
// ChatListener represents the listener for the chat server.
|
||||
type ChatListener interface {
|
||||
ClientAdded(client Client)
|
||||
ClientRemoved(clientID string)
|
||||
MessageSent(message ChatEvent)
|
||||
IsStreamConnected() bool
|
||||
}
|
||||
@@ -1,135 +0,0 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/microcosm-cc/bluemonday"
|
||||
"github.com/teris-io/shortid"
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/renderer/html"
|
||||
"mvdan.cc/xurls"
|
||||
)
|
||||
|
||||
// ChatEvent represents a single chat message.
|
||||
type ChatEvent struct {
|
||||
ClientID string `json:"-"`
|
||||
|
||||
Author string `json:"author,omitempty"`
|
||||
Body string `json:"body,omitempty"`
|
||||
RawBody string `json:"-"`
|
||||
ID string `json:"id"`
|
||||
MessageType EventType `json:"type"`
|
||||
Visible bool `json:"visible"`
|
||||
Timestamp time.Time `json:"timestamp,omitempty"`
|
||||
Ephemeral bool `json:"ephemeral,omitempty"`
|
||||
}
|
||||
|
||||
// Valid checks to ensure the message is valid.
|
||||
func (m ChatEvent) Valid() bool {
|
||||
return m.Author != "" && m.Body != "" && m.ID != ""
|
||||
}
|
||||
|
||||
// SetDefaults will set default values on a chat event object.
|
||||
func (m *ChatEvent) SetDefaults() {
|
||||
id, _ := shortid.Generate()
|
||||
m.ID = id
|
||||
m.Timestamp = time.Now()
|
||||
m.Visible = true
|
||||
}
|
||||
|
||||
// RenderAndSanitizeMessageBody will turn markdown into HTML, sanitize raw user-supplied HTML and standardize
|
||||
// the message into something safe and renderable for clients.
|
||||
func (m *ChatEvent) RenderAndSanitizeMessageBody() {
|
||||
m.RawBody = m.Body
|
||||
|
||||
// Set the new, sanitized and rendered message body
|
||||
m.Body = RenderAndSanitize(m.RawBody)
|
||||
}
|
||||
|
||||
// Empty will return if this message's contents is empty.
|
||||
func (m *ChatEvent) Empty() bool {
|
||||
return m.Body == ""
|
||||
}
|
||||
|
||||
// RenderBody will render markdown to html without any sanitization
|
||||
func (m *ChatEvent) RenderBody() {
|
||||
m.RawBody = m.Body
|
||||
m.Body = RenderMarkdown(m.RawBody)
|
||||
}
|
||||
|
||||
// RenderAndSanitize will turn markdown into HTML, sanitize raw user-supplied HTML and standardize
|
||||
// the message into something safe and renderable for clients.
|
||||
func RenderAndSanitize(raw string) string {
|
||||
rendered := RenderMarkdown(raw)
|
||||
safe := sanitize(rendered)
|
||||
|
||||
// Set the new, sanitized and rendered message body
|
||||
return strings.TrimSpace(safe)
|
||||
}
|
||||
|
||||
func RenderMarkdown(raw string) string {
|
||||
markdown := goldmark.New(
|
||||
goldmark.WithRendererOptions(
|
||||
html.WithUnsafe(),
|
||||
),
|
||||
goldmark.WithExtensions(
|
||||
extension.NewLinkify(
|
||||
extension.WithLinkifyAllowedProtocols([][]byte{
|
||||
[]byte("http:"),
|
||||
[]byte("https:"),
|
||||
}),
|
||||
extension.WithLinkifyURLRegexp(
|
||||
xurls.Strict,
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
trimmed := strings.TrimSpace(raw)
|
||||
var buf bytes.Buffer
|
||||
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
return buf.String()
|
||||
}
|
||||
|
||||
func sanitize(raw string) string {
|
||||
p := bluemonday.StrictPolicy()
|
||||
|
||||
// Require URLs to be parseable by net/url.Parse
|
||||
p.AllowStandardURLs()
|
||||
p.RequireParseableURLs(true)
|
||||
|
||||
// Allow links
|
||||
p.AllowAttrs("href").OnElements("a")
|
||||
|
||||
// Force all URLs to have "noreferrer" in their rel attribute.
|
||||
p.RequireNoReferrerOnLinks(true)
|
||||
|
||||
// Links will get target="_blank" added to them.
|
||||
p.AddTargetBlankToFullyQualifiedLinks(true)
|
||||
|
||||
// Allow breaks
|
||||
p.AllowElements("br", "p")
|
||||
|
||||
// Allow img tags from the the local emoji directory only
|
||||
p.AllowAttrs("src", "alt", "class", "title").Matching(regexp.MustCompile(`(?i)/img/emoji`)).OnElements("img")
|
||||
p.AllowAttrs("class").OnElements("img")
|
||||
|
||||
// Allow bold
|
||||
p.AllowElements("strong")
|
||||
|
||||
// Allow emphasis
|
||||
p.AllowElements("em")
|
||||
|
||||
// Allow code blocks
|
||||
p.AllowElements("code")
|
||||
p.AllowElements("pre")
|
||||
|
||||
return p.Sanitize(raw)
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
package models
|
||||
|
||||
// NameChangeEvent represents a user changing their name in chat.
|
||||
type NameChangeEvent struct {
|
||||
OldName string `json:"oldName"`
|
||||
NewName string `json:"newName"`
|
||||
Image string `json:"image"`
|
||||
Type EventType `json:"type"`
|
||||
ID string `json:"id"`
|
||||
}
|
||||
@@ -1,6 +1,10 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/owncast/owncast/utils"
|
||||
)
|
||||
|
||||
// Webhook is an event that is sent to 3rd party, external services with details about something that took place within an Owncast server.
|
||||
type Webhook struct {
|
||||
@@ -25,7 +29,7 @@ var validEvents = []EventType{
|
||||
// This is not a efficient method.
|
||||
func HasValidEvents(events []EventType) bool {
|
||||
for _, event := range events {
|
||||
if !findItemInSlice(validEvents, event) {
|
||||
if _, foundInSlice := utils.FindInSlice(validEvents, event); !foundInSlice {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user