Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation * Disable chat if the user is disabled/blocked or the server hits max connections * Handle dropping sockets if chat is disabled * Fix origin in automated chat test * Work for updated chat moderation * Chat message markdown rendering and fix tests * Put /api/chat behind a chat user access token. Closes #1085 * Reject blocked username changes * More WIP moderation * Defer configuring chat until we know if it is enabled. Closes #1135 * chat user blocking. Closes #1096 * Add tests around user access for #1096 * Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092 * Delete old chat messages from db as to not hold on to excessive data. Closes #1152 * Add schema migration for messages. Closes #1155 * Commit updated API documentation * Add chat load test * Shared db mutex and db optimizations * Simplify past display name handling * Use a new test db for each test run * Wire up the external messages actions + add tests for them * Move access tokens to be actual users * Run message pruning at launch + fix comparison * Do not return API users in disabled users response * Fix incorrect highlighting. Closes #1160 * Consolidate user table statements * Set the max process connection limit to 70% of maximum * Fix wrong old display name being returned in name change event * Delete the old chat server files * Wire back up the webhooks * Remove unused * Invalidate user cache on changes * Do not send rendered body as RawBody * Some cleanup * Standardize names for external API users to ExternalAPIUser * Do not log token * Checkout branch when building admin for testing * Bundle in dev admin for testing * Some cleanup * Cleanup js logs * Cleanup and standardize event names * Clean up some logging * Update API spec. Closes #1133 * Commit updated API documentation * Change paths to be better named * Commit updated API documentation * Update admin bundle * Fix duplicate event name * Rename scope var * Update admin bundle * Move connected clients controller into admin package * Fix collecting usernames for autocomplete purposes * No longer generate username when it is empty * Sort clients and users by timestamp * Move file to admin controller package * Swap, so the comments stay correct Co-authored-by: Jannik <jannik@outlook.com> * Use explicit type alias Co-authored-by: Jannik <jannik@outlook.com> * Remove commented code. Co-authored-by: Jannik <jannik@outlook.com> * Cleanup test * Remove some extra logging * Add some clarity * Update dev instance of admin for testing * Consolidate lines Co-authored-by: Jannik <jannik@outlook.com> * Remove commented unused vars Co-authored-by: Jannik <jannik@outlook.com> * Until needed do not return IP address with client list * Fix typo of wrong var * Typo led to a bad test. Fix typo and fix test. * Guard against the socket reconnecting on error if previously set to shutdown * Do not log access tokens * Return success message on enable/disable user * Clean up some inactionable error messages. Sent ban message. Sort banned users. * fix styling for when chat is completely disabled * Unused * guard against nil clients * Update dev admin bundle * Do not unhide messages when unblocking user just to be safe. Send removal action from the controller * Add convinience function for getting active connections for a single user * Lock db on these mutations * Cleanup force disconnect using GetClientsForUser and capture client reference explicitly * No longer re-showing banned user messages for safety. Removing this test. * Remove no longer needed comment * Tweaks to forbidden username handling. - Standardize naming to not use "block" but "forbidden" instead. - Pass array over the wire instead of string. - Add API test - Fix default list incorrectly being appended to custom list. * Logging cleanup * Update dev admin bundle * Add an artificial delay in order to visually see message being hidden when testing * Remove the user cache as it is a premature optimization * When connected to chat let the user know their current user details to sync the username in the UI * On connected send current display name back to client. - Move name change out of chat component. - Add additional event type constants. * Fix broken workflow due to typo * Troubleshoot workflow * Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181) * Bump htm from 3.0.4 to 3.1.0 in /build/javascript Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0. - [Release notes](https://github.com/developit/htm/releases) - [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0) --- updated-dependencies: - dependency-name: htm dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Run npm run build and update libraries Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gabe Kangas <gabek@real-ity.com> * Commit updated Javascript packages * Re-send current user info when a rejected name change takes place * All socket writes should be through the send chan and not directly * Seed the random generator * Add keys and indexes to users table * a util to generate consistent emoji markup * console clean up * mod tidy * Commit updated API documentation * Handle the max payload size of a socket message. - Only close socket if x2 greater than the max size. - Send the user a message if a message is too large. - Surface the max size in bytes in the config. * Update admin bundle * Force all events to be sent in their own socket message and do not concatinate in a single message * Update chat embed to register for access token * Use a different access token for embed chat * Update the chat message bubble background color to be bolder * add base tag to open links in new window, closes #1220 * Support text input of :emoji: in chat (#1190) * Initial implementation of emoji injection * fix bookkeeping with multiple emoji * make the emoji lookup case-insensitive * try another solution for Caretposition * add title to emojis minor refactoring * bind moji injection to InputKeyUp * simplify the code replace all found emojis * inject emoji if the modifer is released earlier * more efficient emoji tag search * use json emoji.emoji as url * use createEmojiMarkup() * move emojify() to chat.js * emojify on paste * cleanup emoji titles in paste * update inputText in InputKeyup * mark emoji titles with 2*zwnj this way paste cleanup will not interfere with text which include zwnj * emoji should not change the inputText * Do not show join messages when chat is offline. Closes #1224 - Show stream starting/ending messages in chat. - When stream starts show everyone the welcome message. * Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222 * use maxSocketPayloadSize to calculate total bytes of message payload (#1221) * utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting * add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count Co-authored-by: Gabe Kangas <gabek@real-ity.com> Co-authored-by: Owncast <owncast@owncast.online> Co-authored-by: Jannik <jannik@outlook.com> Co-authored-by: Ginger Wong <omqmail@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
This commit is contained in:
Gabe Kangas
@@ -1,191 +1,317 @@
|
||||
package chat
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
"golang.org/x/net/websocket"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
|
||||
"github.com/owncast/owncast/core/chat/events"
|
||||
"github.com/owncast/owncast/core/data"
|
||||
"github.com/owncast/owncast/core/user"
|
||||
"github.com/owncast/owncast/core/webhooks"
|
||||
"github.com/owncast/owncast/models"
|
||||
"github.com/owncast/owncast/utils"
|
||||
)
|
||||
|
||||
var (
|
||||
_server *server
|
||||
)
|
||||
var _server *ChatServer
|
||||
|
||||
var l = &sync.RWMutex{}
|
||||
type ChatServer struct {
|
||||
mu sync.RWMutex
|
||||
seq uint
|
||||
clients map[uint]*ChatClient
|
||||
maxClientCount uint
|
||||
|
||||
// Server represents the server which handles the chat.
|
||||
type server struct {
|
||||
Clients map[string]*Client
|
||||
// send outbound message payload to all clients
|
||||
outbound chan []byte
|
||||
|
||||
pattern string
|
||||
listener models.ChatListener
|
||||
// receive inbound message payload from all clients
|
||||
inbound chan chatClientEvent
|
||||
|
||||
addCh chan *Client
|
||||
delCh chan *Client
|
||||
sendAllCh chan models.ChatEvent
|
||||
pingCh chan models.PingMessage
|
||||
doneCh chan bool
|
||||
errCh chan error
|
||||
// unregister requests from clients.
|
||||
unregister chan *ChatClient
|
||||
}
|
||||
|
||||
// Add adds a client to the server.
|
||||
func (s *server) add(c *Client) {
|
||||
s.addCh <- c
|
||||
}
|
||||
|
||||
// Remove removes a client from the server.
|
||||
func (s *server) remove(c *Client) {
|
||||
s.delCh <- c
|
||||
}
|
||||
|
||||
// SendToAll sends a message to all of the connected clients.
|
||||
func (s *server) SendToAll(msg models.ChatEvent) {
|
||||
s.sendAllCh <- msg
|
||||
}
|
||||
|
||||
// Err handles an error.
|
||||
func (s *server) err(err error) {
|
||||
s.errCh <- err
|
||||
}
|
||||
|
||||
func (s *server) sendAll(msg models.ChatEvent) {
|
||||
l.RLock()
|
||||
for _, c := range s.Clients {
|
||||
c.write(msg)
|
||||
func NewChat() *ChatServer {
|
||||
server := &ChatServer{
|
||||
clients: map[uint]*ChatClient{},
|
||||
outbound: make(chan []byte),
|
||||
inbound: make(chan chatClientEvent),
|
||||
unregister: make(chan *ChatClient),
|
||||
maxClientCount: handleMaxConnectionCount(),
|
||||
}
|
||||
l.RUnlock()
|
||||
|
||||
return server
|
||||
}
|
||||
|
||||
func (s *server) ping() {
|
||||
ping := models.PingMessage{MessageType: models.PING}
|
||||
|
||||
l.RLock()
|
||||
for _, c := range s.Clients {
|
||||
c.pingch <- ping
|
||||
}
|
||||
l.RUnlock()
|
||||
}
|
||||
|
||||
func (s *server) usernameChanged(msg models.NameChangeEvent) {
|
||||
l.RLock()
|
||||
for _, c := range s.Clients {
|
||||
c.usernameChangeChannel <- msg
|
||||
}
|
||||
l.RUnlock()
|
||||
|
||||
go webhooks.SendChatEventUsernameChanged(msg)
|
||||
}
|
||||
|
||||
func (s *server) userJoined(msg models.UserJoinedEvent) {
|
||||
l.RLock()
|
||||
if s.listener.IsStreamConnected() {
|
||||
for _, c := range s.Clients {
|
||||
c.userJoinedChannel <- msg
|
||||
}
|
||||
}
|
||||
l.RUnlock()
|
||||
|
||||
go webhooks.SendChatEventUserJoined(msg)
|
||||
}
|
||||
|
||||
func (s *server) onConnection(ws *websocket.Conn) {
|
||||
client := NewClient(ws)
|
||||
|
||||
defer func() {
|
||||
s.removeClient(client)
|
||||
|
||||
if err := ws.Close(); err != nil {
|
||||
log.Debugln(err)
|
||||
//s.errCh <- err
|
||||
}
|
||||
}()
|
||||
|
||||
s.add(client)
|
||||
client.listen()
|
||||
}
|
||||
|
||||
// Listen and serve.
|
||||
// It serves client connection and broadcast request.
|
||||
func (s *server) Listen() {
|
||||
http.Handle(s.pattern, websocket.Handler(s.onConnection))
|
||||
|
||||
log.Tracef("Starting the websocket listener on: %s", s.pattern)
|
||||
|
||||
func (s *ChatServer) Run() {
|
||||
for {
|
||||
select {
|
||||
// add new a client
|
||||
case c := <-s.addCh:
|
||||
l.Lock()
|
||||
s.Clients[c.socketID] = c
|
||||
|
||||
if !c.Ignore {
|
||||
s.listener.ClientAdded(c.GetViewerClientFromChatClient())
|
||||
s.sendWelcomeMessageToClient(c)
|
||||
}
|
||||
l.Unlock()
|
||||
|
||||
// remove a client
|
||||
case c := <-s.delCh:
|
||||
s.removeClient(c)
|
||||
case msg := <-s.sendAllCh:
|
||||
if data.GetChatDisabled() {
|
||||
break
|
||||
case client := <-s.unregister:
|
||||
if _, ok := s.clients[client.id]; ok {
|
||||
s.mu.Lock()
|
||||
delete(s.clients, client.id)
|
||||
close(client.send)
|
||||
s.mu.Unlock()
|
||||
}
|
||||
|
||||
if !msg.Empty() {
|
||||
// set defaults before sending msg to anywhere
|
||||
msg.SetDefaults()
|
||||
|
||||
s.listener.MessageSent(msg)
|
||||
s.sendAll(msg)
|
||||
|
||||
// Store in the message history
|
||||
if !msg.Ephemeral {
|
||||
addMessage(msg)
|
||||
}
|
||||
|
||||
// Send webhooks
|
||||
go webhooks.SendChatEvent(msg)
|
||||
}
|
||||
case ping := <-s.pingCh:
|
||||
fmt.Println("PING?", ping)
|
||||
|
||||
case err := <-s.errCh:
|
||||
log.Trace("Error: ", err.Error())
|
||||
|
||||
case <-s.doneCh:
|
||||
return
|
||||
case message := <-s.inbound:
|
||||
s.eventReceived(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *server) removeClient(c *Client) {
|
||||
l.Lock()
|
||||
if _, ok := s.Clients[c.socketID]; ok {
|
||||
delete(s.Clients, c.socketID)
|
||||
|
||||
s.listener.ClientRemoved(c.socketID)
|
||||
log.Tracef("The client was connected for %s and sent %d messages (%s)", time.Since(c.ConnectedAt), c.MessageCount, c.ClientID)
|
||||
// Addclient registers new connection as a User.
|
||||
func (s *ChatServer) Addclient(conn *websocket.Conn, user *user.User, accessToken string, userAgent string) *ChatClient {
|
||||
client := &ChatClient{
|
||||
server: s,
|
||||
conn: conn,
|
||||
User: user,
|
||||
ipAddress: conn.RemoteAddr().String(),
|
||||
accessToken: accessToken,
|
||||
send: make(chan []byte, 256),
|
||||
UserAgent: userAgent,
|
||||
ConnectedAt: time.Now(),
|
||||
}
|
||||
l.Unlock()
|
||||
|
||||
s.mu.Lock()
|
||||
{
|
||||
client.id = s.seq
|
||||
s.clients[client.id] = client
|
||||
s.seq++
|
||||
}
|
||||
s.mu.Unlock()
|
||||
|
||||
log.Traceln("Adding client", client.id, "total count:", len(s.clients))
|
||||
|
||||
go client.writePump()
|
||||
go client.readPump()
|
||||
|
||||
client.sendConnectedClientInfo()
|
||||
|
||||
if getStatus().Online {
|
||||
s.sendUserJoinedMessage(client)
|
||||
s.sendWelcomeMessageToClient(client)
|
||||
}
|
||||
|
||||
return client
|
||||
}
|
||||
|
||||
func (s *server) sendWelcomeMessageToClient(c *Client) {
|
||||
go func() {
|
||||
// Add an artificial delay so people notice this message come in.
|
||||
time.Sleep(7 * time.Second)
|
||||
func (s *ChatServer) sendUserJoinedMessage(c *ChatClient) {
|
||||
userJoinedEvent := events.UserJoinedEvent{}
|
||||
userJoinedEvent.SetDefaults()
|
||||
userJoinedEvent.User = c.User
|
||||
|
||||
welcomeMessage := data.GetServerWelcomeMessage()
|
||||
if welcomeMessage != "" {
|
||||
initialMessage := models.ChatEvent{ClientID: "owncast-server", Author: data.GetServerName(), Body: welcomeMessage, ID: "initial-message-1", MessageType: "SYSTEM", Visible: true, Timestamp: time.Now()}
|
||||
c.write(initialMessage)
|
||||
if err := s.Broadcast(userJoinedEvent.GetBroadcastPayload()); err != nil {
|
||||
log.Errorln("error adding client to chat server", err)
|
||||
}
|
||||
|
||||
// Send chat user joined webhook
|
||||
webhooks.SendChatEventUserJoined(userJoinedEvent)
|
||||
}
|
||||
|
||||
func (s *ChatServer) ClientClosed(c *ChatClient) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
c.close()
|
||||
|
||||
if _, ok := s.clients[c.id]; ok {
|
||||
log.Debugln("Deleting", c.id)
|
||||
delete(s.clients, c.id)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *ChatServer) HandleClientConnection(w http.ResponseWriter, r *http.Request) {
|
||||
if data.GetChatDisabled() {
|
||||
_, _ = w.Write([]byte(events.ChatDisabled))
|
||||
return
|
||||
}
|
||||
|
||||
// Limit concurrent chat connections
|
||||
if uint(len(s.clients)) >= s.maxClientCount {
|
||||
log.Warnln("rejecting incoming client connection as it exceeds the max client count of", s.maxClientCount)
|
||||
_, _ = w.Write([]byte(events.ErrorMaxConnectionsExceeded))
|
||||
return
|
||||
}
|
||||
|
||||
conn, err := upgrader.Upgrade(w, r, nil)
|
||||
if err != nil {
|
||||
log.Debugln(err)
|
||||
return
|
||||
}
|
||||
|
||||
accessToken := r.URL.Query().Get("accessToken")
|
||||
if accessToken == "" {
|
||||
log.Errorln("Access token is required")
|
||||
// Return HTTP status code
|
||||
conn.Close()
|
||||
return
|
||||
}
|
||||
|
||||
// A user is required to use the websocket
|
||||
user := user.GetUserByToken(accessToken)
|
||||
if user == nil {
|
||||
_ = conn.WriteJSON(events.EventPayload{
|
||||
"type": events.ErrorNeedsRegistration,
|
||||
})
|
||||
// Send error that registration is required
|
||||
conn.Close()
|
||||
return
|
||||
}
|
||||
|
||||
// User is disabled therefore we should disconnect.
|
||||
if user.DisabledAt != nil {
|
||||
log.Traceln("Disabled user", user.Id, user.DisplayName, "rejected")
|
||||
_ = conn.WriteJSON(events.EventPayload{
|
||||
"type": events.ErrorUserDisabled,
|
||||
})
|
||||
conn.Close()
|
||||
return
|
||||
}
|
||||
|
||||
userAgent := r.UserAgent()
|
||||
|
||||
s.Addclient(conn, user, accessToken, userAgent)
|
||||
}
|
||||
|
||||
// Broadcast sends message to all connected clients.
|
||||
func (s *ChatServer) Broadcast(payload events.EventPayload) error {
|
||||
data, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
for _, client := range s.clients {
|
||||
if client == nil {
|
||||
continue
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case client.send <- data:
|
||||
default:
|
||||
close(client.send)
|
||||
delete(s.clients, client.id)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *ChatServer) Send(payload events.EventPayload, client *ChatClient) {
|
||||
data, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
log.Errorln(err)
|
||||
return
|
||||
}
|
||||
|
||||
client.send <- data
|
||||
}
|
||||
|
||||
// DisconnectUser will forcefully disconnect all clients belonging to a user by ID.
|
||||
func (s *ChatServer) DisconnectUser(userID string) {
|
||||
s.mu.Lock()
|
||||
clients, err := GetClientsForUser(userID)
|
||||
s.mu.Unlock()
|
||||
|
||||
if err != nil || clients == nil || len(clients) == 0 {
|
||||
log.Debugln("Requested to disconnect user", userID, err)
|
||||
return
|
||||
}
|
||||
|
||||
for _, client := range clients {
|
||||
log.Traceln("Disconnecting client", client.User.Id, "owned by", client.User.DisplayName)
|
||||
|
||||
go func(client *ChatClient) {
|
||||
event := events.UserDisabledEvent{}
|
||||
event.SetDefaults()
|
||||
|
||||
// Send this disabled event specifically to this single connected client
|
||||
// to let them know they've been banned.
|
||||
_server.Send(event.GetBroadcastPayload(), client)
|
||||
|
||||
// Give the socket time to send out the above message.
|
||||
// Unfortunately I don't know of any way to get a real callback to know when
|
||||
// the message was successfully sent, so give it a couple seconds.
|
||||
time.Sleep(2 * time.Second)
|
||||
|
||||
// Forcefully disconnect if still valid.
|
||||
if client != nil {
|
||||
client.close()
|
||||
}
|
||||
}(client)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *ChatServer) eventReceived(event chatClientEvent) {
|
||||
var typecheck map[string]interface{}
|
||||
if err := json.Unmarshal(event.data, &typecheck); err != nil {
|
||||
log.Debugln(err)
|
||||
}
|
||||
|
||||
eventType := typecheck["type"]
|
||||
|
||||
switch eventType {
|
||||
case events.MessageSent:
|
||||
s.userMessageSent(event)
|
||||
|
||||
case events.UserNameChanged:
|
||||
s.userNameChanged(event)
|
||||
|
||||
default:
|
||||
log.Debugln(eventType, "event not found:", typecheck)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *ChatServer) sendWelcomeMessageToClient(c *ChatClient) {
|
||||
// Add an artificial delay so people notice this message come in.
|
||||
time.Sleep(7 * time.Second)
|
||||
|
||||
welcomeMessage := utils.RenderSimpleMarkdown(data.GetServerWelcomeMessage())
|
||||
|
||||
if welcomeMessage != "" {
|
||||
s.sendSystemMessageToClient(c, welcomeMessage)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *ChatServer) sendAllWelcomeMessage() {
|
||||
welcomeMessage := utils.RenderSimpleMarkdown(data.GetServerWelcomeMessage())
|
||||
|
||||
if welcomeMessage != "" {
|
||||
clientMessage := events.SystemMessageEvent{
|
||||
Event: events.Event{},
|
||||
MessageEvent: events.MessageEvent{
|
||||
Body: welcomeMessage,
|
||||
},
|
||||
}
|
||||
clientMessage.SetDefaults()
|
||||
_ = s.Broadcast(clientMessage.GetBroadcastPayload())
|
||||
}
|
||||
}
|
||||
|
||||
func (s *ChatServer) sendSystemMessageToClient(c *ChatClient, message string) {
|
||||
clientMessage := events.SystemMessageEvent{
|
||||
Event: events.Event{},
|
||||
MessageEvent: events.MessageEvent{
|
||||
Body: message,
|
||||
},
|
||||
}
|
||||
clientMessage.SetDefaults()
|
||||
s.Send(clientMessage.GetBroadcastPayload(), c)
|
||||
}
|
||||
|
||||
func (s *ChatServer) sendActionToClient(c *ChatClient, message string) {
|
||||
clientMessage := events.ActionEvent{
|
||||
MessageEvent: events.MessageEvent{
|
||||
Body: message,
|
||||
},
|
||||
}
|
||||
clientMessage.SetDefaults()
|
||||
clientMessage.RenderBody()
|
||||
s.Send(clientMessage.GetBroadcastPayload(), c)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user