Chat refactor + persistent backing chat users (#1163)

* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>

webroot/js/components/chat/chat.js

@@ -8,13 +8,12 @@ import { CALLBACKS, SOCKET_MESSAGE_TYPES } from '../../utils/websocket.js';
 import {
   jumpToBottom,
   debounce,
-  getLocalStorage,
+  setLocalStorage,
 } from '../../utils/helpers.js';
 import { extraUserNamesFromMessageHistory } from '../../utils/chat.js';
 import {
   URL_CHAT_HISTORY,
   MESSAGE_JUMPTOBOTTOM_BUFFER,
-  KEY_CUSTOM_USERNAME_SET,
 } from '../../utils/constants.js';
 
 export default class Chat extends Component {
@@ -33,6 +32,7 @@ export default class Chat extends Component {
     this.websocket = null;
     this.receivedFirstMessages = false;
     this.receivedMessageUpdate = false;
+    this.hasFetchedHistory = false;
 
     this.windowBlurred = false;
     this.numMessagesSinceBlur = 0;
@@ -52,7 +52,6 @@ export default class Chat extends Component {
 
   componentDidMount() {
     this.setupWebSocketCallbacks();
-    this.getChatHistory();
 
     window.addEventListener('resize', this.handleWindowResize);
 
@@ -93,23 +92,25 @@ export default class Chat extends Component {
 
   componentDidUpdate(prevProps, prevState) {
     const { username: prevName } = prevProps;
-    const { username } = this.props;
+    const { username, accessToken } = this.props;
 
     const { messages: prevMessages } = prevState;
     const { messages } = this.state;
 
-    // if username updated, send a message
-    if (prevName !== username) {
-      this.sendUsernameChange(prevName, username);
-    }
-
     // scroll to bottom of messages list when new ones come in
     if (messages.length !== prevMessages.length) {
       this.setState({
         newMessagesReceived: true,
       });
     }
+
+    // Fetch chat history
+    if (!this.hasFetchedHistory && accessToken) {
+      this.hasFetchedHistory = true;
+      this.getChatHistory(accessToken);
+    }
   }
+
   componentWillUnmount() {
     window.removeEventListener('resize', this.handleWindowResize);
     if (!this.props.messagesOnly) {
@@ -138,8 +139,8 @@ export default class Chat extends Component {
   }
 
   // fetch chat history
-  getChatHistory() {
-    fetch(URL_CHAT_HISTORY)
+  getChatHistory(accessToken) {
+    fetch(URL_CHAT_HISTORY + `?accessToken=${accessToken}`)
       .then((response) => {
         if (!response.ok) {
           throw new Error(`Network response was not ok ${response.ok}`);
@@ -153,30 +154,20 @@ export default class Chat extends Component {
           messages: this.state.messages.concat(data),
           chatUserNames,
         });
+        this.scrollToBottom();
       })
       .catch((error) => {
         this.handleNetworkingError(`Fetch getChatHistory: ${error}`);
       });
   }
 
-  sendUsernameChange(oldName, newName) {
-    clearTimeout(this.sendUserJoinedEvent);
-
-    const nameChange = {
-      type: SOCKET_MESSAGE_TYPES.NAME_CHANGE,
-      oldName,
-      newName,
-    };
-    this.websocket.send(nameChange);
-  }
-
   receivedWebsocketMessage(message) {
     this.handleMessage(message);
   }
 
   handleNetworkingError(error) {
     // todo: something more useful
-    console.log(error);
+    console.error('chat error', error);
   }
 
   // handle any incoming message
@@ -247,13 +238,6 @@ export default class Chat extends Component {
     this.setState({
       webSocketConnected: true,
     });
-
-    const hasPreviouslySetCustomUsername = getLocalStorage(
-      KEY_CUSTOM_USERNAME_SET
-    );
-    if (hasPreviouslySetCustomUsername && !this.props.ignoreClient) {
-      this.sendJoinedMessage();
-    }
   }
 
   websocketDisconnected() {
@@ -269,38 +253,20 @@ export default class Chat extends Component {
     const { username } = this.props;
     const message = {
       body: content,
-      author: username,
-      type: SOCKET_MESSAGE_TYPES.CHAT,
+			type: SOCKET_MESSAGE_TYPES.CHAT,
     };
     this.websocket.send(message);
   }
 
-  sendJoinedMessage() {
-    const { username } = this.props;
-    const message = {
-      username: username,
-      type: SOCKET_MESSAGE_TYPES.USER_JOINED,
-    };
-
-    // Artificial delay so people who join and immediately
-    // leave don't get counted.
-    this.sendUserJoinedEvent = setTimeout(
-      function () {
-        this.websocket.send(message);
-      }.bind(this),
-      5000
-    );
-  }
-
   updateAuthorList(message) {
     const { type } = message;
     const nameList = this.state.chatUserNames;
 
     if (
       type === SOCKET_MESSAGE_TYPES.CHAT &&
-      !nameList.includes(message.author)
+      !nameList.includes(message.user.displayName)
     ) {
-      return nameList.push(message.author);
+      return nameList.push(message.user.displayName);
     } else if (type === SOCKET_MESSAGE_TYPES.NAME_CHANGE) {
       const { oldName, newName } = message;
       const oldNameIndex = nameList.indexOf(oldName);
@@ -373,7 +339,7 @@ export default class Chat extends Component {
   }
 
   render(props, state) {
-    const { username, messagesOnly, chatInputEnabled } = props;
+    const { username, messagesOnly, chatInputEnabled, inputMaxBytes } = props;
     const { messages, chatUserNames, webSocketConnected } = state;
 
     const messageList = messages
@@ -416,6 +382,7 @@ export default class Chat extends Component {
             chatUserNames=${chatUserNames}
             inputEnabled=${webSocketConnected && chatInputEnabled}
             handleSendMessage=${this.submitChat}
+            inputMaxBytes=${inputMaxBytes}
           />
         </div>
       </section>