LogalDeveloper/owncast
0
Fork 0
Code Activity

Chat refactor + persistent backing chat users (#1163)

Browse Source 
* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
This commit is contained in:
Gabe Kangas
2021-07-19 19:22:29 -07:00
committed by GitHub
parent e3dc736cf4
commit b6f68628c0
88 changed files with 10691 additions and 2281 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
test/automated/chat.test.js
View File

@@ -2,52 +2,39 @@ const { test } = require('@jest/globals');
var request = require('supertest');
request = request('http://127.0.0.1:8080');
const WebSocket = require('ws');
var ws;
const registerChat = require('./lib/chat').registerChat;
const sendChatMessage = require('./lib/chat').sendChatMessage;
const testMessageId = Math.random().toString(36).substring(7);
const username = 'user' + Math.floor(Math.random() * 100);
var userDisplayName;
const message = Math.floor(Math.random() * 100) + ' test 123';
const messageRaw = message + ' *and some markdown too*';
const messageMarkdown = '<p>' + message + ' <em>and some markdown too</em></p>'
const date = new Date().toISOString();
const testMessage = {
author: username,
body: messageRaw,
id: testMessageId,
type: 'CHAT',
visible: true,
timestamp: date,
body: message,
type: 'CHAT',
};
test('can send a chat message', (done) => {
ws = new WebSocket('ws://127.0.0.1:8080/entry', {
origin: 'http://localhost',
test('can send a chat message', async (done) => {
const registration = await registerChat();
const accessToken = registration.accessToken;
userDisplayName = registration.displayName;
sendChatMessage(testMessage, accessToken, done);
});
function onOpen() {
ws.send(JSON.stringify(testMessage), function() {
ws.close();
done();
});
}
test('can fetch chat messages', async (done) => {
const res = await request
.get('/api/admin/chat/messages')
.auth('admin', 'abc123')
.expect(200);
ws.on('open', onOpen);
});
test('can fetch chat messages', (done) => {
request.get('/api/admin/chat/messages').auth('admin', 'abc123').expect(200)
.then((res) => {
const message = res.body.filter(function(msg) {
return msg.id = testMessageId;
})[0];
expect(message.author).toBe(testMessage.author);
expect(message.body).toBe(messageMarkdown);
expect(message.date).toBe(testMessage.date);
expect(message.type).toBe(testMessage.type);
done();
});
const expectedBody = `<p>${testMessage.body}</p>`
const message = res.body.filter(function (msg) {
return msg.body === expectedBody
})[0];
expect(message.body).toBe(expectedBody);
expect(message.user.displayName).toBe(userDisplayName);
expect(message.type).toBe(testMessage.type);
done();
});

36
test/automated/chatmoderation.test.js
View File

@@ -2,38 +2,26 @@ const { test } = require('@jest/globals');
var request = require('supertest');
request = request('http://127.0.0.1:8080');
const WebSocket = require('ws');
var ws;
const registerChat = require('./lib/chat').registerChat;
const sendChatMessage = require('./lib/chat').sendChatMessage;
const testVisibilityMessage = {
author: "username",
body: "message " + Math.floor(Math.random() * 100),
type: 'CHAT',
visible: true,
timestamp: new Date().toISOString()
};
test('can send a chat message', (done) => {
ws = new WebSocket('ws://127.0.0.1:8080/entry', {
origin: 'http://localhost',
});
function onOpen() {
ws.send(JSON.stringify(testVisibilityMessage), function () {
ws.close();
done();
});
}
ws.on('open', onOpen);
});
var messageId;
test('can send a chat message', async (done) => {
const registration = await registerChat();
const accessToken = registration.accessToken;
sendChatMessage(testVisibilityMessage, accessToken, done);
});
test('verify we can make API call to mark message as hidden', async (done) => {
const res = await request.get('/api/admin/chat/messages').auth('admin', 'abc123').expect(200)
const message = res.body[0];
messageId = message.id;
const messageId = message.id;
await request.post('/api/admin/chat/updatemessagevisibility')
.auth('admin', 'abc123')
.send({ "idArray": [messageId], "visible": false }).expect(200);
@@ -46,9 +34,9 @@ test('verify message has become hidden', async (done) => {
.auth('admin', 'abc123')
const message = res.body.filter(obj => {
return obj.id === messageId;
return obj.body === `<p>${testVisibilityMessage.body}</p>`;
});
expect(message.length).toBe(1);
expect(message[0].visible).toBe(false);
expect(message[0].hiddenAt).toBeTruthy();
done();
});

66
test/automated/chatusers.test.js Normal file
View File

@@ -0,0 +1,66 @@
const { test } = require('@jest/globals');
var request = require('supertest');
request = request('http://127.0.0.1:8080');
const registerChat = require('./lib/chat').registerChat;
const sendChatMessage = require('./lib/chat').sendChatMessage;
const testVisibilityMessage = {
body: "message " + Math.floor(Math.random() * 100),
type: 'CHAT',
};
var userId
var accessToken
test('can register a user', async (done) => {
const registration = await registerChat();
userId = registration.id;
accessToken = registration.accessToken;
done();
});
test('can send a chat message', async (done) => {
sendChatMessage(testVisibilityMessage, accessToken, done);
});
test('can disable a user', async (done) => {
// To allow for visually being able to see the test hiding the
// message add a short delay.
await new Promise((r) => setTimeout(r, 1500));
await request.post('/api/admin/chat/users/setenabled').send({ "userId": userId, "enabled": false })
.auth('admin', 'abc123').expect(200);
done();
});
test('verify user is disabled', async (done) => {
const response = await request.get('/api/admin/chat/users/disabled').auth('admin', 'abc123').expect(200);
const tokenCheck = response.body.filter((user) => user.id === userId)
expect(tokenCheck).toHaveLength(1);
done();
});
test('verify messages from user are hidden', async (done) => {
const response = await request.get('/api/admin/chat/messages')
.auth('admin', 'abc123')
.expect(200);
const message = response.body.filter(obj => {
return obj.user.id === userId;
});
expect(message[0].hiddenAt).toBeTruthy();
done();
});
test('can re-enable a user', async (done) => {
await request.post('/api/admin/chat/users/setenabled').send({ "userId": userId, "enabled": true })
.auth('admin', 'abc123').expect(200);
done();
});
test('verify user is enabled', async (done) => {
const response = await request.get('/api/admin/chat/users/disabled').auth('admin', 'abc123').expect(200);
const tokenCheck = response.body.filter((user) => user.id === userId)
expect(tokenCheck).toHaveLength(0);
done();
});

8
test/automated/configmanagement.test.js
View File

@@ -31,6 +31,8 @@ const s3Config = {
region: randomString(),
};
const forbiddenUsernames = [randomString(), randomString(), randomString()];
test('set server name', async (done) => {
const res = await sendConfigChangeRequest('name', serverName);
done();
@@ -81,6 +83,11 @@ test('set s3 configuration', async (done) => {
done();
});
test('set forbidden usernames', async (done) => {
const res = await sendConfigChangeRequest('chat/forbiddenusernames', forbiddenUsernames);
done();
});
test('verify updated config values', async (done) => {
const res = await request.get('/api/config');
expect(res.body.name).toBe(serverName);
@@ -122,6 +129,7 @@ test('admin configuration is correct', (done) => {
expect(res.body.instanceDetails.socialHandles).toStrictEqual(
socialHandles
);
expect(res.body.forbiddenUsernames).toStrictEqual(forbiddenUsernames);
expect(res.body.videoSettings.latencyLevel).toBe(latencyLevel);
expect(res.body.videoSettings.videoQualityVariants[0].framerate).toBe(

53
test/automated/integrations.test.js
View File

@@ -49,39 +49,53 @@ test('check that webhook was deleted', (done) => {
});
test('create access token', async (done) => {
const name = 'test token';
const scopes = ['CAN_SEND_SYSTEM_MESSAGES'];
const name = 'Automated integration test';
const scopes = ['CAN_SEND_SYSTEM_MESSAGES', 'CAN_SEND_MESSAGES'];
const res = await sendIntegrationsChangePayload('accesstokens/create', {
name: name,
scopes: scopes,
});
expect(res.body.token).toBeTruthy();
expect(res.body.timestamp).toBeTruthy();
expect(res.body.name).toBe(name);
expect(res.body.accessToken).toBeTruthy();
expect(res.body.createdAt).toBeTruthy();
expect(res.body.displayName).toBe(name);
expect(res.body.scopes).toStrictEqual(scopes);
accessToken = res.body.token;
accessToken = res.body.accessToken;
done();
});
test('check access tokens', (done) => {
request.get('/api/admin/accesstokens')
test('check access tokens', async (done) => {
const res = await request.get('/api/admin/accesstokens')
.auth('admin', 'abc123').expect(200)
.then((res) => {
expect(res.body).toHaveLength(1);
expect(res.body[0].token).toBe(accessToken);
done();
});
const tokenCheck = res.body.filter((token) => token.accessToken === accessToken)
expect(tokenCheck).toHaveLength(1);
done();
});
test('send a system message using access token', async (done) => {
const payload = {body: 'test 1234'};
const payload = {body: 'This is a test system message from the automated integration test'};
const res = await request.post('/api/integrations/chat/system')
.set('Authorization', 'Bearer ' + accessToken)
.send(payload).expect(200);
done();
});
test('send an external integration message using access token', async (done) => {
const payload = {body: 'This is a test external message from the automated integration test'};
const res = await request.post('/api/integrations/chat/send')
.set('Authorization', 'Bearer ' + accessToken)
.send(payload).expect(200);
done();
});
test('send an external integration action using access token', async (done) => {
const payload = {body: 'This is a test external action from the automated integration test'};
const res = await request.post('/api/integrations/chat/action')
.set('Authorization', 'Bearer ' + accessToken)
.send(payload).expect(200);
done();
});
test('delete access token', async (done) => {
const res = await sendIntegrationsChangePayload('accesstokens/delete', {
token: accessToken,
@@ -90,13 +104,12 @@ test('delete access token', async (done) => {
done();
});
test('check token delete was successful', (done) => {
request.get('/api/admin/accesstokens')
test('check token delete was successful', async (done) => {
const res = await request.get('/api/admin/accesstokens')
.auth('admin', 'abc123').expect(200)
.then((res) => {
expect(res.body).toHaveLength(0);
done();
});
const tokenCheck = res.body.filter((token) => token.accessToken === accessToken)
expect(tokenCheck).toHaveLength(0);
done();
});
async function sendIntegrationsChangePayload(endpoint, payload) {

33
test/automated/lib/chat.js Normal file
View File

@@ -0,0 +1,33 @@
var request = require('supertest');
request = request('http://127.0.0.1:8080');
const WebSocket = require('ws');
async function registerChat() {
try {
const response = await request.post('/api/chat/register');
return response.body;
} catch (e) {
console.error(e);
}
}
function sendChatMessage(message, accessToken, done) {
const ws = new WebSocket(
`ws://localhost:8080/ws?accessToken=${accessToken}`,
{
origin: 'http://localhost:8080',
}
);
function onOpen() {
ws.send(JSON.stringify(message), function () {
ws.close();
done();
});
}
ws.on('open', onOpen);
}
module.exports.sendChatMessage = sendChatMessage;
module.exports.registerChat = registerChat;

6314
test/automated/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff