0
Gabe Kangas b6f68628c0
Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
2021-07-19 19:22:29 -07:00

634 lines
17 KiB
Go

package admin
import (
"encoding/base64"
"encoding/json"
"fmt"
"io/ioutil"
"net"
"net/http"
"path/filepath"
"reflect"
"strings"
"github.com/owncast/owncast/controllers"
"github.com/owncast/owncast/core/chat"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/user"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
)
// ConfigValue is a container object that holds a value, is encoded, and saved to the database.
type ConfigValue struct {
Value interface{} `json:"value"`
}
// SetTags will handle the web config request to set tags.
func SetTags(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValues, success := getValuesFromRequest(w, r)
if !success {
return
}
tagStrings := make([]string, 0)
for _, tag := range configValues {
tagStrings = append(tagStrings, tag.Value.(string))
}
if err := data.SetServerMetadataTags(tagStrings); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetStreamTitle will handle the web config request to set the current stream title.
func SetStreamTitle(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
value := configValue.Value.(string)
if err := data.SetStreamTitle(value); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
if value != "" {
sendSystemChatAction(fmt.Sprintf("Stream title changed to **%s**", value), true)
}
controllers.WriteSimpleResponse(w, true, "changed")
}
func ExternalSetStreamTitle(integration user.ExternalAPIUser, w http.ResponseWriter, r *http.Request) {
SetStreamTitle(w, r)
}
func sendSystemChatAction(messageText string, ephemeral bool) {
if err := chat.SendSystemAction(messageText, ephemeral); err != nil {
log.Errorln(err)
}
}
// SetServerName will handle the web config request to set the server's name.
func SetServerName(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetServerName(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetServerSummary will handle the web config request to set the about/summary text.
func SetServerSummary(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetServerSummary(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetServerWelcomeMessage will handle the web config request to set the welcome message text.
func SetServerWelcomeMessage(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetServerWelcomeMessage(strings.TrimSpace(configValue.Value.(string))); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetExtraPageContent will handle the web config request to set the page markdown content.
func SetExtraPageContent(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetExtraPageBodyContent(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetStreamKey will handle the web config request to set the server stream key.
func SetStreamKey(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetStreamKey(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetLogo will handle a new logo image file being uploaded.
func SetLogo(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
s := strings.SplitN(configValue.Value.(string), ",", 2)
if len(s) < 2 {
controllers.WriteSimpleResponse(w, false, "Error splitting base64 image data.")
return
}
bytes, err := base64.StdEncoding.DecodeString(s[1])
if err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
splitHeader := strings.Split(s[0], ":")
if len(splitHeader) < 2 {
controllers.WriteSimpleResponse(w, false, "Error splitting base64 image header.")
return
}
contentType := strings.Split(splitHeader[1], ";")[0]
extension := ""
if contentType == "image/svg+xml" {
extension = ".svg"
} else if contentType == "image/gif" {
extension = ".gif"
} else if contentType == "image/png" {
extension = ".png"
} else if contentType == "image/jpeg" {
extension = ".jpeg"
}
if extension == "" {
controllers.WriteSimpleResponse(w, false, "Missing or invalid contentType in base64 image.")
return
}
imgPath := filepath.Join("data", "logo"+extension)
if err := ioutil.WriteFile(imgPath, bytes, 0600); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
if err := data.SetLogoPath("logo" + extension); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetNSFW will handle the web config request to set the NSFW flag.
func SetNSFW(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetNSFW(configValue.Value.(bool)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetFfmpegPath will handle the web config request to validate and set an updated copy of ffmpg.
func SetFfmpegPath(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
path := configValue.Value.(string)
if err := utils.VerifyFFMpegPath(path); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
if err := data.SetFfmpegPath(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "changed")
}
// SetWebServerPort will handle the web config request to set the server's HTTP port.
func SetWebServerPort(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if port, ok := configValue.Value.(float64); ok {
if (port < 1) || (port > 65535) {
controllers.WriteSimpleResponse(w, false, "Port number must be between 1 and 65535")
return
}
if err := data.SetHTTPPortNumber(port); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
} else {
controllers.WriteSimpleResponse(w, true, "HTTP port set")
return
}
}
controllers.WriteSimpleResponse(w, false, "Invalid type or value, port must be a number")
}
// SetWebServerIP will handle the web config request to set the server's HTTP listen address.
func SetWebServerIP(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if input, ok := configValue.Value.(string); ok {
if ip := net.ParseIP(input); ip != nil {
if err := data.SetHTTPListenAddress(ip.String()); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
} else {
controllers.WriteSimpleResponse(w, true, "HTTP listen address set")
return
}
} else {
controllers.WriteSimpleResponse(w, false, "Invalid IP address")
return
}
}
controllers.WriteSimpleResponse(w, false, "Invalid type or value, IP address must be a string")
}
// SetRTMPServerPort will handle the web config request to set the inbound RTMP port.
func SetRTMPServerPort(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetRTMPPortNumber(configValue.Value.(float64)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "rtmp port set")
}
// SetServerURL will handle the web config request to set the full server URL.
func SetServerURL(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetServerURL(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "server url set")
}
// SetDirectoryEnabled will handle the web config request to enable or disable directory registration.
func SetDirectoryEnabled(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetDirectoryEnabled(configValue.Value.(bool)); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "directory state changed")
}
// SetStreamLatencyLevel will handle the web config request to set the stream latency level.
func SetStreamLatencyLevel(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
return
}
if err := data.SetStreamLatencyLevel(configValue.Value.(float64)); err != nil {
controllers.WriteSimpleResponse(w, false, "error setting stream latency "+err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "set stream latency")
}
// SetS3Configuration will handle the web config request to set the storage configuration.
func SetS3Configuration(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
type s3ConfigurationRequest struct {
Value models.S3 `json:"value"`
}
decoder := json.NewDecoder(r.Body)
var newS3Config s3ConfigurationRequest
if err := decoder.Decode(&newS3Config); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update s3 config with provided values")
return
}
if newS3Config.Value.Enabled {
if newS3Config.Value.Endpoint == "" || !utils.IsValidUrl((newS3Config.Value.Endpoint)) {
controllers.WriteSimpleResponse(w, false, "s3 support requires an endpoint")
return
}
if newS3Config.Value.AccessKey == "" || newS3Config.Value.Secret == "" {
controllers.WriteSimpleResponse(w, false, "s3 support requires an access key and secret")
return
}
if newS3Config.Value.Region == "" {
controllers.WriteSimpleResponse(w, false, "s3 support requires a region and endpoint")
return
}
if newS3Config.Value.Bucket == "" {
controllers.WriteSimpleResponse(w, false, "s3 support requires a bucket created for storing public video segments")
return
}
}
data.SetS3Config(newS3Config.Value)
controllers.WriteSimpleResponse(w, true, "storage configuration changed")
}
// SetStreamOutputVariants will handle the web config request to set the video output stream variants.
func SetStreamOutputVariants(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
type streamOutputVariantRequest struct {
Value []models.StreamOutputVariant `json:"value"`
}
decoder := json.NewDecoder(r.Body)
var videoVariants streamOutputVariantRequest
if err := decoder.Decode(&videoVariants); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update video config with provided values "+err.Error())
return
}
if err := data.SetStreamOutputVariants(videoVariants.Value); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update video config with provided values "+err.Error())
return
}
controllers.WriteSimpleResponse(w, true, "stream output variants updated")
}
// SetSocialHandles will handle the web config request to set the external social profile links.
func SetSocialHandles(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
type socialHandlesRequest struct {
Value []models.SocialHandle `json:"value"`
}
decoder := json.NewDecoder(r.Body)
var socialHandles socialHandlesRequest
if err := decoder.Decode(&socialHandles); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update social handles with provided values")
return
}
if err := data.SetSocialHandles(socialHandles.Value); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update social handles with provided values")
return
}
controllers.WriteSimpleResponse(w, true, "social handles updated")
}
// SetChatDisabled will disable chat functionality.
func SetChatDisabled(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
controllers.WriteSimpleResponse(w, false, "unable to update chat disabled")
return
}
data.SetChatDisabled(configValue.Value.(bool))
controllers.WriteSimpleResponse(w, true, "chat disabled status updated")
}
// SetVideoCodec will change the codec used for video encoding.
func SetVideoCodec(w http.ResponseWriter, r *http.Request) {
if !requirePOST(w, r) {
return
}
configValue, success := getValueFromRequest(w, r)
if !success {
controllers.WriteSimpleResponse(w, false, "unable to change video codec")
return
}
if err := data.SetVideoCodec(configValue.Value.(string)); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update codec")
return
}
controllers.WriteSimpleResponse(w, true, "video codec updated")
}
// SetExternalActions will set the 3rd party actions for the web interface.
func SetExternalActions(w http.ResponseWriter, r *http.Request) {
type externalActionsRequest struct {
Value []models.ExternalAction `json:"value"`
}
decoder := json.NewDecoder(r.Body)
var actions externalActionsRequest
if err := decoder.Decode(&actions); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update external actions with provided values")
return
}
if err := data.SetExternalActions(actions.Value); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update external actions with provided values")
}
controllers.WriteSimpleResponse(w, true, "external actions update")
}
// SetCustomStyles will set the CSS string we insert into the page.
func SetCustomStyles(w http.ResponseWriter, r *http.Request) {
customStyles, success := getValueFromRequest(w, r)
if !success {
controllers.WriteSimpleResponse(w, false, "unable to update custom styles")
return
}
data.SetCustomStyles(customStyles.Value.(string))
controllers.WriteSimpleResponse(w, true, "custom styles updated")
}
// SetForbiddenUsernameList will set the list of usernames we do not allow to use.
func SetForbiddenUsernameList(w http.ResponseWriter, r *http.Request) {
type forbiddenUsernameListRequest struct {
Value []string `json:"value"`
}
decoder := json.NewDecoder(r.Body)
var request forbiddenUsernameListRequest
if err := decoder.Decode(&request); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to update forbidden usernames with provided values")
return
}
if err := data.SetForbiddenUsernameList(request.Value); err != nil {
controllers.WriteSimpleResponse(w, false, err.Error())
}
controllers.WriteSimpleResponse(w, true, "forbidden username list updated")
}
func requirePOST(w http.ResponseWriter, r *http.Request) bool {
if r.Method != controllers.POST {
controllers.WriteSimpleResponse(w, false, r.Method+" not supported")
return false
}
return true
}
func getValueFromRequest(w http.ResponseWriter, r *http.Request) (ConfigValue, bool) {
decoder := json.NewDecoder(r.Body)
var configValue ConfigValue
if err := decoder.Decode(&configValue); err != nil {
log.Warnln(err)
controllers.WriteSimpleResponse(w, false, "unable to parse new value")
return configValue, false
}
return configValue, true
}
func getValuesFromRequest(w http.ResponseWriter, r *http.Request) ([]ConfigValue, bool) {
var values []ConfigValue
decoder := json.NewDecoder(r.Body)
var configValue ConfigValue
if err := decoder.Decode(&configValue); err != nil {
controllers.WriteSimpleResponse(w, false, "unable to parse array of values")
return values, false
}
object := reflect.ValueOf(configValue.Value)
for i := 0; i < object.Len(); i++ {
values = append(values, ConfigValue{Value: object.Index(i).Interface()})
}
return values, true
}