045a0a2afd
* Support webfinger requests for the live account. Closes https://github.com/owncast/owncast/issues/1193 * Support for actor requests. Returns response for live actor. Closes https://github.com/owncast/owncast/issues/1203 * Handle follow and unfollow requests. Closes https://github.com/owncast/owncast/issues/1191 and https://github.com/owncast/owncast/issues/1205 and https://github.com/owncast/owncast/issues/1206 and https://github.com/owncast/owncast/issues/1194 * Add basic support for sending out text activities. For https://github.com/owncast/owncast/issues/1192 * Some error handling and passing of dynamic local account names. * Add hardcoded example image attachment to test post * Centralize the map of accounts and inboxes * No longer disable the preview generator based on YP toggle * Send a federated message to followers when stream starts. For https://github.com/owncast/owncast/issues/1192 * Placeholder for attaching tags * Add image description * Save and get to outbox persistence. Return using outbox endpoint for actor * Pass payloads to be handled through the gochan * Handle undo follow requests explitly, not all undo requests * Add API for manually sending simple federated messages. Closes #1215 * Verify inbox requests. Closes #1321 * Add route to fetch a single AP object by ID. For #1329 * Add responses to fediverse nodeinfo requests * Set and get federation config values for admin * Handle host-meta requests * Do not send out message if disabled. Use saved go live message. * Require AP-compatible content types for AP-related requests * Rename ap models to apmodels for clarity * Change how content type matching takes place. * io -> ioutil * Add stub delete activity callback * Handle likes and announces to surface engagement in chat. Part of #1229 * Append url to go live posts * Do not require specific content types for nodeinfo requests * Add follow engagement chat message via AP * add owncast user-agent to requests * Set note visibility to public (for now) * Fix saving/fetching a single object * Add support for x-nodeinfo2 responses * Point to the dev admin branch for ap * Bundle in dev admin for testing * Add error logging * Add AP middleware back * Point to the new external compatible logo endpoint * Clean up more AP logging to help testing * Tweak go live text and link hashtags * Fix bug in fetching init time * Send update actor activities when server details/profile is updated * Add federation config overview to web client config * Add additional actor properties * Make the AP middleware checking more flexible when looking at types * First pass at remote fediverse follow flow. For #1371 * Added a basic AP actor followers endpoint * WIP client followers API * Add profile-page reference to webfinger response * Add aliases to webfinger response * Fix content-type returned to be expected activitypub+json * First pass at followers api * Point at local dev copy of go-fed/activity * Add custom toot Hashtag objects to posts * Store additional user details to followers table * Fix AP followers endpoint. Closes #1204 * Add owncast hashtag as an invisible tag to go live posts * Reject AP requests when it is disabled * Add actor util for generating full account user from person object * Verify inbox requests before performing any other work * Accept actor update requests * Fix linter errors in federation branch * Migrate AP SQL to sqlc for type safe queries * Use the @unclearParadigm REST parameter helper * Fix verifying post ID on AP engagement * WIP privacy/request approval * Style the remote follow modal * First pass at a followers list component w/ mock data. #1370 * Revert "Use the @unclearParadigm REST parameter helper" This reverts commit c8af8a413f6f53e7d1a15a7d823ff28be2db3c23. * Fix get followers API * Add support for requiring approval. Closes https://github.com/owncast/owncast/issues/1208 * Handle Applications as Actors partly for PeerTube support * add temp todo list * check route on load, this might change later * style followers * account for just 1 tab case * Remove mock data. Allow showing follow button even when there are no external actions defined * Point to actual followers API * Support fallback img for follower views * Remove duplicate verification. Add some additional verbose logging * Bundle dev admin * Add type to host-meta webfinger template response * Tweak remote follow modal content * WIP federation followers refactor * Do not send pointer to middleware * Update admin * Add setting for toggling displaying fediverse engagement. Closes #1404 * Add in-development admin * Do not enable cors on admin followers api * Add db migration for updating messages table * Enable empty string go live messages to disable * Remove debug messages * Rework some ActivityPub handling. Create new Actor->Person handling. Create new Actor->Service handling. Add engagement handlers to send chat events and store event objects. Store inbound activities to new ap_inbound_activities table. * Support federated engagement events. Store them in the messages table and surface them via chat events. * Support federated event engatement in the chat * Tweak web UI followers handling * Point go.mod at remote fork instead of local * Update admin * Merged in develop. Couple fixes * Update dev admin * Update fedi engagement posts. - Fix incorrect action text. - Add action icons. * Set public as to instead of cc for ap msg * Updated styling for federated actions in chat * Add support for blocking federated domains. Closes #1209 * Force checking of https in verify step * Update dev admin * Return user scopes in chat history api. Closes #1586 * Update dev admin * Add AP outbound request worker pool. Closes #1571 * Disable (temporarily?) owncast tag on AP posts * Consolidate creating activity+notes in outbound AP messages * Add inbox worker pool. Closes #1570 * Update dev admin bundle * Clean up some logs * Re-enable inbound verfication * Save full IRI to outbox instead of path * Reject if full IRI is not found in outbox * Use full ActivityPub user account in chat event * Fix and expand follower APIs - Add missing IDs to AP follower endpoints - Split AP follower endpoints into initial request and pages. - Support pagination in AP requests. * Include IRI in error message * Hide chat toggle when chat is hidden. Closes #1606 * Updates to followers pagination * Set default go live message * Remove log * indirect -> direct import * Updates for inbound federated event handling. - Keep track of existing events and reject duplicates. - Change what is sent to chat for surfing federated engagement. - Keep track if outbound events are automated "go live" events or not. * Update chat federated engagement. * Update dev admin. * Move from being a person to a bot (service). Closes #1619 * Only set server init date if not already set * Only save notes to outbox able * Rework private-mode followers/approvals * API for returning a list of federated actions for #1573 * Fix too-small follower cells and jumpy tabs. Closes #1616 and closes #1516 * Fix shortcuts getting fired on inputs. Fixes #1489 and #1201 * Add spinner, autoclose + other fixes to follow modal. Fixes #1593 * Fix fetching a single object by IRI * SendFederationMessage -> SendFederatedMessage * Autolink and create tag objects from manual posts. Closes #1620 * Update dev admin bundle * Handle engagement from non-automated/live posts * Reject federated engagement actions if they do not match a local post * Update dev admin bundle * A bunch of cleanup * Fix unused assignments and logic * Remove unused function * Add content warning and sentive content flag if stream is NSFW. Closes #1624 * Disable fetching objects by IRI when in private mode. Closes #1623 * Update the error message of the remote follow dialog. closes #1622 * Update dev admin * Fix NREs throwing in test content * Fix query that wasn't properly filtering out hidden messages * Test against user being disabled instead of message visibility * Fix automated test NRE * Update comment * Adjust federated engagement chat views. Closes #1617 * Add additional index to users table * Add support for removing followers/requests. Closes #1630 * Reject federated actions from blocked actors. #1631 * Use fallback avatar if it fails to load. Closes #1635 * Fix styling of follower list. Closes #1636 * Add basic blurb stating they should follow the server. Closes #1641 * Update dev admin * Set default go live message in migration. Closes #1642 * Reset the messages table on 0.0.11 schema migration * Fix js error with moderation actions. Closes #1621 * Add a bit more clarification on follow modal. Closes #1599 * Remove todos * Split out actor and domain blocking checks * Check for errors on default values being set * Clean up actor rejection due to being blocked * Update dev admin * Add colon to error to make it easier to read * Remove markdown rendering of go live message. Reorganize text. Remove content warning. Closes #1645 * Break out the sort+render messages logic so it can be fired on visibility change. Closes #1643 * Do not send profile updates if federation is disabled * Save follow references to inbound activities table * Update dev admin * Add blocked actor test * Remove the overloaded term of Follow from social links * Fix test running in memory only * Remove "just" in engagement messags * Replace star with heart for like action. * Update dev admin * Explicitly set cc as public * Remove overly using the stream name in fediverse engagement messages * Some federated/follow UI tweaks * Remove explicit cc and bcc as they are not required * Explicitly set the audience * Remove extra margin * Add Join Fediverse button to follow modal. Closes #1651 * Do not allow multiple follows to send multiple events. Closes #1650 * Give events a min height * Do not allow old posts to be liked/shared. Closes #1652 * Remove value from log message * Alert followers on private mode toggle * Ignore clicks to follow button if disabled * Remove underline from action buttons * Add moderator icon to join message * Update admin * Post-merge remove unused var * Remove pointing at feature branch Co-authored-by: Ginger Wong <omqmail@gmail.com>
331 lines
8.1 KiB
Go
331 lines
8.1 KiB
Go
package utils
|
|
|
|
import (
|
|
"bytes"
|
|
"errors"
|
|
"fmt"
|
|
"math/rand"
|
|
"net/url"
|
|
"os"
|
|
"os/exec"
|
|
"path"
|
|
"path/filepath"
|
|
"strings"
|
|
|
|
"github.com/mssola/user_agent"
|
|
log "github.com/sirupsen/logrus"
|
|
"github.com/yuin/goldmark"
|
|
"github.com/yuin/goldmark/extension"
|
|
"github.com/yuin/goldmark/renderer/html"
|
|
"mvdan.cc/xurls"
|
|
)
|
|
|
|
// DoesFileExists checks if the file exists.
|
|
func DoesFileExists(name string) bool {
|
|
if _, err := os.Stat(name); err == nil {
|
|
return true
|
|
} else if os.IsNotExist(err) {
|
|
return false
|
|
} else {
|
|
log.Errorln(err)
|
|
return false
|
|
}
|
|
}
|
|
|
|
// GetRelativePathFromAbsolutePath gets the relative path from the provided absolute path.
|
|
func GetRelativePathFromAbsolutePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
file := pathComponents[len(pathComponents)-1]
|
|
|
|
return filepath.Join(variant, file)
|
|
}
|
|
|
|
// GetIndexFromFilePath is a utility that will return the index/key/variant name in a full path.
|
|
func GetIndexFromFilePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
|
|
return variant
|
|
}
|
|
|
|
// Copy copies the file to destination.
|
|
func Copy(source, destination string) error {
|
|
input, err := os.ReadFile(source) // nolint
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return os.WriteFile(destination, input, 0600)
|
|
}
|
|
|
|
// Move moves the file to destination.
|
|
func Move(source, destination string) error {
|
|
return os.Rename(source, destination)
|
|
}
|
|
|
|
// IsUserAgentABot returns if a web client user-agent is seen as a bot.
|
|
func IsUserAgentABot(userAgent string) bool {
|
|
if userAgent == "" {
|
|
return false
|
|
}
|
|
|
|
botStrings := []string{
|
|
"mastodon",
|
|
"pleroma",
|
|
"applebot",
|
|
"whatsapp",
|
|
"matrix",
|
|
"synapse",
|
|
"element",
|
|
"rocket.chat",
|
|
"duckduckbot",
|
|
}
|
|
|
|
for _, botString := range botStrings {
|
|
if strings.Contains(strings.ToLower(userAgent), botString) {
|
|
return true
|
|
}
|
|
}
|
|
|
|
ua := user_agent.New(userAgent)
|
|
return ua.Bot()
|
|
}
|
|
|
|
// IsUserAgentAPlayer returns if a web client user-agent is seen as a media player.
|
|
func IsUserAgentAPlayer(userAgent string) bool {
|
|
if userAgent == "" {
|
|
return false
|
|
}
|
|
|
|
playerStrings := []string{
|
|
"mpv",
|
|
"player",
|
|
"vlc",
|
|
"applecoremedia",
|
|
}
|
|
|
|
for _, playerString := range playerStrings {
|
|
if strings.Contains(strings.ToLower(userAgent), playerString) {
|
|
return true
|
|
}
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// RenderSimpleMarkdown will return HTML without sanitization or specific formatting rules.
|
|
func RenderSimpleMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
log.Fatalln(err)
|
|
}
|
|
|
|
return buf.String()
|
|
}
|
|
|
|
// RenderPageContentMarkdown will return HTML specifically handled for the user-specified page content.
|
|
func RenderPageContentMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.GFM,
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
log.Fatalln(err)
|
|
}
|
|
|
|
return strings.TrimSpace(buf.String())
|
|
}
|
|
|
|
// GetCacheDurationSecondsForPath will return the number of seconds to cache an item.
|
|
func GetCacheDurationSecondsForPath(filePath string) int {
|
|
filename := path.Base(filePath)
|
|
fileExtension := path.Ext(filePath)
|
|
|
|
if filename == "thumbnail.jpg" || filename == "preview.gif" {
|
|
// Thumbnails & preview gif re-generate during live
|
|
return 20
|
|
} else if fileExtension == ".js" || fileExtension == ".css" {
|
|
// Cache javascript & CSS
|
|
return 60 * 60 * 3
|
|
} else if fileExtension == ".ts" {
|
|
// Cache video segments as long as you want. They can't change.
|
|
// This matters most for local hosting of segments for recordings
|
|
// and not for live or 3rd party storage.
|
|
return 31557600
|
|
} else if fileExtension == ".m3u8" {
|
|
return 0
|
|
} else if fileExtension == ".jpg" || fileExtension == ".png" || fileExtension == ".gif" || fileExtension == ".svg" {
|
|
return 60 * 60 * 24 * 7
|
|
}
|
|
|
|
// Default cache length in seconds
|
|
return 60 * 60 * 2
|
|
}
|
|
|
|
// IsValidURL will return if a URL string is a valid URL or not.
|
|
func IsValidURL(urlToTest string) bool {
|
|
if _, err := url.ParseRequestURI(urlToTest); err != nil {
|
|
return false
|
|
}
|
|
|
|
u, err := url.Parse(urlToTest)
|
|
if err != nil || u.Scheme == "" || u.Host == "" {
|
|
return false
|
|
}
|
|
|
|
return true
|
|
}
|
|
|
|
// ValidatedFfmpegPath will take a proposed path to ffmpeg and return a validated path.
|
|
func ValidatedFfmpegPath(ffmpegPath string) string {
|
|
if ffmpegPath != "" {
|
|
if err := VerifyFFMpegPath(ffmpegPath); err == nil {
|
|
return ffmpegPath
|
|
}
|
|
log.Warnln(ffmpegPath, "is an invalid path to ffmpeg will try to use a copy in your path, if possible")
|
|
}
|
|
|
|
// First look to see if ffmpeg is in the current working directory
|
|
localCopy := "./ffmpeg"
|
|
hasLocalCopyError := VerifyFFMpegPath(localCopy)
|
|
if hasLocalCopyError == nil {
|
|
// No error, so all is good. Use the local copy.
|
|
return localCopy
|
|
}
|
|
|
|
cmd := exec.Command("which", "ffmpeg")
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
log.Fatalln("Unable to determine path to ffmpeg. Please specify it in the config file.")
|
|
}
|
|
|
|
path := strings.TrimSpace(string(out))
|
|
return path
|
|
}
|
|
|
|
// VerifyFFMpegPath verifies that the path exists, is a file, and is executable.
|
|
func VerifyFFMpegPath(path string) error {
|
|
stat, err := os.Stat(path)
|
|
|
|
if os.IsNotExist(err) {
|
|
return errors.New("ffmpeg path does not exist")
|
|
}
|
|
|
|
if err != nil {
|
|
return fmt.Errorf("error while verifying the ffmpeg path: %s", err.Error())
|
|
}
|
|
|
|
if stat.IsDir() {
|
|
return errors.New("ffmpeg path can not be a folder")
|
|
}
|
|
|
|
mode := stat.Mode()
|
|
// source: https://stackoverflow.com/a/60128480
|
|
if mode&0111 == 0 {
|
|
return errors.New("ffmpeg path is not executable")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// CleanupDirectory removes the directory and makes it fresh again. Throws fatal error on failure.
|
|
func CleanupDirectory(path string) {
|
|
log.Traceln("Cleaning", path)
|
|
if err := os.RemoveAll(path); err != nil {
|
|
log.Fatalln("Unable to remove directory. Please check the ownership and permissions", err)
|
|
}
|
|
if err := os.MkdirAll(path, 0750); err != nil {
|
|
log.Fatalln("Unable to create directory. Please check the ownership and permissions", err)
|
|
}
|
|
}
|
|
|
|
// FindInSlice will return if a string is in a slice, and the index of that string.
|
|
func FindInSlice(slice []string, val string) (int, bool) {
|
|
for i, item := range slice {
|
|
if item == val {
|
|
return i, true
|
|
}
|
|
}
|
|
return -1, false
|
|
}
|
|
|
|
// StringSliceToMap is a convinience function to convert a slice of strings into
|
|
// a map using the string as the key.
|
|
func StringSliceToMap(stringSlice []string) map[string]interface{} {
|
|
stringMap := map[string]interface{}{}
|
|
|
|
for _, str := range stringSlice {
|
|
stringMap[str] = true
|
|
}
|
|
|
|
return stringMap
|
|
}
|
|
|
|
// StringMapKeys returns a slice of string keys from a map.
|
|
func StringMapKeys(stringMap map[string]interface{}) []string {
|
|
stringSlice := []string{}
|
|
for k := range stringMap {
|
|
stringSlice = append(stringSlice, k)
|
|
}
|
|
return stringSlice
|
|
}
|
|
|
|
// GenerateRandomDisplayColor will return a random _hue_ to be used when displaying a user.
|
|
// The UI should determine the right saturation and lightness in order to make it look right.
|
|
func GenerateRandomDisplayColor() int {
|
|
rangeLower := 0
|
|
rangeUpper := 360
|
|
return rangeLower + rand.Intn(rangeUpper-rangeLower+1) //nolint
|
|
}
|
|
|
|
// GetHostnameFromURL will return the hostname component from a URL string.
|
|
func GetHostnameFromURL(u url.URL) string {
|
|
return u.Host
|
|
}
|
|
|
|
// GetHostnameFromURLString will return the hostname component from a URL object.
|
|
func GetHostnameFromURLString(s string) string {
|
|
u, err := url.Parse(s)
|
|
|
|
if err != nil {
|
|
return ""
|
|
}
|
|
|
|
return u.Host
|
|
}
|