Compare commits

..
1217 changed files with 66178 additions and 81772 deletions
-1
View File
@@ -1 +0,0 @@
test/automated/api/node_modules
@@ -6,8 +6,7 @@ body:
value: |
Thanks for helping by reporting issues and sharing ideas you might have!
While no idea is a bad idea, some might make more sense for Owncast than others.
Take a look at the [Owncast product definition](https://docs.owncast.dev/project-definition) to see what our focus is and how your requests might align.
Note: Please file issues in English so your message can be interpreted as you intended.
Take a look at the [Owncast product definition](https://github.com/owncast/owncast/blob/develop/docs/product-definition.md) to see what our focus is and how your requests might align.
- type: textarea
id: issue-body
-55
View File
@@ -1,55 +0,0 @@
<!-- REQUIRED-CHECKLIST:START - Do not remove this section -->
## Required checklist
**Do not remove this section.** These checkboxes are required and validated.
- [ ] I have personally tested these changes and verified they work as intended.
- [ ] I understand the code I'm submitting and can explain how it works if asked.
- [ ] I included a screenshot, logs, example payload to demonstrate the change, or it really doesn't need it.
- [ ] This is a frontend change and it supports [translations](https://docs.owncast.dev/web-translations), or it's not a frontend change.
- [ ] The code has been run through the proper linters and/or formatters for the language.
- [ ] There is an issue discussing this change and it's assigned to me.
<!-- REQUIRED-CHECKLIST:END -->
---
# Read first
## We manage everything through issues, not PRs.
A PR is a **solution**. A solution without a **problem** breaks release notes. **Please open an issue** so the list of problems that were fixed can be correctly listed and be attributed to you.
If this is an unsolicited change, or there is no existing issue filed for it, **please open a GitHub issue before creating a pull request**. This will allow us to discuss the motivations and the big picture behind the change first. It's possible there may be other solutions that should be discussed for what you think should be built. It is possible your change will be rejected unless some discussion around your proposal happens first. While creating this PR means you probably already did the work, **it still makes sense to file an issue now**, and into the future when you have proposed changes.
Additionally, when attributing credit in releases, we use issues to determine who worked on each version. If there isn't an issue assigned to you, you may not get credit in release notes.
## Resources
- [Developing Owncast](https://docs.owncast.dev/development)
- [How We Develop Frontend React Components](https://docs.owncast.dev/develop-frontend-components)
- [Supporting Translations](https://docs.owncast.dev/web-translations)
## Description
Once there is an issue filed, a PR can be linked to it.
Please include a summary of the change in the PR and which issue number is fixed, including relevant motivation and context. Mark this as a Draft or WIP and write up some details later and start a conversation, even if your PR is not ready for review or you haven't yet provided all the information.
Fixes # (issue)
## Screenshot Examples or Logs
If this is a frontend change, please include a screenshot of the change. If this is a backend change, please include relevant logs or examples of the change in action if applicable.
---
Some things you might want to mention:
1. Why are you making the change?
2. Explain how it works and decisions you made.
3. If you're fixing something, what was wrong? How should we stop from having this issue happen again?
4. If this is a new feature or addition to functionality, why should it be added? What are the use cases? Who was asking for this functionality?
5. If this is a frontend change, does the text support [translation](https://owncast.notion.site/web-translations)?
Thank you so much for contributing to Owncast! 🎉
@@ -0,0 +1,19 @@
Please include a summary of the change and which issue number is fixed, including relevant motivation and context. Feel free to mark this as a Draft or WIP and write up some details later.
If there is no issue filed for this particular change it's highly recommended you file one. While creating this PR means you probably already did the work, in the future make sure an issue is filed beforehand so changes, fixes and features can be discussed ahead of time.
# Description
Fixes # (issue)
---
Some things you might want to mention:
1. Why are you making the change?
2. Explain how it works and decisions you made.
3. If you're fixing something, what was wrong? How should we stop from having this issue happen again?
4. If this is a new feature or addition to functionality, why should it be added? What are the use cases? Who was asking for this functionality?
If this is an unsolicited change or have no issue associated please do your best to detail the motivations behind this PR, and think about filing an issue to discuss changes ahead of time in the future.
-73
View File
@@ -1,73 +0,0 @@
This is a repository consisting of a Go backend and a React frontend. It supports both an internal web application that is public facing, an internal admin web application, internal-only APIs for powering these web interfaces, and a set of APIs for third parties to take advantage of.
## Code Standards
- UI component standards can be found in https://docs.owncast.dev/develop-frontend-components
### Required Before Each Commit
- Ensure all code is formatted using `golangci-lint` for Go, `stylelint` for CSS, and `prettier` for JavaScript/React.
- Fix any formatting or linting errors.
### Development Flow
- Build: Ensure the code builds successfully using `go build` for Go and `npm run build` for React in the `web` directory.
- Tests: Write unit tests for Go code and Javascript logic. Use `go test` and `npm run test`.
- UI components: Components should be standalone and reusable and show up in Storybook to enable testing, prototyping, and iteration.
- API: All APIs should be documented using OpenAPI specifications. Use `build/gen-api.sh` to generate the stubs, and then fill them in with your actual API implementation.
- If you made new UI components or made changes ensure that Storybook still builds by running `npm run build-storybook`.
## Repository Structure
- `web`: The web source code for the React frontend.
- Root of the repo: Go source code for the backend.
- `static/web`: Generated static files for the web application. Do not edit or commit files in this directory directly; they are generated from the `web` directory. Ignore this.
- `test/automated/api`: A series of automated integration tests for API endpoints written in Javascript.
- `test/automated/browser`: A series of automated browser UI tests for actual real-world browser interaction.
- `build/web`: Script to build and bundle the web application.
## Testing web frontend
Testing the web frontend requires running the frontend development server and the backend service together. The frontend development server must be started using `npm run dev` in the `web` directory, which will serve the web application at `http://localhost:3000`. The backend service can be started using `go run main.go` in the root of the repository.
Do not access the web application via http://localhost:8080 or build the web project to copy files to the `static/web` directory for local development.
## Code Reviews
- When performing a code review for a UI component take a screenshot and attach it to your comments to make the review more detailed.
- When performing a code review for an API provide the payload for the API and attach it to the code review comment.
- When performing a code review check for all linters and formatters for in code and put in the code review comment a suggestion on how to fix each line.
## Key Guidelines
1. All APIs are to be documented using OpenAPI specifications and code is to be generated using `build/gen-api.sh`. Additional details can be found at https://docs.owncast.dev/api-web-routing.
2. Write API tests for all new endpoints in the `test/automated/api` directory.
3. Use the `test/automated/browser` directory for browser-based tests for new functionality that simulate user interactions.
4. All user-facing frontend UI strings need to support localization. Use the `Translation` component to show most displayable strings. To create dynamic translated strings use the `next-export-i18n` library and the `t()` function. But use the `Translation` component unless there is a reason not to as it allows you to set default text. Read https://docs.owncast.dev/web-translations for more details. Test localization by adding "?lang=XX" with XX being a country code, such as "de" for German. Strings that have not yet been translated will not show as changed, but it's good to test anyway to make sure that previously translated strings have not been broken or regressed in any way. Screenshots with some additional languages can be helpful in showing this.
5. For UI component changes, a before and after screenshot of the component should always be added to the pull request to help with review. Additionally a link to the PR's Storybook on Chromatic via the PR's Chromatic job should be included to help with review.
6. For API changes a before and after example of the API response should be added to the pull request to help with review.
7. For backend changes, a before and after example of logs to demonstrate the change should be added to the pull request to help with review.
8. Do not build the web project or copy the files to the `static/web` directory for local development.
9. When running the frontend development server, the local backend service must also be running. You can run the backend service using `go run main.go` in the root of the repository.
10. The credentials for the backend development backend are username: admin and password: abc123 and uses HTTP Basic Auth. This is used for the admin web application and the admin APIs.
11. The admin is found at `/admin`.
12. If a live stream video is needed to run, you can run `./test/ocTestStream.sh` to start an actual stream that will begin streaming from the local development server.
13. You should never commit the `static/web` directory to the repository. It is generated from the `web` directory and should be ignored in your commits.
14. Don't use emoji in code comments or commit messages. That's lame.
## Screenshots
Take a screenshot after every UI change without being asked. Screenshots should be automatically taken after every code change that affects UI.
After every UI change we should be able to see in the PR comments, visually, exactly what the change is. This helps with code review and ensures that the UI changes are clear and understandable. This should happen every time, automatically, without being asked, and never fail or be skipped.
These screenshots should be displayed inline in the PR comments, and not as attachments. This allows reviewers to see the changes without having to download or open files separately.
- If it is a standalone UI component, take a screenshot of the component in Storybook if it is available.
- When taking a screenshot of a component in Storybook, make sure to hide all the controls that are not relevant to the component itself. This means hiding the knobs, actions, and any other controls that are not part of the component's visual representation and would get in the way of the screenshot.
- If it is something that doesn't exist in Storybook, take a screenshot of the component in the live web application using the local development server.
- Most things in the admin do not exist in Storybook, so screenshots should be taken of the admin web application.
- When taking a screenshot of the web frontend or the admin web application, an instance of the Owncast backend service needs to be running locally by running `go run main.go` in the root of the repository as well.
- When taking screenshots for PR documentation, create temporary files in /tmp directory or use patterns like _screenshot_.js and _screenshot_.png that are excluded by .gitignore.
- Screnshots should be taken using the web dev server at `http://localhost:3000` and not the production build at `http://localhost:8080`.
- Never commit temporary screenshot scripts or image files to the repository - they should only be used locally and uploaded directly to GitHub for PR comments.
- Double check that the screenshots are attached to the PR comments. Copilot often forgets to do this or says it did it but doesn't actually do it. If it doesn't do it, it should continue to try until it succeeds.
+31
View File
@@ -0,0 +1,31 @@
# Number of days of inactivity before an issue becomes stale
daysUntilStale: 60
# Number of days of inactivity before a stale issue is closed
daysUntilClose: 7
# Issues with these labels will never be considered stale
exemptLabels:
- backlog
# Label to use when marking an issue as stale
staleLabel: stale
# Comment to post when marking an issue as stale. Set to `false` to disable
markComment: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. If this
was a feature request that others have shown no interest in then it's
likely to not get implemented due to lack of interest. If others also
want to see this feature then now is the time to say something!
Thank you for your contributions.
# Comment to post when closing a stale issue. Set to `false` to disable
closeComment: false
exemptMilestones: true
# Since old PRs are less useful than old issues ping them sooner.
pulls:
daysUntilStale: 30
markComment: >
This pull request has not had any activity in 30 days. Since things move fast it's best
to get PRs merged in. If this PR addresses a previously filed issue that needs to be
resolved please work to get it merged in, or allow somebody else to work on a fix.
This PR will be closed if no further activity occurs. Thank you for your contributions!
exemptLabels:
- bot
+1 -10
View File
@@ -13,16 +13,7 @@ jobs:
name: GitHub actions
runs-on: ubuntu-latest
steps:
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- uses: actions/checkout@v4
- uses: docker://rhysd/actionlint:latest
with:
-130
View File
@@ -1,130 +0,0 @@
name: ActivityPub Tests
on:
push:
paths:
- "activitypub/**"
- "core/chat/**"
- "config/**"
- "main.go"
- "webserver/**"
- "test/automated/activitypub/**"
- ".github/workflows/activitypub-tests.yaml"
pull_request:
paths:
- "activitypub/**"
- "core/chat/**"
- "config/**"
- "main.go"
- "webserver/**"
- "test/automated/activitypub/**"
- ".github/workflows/activitypub-tests.yaml"
jobs:
federation-test:
runs-on: ubuntu-latest
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: "same_content_newer"
- name: Check out repository code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: "^1.23"
cache: true
- name: Install Caddy
run: |
sudo apt-get update
sudo apt-get install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt-get update
sudo apt-get install -y caddy
- name: Install dependencies
run: |
sudo apt-get install -y snac2 ffmpeg
- name: Run setup
run: |
cd test/automated/activitypub
sudo ./setup.sh
- name: Run federation test
run: |
cd test/automated/activitypub
CI=true USER_COUNT=20 ./test-federation.sh
chat-sanitization-test:
runs-on: ubuntu-latest
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: "same_content_newer"
- name: Check out repository code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: "^1.23"
cache: true
- name: Install Caddy
run: |
sudo apt-get update
sudo apt-get install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt-get update
sudo apt-get install -y caddy
- name: Install dependencies
run: |
sudo apt-get install -y snac2 ffmpeg
- name: Run setup
run: |
cd test/automated/activitypub
sudo ./setup.sh
- name: Run chat sanitization test
run: |
cd test/automated/activitypub
CI=true ./test-chat-sanitization.sh
follower-validation-test:
runs-on: ubuntu-latest
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: "same_content_newer"
- name: Check out repository code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: "^1.23"
cache: true
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y sqlite3 ffmpeg jq
- name: Run follower validation test
timeout-minutes: 15
run: |
cd test/automated/activitypub
./test-follower-validation.sh
@@ -1,50 +0,0 @@
name: Extract Default Translations
on:
push:
paths:
- 'web/**/*.ts'
- 'web/**/*.tsx'
- 'web/**/*.js'
- 'web/**/*.jsx'
jobs:
extract-and-commit:
runs-on: ubuntu-latest
defaults:
run:
working-directory: web
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
persist-credentials: true
fetch-depth: 0 # Required to push back to same branch
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: '24'
cache: 'npm'
cache-dependency-path: web/package-lock.json
- name: Install dependencies
run: npm ci
- name: Run translation extraction
run: npm run translate
- name: Commit and push changes
run: |
git config --global user.name "Owncast default web localizations"
git config --global user.email "owncast@owncast.online"
if [ -n "$(git status --porcelain i18n/)" ]; then
git add i18n/
git commit -m "chore: update extracted translations"
git push origin HEAD
else
echo "No translation changes to commit."
fi
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
issues: write
steps:
- name: Add comment
uses: peter-evans/create-or-update-comment@57232238742e38b2ccc27136ce596ccae7ca28b4
uses: peter-evans/create-or-update-comment@71ac479718f8aed504782bc920d802da994c05fa
with:
issue-number: ${{ github.event.issue.number }}
body: |
@@ -1,13 +1,3 @@
# This workflow runs automated API integration tests using Earthly
# Triggers:
# - Push/PR with Go file changes
# Skips:
# - Web-only changes
# Uses:
# - Earthly for reproducible builds
# - QEMU for cross-platform testing
# - Retries up to 3 times for flaky tests
name: Automated API tests
on:
@@ -28,51 +18,23 @@ jobs:
with:
concurrent_skipping: 'same_content_newer'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
files_yaml: |
src:
- '**/*.{go,mod,sum}'
- uses: earthly/actions-setup@v1
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
with:
version: 'latest' # or pin to an specific version, e.g. "v0.6.10"
- name: Earthly version
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: earthly --version
- name: Set up QEMU
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
id: qemu
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:latest
platforms: all
- uses: actions/checkout@v4
- name: Run API tests
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
+8 -55
View File
@@ -1,27 +1,14 @@
# This workflow runs browser-based tests using Cypress
# Triggers:
# - Push/PR with web, Go backend, or browser test changes
# Note: Browser tests require both frontend and backend
name: Browser Tests
on:
push:
paths:
- 'web/**'
- '**.go'
- 'go.mod'
- 'go.sum'
- 'test/automated/browser/**'
- '.github/workflows/browser-testing.yml'
pull_request:
paths:
- 'web/**'
- '**.go'
- 'go.mod'
- 'go.sum'
- 'test/automated/browser/**'
- '.github/workflows/browser-testing.yml'
jobs:
cypress-run:
@@ -32,46 +19,15 @@ jobs:
with:
concurrent_skipping: 'same_content_newer'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
files_yaml: |
testable:
- 'web/**/*.{tsx,ts,jsx,js,css,scss}'
- 'test/automated/browser/**'
- '**/*.go'
- 'go.mod'
- 'go.sum'
docs:
- '**/*.md'
- uses: actions/setup-node@v6
if: steps.changed-files-yaml.outputs.testable_any_changed == 'true'
with:
node-version: '24.12.0'
node-version: latest
- name: Cache node modules
if: steps.changed-files-yaml.outputs.testable_any_changed == 'true'
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-browser-tests
with:
@@ -82,18 +38,15 @@ jobs:
${{ runner.os }}-build-
${{ runner.os }}-
- uses: actions/setup-go@v6
if: steps.changed-files-yaml.outputs.testable_any_changed == 'true'
- uses: actions/setup-go@v5
with:
go-version: '1.26.2'
go-version: '1.22'
cache: true
- name: Install Google Chrome
if: steps.changed-files-yaml.outputs.testable_any_changed == 'true'
run: sudo apt-get update && sudo apt-get install google-chrome-stable
- name: Run Browser tests
if: steps.changed-files-yaml.outputs.testable_any_changed == 'true'
uses: nick-fields/retry@v3
with:
timeout_minutes: 20
+5 -39
View File
@@ -1,26 +1,9 @@
# This workflow builds and deploys Storybook component documentation
# Triggers:
# - Push to develop branch with changes to:
# - web/stories/** (story files)
# - web/components/** (component source)
# - web/.storybook/** (Storybook config)
# - web/i18n/** (translations)
# Output:
# - Generates updated story files
# - Builds Storybook to docs/components
# - Dispatches event to owncast.github.io for deployment
name: Build and Deploy Components+Style Guide
on:
push:
branches:
- develop
paths: [
'web/stories/**',
'web/components/**',
'web/.storybook/**',
'web/i18n/**',
] # Trigger the action only when files change in the folders defined here
paths: ['web/stories/**', 'web/components/**', 'web/.storybook/**'] # Trigger the action only when files change in the folders defined here
jobs:
build-and-deploy:
@@ -28,26 +11,11 @@ jobs:
if: github.repository == 'owncast/owncast'
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Checkout
uses: actions/checkout@v4
- name: Cache node modules
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-bundle-web-app
with:
@@ -68,7 +36,6 @@ jobs:
npm run build-storybook -- -o ../docs/components
- name: Commit changes
if: github.repository == 'owncast/owncast'
uses: EndBug/add-and-commit@v9
with:
author_name: Owncast
@@ -80,8 +47,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Dispatch event to web site
if: github.repository == 'owncast/owncast'
uses: peter-evans/repository-dispatch@v4
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.BUNDLE_STORYBOOK_OWNCAST_ONLINE }}
repository: owncast/owncast.github.io
+14 -58
View File
@@ -1,44 +1,22 @@
# .github/workflows/chromatic.yml
# This workflow publishes Storybook to Chromatic for visual regression testing
# Triggers:
# - Push/PR with changes to web components, stories, or styles
# Skips:
# - Static files
# - Renovate bot PRs
# Features:
# - Uses onlyChanged for efficient testing
# - Publishes visual diffs for review
# Workflow name
name: 'Chromatic'
on:
push:
paths:
- 'web/components/**'
- 'web/stories/**'
- 'web/.storybook/**'
- 'web/**/*.{css,scss}'
- 'web/i18n/**'
- 'web/package.json'
- 'web/package-lock.json'
- '.github/workflows/chromatic.yml'
pull_request:
- web/**
pull_request_target:
paths:
- 'web/components/**'
- 'web/stories/**'
- 'web/.storybook/**'
- 'web/**/*.{css,scss}'
- 'web/i18n/**'
- 'web/package.json'
- 'web/package-lock.json'
- '.github/workflows/chromatic.yml'
- web/**
# List of jobs
jobs:
chromatic-deployment:
# Operating System
runs-on: ubuntu-latest
if: github.repository == 'owncast/owncast'
defaults:
run:
@@ -49,47 +27,25 @@ jobs:
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Check out repository code (Push)
if: github.event_name == 'push'
uses: actions/checkout@v6
- name: Check out pull request code (PR)
if: github.event_name == 'pull_request'
uses: actions/checkout@v6
- name: Check out code
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' }}
uses: actions/checkout@v4
with:
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
path: 'web'
files_ignore: |
static/**
web/next.config.js
files_yaml: |
src:
- '**/*.{js,ts,tsx,jsx,md}'
- name: Install dependencies
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' && steps.changed-files-yaml.outputs.src_any_changed == 'true'}}
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' }}
run: npm install
- name: Publish to Chromatic
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' && steps.changed-files-yaml.outputs.src_any_changed == 'true' }}
uses: chromaui/action@v16
if: ${{ github.actor != 'renovate[bot]' && github.actor != 'renovate' }}
uses: chromaui/action@v11
# Chromatic GitHub Action options
with:
workingDir: web
projectToken: f47410569b62
onlyChanged: true
branch: ${{ github.head_ref || github.ref_name }}
sha: ${{ github.event.pull_request.head.sha || github.sha }}
+11 -29
View File
@@ -14,23 +14,13 @@ name: 'CodeQL'
on:
push:
branches: [develop]
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- 'web/**/*.{js,ts,tsx,jsx}'
- '.github/workflows/codeql-analysis.yml'
- '.github/codeql/**'
paths-ignore:
- 'static/**'
pull_request:
# The branches below must be a subset of the branches above
branches: [develop]
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- 'web/**/*.{js,ts,tsx,jsx}'
- '.github/workflows/codeql-analysis.yml'
- '.github/codeql/**'
paths-ignore:
- 'static/**'
jobs:
analyze:
@@ -46,25 +36,17 @@ jobs:
# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
steps:
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Checkout repository
uses: actions/checkout@v4
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- uses: actions/setup-go@v6
- uses: actions/setup-go@v5
with:
go-version: '1.26.2'
go-version: '1.22'
cache: true
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/${{ matrix.language }}.yml
@@ -76,7 +58,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v4
uses: github/codeql-action/autobuild@v3
# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -90,4 +72,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@v3
+1 -10
View File
@@ -19,16 +19,7 @@ jobs:
container:
image: aquasec/trivy
steps:
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- uses: actions/checkout@v4
- name: Check critical issues
run: trivy config --exit-code 1 --severity "HIGH,CRITICAL" ./Dockerfile
+3 -33
View File
@@ -1,14 +1,6 @@
# See https://docs.earthly.dev/ci-integration/vendor-specific-guides/gh-actions-integration
# for details.
# This workflow builds the development container image
# Triggers:
# - Nightly scheduled builds (2 AM UTC)
# - Push/PR to develop with relevant file changes
# Skips:
# - Documentation-only changes
# - Web-only changes (web/** without Go changes)
name: Build development container
on:
@@ -17,25 +9,9 @@ on:
push:
branches:
- develop
paths:
- '**/*.go'
- 'go.mod'
- 'go.sum'
- 'Dockerfile'
- 'Earthfile'
- 'build/**'
- '.github/workflows/container.yaml'
pull_request:
branches:
- develop
paths:
- '**/*.go'
- 'go.mod'
- 'go.sum'
- 'Dockerfile'
- 'Earthfile'
- 'build/**'
- '.github/workflows/container.yaml'
jobs:
Earthly:
@@ -61,16 +37,10 @@ jobs:
image: tonistiigi/binfmt:latest
platforms: all
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- name: Checkout repo
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push' || github.event_name == 'schedule'
fetch-depth: 0
- name: Build and push
if: ${{ github.event_name == 'schedule' && env.GH_CR_PAT != null }}
-47
View File
@@ -1,47 +0,0 @@
name: 'Copilot Setup Steps'
# Automatically run the setup steps when they are changed to allow for easy validation, and
# allow manual testing through the repository's "Actions" tab
on:
workflow_dispatch:
push:
paths:
- .github/workflows/copilot-setup-steps.yml
pull_request:
paths:
- .github/workflows/copilot-setup-steps.yml
jobs:
# The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot.
copilot-setup-steps:
runs-on: ubuntu-latest
# Set the permissions to the lowest permissions possible needed for your steps.
# Copilot will be given its own token for its operations.
permissions:
# If you want to clone the repository as part of your setup steps, for example to install dependencies, you'll need the `contents: read` permission. If you don't clone the repository in your setup steps, Copilot will do this for you automatically after the steps complete.
contents: read
# You can define any steps you want, and they will run before the agent starts.
# If you do not check out your code, Copilot will do this for you.
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Install go
uses: actions/setup-go@v6
with:
go-version: '^1'
cache: true
- name: Install ffmpeg
run: sudo apt-get install -y ffmpeg
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: '24'
cache: 'npm'
- name: Install JavaScript dependencies
run: npm ci
working-directory: web
-69
View File
@@ -1,69 +0,0 @@
# This workflow checks CSS/SCSS code quality and formatting
# Triggers:
# - Push/PR with changes to web/ directory
# Checks:
# - Prettier formatting for CSS/SCSS files
# - Stylelint rules for code quality
# Note: Only runs when CSS or SCSS files are changed
name: CSS Lint and Formatting
on:
push:
paths:
- 'web/**'
pull_request:
paths:
- 'web/**'
jobs:
css-lint:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./web
steps:
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
path: 'web'
files_yaml: |
src:
- '**/*.{css,scss}'
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: '24.12.0'
- name: Install dependencies
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npm install
- name: Run Prettier
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx prettier --check ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Run Stylelint
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx stylelint ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
@@ -9,16 +9,8 @@ jobs:
name: Generate API Documentation
runs-on: ubuntu-latest
steps:
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Checkout repo
uses: actions/checkout@v4
- name: Run redoc on openapi.yaml
run: |
+9 -26
View File
@@ -1,27 +1,11 @@
# This workflow runs golangci-lint to check Go code quality
# Triggers:
# - Push/PR with Go file changes
# Skips:
# - Web-only changes
# - Dependabot PRs (separate workflow)
# Note: Only reports on new issues to reduce noise
name: Lint
on:
push:
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- '.golangci.yml'
- '.github/workflows/go-lint.yml'
paths-ignore:
- 'web/**'
pull_request:
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- '.golangci.yml'
- '.github/workflows/go-lint.yml'
paths-ignore:
- 'web/**'
permissions:
contents: read
@@ -38,18 +22,17 @@ jobs:
with:
concurrent_skipping: 'same_content_newer'
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v6
- uses: actions/setup-go@v5
with:
go-version: '1.26.2'
go-version: '1.22'
cache: true
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: golangci-lint
uses: golangci/golangci-lint-action@v9
uses: golangci/golangci-lint-action@v6
with:
version: v2.11.4
only-new-issues: true
args: --timeout=3m
+8 -39
View File
@@ -1,11 +1,3 @@
# This workflow runs Go unit tests across multiple OS and Go versions
# Triggers:
# - Push/PR with Go file changes (*.go, go.mod, go.sum)
# - Skips when only web/ directory changes
# Test Matrix:
# - PRs: Latest Go on Ubuntu only (fast feedback)
# - Develop/Main: Full matrix (ensure cross-platform compatibility)
name: Go Tests
on:
@@ -20,22 +12,13 @@ jobs:
test:
strategy:
matrix:
go-version: ${{ github.event_name == 'pull_request' && fromJSON('["1.24.x"]') || fromJSON('["1.23.x", "1.24.x"]') }}
os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest", "macos-latest", "windows-latest", "ubuntu-24.04-arm"]') }}
go-version: [1.21.x, 1.22.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
files_yaml: |
src:
- '**/*.{go,mod,sum}'
- uses: actions/cache@v5
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@@ -45,19 +28,15 @@ jobs:
go-test-
- name: Install go
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: actions/setup-go@v6
uses: actions/setup-go@v5
with:
go-version: '^1'
cache: true
- name: Run tests
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: go test ./...
test-bsds:
# Only run BSD tests on push events (not PRs) for efficiency
if: github.event_name == 'push'
runs-on: macos-latest
strategy:
matrix:
@@ -68,17 +47,9 @@ jobs:
version: 6.8
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
files_yaml: |
src:
- '**/*.{go,mod,sum}'
- uses: actions/cache@v5
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@@ -88,12 +59,10 @@ jobs:
go-test-
- name: Install go
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: actions/setup-go@v6
uses: actions/setup-go@v5
with:
go-version: '^1'
cache: true
- name: Run tests
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: go test ./...
@@ -1,47 +0,0 @@
name: Hacktoberfest Reminder
on:
schedule:
# Run on September 15th at 9:00 AM UTC every year
- cron: '0 9 15 9 *'
workflow_dispatch: # Allow manual triggering for testing
jobs:
create-reminder-issue:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Create Hacktoberfest reminder issue
uses: dacbd/create-issue-action@v2
with:
token: ${{ github.token }}
title: 'Reminder: Update Hacktoberfest branding for this year'
body: |
## Hacktoberfest Update Reminder
It's time to review and update the Hacktoberfest page on the Owncast website!
### What needs to be updated:
- **Branding/Images**: Hacktoberfest releases new branding each year. Check the official [Hacktoberfest website](https://hacktoberfest.com/) for current year branding assets.
- **Year**: Update any references to the year (e.g., "Hacktoberfest 2024" → "Hacktoberfest 2025")
- **Links**: Verify all links point to the current year's Hacktoberfest pages
- **Text/Copy**: Review any text for accuracy and updates from Hacktoberfest organizers
### Resources:
- Hacktoberfest website: https://hacktoberfest.com/
### Action Items:
- [ ] Download new Hacktoberfest branding assets
- [ ] Update images on the website
- [ ] Update year references
- [ ] Verify all links are working
- [ ] Test the updated page
---
_This issue was automatically created by a scheduled GitHub Action._
labels: hacktoberfest,good first issue,help wanted,documentation
+8 -53
View File
@@ -1,28 +1,12 @@
# This workflow tests HLS video streaming functionality
# Triggers:
# - Push/PR with Go file changes or HLS test changes
# Tests:
# - Local storage backend
# - S3 storage backend (when secrets available)
# - Retries up to 3 times for flaky network tests
name: HLS tests
on:
push:
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- 'test/automated/hls/**'
- '.github/workflows/hls-tests.yml'
paths-ignore:
- 'web/**'
pull_request:
paths:
- '**.go'
- 'go.mod'
- 'go.sum'
- 'test/automated/hls/**'
- '.github/workflows/hls-tests.yml'
paths-ignore:
- 'web/**'
env:
S3_BUCKET: ${{ secrets.S3_BUCKET }}
@@ -40,41 +24,14 @@ jobs:
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
with:
files_yaml: |
src:
- '**/*.{go,mod,sum}'
- uses: actions/setup-go@v6
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
with:
go-version: '1.26.2'
go-version: '1.22'
cache: true
- name: Cache node modules
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-hls-tests
with:
@@ -86,7 +43,6 @@ jobs:
${{ runner.os }}-
- name: Local stroage
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
@@ -94,7 +50,6 @@ jobs:
command: cd test/automated/hls && ./run.sh
- name: S3 storage
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
@@ -1,44 +1,20 @@
# This workflow handles Javascript formatting, linting, testing, and building
# Triggers:
# - Push/PR with changes to web/ directory source code
# Skips:
# - Markdown files
# - Static/generated files
# Jobs:
# 1. formatting: ESLint and Prettier formatting
# 2. unused-code: Knip checks for unused code and dependencies
# 3. web-bundle: Builds and bundles the web app (only on develop branch)
# Note: Auto-fixes and commits formatting issues on push events
name: Javascript
# This action works with pull requests and pushes
on:
push:
paths:
- 'web/**/*.{js,ts,tsx,jsx,json,css,scss}'
- 'web/package.json'
- 'web/package-lock.json'
- 'web/.eslintrc.js'
- 'web/.prettierrc.js'
- 'web/tsconfig.json'
- 'web/next.config.js'
- '.github/workflows/javascript-format-test-build.yml'
- web/**
- '!**.md'
pull_request:
paths:
- 'web/**/*.{js,ts,tsx,jsx,json,css,scss}'
- 'web/package.json'
- 'web/package-lock.json'
- 'web/.eslintrc.js'
- 'web/.prettierrc.js'
- 'web/tsconfig.json'
- 'web/next.config.js'
- '.github/workflows/javascript-format-test-build.yml'
- web/**
- '!**.md'
jobs:
formatting:
name: Code formatting
if: github.actor != 'renovate[bot]' && github.actor != 'dependabot[bot]'
runs-on: ubuntu-latest
defaults:
run:
@@ -52,32 +28,18 @@ jobs:
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- name: Checkout
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Setup Nodejs
uses: actions/setup-node@v6
with:
node-version: '24.12.0'
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
persist-credentials: true
- name: Get changed files
id: changed-files-yaml
uses: tj-actions/changed-files@v47
uses: tj-actions/changed-files@v44
with:
path: 'web'
files_ignore: |
@@ -85,10 +47,10 @@ jobs:
web/next.config.js
files_yaml: |
src:
- '**/*.{js,ts,tsx,jsx,md}'
- '**/*.{js,ts,tsx,jsx,css,md}'
- name: Cache node modules
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-bundle-web-app
with:
@@ -102,27 +64,19 @@ jobs:
- name: Install Dependencies
run: npm install
- name: Lint and fix
if: steps.changed-files-yaml.outputs.src_any_changed == 'true' && github.event_name != 'pull_request'
- name: Lint
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx eslint --fix ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Lint
if: steps.changed-files-yaml.outputs.src_any_changed == 'true' && github.event_name == 'pull_request'
run: npx eslint ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Prettier formatting
if: steps.changed-files-yaml.outputs.src_any_changed == 'true' && github.event_name == 'pull_request'
- name: Prettier
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
run: npx prettier --write ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Prettier check
if: steps.changed-files-yaml.outputs.src_any_changed == 'true' && github.event_name != 'pull_request'
run: npx prettier ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}
- name: Debug changed files output
run: 'pwd && echo "Changed files: ${{ steps.changed-files-yaml.outputs.src_all_changed_files }}"'
- name: Commit changes
if: steps.changed-files-yaml.outputs.src_any_changed == 'true' && github.event_name != 'pull_request'
if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
uses: EndBug/add-and-commit@v9
with:
author_name: Owncast
@@ -134,7 +88,6 @@ jobs:
unused-code:
name: Test for unused code
if: github.actor != 'renovate[bot]' && github.actor != 'dependabot[bot]'
runs-on: ubuntu-latest
defaults:
run:
@@ -148,31 +101,16 @@ jobs:
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- name: Checkout
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Fetch base branch for comparison
if: github.event_name == 'pull_request'
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}.git || true
git fetch upstream ${{ github.event.pull_request.base.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- name: Setup Nodejs
uses: actions/setup-node@v6
with:
node-version: '24.12.0'
- name: Cache node modules
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-bundle-web-app
with:
@@ -189,17 +127,13 @@ jobs:
- name: Check for unused JS code and dependencies
run: npx knip --include dependencies,files,exports
- name: Run tests
working-directory: ./web
run: npm test
# After any formatting and linting is complete we can run the build
# and bundle step. This both will verify that the build is successful as
# well as commiting the updated static files into the repository for use.
web-bundle:
name: Build and bundle web project
runs-on: ubuntu-latest
if: github.repository == 'owncast/owncast' && !cancelled()
if: github.repository == 'owncast/owncast'
needs: [formatting, unused-code]
steps:
- id: skip_check
@@ -209,13 +143,8 @@ jobs:
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Setup Nodejs
uses: actions/setup-node@v6
with:
node-version: '24.12.0'
- name: Cache node modules
uses: actions/cache@v5
uses: actions/cache@v4
env:
cache-name: cache-node-modules-bundle-web-app
with:
@@ -226,16 +155,13 @@ jobs:
${{ runner.os }}-build-
${{ runner.os }}-
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
- name: Checkout
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
# Make sure the actual branch is checked out when running on pull requests
ref: ${{ github.event.pull_request.head.ref }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Bundle web app (next.js build)
run: build/web/bundleWeb.sh
+45
View File
@@ -0,0 +1,45 @@
name: Javascript Tests
on:
push:
paths:
- 'web/**'
pull_request:
paths:
- 'web/**'
jobs:
jest-run:
runs-on: ubuntu-latest
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18.9.0
- name: Cache node modules
uses: actions/cache@v4
env:
cache-name: cache-node-modules-javascript-tests
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Install Dependencies
working-directory: ./web
run: npm install
- name: Run tests
working-directory: ./web
run: npm test
-112
View File
@@ -1,112 +0,0 @@
# This workflow validates that required PR checklist items are checked
# Triggers:
# - PR opened, edited, or reopened
# Skips:
# - Organization members
# - Bot PRs (dependabot, renovate, etc.)
# Note: Looks for content between REQUIRED-CHECKLIST:START and REQUIRED-CHECKLIST:END
# markers and fails if any unchecked boxes exist in that section.
name: PR Checklist
on:
pull_request:
types: [opened, edited, reopened]
permissions:
contents: read
pull-requests: write
jobs:
validate-checklist:
name: Validate PR checklist
runs-on: ubuntu-latest
if: ${{ !contains(github.actor, '[bot]') }}
steps:
- name: Check organization membership
id: membership
uses: actions/github-script@v8
with:
script: |
try {
await github.rest.orgs.checkMembershipForUser({
org: 'owncast',
username: context.payload.pull_request.user.login
});
console.log('User is an organization member, skipping checklist validation');
return true;
} catch (error) {
if (error.status === 404 || error.status === 302) {
console.log('User is not an organization member, will validate checklist');
return false;
}
throw error;
}
- name: Validate required checkboxes
id: validate
if: ${{ steps.membership.outputs.result == 'false' }}
uses: actions/github-script@v8
with:
script: |
const prBody = context.payload.pull_request.body || '';
// Extract content between the markers
const markerPattern = /<!-- REQUIRED-CHECKLIST:START[\s\S]*?-->([\s\S]*?)<!-- REQUIRED-CHECKLIST:END -->/;
const match = prBody.match(markerPattern);
if (!match) {
core.setOutput('missing_checklist', 'true');
core.setFailed(
'## PR Checklist Validation Failed\n\n' +
'The required checklist section is missing from your PR description.\n\n' +
'Please do not remove the "Required checklist" section from the PR template. ' +
'If you need to restore this section, please edit your PR description to include it from [the original template](https://raw.githubusercontent.com/owncast/owncast/refs/heads/develop/.github/PULL_REQUEST_TEMPLATE.MD).'
);
return;
}
const checklistSection = match[1];
// Find all unchecked boxes in the section
const uncheckedPattern = /-\s*\[\s*\]\s*(.+)/g;
const uncheckedItems = [];
let uncheckedMatch;
while ((uncheckedMatch = uncheckedPattern.exec(checklistSection)) !== null) {
uncheckedItems.push(uncheckedMatch[1].trim());
}
if (uncheckedItems.length > 0) {
let message = '## PR Checklist Validation Failed\n\n';
message += 'The following required checklist items are not checked:\n\n';
for (const item of uncheckedItems) {
message += `- [ ] ${item}\n`;
}
message += '\nPlease check all items in the required checklist section to confirm you have completed them.';
core.setFailed(message);
} else {
console.log('All required checklist items are checked');
}
- name: Comment on PR about missing checklist
if: ${{ always() && steps.validate.outputs.missing_checklist == 'true' }}
uses: actions/github-script@v8
with:
script: |
const body =
'## PR Checklist Missing\n\n' +
'It looks like the **required checklist section** was removed from your PR description. ' +
'This section is needed for the PR checks to pass.\n\n' +
'Please edit your PR description and restore the checklist from ' +
'[the PR template](https://raw.githubusercontent.com/owncast/owncast/refs/heads/develop/.github/PULL_REQUEST_TEMPLATE.MD). ' +
'Once restored, check off each item to confirm you have completed them.\n\n' +
'The checklist section begins with `<!-- REQUIRED-CHECKLIST:START -->` and ends with `<!-- REQUIRED-CHECKLIST:END -->`.';
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
body: body
});
+58
View File
@@ -0,0 +1,58 @@
name: Take nightly screenshots
on:
schedule:
- cron: '0 4 * * *'
env:
BROWSERSTACK_KEY: ${{ secrets.BROWSERSTACK_KEY }}
BROWSERSTACK_PASSWORD: ${{ secrets.BROWSERSTACK_PASSWORD }}
BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
TEST_URL: http://localhost:8080
jobs:
Screenshots:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.22'
cache: true
- name: Cache node modules
uses: actions/cache@v4
env:
cache-name: cache-node-modules-screenshots
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('test/automated/screenshots/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Automate screenshots
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
max_attempts: 4
command: cd test/automated/screenshots && ./run.sh
- name: Commit changes
uses: EndBug/add-and-commit@v9
with:
author_name: Owncast
author_email: owncast@owncast.online
message: 'Commit screenshots'
add: '*.png'
pull: '--rebase --autostash'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Dispatch event to web site
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.BUNDLE_STORYBOOK_OWNCAST_ONLINE }}
repository: owncast/owncast.github.io
event-type: bundle-components-library
+1 -24
View File
@@ -1,11 +1,3 @@
# This workflow checks shell scripts for common issues using shellcheck
# Triggers:
# - Push/PR to develop with changes to .sh files
# Checks:
# - All shell scripts recursively
# - Reports info-level severity and above
# - Follows sourced files with -x flag
name: Lint
on:
@@ -28,22 +20,7 @@ jobs:
container:
image: docker.io/ubuntu:24.04
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
concurrent_skipping: 'same_content_newer'
cancel_others: 'true'
skip_after_successful_duplicate: 'true'
- name: Check out pull request code
uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Check out repository code
uses: actions/checkout@v6
if: github.event_name == 'push'
- uses: actions/checkout@v4
- name: Install shellcheck
run: apt update && apt install -y shellcheck bash && shellcheck --version
-49
View File
@@ -1,49 +0,0 @@
name: 'Close stale issues and PRs'
on:
schedule:
- cron: '30 */2 * * *'
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v10
with:
exempt-all-milestones: true
days-before-issue-stale: 60
days-before-issue-close: 67
exempt-issue-labels: backlog,long-lived,bot
exempt-all-issue-milestones: true
stale-issue-message: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. If this
was a feature request that others have shown no interest in, then it's
unlikely to get implemented due to lack of interest. If others also
want to see this feature then now is the time to say something! If this
is a bug report or you have questions that still need answering, please say
something. Feel free to drop by [our chat](https://owncast.rocket.chat) if
you'd like to discuss in real-time with people.
close-issue-message: >
This issue has been automatically closed due to inactivity. This isn't done
to be a jerk, or because the project doesn't care. But simply to keep the focus
on things that are actively discussed, and has continued interest from the community and
Owncast developers. Feel free to to comment if there is still discussion to be
had, or if you plan to work on it. Feel free to drop by [our chat](https://owncast.rocket.chat)
if you'd like to discuss in real-time with people. Thank you for being involved!
days-before-pr-stale: 30
days-before-pr-close: 37
exempt-pr-labels: backlog,long-lived,bot
exempt-all-pr-milestones: true
stale-pr-message: >
This pull request has not had any activity in 30 days. If it has been abandoned
no future actions are necessary, it will be automatically closed. If this is a PR
with no clear plan on how to move forward on it getting into the project, then
further discussion is needed. Now is a good time to discuss if this is still
something that should be worked on. If this PR is idle simply because nobody
has reviewed it, then feel free to ping somebody. However, if this PR is not linked to an
existing issue regarding something that was previously determined to be important, then even
more discussion needs to take place before it can get anywhere.
This PR will be closed if no further activity occurs. Thank you for your contributions!
close-pr-message: 'This PR was closed because it has been stalled for 10 days with no activity.'
-136
View File
@@ -1,136 +0,0 @@
# This workflow manages internationalization (i18n) with Crowdin
# Triggers:
# - Hourly schedule to sync translations
# - Push with changes to translation files or config
# Process:
# 1. Extracts translatable strings from source code
# 2. Uploads source strings to Crowdin
# 3. Downloads translated strings from Crowdin
# 4. Creates PR with updated translations
# Note: Uses concurrency control to prevent overlapping runs
name: Translation job
on:
schedule:
# Run the workflow every hour
- cron: '0 * * * *'
push:
branches:
- develop
paths:
- 'web/i18n/en/translation.json'
- 'web/**/*.tsx'
- 'web/**/*.js'
- 'crowdin.yml'
- '.github/workflows/translations.yml'
- 'web/i18next-parser.config.mjs'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
generate-translations:
defaults:
run:
working-directory: ./web
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0
- name: Configure Git
run: |
git config --global user.name "Owncast"
git config --global user.email "owncast@owncast.online"
- name: Install dependencies
run: npm install
- name: Generate translation files
run: npm run translate
- name: Upload sources to Crowdin
uses: crowdin/github-action@v2
with:
upload_sources: true
upload_translations: false
download_translations: false
config: crowdin.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
- name: Download translations from Crowdin
uses: crowdin/github-action@v2
with:
upload_sources: false
upload_translations: false
download_translations: true
skip_untranslated_strings: true
export_only_approved: false
push_translations: false
commit_message: 'chore(i18n): update translations from Crowdin'
localization_branch_name: crowdin-translations
create_pull_request: false
config: crowdin.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
- name: Check for translation changes
id: changes
run: |
cd ..
git add web/i18n/
if git diff --cached --quiet; then
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No translation changes detected"
else
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "Translation changes detected"
fi
- name: Create Pull Request
if: steps.changes.outputs.has_changes == 'true'
id: create_pr
uses: peter-evans/create-pull-request@v8
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: crowdin-translations
base: develop
title: 'chore(i18n): update translations from Crowdin'
body: |
This PR contains updated translations downloaded from Crowdin.
**Changes include:**
- Updated translation files in `web/i18n/`
- Translations synchronized from Crowdin project
Please review the changes and merge if they look correct.
---
This PR was created automatically by the translation workflow.
commit-message: 'chore(i18n): update translations from Crowdin'
author: 'Owncast <owncast@owncast.online>'
committer: 'Owncast <owncast@owncast.online>'
add-paths: |
web/i18n/**
delete-branch: true
- name: Log PR creation result
if: steps.create_pr.outputs.pull-request-number
run: |
echo "Created PR #${{ steps.create_pr.outputs.pull-request-number }}"
echo "PR URL: ${{ steps.create_pr.outputs.pull-request-url }}"
- name: No changes detected
if: steps.changes.outputs.has_changes == 'false'
run: echo "No translation changes to commit"
+1 -3
View File
@@ -4,7 +4,6 @@
*.dll
*.so
*.dylib
.DS_Store
# Test binary, built with `go test -c`
*.test
@@ -40,11 +39,10 @@ backup/
!core/data
test/test.db
test/automated/browser/screenshots
lefthook-local.yml
lefthook.yml
test/automated/browser/cypress/screenshots
test/automated/browser/cypress/videos
web/style-definitions/build/
web/public/sw.js
web/public/workbox-*.js
bin/
+66 -49
View File
@@ -1,78 +1,95 @@
version: "2"
run:
go: "1.25.3"
modules-download-mode: readonly
tests: false
modules-download-mode: readonly
# Define the Go version limit.
# Mainly related to generics support in go1.18.
# Default: use Go version from the go.mod file, fallback on the env var `GOVERSION`, fallback on 1.18
go: '1.22'
issues:
# The linter has a default list of ignorable errors. Turning this on will enable that list.
exclude-use-default: false
# Maximum issues count per one linter. Set to 0 to disable. Default is 50.
max-issues-per-linter: 0
# Maximum count of issues with the same text. Set to 0 to disable. Default is 3.
max-same-issues: 0
exclude:
- Subprocess launch(ed with variable|ing should be audited)
- Error return value of .((os\.)?std(out|err)\..*|.*Close|.*Flush|os\.Remove(All)?|.*print(f|ln)?|os\.(Un)?Setenv). is not checked
- G307 # Allow closing files as a defer without checking error.
- composite literal uses unkeyed fields
linters:
enable:
- bodyclose
- copyloopvar
- cyclop
- dupl
- forbidigo
- errcheck
- exportloopref
- goconst
- gocritic
- godot
- godox
- goimports
- goprintffuncname
- gosec
- govet
- misspell
- nakedret
- prealloc
- revive
- rowserrcheck
- sqlclosecheck
- staticcheck
- unconvert
- unparam
- wastedassign
- whitespace
settings:
- nakedret
- cyclop
- gosimple
- unused
- exportloopref
- gocritic
- forbidigo
- unparam
- wastedassign
linters-settings:
govet:
disable:
- composite
cyclop:
# the maximal code complexity to report. default is 10. eventually work our way to that.
max-complexity: 15
package-average: 0
dupl:
threshold: 200
forbidigo:
forbid:
- pattern: ^(fmt\.Print(|f|ln)|print|println)
- pattern: ^panic.*$
# the max average package complexity. If it's higher than 0.0 (float) the check is enabled (default 0.0)
package-average: 0.0
# should ignore tests
skip-tests: true
gosimple:
# Select the Go version to target. The default is '1.13'.
go: '1.22'
# https://staticcheck.io/docs/options#checks
checks: ['all']
gocritic:
disabled-checks:
- ifElseChain
- exitAfterDefer
revive:
rules:
- name: package-comments
disabled: true
exclusions:
generated: lax
rules:
- path: (.+)\.go$
text: Subprocess launch(ed with variable|ing should be audited)
- path: (.+)\.go$
text: Error return value of .((os\.)?std(out|err)\..*|.*Close|.*Flush|os\.Remove(All)?|.*print(f|ln)?|os\.(Un)?Setenv). is not checked
- path: (.+)\.go$
text: G307
- path: (.+)\.go$
text: composite literal uses unkeyed fields
- path: (.+)\.go$
text: 'SA1019.*aws-sdk-go.*deprecated'
- linters:
- cyclop
path: (.+)_test\.go
paths:
- third_party$
- builtin$
- examples$
issues:
max-issues-per-linter: 0
max-same-issues: 0
formatters:
enable:
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
forbidigo:
# Forbid the following identifiers (identifiers are written using regexp):
forbid:
# Logging via Print bypasses our logging framework.
- ^(fmt\.Print(|f|ln)|print|println)
- ^panic.*$
dupl:
# tokens count to trigger issue, 150 by default
threshold: 200
-127
View File
@@ -1,127 +0,0 @@
# Owncast AI Agent Instructions
This is a repository consisting of a Go backend and a React frontend. It supports both an internal web application that is public facing, an internal admin web application, internal-only APIs for powering these web interfaces, and a set of APIs for third parties to take advantage of.
## Quick Reference
| Task | Command |
| ----------------------- | ----------------------------------- |
| Build backend | `go build -o owncast .` |
| Run backend | `go run main.go` |
| Run frontend dev server | `cd web && npm run dev` |
| Run Go tests | `go test ./...` |
| Run JS tests | `cd web && npm test` |
| Lint Go | `make lint` |
| Format Go | `make fmt` |
| Lint JS/CSS | `cd web && npm run lint` |
| Format JS/CSS | `cd web && npm run format` |
| Full web CI check | `cd web && npm run check` |
| Build Storybook | `cd web && npm run build-storybook` |
| Generate API code | `make api-generate` |
| Generate DB code | `make sqlc` |
| Install dev tools | `make install-tools` |
| Install git hooks | `make install-hooks` |
| Start test stream | `./test/ocTestStream.sh` |
## Code Standards
- UI component standards can be found in https://docs.owncast.dev/develop-frontend-components
### Required Before Each Commit
- Ensure all code is formatted using `golangci-lint` for Go, `stylelint` for CSS, and `prettier` for JavaScript/React.
- Fix any formatting or linting errors.
### Development Flow
- Build: Ensure the code builds successfully using `go build` for Go and `npm run build` for React in the `web` directory.
- Tests: Write unit tests for Go code and Javascript logic. Use `go test` and `npm run test`.
- UI components: Components should be standalone and reusable and show up in Storybook to enable testing, prototyping, and iteration.
- API: All APIs should be documented using OpenAPI specifications. Use `build/gen-api.sh` to generate the stubs, and then fill them in with your actual API implementation.
- If you made new UI components or made changes ensure that Storybook still builds by running `npm run build-storybook`.
## Repository Structure
- `web/`: The web source code for the React frontend.
- Root of the repo: Go source code for the backend.
- `static/web/`: Generated static files for the web application. Do not edit or commit files in this directory directly; they are generated from the `web` directory. Ignore this.
- `test/automated/api/`: A series of automated integration tests for API endpoints written in Javascript.
- `test/automated/browser/`: A series of automated browser UI tests for actual real-world browser interaction.
- `build/`: Build and code generation scripts.
### Backend Structure
- `main.go`: Entry point. Initializes logging, database, config, core services, metrics, and the HTTP router.
- `webserver/router/`: Chi (v5) HTTP router with HTTP/2 support. Routes organized under `/api/`, `/api/admin/`, federation endpoints, `/hls/`, and `/ws`.
- `webserver/handlers/`: Request handlers for web, admin, static files, HLS streaming.
- `webserver/handlers/generated/`: Auto-generated API types and Chi server stubs from OpenAPI spec. Do not hand-edit.
- `webserver/router/middleware/`: Authentication (`RequireAdminAuth()` for admin endpoints via HTTP Basic Auth) and ActivityPub content negotiation middleware.
- `persistence/`: Repository pattern implementations (ConfigRepository, UserRepository, ChatMessageRepository, WebhookRepository, AuthRepository). All database access goes through these.
- `db/`: sqlc-generated type-safe database code. Schema in `db/schema.sql`, queries in `db/query.sql`. Run `make sqlc` after modifying.
- `core/`: Core streaming logic including transcoder, chat, webhooks, storage providers, playlist management.
- `core/storageproviders/`: Video storage backends (local filesystem and S3).
- `core/transcoder/`: FFmpeg-based video transcoding.
- `activitypub/`: Federation support (controllers, inbox/outbox, HTTP signatures, WebFinger, NodeInfo, worker pool).
- `models/`: Shared data models.
- `config/`: Configuration package with defaults and constants.
- `tools/`: Separate `go.mod` for development tool dependencies installed to `./bin/`.
### Frontend Structure
- `web/pages/`: Next.js 14 pages with static export.
- `web/components/`: React components organized by domain (`admin/`, `chat/`, `common/`, `layouts/`, `modals/`, `ui/`, `video/`, `action-buttons/`).
- `web/components/stores/`: Recoil state management stores.
- `web/interfaces/`: TypeScript interfaces.
- `web/i18n/`: Localization files (next-export-i18n).
- `web/style-definitions/`: Design token definitions.
### Key Technical Details
- **Go version**: 1.24+ (see `go.mod`)
- **Database**: SQLite (single file, no external DB server needed)
- **Frontend framework**: Next.js 14, React 18, TypeScript, static export
- **UI library**: Ant Design v4 with Less theming
- **State management**: Recoil
- **Styling**: SCSS with component-scoped files, Less for Ant Design
- **API spec**: OpenAPI 3.1 at `openapi.yaml`
- **Commit style**: Conventional Commits (`type(scope): description`)
- **Branches**: `develop` is the development branch; `master` is the production/release branch
- **CI**: GitHub Actions for Go tests, JS linting/testing, browser tests, Storybook builds, Chromatic visual regression, CodeQL security scanning
- **Go linting**: golangci-lint with cyclop (max complexity 15), dupl (threshold 200), gosec, gocritic, forbidigo (no `fmt.Print*`, `print`, `println`, or `panic`), and others. Full config in `.golangci.yml`.
- **JS linting**: ESLint with airbnb config, Prettier, Stylelint, knip for unused code detection
- **Makefile**: For common tasks like building, testing, linting, code generation, and installing dev tools/hooks. Run `make help` for a list of functions.
## Testing Web Frontend
Testing the web frontend requires running the frontend development server and the backend service together. The frontend development server must be started using `npm run dev` in the `web` directory, which will serve the web application at `http://localhost:3000`. The backend service can be started using `go run main.go` in the root of the repository.
Do not access the web application via `http://localhost:8080` or build the web project to copy files to the `static/web` directory for local development.
## Key Guidelines
1. All APIs are to be documented using OpenAPI specifications and code is to be generated using `build/gen-api.sh`. Additional details can be found at https://docs.owncast.dev/api-web-routing.
2. Write API tests for all new endpoints in the `test/automated/api` directory.
3. Use the `test/automated/browser` directory for browser-based tests for new functionality that simulate user interactions.
4. All user-facing frontend UI strings need to support localization. Use the `Translation` component to show most displayable strings. To create dynamic translated strings use the `next-export-i18n` library and the `t()` function. But use the `Translation` component unless there is a reason not to as it allows you to set default text. Read https://docs.owncast.dev/web-translations for more details. Test localization by adding "?lang=XX" with XX being a country code, such as "de" for German. Strings that have not yet been translated will not show as changed, but it's good to test anyway to make sure that previously translated strings have not been broken or regressed in any way.
5. A link to the PR's Storybook on Chromatic via the PR's Chromatic job should be included to help with review.
6. For API changes a before and after example of the API response should be added to the pull request to help with review.
7. For backend changes, a before and after example of logs to demonstrate the change should be added to the pull request to help with review.
8. Do not build the web project or copy the files to the `static/web` directory for local development.
9. When running the frontend development server, the local backend service must also be running. You can run the backend service using `go run main.go` in the root of the repository.
10. The credentials for the backend development backend are username: admin and password: abc123 and uses HTTP Basic Auth. This is used for the admin web application and the admin APIs.
11. The admin is found at `/admin`.
12. If a live stream video is needed to run, you can run `./test/ocTestStream.sh` to start an actual stream that will begin streaming from the local development server.
13. You should never commit the `static/web` directory to the repository. It is generated from the `web` directory and should be ignored in your commits.
14. Don't use emoji in code comments or commit messages. That's lame.
15. User interface components should use Ant Design v4 components, not Ant Design v5.
16. Database access should be through the repository patterns.
17. Any substantial new features or changes around Federation or ActivityPub should be documented in FEDERATION.md.
## External Documentation
- Project documentation: https://docs.owncast.dev
- Frontend component guide: https://docs.owncast.dev/develop-frontend-components
- API and routing: https://docs.owncast.dev/api-web-routing
- Localization: https://docs.owncast.dev/web-translations
- Contributing guide: https://docs.owncast.dev/contributor-guide
- Federation protocol: `FEDERATION.md`
-15
View File
@@ -7,18 +7,3 @@ We abide by our [Code of Conduct](https://owncast.online/contribute/) and feel s
We’ve been very lucky to have this so far, so maybe you can help us with your skills and passion, too!
There is a larger, more detailed, and more up-to-date [guide for helping contribute to Owncast on our website](https://owncast.online/help/).
## Before Submitting a PR
For web changes, from the `web/` directory:
```bash
npm run check # Verify your code will pass CI
npm run lint && npm run format # Auto-fix errors
```
For Go changes, from the repository root:
```bash
make lint && make fmt
```
+1 -1
View File
@@ -22,7 +22,7 @@ ENV NAME=${NAME}
RUN CGO_ENABLED=1 GOOS=linux go build -a -installsuffix cgo -ldflags "-extldflags \"-static\" -s -w -X github.com/owncast/owncast/config.GitCommit=$GIT_COMMIT -X github.com/owncast/owncast/config.VersionNumber=$VERSION -X github.com/owncast/owncast/config.BuildPlatform=$NAME" -o owncast .
# Create the image by copying the result of the build into a new alpine image
FROM alpine:3.23.3
FROM alpine:3.20.1
RUN apk update && apk add --no-cache ffmpeg ffmpeg-libs ca-certificates && update-ca-certificates
RUN addgroup -g 101 -S owncast && adduser -u 101 -S owncast -G owncast
+14 -69
View File
@@ -1,6 +1,6 @@
VERSION --new-platform 0.6
FROM --platform=linux/amd64 alpine:3.23.3
FROM --platform=linux/amd64 alpine:3.15.5
ARG version=develop
WORKDIR /build
@@ -16,12 +16,11 @@ docker-all:
crosscompiler:
# This image is missing a few platforms, so we'll add them locally
FROM --platform=linux/amd64 ghcr.io/gabek/go-crosscompile:latest
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
RUN apk add --update --no-cache tar gzip upx >> /dev/null
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/armv7l-linux-musleabihf-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/i686-linux-musl-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/x86_64-linux-musl-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
RUN curl -sfL "https://owncast-infra.nyc3.cdn.digitaloceanspaces.com/build/aarch64-linux-musl-cross.tgz" | tar zxf - -C /usr/ --strip-components=1
code:
FROM --platform=linux/amd64 +crosscompiler
@@ -84,40 +83,11 @@ build:
# See https://github.com/upx/upx/issues/612
IF [ "$GOOS" != "darwin" ]
RUN upx --best --lzma owncast
# Test the binary integrity
# Test the binary
RUN upx -t owncast
END
# Sanity check: verify the binary runs without immediate crash.
# We can only run Linux binaries in this container. macOS binaries are skipped.
# The check downloads a static ffmpeg (required dependency), starts the binary,
# waits 3 seconds, then verifies the process is still running.
IF [ "$GOOS" = "linux" ]
IF [ "$TARGETPLATFORM" = "linux/amd64" ]
# Native architecture - run directly
RUN curl -sL -o /usr/local/bin/ffmpeg https://github.com/eugeneware/ffmpeg-static/releases/download/b6.1.1/ffmpeg-linux-x64 && chmod +x /usr/local/bin/ffmpeg
RUN ./owncast & PID=$!; sleep 3; if kill -0 $PID 2>/dev/null; then kill $PID; echo "Sanity check passed: binary runs on linux/amd64"; else echo "Sanity check FAILED: binary crashed on linux/amd64"; exit 1; fi
ELSE IF [ "$TARGETPLATFORM" = "linux/arm64" ]
# ARM64 - use QEMU
RUN apk add --no-cache qemu-aarch64 >> /dev/null
RUN curl -sL -o /usr/local/bin/ffmpeg https://github.com/eugeneware/ffmpeg-static/releases/download/b6.1.1/ffmpeg-linux-arm64 && chmod +x /usr/local/bin/ffmpeg
RUN qemu-aarch64 ./owncast & PID=$!; sleep 3; if kill -0 $PID 2>/dev/null; then kill $PID; echo "Sanity check passed: binary runs on linux/arm64"; else echo "Sanity check FAILED: binary crashed on linux/arm64"; exit 1; fi
ELSE IF [ "$TARGETPLATFORM" = "linux/arm/v7" ]
# ARMv7 - use QEMU
RUN apk add --no-cache qemu-arm >> /dev/null
RUN curl -sL -o /usr/local/bin/ffmpeg https://github.com/eugeneware/ffmpeg-static/releases/download/b6.1.1/ffmpeg-linux-arm && chmod +x /usr/local/bin/ffmpeg
RUN qemu-arm ./owncast & PID=$!; sleep 3; if kill -0 $PID 2>/dev/null; then kill $PID; echo "Sanity check passed: binary runs on linux/arm/v7"; else echo "Sanity check FAILED: binary crashed on linux/arm/v7"; exit 1; fi
ELSE IF [ "$TARGETPLATFORM" = "linux/386" ]
# 32-bit x86 - use QEMU
RUN apk add --no-cache qemu-i386 >> /dev/null
RUN curl -sL -o /usr/local/bin/ffmpeg https://github.com/eugeneware/ffmpeg-static/releases/download/b6.1.1/ffmpeg-linux-ia32 && chmod +x /usr/local/bin/ffmpeg
RUN qemu-i386 ./owncast & PID=$!; sleep 3; if kill -0 $PID 2>/dev/null; then kill $PID; echo "Sanity check passed: binary runs on linux/386"; else echo "Sanity check FAILED: binary crashed on linux/386"; exit 1; fi
END
ELSE
RUN echo "Skipping sanity check for $TARGETPLATFORM (cannot execute on Linux)"
END
SAVE ARTIFACT --keep-ts owncast owncast
SAVE ARTIFACT owncast owncast
package:
RUN apk add --update --no-cache zip >> /dev/null
@@ -139,19 +109,21 @@ package:
ARG NAME=custom
END
COPY --keep-ts (+build/owncast --platform $TARGETPLATFORM) /build/dist/owncast
COPY (+build/owncast --platform $TARGETPLATFORM) /build/dist/owncast
ENV ZIPNAME owncast-$version-$NAME.zip
RUN cd /build/dist && zip -r -q -8 /build/dist/owncast.zip .
SAVE ARTIFACT --keep-ts /build/dist/owncast.zip owncast.zip AS LOCAL dist/$ZIPNAME
docker-image:
# Internal target that builds the docker image. Used by +docker for testing.
docker:
# Multiple image names can be tagged at once. They should all be passed
# in as space separated strings using the full account/repo:tag format.
# https://github.com/earthly/earthly/blob/aea38448fa9c0064b1b70d61be717ae740689fb9/docs/earthfile/earthfile.md#assigning-multiple-image-names
ARG TARGETPLATFORM
FROM --platform=$TARGETPLATFORM alpine:3.23.3
FROM --platform=$TARGETPLATFORM alpine:3.15.5
RUN apk update && apk add --no-cache ffmpeg ffmpeg-libs ca-certificates unzip && update-ca-certificates
RUN addgroup -g 101 -S owncast && adduser -u 101 -S owncast -G owncast
WORKDIR /app
COPY --keep-ts --platform=$TARGETPLATFORM +package/owncast.zip /app
COPY --platform=$TARGETPLATFORM +package/owncast.zip /app
RUN unzip -x owncast.zip && mkdir data
# temporarily disable until we figure out how to move forward
@@ -161,34 +133,7 @@ docker-image:
ENTRYPOINT ["/app/owncast"]
EXPOSE 8080 1935
docker:
# Multiple image names can be tagged at once. They should all be passed
# in as space separated strings using the full account/repo:tag format.
# https://github.com/earthly/earthly/blob/aea38448fa9c0064b1b70d61be717ae740689fb9/docs/earthfile/earthfile.md#assigning-multiple-image-names
ARG TARGETPLATFORM
ARG images=ghcr.io/owncast/owncast:testing
# Sanity check: run the container to verify it starts without crashing.
# Only run for linux/amd64 as other architectures would need QEMU emulation.
IF [ "$TARGETPLATFORM" = "linux/amd64" ]
WITH DOCKER --load owncast:sanity-test=+docker-image
RUN docker run -d --name owncast-sanity-test owncast:sanity-test && \
sleep 5 && \
if docker ps | grep -q owncast-sanity-test; then \
echo "Docker sanity check passed: container runs on $TARGETPLATFORM"; \
docker stop owncast-sanity-test; \
else \
echo "Docker sanity check FAILED: container crashed on $TARGETPLATFORM"; \
docker logs owncast-sanity-test; \
exit 1; \
fi
END
ELSE
RUN echo "Skipping docker sanity check for $TARGETPLATFORM (only runs on linux/amd64)"
END
FROM --platform=$TARGETPLATFORM +docker-image
RUN echo "Saving images: ${images}"
# Tag this image with the list of names
@@ -201,13 +146,13 @@ dockerfile:
FROM DOCKERFILE -f Dockerfile .
unit-tests:
FROM --platform=linux/amd64 ghcr.io/gabek/go-crosscompile:latest
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
COPY . /build
WORKDIR /build
RUN go test ./...
api-tests:
FROM --platform=linux/amd64 ghcr.io/gabek/go-crosscompile:latest
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
RUN apk add npm font-noto && fc-cache -f
COPY . /build
WORKDIR /build/test/automated/api
@@ -215,7 +160,7 @@ api-tests:
RUN ./run.sh
hls-tests:
FROM --platform=linux/amd64 ghcr.io/gabek/go-crosscompile:latest
FROM --platform=linux/amd64 bdwyertech/go-crosscompile
RUN apk add npm font-noto && fc-cache -f
COPY . /build
WORKDIR /build/test/automated/hls
-134
View File
@@ -1,134 +0,0 @@
# Federation
This document describes how Owncast federates with other ActivityPub servers, following [FEP-67ff](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md).
## Supported Federation Protocols and Standards
- [ActivityPub](https://www.w3.org/TR/activitypub/) (Server-to-Server)
- [WebFinger](https://webfinger.net/)
- [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures)
- [NodeInfo](https://nodeinfo.diaspora.software/)
## Supported FEPs
- [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md)
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
## Actor
Owncast servers present as a single `Service` actor (not `Person`). Each Owncast instance has one actor representing the live stream.
The actor profile includes:
- `icon`: Server logo/avatar
- `image`: Profile banner image
- `summary`: Server description (HTML)
- `attachment`: Array of `PropertyValue` objects containing social links and metadata
- `tag`: Hashtags describing the server content
- `manuallyApprovesFollowers`: `true` if the server operates in private mode
- `discoverable`: Always `true`
Owncast actors do not follow other actors. The `following` collection endpoint returns 404.
## Activities
| Activity | Object | Send | Receive | Notes |
|----------|--------|:----:|:-------:|-------|
| `Create` | `Note` | Yes | No | Sent on go-live and manual fediverse posts. Incoming posts are ignored. |
| `Update` | `Service` | Yes | No | Sent when server profile changes. |
| `Update` | `Person` | No | Yes | Updates cached follower profile information. |
| `Follow` | - | No | Yes | Queued for approval if private mode is enabled. |
| `Accept` | `Follow` | Yes | No | Sent automatically unless in private mode. |
| `Undo` | `Follow` | No | Yes | Removes the follower. |
| `Like` | `Note` | No | Yes | Optionally displayed in live chat. |
| `Announce` | `Note` | No | Yes | Optionally displayed in live chat. |
Owncast is a broadcast-only service. It does not follow other actors or accept incoming posts.
## Notes
Posts from Owncast are `Note` objects with:
- `content`: HTML-formatted text with linked hashtags
- `attachment`: `Image` object with stream thumbnail (for go-live posts)
- `tag`: Array containing `Hashtag` and `Mention` objects
- `sensitive`: `true` if the stream is marked NSFW
### Go-Live Announcements
When a stream starts, Owncast sends a `Create(Note)` containing:
- The configured go-live message (or default announcement)
- A thumbnail image of the stream
- Hashtags configured for the server
- A link to watch the stream
### Hashtags
Hashtags use `toot:Hashtag` type and link to `https://directory.owncast.online/tags/{tag}` for discovery across Owncast instances.
## Addressing
Public posts are addressed to:
```json
{
"to": ["https://www.w3.org/ns/activitystreams#Public"],
"cc": ["{actor}/followers"]
}
```
Owncast supports sending direct messages to specific actors (used for replying to engagement). These include the recipient in `cc` with a corresponding `Mention` tag for Mastodon compatibility.
## HTTP Signatures
**Outbound**: All POST requests are signed using RSA-SHA256. Signed headers: `(request-target)`, `host`, `date`, `digest`.
**Inbound**: All POST requests to the inbox must have a valid signature. Unsigned requests are rejected.
**GET requests** to actor, outbox, and followers endpoints do not require signatures.
## WebFinger
Actor discovery via `/.well-known/webfinger?resource=acct:{username}@{domain}`
Response links include:
- `self` (application/activity+json): Actor document
- `http://webfinger.net/rel/profile-page`: Web profile
- `http://webfinger.net/rel/avatar`: Profile image
- `alternate` (application/x-mpegURL): HLS stream URL
The HLS stream link allows clients to discover the live stream URL directly from WebFinger.
## NodeInfo
Available at `/.well-known/nodeinfo` with NodeInfo 2.0 at `/nodeinfo/2.0`.
Additional endpoints:
- `/.well-known/x-nodeinfo2`: Extended format
- `/api/v1/instance`: Mastodon-compatible instance information
- `/.well-known/host-meta`: WebFinger discovery
## Private Mode
Owncast can operate in private mode where:
- `manuallyApprovesFollowers` is `true` on the actor
- Follow requests are queued for admin approval
- `Accept` is only sent after manual approval
## Blocking
Owncast supports blocking at the domain and individual actor level. Blocked entities:
- Cannot deliver to the inbox (requests rejected)
- Do not receive activities from the server
## Interoperability Notes
- Owncast uses `Service` actor type, not `Person`
- There is no `following` collection (Owncast does not follow accounts)
- Owncast does not accept incoming posts (`Create` from remote actors is ignored)
- Engagement (`Like`, `Announce`) can be displayed in the live chat if enabled
- The `sensitive` flag indicates NSFW content for the entire stream
## Additional Resources
- [Owncast Documentation](https://owncast.online/docs/)
- [ActivityPub Specification](https://www.w3.org/TR/activitypub/)
- [Fediverse Enhancement Proposals](https://codeberg.org/fediverse/fep)
-82
View File
@@ -1,82 +0,0 @@
# Development tools are managed in tools/go.mod and installed to ./bin
GOBIN := $(shell pwd)/bin
# Tool binaries
LEFTHOOK := $(GOBIN)/lefthook
GOLANGCI_LINT := $(GOBIN)/golangci-lint
GOFUMPT := $(GOBIN)/gofumpt
SQLC := $(GOBIN)/sqlc
OAPI_CODEGEN := $(GOBIN)/oapi-codegen
.PHONY: install-tools install-hooks lint fmt sqlc api-generate build test clean
## Install all development tools to ./bin
install-tools: $(LEFTHOOK) $(GOLANGCI_LINT) $(GOFUMPT) $(SQLC) $(OAPI_CODEGEN)
$(LEFTHOOK):
GOBIN=$(GOBIN) go install -C tools github.com/evilmartians/lefthook
$(GOLANGCI_LINT):
GOBIN=$(GOBIN) go install -C tools github.com/golangci/golangci-lint/v2/cmd/golangci-lint
$(GOFUMPT):
GOBIN=$(GOBIN) go install -C tools mvdan.cc/gofumpt
$(SQLC):
GOBIN=$(GOBIN) go install -C tools github.com/sqlc-dev/sqlc/cmd/sqlc
$(OAPI_CODEGEN):
GOBIN=$(GOBIN) go install -C tools github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen
## Install git hooks using lefthook
install-hooks: $(LEFTHOOK)
$(LEFTHOOK) install
@echo "Patching hooks to use ./bin/lefthook..."
@for hook in .git/hooks/*; do \
if [ -f "$$hook" ] && grep -q 'call_lefthook' "$$hook" && ! grep -q 'export LEFTHOOK_BIN' "$$hook"; then \
sed -i.bak 's|call_lefthook|export LEFTHOOK_BIN="$$(git rev-parse --show-toplevel)/bin/lefthook"; call_lefthook|' "$$hook" && rm -f "$$hook.bak"; \
fi \
done
@echo "Hooks installed successfully"
## Run golangci-lint
lint: $(GOLANGCI_LINT)
$(GOLANGCI_LINT) run ./...
## Format Go code with gofumpt
fmt: $(GOFUMPT)
$(GOFUMPT) -l -w .
## Generate database models with sqlc
sqlc: $(SQLC)
$(SQLC) generate
## Generate API code from OpenAPI spec
api-generate: $(OAPI_CODEGEN)
./build/gen-api.sh
## Build the application
build:
go build -o owncast .
## Run tests
test:
go test ./...
## Clean build artifacts
clean:
rm -rf bin/
rm -f owncast
## Show help
help:
@echo "Available targets:"
@echo " install-tools - Install all development tools to ./bin"
@echo " install-hooks - Install git hooks using lefthook"
@echo " lint - Run golangci-lint"
@echo " fmt - Format Go code with gofumpt"
@echo " sqlc - Generate database models"
@echo " api-generate - Generate API code from OpenAPI spec"
@echo " build - Build the application"
@echo " test - Run tests"
@echo " clean - Remove build artifacts and tools"
+2 -7
View File
@@ -28,6 +28,7 @@
<a href="https://github.com/owncast/owncast/issues">Report Bug</a>
</p>
<!-- TABLE OF CONTENTS -->
## Table of Contents
@@ -105,7 +106,7 @@ The Owncast backend is a service written in Go.
1. Ensure you have prerequisites installed.
- C compiler, such as [GCC compiler](https://gcc.gnu.org/install/download.html) or a [Musl-compatible compiler](https://musl.libc.org/)
- [ffmpeg](https://ffmpeg.org/download.html)
1. Install the [Go toolchain](https://golang.org/dl/) (1.24 or above).
1. Install the [Go toolchain](https://golang.org/dl/) (1.22 or above).
1. Clone the repo. `git clone https://github.com/owncast/owncast`
1. `go run main.go` will run from the source.
1. Visit `http://yourserver:8080` to access the web interface or `http://yourserver:8080/admin` to access the admin.
@@ -132,7 +133,6 @@ If you're new to the project, maybe you'd be interested in looking at [![Good Fi
There is a larger, more detailed, and more up-to-date [guide for helping contribute to Owncast on our website](https://owncast.online/help/).
### Donors
The Owncast project is possible thanks to the people who make a donation to support us and our work.
Thank you to all our donors who help keep Owncast running by donating on OpenCollective. You can support this project by [becoming a backer/sponsor](https://opencollective.com/owncast#suppor).
@@ -171,11 +171,6 @@ style="vertical-align: middle;margin-left:5px" width="147" height="26"
<img src="https://owncast.online/images/sponsors/chromatic.png" height="26" style="vertical-align: middle;margin-left:5px">
</a>
</li>
<li>Infrastructure and hosting by
<a href="https://digitalocean.com?utm_medium=opensource&utm_source=owncast" target="_blank">
<img src="https://owncast.online/images/sponsors/digitalocean.svg" height="26" style="vertical-align: middle;margin-left:5px">
</a>
</li>
</ul>
<!-- CONTACT -->
+6 -47
View File
@@ -3,12 +3,9 @@ package activitypub
import (
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/inbox"
"github.com/owncast/owncast/activitypub/jobs"
"github.com/owncast/owncast/activitypub/outbox"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/workerpool"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
@@ -17,55 +14,19 @@ import (
// Start will initialize and start the federation support.
func Start(datastore *data.Datastore) {
configRepository := configrepository.Get()
persistence.Setup(datastore)
outboundWorkerPoolSize := getOutboundWorkerPoolSize()
workerpool.InitOutboundWorkerPool(outboundWorkerPoolSize)
workerpool.InitOutboundWorkerPool()
inbox.InitInboxWorkerPool()
// Generate the keys for signing federated activity if needed.
if configRepository.GetPrivateKey() == "" {
if data.GetPrivateKey() == "" {
privateKey, publicKey, err := crypto.GenerateKeys()
_ = configRepository.SetPrivateKey(string(privateKey))
_ = configRepository.SetPublicKey(string(publicKey))
_ = data.SetPrivateKey(string(privateKey))
_ = data.SetPublicKey(string(publicKey))
if err != nil {
log.Errorln("Unable to get private key", err)
}
}
// Start follower validation background job.
jobs.StartFollowerValidationJob()
}
func getOutboundWorkerPoolSize() int {
// Use a reasonable fixed worker pool size instead of scaling with followers
// This prevents excessive resource usage when streamers have many followers
const (
minWorkers = 10 // Minimum workers for small instances
maxWorkers = 50 // Maximum workers to prevent resource exhaustion
defaultWorkers = 20 // Default for most instances
)
followersRepo := followersrepository.Get()
var followerCount int64
fc, err := followersRepo.GetCount()
if err != nil {
log.Errorln("Unable to get follower count", err)
return defaultWorkers
}
followerCount = fc
// Scale more conservatively: start with base workers, add 1 worker per 100 followers
// This gives a much more reasonable scaling than the previous followerCount * 5
workers := minWorkers + int(followerCount/100)
if workers > maxWorkers {
workers = maxWorkers
}
log.Debugf("Initializing ActivityPub outbound worker pool with %d workers for %d followers", workers, followerCount)
return workers
}
// SendLive will send a "Go Live" message to followers.
@@ -85,12 +46,10 @@ func SendDirectFederatedMessage(message, account string) error {
// GetFollowerCount will return the local tracked follower count.
func GetFollowerCount() (int64, error) {
followersRepo := followersrepository.Get()
return followersRepo.GetCount()
return persistence.GetFollowerCount()
}
// GetPendingFollowRequests will return the pending follow requests.
func GetPendingFollowRequests() ([]models.Follower, error) {
followersRepo := followersrepository.Get()
return followersRepo.GetPendingFollowRequests()
return persistence.GetPendingFollowRequests()
}
+3 -21
View File
@@ -6,7 +6,7 @@ import (
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
)
// PrivacyAudience represents the audience for an activity.
@@ -31,20 +31,6 @@ func MakeNotePublic(note vocab.ActivityStreamsNote) vocab.ActivityStreamsNote {
return note
}
func MakeAddressingToFollowers(followers_iri *url.URL, public bool) (vocab.ActivityStreamsToProperty, vocab.ActivityStreamsCcProperty) {
to := streams.NewActivityStreamsToProperty()
cc := streams.NewActivityStreamsCcProperty()
if public {
public, _ := url.Parse(PUBLIC)
to.AppendIRI(public)
cc.AppendIRI(followers_iri)
} else {
to.AppendIRI(followers_iri)
}
return to, cc
}
// MakeNoteDirect sets the required properties to make this note seen as a
// direct message.
func MakeNoteDirect(note vocab.ActivityStreamsNote, toIRI *url.URL) vocab.ActivityStreamsNote {
@@ -101,10 +87,8 @@ func MakeActivityDirect(activity vocab.ActivityStreamsCreate, toIRI *url.URL) vo
// MakeActivityPublic sets the required properties to make this activity
// seen as public.
func MakeActivityPublic(activity vocab.ActivityStreamsCreate) vocab.ActivityStreamsCreate {
configRepository := configrepository.Get()
// TO the public if we're not treating ActivityPub as "private".
if !configRepository.GetFederationIsPrivate() {
if !data.GetFederationIsPrivate() {
public, _ := url.Parse(PUBLIC)
to := streams.NewActivityStreamsToProperty()
@@ -137,9 +121,7 @@ func MakeUpdateActivity(activityID *url.URL) vocab.ActivityStreamsUpdate {
activity.SetJSONLDId(id)
// CC the public if we're not treating ActivityPub as "private".
configRepository := configrepository.Get()
if !configRepository.GetFederationIsPrivate() {
if !data.GetFederationIsPrivate() {
public, _ := url.Parse(PUBLIC)
cc := streams.NewActivityStreamsCcProperty()
cc.AppendIRI(public)
+51 -167
View File
@@ -9,8 +9,8 @@ import (
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/persistence/configrepository"
log "github.com/sirupsen/logrus"
)
@@ -26,8 +26,6 @@ type ActivityPubActor struct {
FollowRequestIri *url.URL
// Inbox is the inbox URL of the remote follower
Inbox *url.URL
// SharedInbox is the shared inbox URL of the remote server (optional)
SharedInbox *url.URL
// Image is the avatar image of the Actor.
Image *url.URL
// DisabledAt is the time, if any, this follower was blocked/removed.
@@ -40,156 +38,6 @@ type ActivityPubActor struct {
FullUsername string
}
// ErrActorMissingRequiredField is returned when an actor is missing a required field.
var ErrActorMissingRequiredField = errors.New("actor missing required field")
// Validate checks that required fields are present on the actor.
// Returns an error if ActorIri or Inbox are nil.
func (a *ActivityPubActor) Validate() error {
if a.ActorIri == nil {
return fmt.Errorf("%w: ActorIri is required", ErrActorMissingRequiredField)
}
if a.Inbox == nil {
return fmt.Errorf("%w: Inbox is required", ErrActorMissingRequiredField)
}
return nil
}
// IsValid returns true if the actor has all required fields.
func (a *ActivityPubActor) IsValid() bool {
return a.Validate() == nil
}
// ActorIriString returns the string representation of ActorIri, or empty string if nil.
func (a *ActivityPubActor) ActorIriString() string {
if a.ActorIri == nil {
return ""
}
return a.ActorIri.String()
}
// InboxString returns the string representation of Inbox, or empty string if nil.
func (a *ActivityPubActor) InboxString() string {
if a.Inbox == nil {
return ""
}
return a.Inbox.String()
}
// SharedInboxString returns the string representation of SharedInbox, or empty string if nil.
func (a *ActivityPubActor) SharedInboxString() string {
if a.SharedInbox == nil {
return ""
}
return a.SharedInbox.String()
}
// ImageString returns the string representation of Image, or empty string if nil.
func (a *ActivityPubActor) ImageString() string {
if a.Image == nil {
return ""
}
return a.Image.String()
}
// FollowRequestIriString returns the string representation of FollowRequestIri, or empty string if nil.
func (a *ActivityPubActor) FollowRequestIriString() string {
if a.FollowRequestIri == nil {
return ""
}
return a.FollowRequestIri.String()
}
// ActorIriHostname returns the hostname of ActorIri, or empty string if nil.
func (a *ActivityPubActor) ActorIriHostname() string {
if a.ActorIri == nil {
return ""
}
return a.ActorIri.Hostname()
}
// NewActivityPubActor creates a new ActivityPubActor with required fields.
// Returns an error if actorIri or inbox are nil.
func NewActivityPubActor(actorIri, inbox *url.URL) (*ActivityPubActor, error) {
if actorIri == nil {
return nil, fmt.Errorf("%w: actorIri is required", ErrActorMissingRequiredField)
}
if inbox == nil {
return nil, fmt.Errorf("%w: inbox is required", ErrActorMissingRequiredField)
}
return &ActivityPubActor{
ActorIri: actorIri,
Inbox: inbox,
}, nil
}
// validateEntityRequiredFields checks that all required fields are present on the entity.
func validateEntityRequiredFields(entity ExternalEntity) error {
if entity.GetJSONLDId() == nil || entity.GetJSONLDId().Get() == nil {
return fmt.Errorf("%w: entity is missing actor IRI", ErrActorMissingRequiredField)
}
if entity.GetActivityStreamsInbox() == nil || entity.GetActivityStreamsInbox().GetIRI() == nil {
return fmt.Errorf("%w: entity is missing inbox", ErrActorMissingRequiredField)
}
if entity.GetActivityStreamsPreferredUsername() == nil || entity.GetActivityStreamsPreferredUsername().GetXMLSchemaString() == "" {
return fmt.Errorf("%w: entity is missing preferred username", ErrActorMissingRequiredField)
}
if entity.GetW3IDSecurityV1PublicKey() == nil || entity.GetW3IDSecurityV1PublicKey().Len() == 0 {
return fmt.Errorf("%w: entity is missing public key", ErrActorMissingRequiredField)
}
return nil
}
// getNameFromEntity extracts the optional name from an entity.
func getNameFromEntity(entity ExternalEntity) string {
nameProp := entity.GetActivityStreamsName()
if nameProp == nil || nameProp.Empty() {
return ""
}
return nameProp.At(0).GetXMLSchemaString()
}
// getSharedInboxFromEntity extracts the optional shared inbox URL from an entity.
func getSharedInboxFromEntity(entity ExternalEntity) *url.URL {
endpointsProp := entity.GetActivityStreamsEndpoints()
if endpointsProp == nil || !endpointsProp.IsActivityStreamsEndpoints() {
return nil
}
endpoints := endpointsProp.Get()
if endpoints == nil {
return nil
}
sharedInboxProp := endpoints.GetActivityStreamsSharedInbox()
if sharedInboxProp == nil || !sharedInboxProp.HasAny() {
return nil
}
return sharedInboxProp.Get()
}
// NewActivityPubActorFromEntity creates a new ActivityPubActor from an external entity
// with validation of required fields.
func NewActivityPubActorFromEntity(entity ExternalEntity) (*ActivityPubActor, error) {
if err := validateEntityRequiredFields(entity); err != nil {
return nil, err
}
apActor := &ActivityPubActor{
ActorIri: entity.GetJSONLDId().Get(),
Inbox: entity.GetActivityStreamsInbox().GetIRI(),
SharedInbox: getSharedInboxFromEntity(entity),
Name: getNameFromEntity(entity),
Username: entity.GetActivityStreamsPreferredUsername().GetXMLSchemaString(),
FullUsername: GetFullUsernameFromExternalEntity(entity),
W3IDSecurityV1PublicKey: entity.GetW3IDSecurityV1PublicKey(),
Image: GetImageFromIcon(entity.GetActivityStreamsIcon()),
}
return apActor, nil
}
// DeleteRequest represents a request for delete.
type DeleteRequest struct {
ActorIri string
@@ -203,7 +51,45 @@ type ExternalEntity interface {
GetActivityStreamsPreferredUsername() vocab.ActivityStreamsPreferredUsernameProperty
GetActivityStreamsIcon() vocab.ActivityStreamsIconProperty
GetW3IDSecurityV1PublicKey() vocab.W3IDSecurityV1PublicKeyProperty
GetActivityStreamsEndpoints() vocab.ActivityStreamsEndpointsProperty
}
// MakeActorFromExernalAPEntity takes a full ActivityPub entity and returns our
// internal representation of an actor.
func MakeActorFromExernalAPEntity(entity ExternalEntity) (*ActivityPubActor, error) {
// Username is required (but not a part of the official ActivityPub spec)
if entity.GetActivityStreamsPreferredUsername() == nil || entity.GetActivityStreamsPreferredUsername().GetXMLSchemaString() == "" {
return nil, errors.New("remote activitypub entity does not have a preferred username set, rejecting")
}
username := GetFullUsernameFromExternalEntity(entity)
// Key is required
if entity.GetW3IDSecurityV1PublicKey() == nil {
return nil, errors.New("remote activitypub entity does not have a public key set, rejecting")
}
// Name is optional
var name string
if entity.GetActivityStreamsName() != nil && !entity.GetActivityStreamsName().Empty() {
name = entity.GetActivityStreamsName().At(0).GetXMLSchemaString()
}
// Image is optional
var image *url.URL
if entity.GetActivityStreamsIcon() != nil && !entity.GetActivityStreamsIcon().Empty() && entity.GetActivityStreamsIcon().At(0).GetActivityStreamsImage() != nil {
image = entity.GetActivityStreamsIcon().At(0).GetActivityStreamsImage().GetActivityStreamsUrl().Begin().GetIRI()
}
apActor := ActivityPubActor{
ActorIri: entity.GetJSONLDId().Get(),
Inbox: entity.GetActivityStreamsInbox().GetIRI(),
Name: name,
Username: entity.GetActivityStreamsPreferredUsername().GetXMLSchemaString(),
FullUsername: username,
W3IDSecurityV1PublicKey: entity.GetW3IDSecurityV1PublicKey(),
Image: image,
}
return &apActor, nil
}
// MakeActorPropertyWithID will return an actor property filled with the provided IRI.
@@ -215,13 +101,11 @@ func MakeActorPropertyWithID(idIRI *url.URL) vocab.ActivityStreamsActorProperty
// MakeServiceForAccount will create a new local actor service with the the provided username.
func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
configRepository := configrepository.Get()
actorIRI := MakeLocalIRIForAccount(accountName)
person := streams.NewActivityStreamsService()
nameProperty := streams.NewActivityStreamsNameProperty()
nameProperty.AppendXMLSchemaString(configRepository.GetServerName())
nameProperty.AppendXMLSchemaString(data.GetServerName())
person.SetActivityStreamsName(nameProperty)
preferredUsernameProperty := streams.NewActivityStreamsPreferredUsernameProperty()
@@ -235,7 +119,7 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
person.SetActivityStreamsInbox(inboxProp)
needsFollowApprovalProperty := streams.NewActivityStreamsManuallyApprovesFollowersProperty()
needsFollowApprovalProperty.Set(configRepository.GetFederationIsPrivate())
needsFollowApprovalProperty.Set(data.GetFederationIsPrivate())
person.SetActivityStreamsManuallyApprovesFollowers(needsFollowApprovalProperty)
outboxIRI := MakeLocalIRIForResource("/user/" + accountName + "/outbox")
@@ -268,7 +152,7 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
publicKeyProp.AppendW3IDSecurityV1PublicKey(publicKeyType)
person.SetW3IDSecurityV1PublicKey(publicKeyProp)
if t, err := configRepository.GetServerInitTime(); t != nil {
if t, err := data.GetServerInitTime(); t != nil {
publishedDateProp := streams.NewActivityStreamsPublishedProperty()
publishedDateProp.Set(t.Time)
person.SetActivityStreamsPublished(publishedDateProp)
@@ -279,8 +163,8 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
// Profile properties
// Avatar
uniquenessString := configRepository.GetLogoUniquenessString()
userAvatarURLString := configRepository.GetServerURL() + "/logo/external"
uniquenessString := data.GetLogoUniquenessString()
userAvatarURLString := data.GetServerURL() + "/logo/external"
userAvatarURL, err := url.Parse(userAvatarURLString)
userAvatarURL.RawQuery = "uc=" + uniquenessString
if err != nil {
@@ -311,14 +195,14 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
// Profile bio
summaryProperty := streams.NewActivityStreamsSummaryProperty()
summaryProperty.AppendXMLSchemaString(configRepository.GetServerSummary())
summaryProperty.AppendXMLSchemaString(data.GetServerSummary())
person.SetActivityStreamsSummary(summaryProperty)
// Links
if serverURL := configRepository.GetServerURL(); serverURL != "" {
if serverURL := data.GetServerURL(); serverURL != "" {
addMetadataLinkToProfile(person, "Stream", serverURL)
}
for _, link := range configRepository.GetSocialHandles() {
for _, link := range data.GetSocialHandles() {
addMetadataLinkToProfile(person, link.Platform, link.URL)
}
@@ -336,7 +220,7 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
// Tags
tagProp := streams.NewActivityStreamsTagProperty()
for _, tagString := range configRepository.GetServerMetadataTags() {
for _, tagString := range data.GetServerMetadataTags() {
hashtag := MakeHashtag(tagString)
tagProp.AppendTootHashtag(hashtag)
}
@@ -345,7 +229,7 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
// Work around an issue where a single attachment will not serialize
// as an array, so add another item to the mix.
if len(configRepository.GetSocialHandles()) == 1 {
if len(data.GetSocialHandles()) == 1 {
addMetadataLinkToProfile(person, "Owncast", "https://owncast.online")
}
@@ -355,7 +239,7 @@ func MakeServiceForAccount(accountName string) vocab.ActivityStreamsService {
// GetFullUsernameFromExternalEntity will return the full username from an
// internal representation of an ExternalEntity. Returns user@host.tld.
func GetFullUsernameFromExternalEntity(entity ExternalEntity) string {
hostname := GetHostnameFromJSONLDId(entity.GetJSONLDId())
hostname := entity.GetJSONLDId().GetIRI().Hostname()
username := entity.GetActivityStreamsPreferredUsername().GetXMLSchemaString()
fullUsername := fmt.Sprintf("%s@%s", username, hostname)
+30 -468
View File
@@ -1,7 +1,6 @@
package apmodels
import (
"errors"
"io/ioutil"
"net/url"
"os"
@@ -10,7 +9,6 @@ import (
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/persistence/configrepository"
)
func makeFakeService() vocab.ActivityStreamsService {
@@ -47,8 +45,6 @@ func makeFakeService() vocab.ActivityStreamsService {
service.SetActivityStreamsIcon(icon)
publicKeyProperty := streams.NewW3IDSecurityV1PublicKeyProperty()
publicKeyType := streams.NewW3IDSecurityV1PublicKey()
publicKeyProperty.AppendW3IDSecurityV1PublicKey(publicKeyType)
service.SetW3IDSecurityV1PublicKey(publicKeyProperty)
return service
@@ -59,15 +55,41 @@ func TestMain(m *testing.M) {
if err != nil {
panic(err)
}
data.SetupPersistence(dbFile.Name())
configRepository := configrepository.Get()
configRepository.SetServerURL("https://my.cool.site.biz")
data.SetServerURL("https://my.cool.site.biz")
m.Run()
}
func TestMakeActorFromExternalAPEntity(t *testing.T) {
service := makeFakeService()
actor, err := MakeActorFromExernalAPEntity(service)
if err != nil {
t.Error(err)
}
if actor.ActorIri != service.GetJSONLDId().GetIRI() {
t.Errorf("actor.ID = %v, want %v", actor.ActorIri, service.GetJSONLDId().GetIRI())
}
if actor.Name != service.GetActivityStreamsName().At(0).GetXMLSchemaString() {
t.Errorf("actor.Name = %v, want %v", actor.Name, service.GetActivityStreamsName().At(0).GetXMLSchemaString())
}
if actor.Username != service.GetActivityStreamsPreferredUsername().GetXMLSchemaString() {
t.Errorf("actor.Username = %v, want %v", actor.Username, service.GetActivityStreamsPreferredUsername().GetXMLSchemaString())
}
if actor.Inbox != service.GetActivityStreamsInbox().GetIRI() {
t.Errorf("actor.Inbox = %v, want %v", actor.Inbox.String(), service.GetActivityStreamsInbox().GetIRI())
}
if actor.Image != service.GetActivityStreamsIcon().At(0).GetActivityStreamsImage().GetActivityStreamsUrl().At(0).GetIRI() {
t.Errorf("actor.Image = %v, want %v", actor.Image, service.GetActivityStreamsIcon().At(0).GetActivityStreamsImage().GetActivityStreamsUrl().At(0).GetIRI())
}
}
func TestMakeActorPropertyWithID(t *testing.T) {
iri, _ := url.Parse("https://fake.fediverse.server/user/mrfoo")
actor := MakeActorPropertyWithID(iri)
@@ -155,463 +177,3 @@ func TestMakeServiceForAccount(t *testing.T) {
t.Errorf("actor.URL = %v, want %v", person.GetActivityStreamsUrl().At(0).GetIRI().String(), expectedIRI)
}
}
// Tests for nil-safe accessor methods
func TestActorIriStringWithNilValue(t *testing.T) {
actor := ActivityPubActor{}
result := actor.ActorIriString()
if result != "" {
t.Errorf("ActorIriString() with nil ActorIri = %v, want empty string", result)
}
}
func TestActorIriStringWithValue(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
actor := ActivityPubActor{ActorIri: iri}
result := actor.ActorIriString()
if result != "https://example.com/user/test" {
t.Errorf("ActorIriString() = %v, want %v", result, "https://example.com/user/test")
}
}
func TestInboxStringWithNilValue(t *testing.T) {
actor := ActivityPubActor{}
result := actor.InboxString()
if result != "" {
t.Errorf("InboxString() with nil Inbox = %v, want empty string", result)
}
}
func TestInboxStringWithValue(t *testing.T) {
inbox, _ := url.Parse("https://example.com/user/test/inbox")
actor := ActivityPubActor{Inbox: inbox}
result := actor.InboxString()
if result != "https://example.com/user/test/inbox" {
t.Errorf("InboxString() = %v, want %v", result, "https://example.com/user/test/inbox")
}
}
func TestImageStringWithNilValue(t *testing.T) {
actor := ActivityPubActor{}
result := actor.ImageString()
if result != "" {
t.Errorf("ImageString() with nil Image = %v, want empty string", result)
}
}
func TestImageStringWithValue(t *testing.T) {
image, _ := url.Parse("https://example.com/avatar.png")
actor := ActivityPubActor{Image: image}
result := actor.ImageString()
if result != "https://example.com/avatar.png" {
t.Errorf("ImageString() = %v, want %v", result, "https://example.com/avatar.png")
}
}
func TestFollowRequestIriStringWithNilValue(t *testing.T) {
actor := ActivityPubActor{}
result := actor.FollowRequestIriString()
if result != "" {
t.Errorf("FollowRequestIriString() with nil FollowRequestIri = %v, want empty string", result)
}
}
func TestFollowRequestIriStringWithValue(t *testing.T) {
followIri, _ := url.Parse("https://example.com/follow/123")
actor := ActivityPubActor{FollowRequestIri: followIri}
result := actor.FollowRequestIriString()
if result != "https://example.com/follow/123" {
t.Errorf("FollowRequestIriString() = %v, want %v", result, "https://example.com/follow/123")
}
}
func TestActorIriHostnameWithNilValue(t *testing.T) {
actor := ActivityPubActor{}
result := actor.ActorIriHostname()
if result != "" {
t.Errorf("ActorIriHostname() with nil ActorIri = %v, want empty string", result)
}
}
func TestActorIriHostnameWithValue(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
actor := ActivityPubActor{ActorIri: iri}
result := actor.ActorIriHostname()
if result != "example.com" {
t.Errorf("ActorIriHostname() = %v, want %v", result, "example.com")
}
}
// Tests for Validate() and IsValid() methods
func TestValidateWithAllNilFields(t *testing.T) {
actor := ActivityPubActor{}
err := actor.Validate()
if err == nil {
t.Error("Validate() with all nil fields should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("Validate() error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestValidateWithNilActorIri(t *testing.T) {
inbox, _ := url.Parse("https://example.com/inbox")
actor := ActivityPubActor{Inbox: inbox}
err := actor.Validate()
if err == nil {
t.Error("Validate() with nil ActorIri should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("Validate() error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestValidateWithNilInbox(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
actor := ActivityPubActor{ActorIri: iri}
err := actor.Validate()
if err == nil {
t.Error("Validate() with nil Inbox should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("Validate() error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestValidateWithRequiredFields(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
inbox, _ := url.Parse("https://example.com/inbox")
actor := ActivityPubActor{ActorIri: iri, Inbox: inbox}
err := actor.Validate()
if err != nil {
t.Errorf("Validate() with required fields should not return error, got %v", err)
}
}
func TestIsValidWithInvalidActor(t *testing.T) {
actor := ActivityPubActor{}
if actor.IsValid() {
t.Error("IsValid() with invalid actor should return false")
}
}
func TestIsValidWithValidActor(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
inbox, _ := url.Parse("https://example.com/inbox")
actor := ActivityPubActor{ActorIri: iri, Inbox: inbox}
if !actor.IsValid() {
t.Error("IsValid() with valid actor should return true")
}
}
// Tests for NewActivityPubActor constructor
func TestNewActivityPubActorWithNilActorIri(t *testing.T) {
inbox, _ := url.Parse("https://example.com/inbox")
_, err := NewActivityPubActor(nil, inbox)
if err == nil {
t.Error("NewActivityPubActor with nil actorIri should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActor error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorWithNilInbox(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
_, err := NewActivityPubActor(iri, nil)
if err == nil {
t.Error("NewActivityPubActor with nil inbox should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActor error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorWithBothNil(t *testing.T) {
_, err := NewActivityPubActor(nil, nil)
if err == nil {
t.Error("NewActivityPubActor with both nil should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActor error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorWithValidArgs(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
inbox, _ := url.Parse("https://example.com/inbox")
actor, err := NewActivityPubActor(iri, inbox)
if err != nil {
t.Errorf("NewActivityPubActor with valid args should not return error, got %v", err)
}
if actor == nil {
t.Error("NewActivityPubActor with valid args should return non-nil actor")
}
if actor.ActorIri != iri {
t.Errorf("actor.ActorIri = %v, want %v", actor.ActorIri, iri)
}
if actor.Inbox != inbox {
t.Errorf("actor.Inbox = %v, want %v", actor.Inbox, inbox)
}
}
// Tests for NewActivityPubActorFromEntity with invalid entities
func makeFakeServiceWithoutUsername() vocab.ActivityStreamsService {
iri, _ := url.Parse("https://fake.fediverse.server/user/mrfoo")
inbox, _ := url.Parse("https://fake.fediverse.server/user/mrfoo/inbox")
service := streams.NewActivityStreamsService()
id := streams.NewJSONLDIdProperty()
id.Set(iri)
service.SetJSONLDId(id)
inboxProp := streams.NewActivityStreamsInboxProperty()
inboxProp.SetIRI(inbox)
service.SetActivityStreamsInbox(inboxProp)
publicKeyProperty := streams.NewW3IDSecurityV1PublicKeyProperty()
service.SetW3IDSecurityV1PublicKey(publicKeyProperty)
return service
}
func makeFakeServiceWithoutPublicKey() vocab.ActivityStreamsService {
iri, _ := url.Parse("https://fake.fediverse.server/user/mrfoo")
inbox, _ := url.Parse("https://fake.fediverse.server/user/mrfoo/inbox")
username := "foodawg"
service := streams.NewActivityStreamsService()
id := streams.NewJSONLDIdProperty()
id.Set(iri)
service.SetJSONLDId(id)
preferredUsernameProperty := streams.NewActivityStreamsPreferredUsernameProperty()
preferredUsernameProperty.SetXMLSchemaString(username)
service.SetActivityStreamsPreferredUsername(preferredUsernameProperty)
inboxProp := streams.NewActivityStreamsInboxProperty()
inboxProp.SetIRI(inbox)
service.SetActivityStreamsInbox(inboxProp)
return service
}
func makeFakeServiceWithEmptyPublicKey() vocab.ActivityStreamsService {
iri, _ := url.Parse("https://fake.fediverse.server/user/mrfoo")
inbox, _ := url.Parse("https://fake.fediverse.server/user/mrfoo/inbox")
username := "foodawg"
service := streams.NewActivityStreamsService()
id := streams.NewJSONLDIdProperty()
id.Set(iri)
service.SetJSONLDId(id)
preferredUsernameProperty := streams.NewActivityStreamsPreferredUsernameProperty()
preferredUsernameProperty.SetXMLSchemaString(username)
service.SetActivityStreamsPreferredUsername(preferredUsernameProperty)
inboxProp := streams.NewActivityStreamsInboxProperty()
inboxProp.SetIRI(inbox)
service.SetActivityStreamsInbox(inboxProp)
// Set an empty public key property (Len() == 0)
publicKeyProperty := streams.NewW3IDSecurityV1PublicKeyProperty()
service.SetW3IDSecurityV1PublicKey(publicKeyProperty)
return service
}
func makeFakeServiceWithoutId() vocab.ActivityStreamsService {
inbox, _ := url.Parse("https://fake.fediverse.server/user/mrfoo/inbox")
username := "foodawg"
service := streams.NewActivityStreamsService()
preferredUsernameProperty := streams.NewActivityStreamsPreferredUsernameProperty()
preferredUsernameProperty.SetXMLSchemaString(username)
service.SetActivityStreamsPreferredUsername(preferredUsernameProperty)
inboxProp := streams.NewActivityStreamsInboxProperty()
inboxProp.SetIRI(inbox)
service.SetActivityStreamsInbox(inboxProp)
publicKeyProperty := streams.NewW3IDSecurityV1PublicKeyProperty()
service.SetW3IDSecurityV1PublicKey(publicKeyProperty)
return service
}
func makeFakeServiceWithoutInbox() vocab.ActivityStreamsService {
iri, _ := url.Parse("https://fake.fediverse.server/user/mrfoo")
username := "foodawg"
service := streams.NewActivityStreamsService()
id := streams.NewJSONLDIdProperty()
id.Set(iri)
service.SetJSONLDId(id)
preferredUsernameProperty := streams.NewActivityStreamsPreferredUsernameProperty()
preferredUsernameProperty.SetXMLSchemaString(username)
service.SetActivityStreamsPreferredUsername(preferredUsernameProperty)
publicKeyProperty := streams.NewW3IDSecurityV1PublicKeyProperty()
service.SetW3IDSecurityV1PublicKey(publicKeyProperty)
return service
}
func TestNewActivityPubActorFromEntityWithoutUsername(t *testing.T) {
service := makeFakeServiceWithoutUsername()
_, err := NewActivityPubActorFromEntity(service)
if err == nil {
t.Error("NewActivityPubActorFromEntity without username should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActorFromEntity error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorFromEntityWithoutPublicKey(t *testing.T) {
service := makeFakeServiceWithoutPublicKey()
_, err := NewActivityPubActorFromEntity(service)
if err == nil {
t.Error("NewActivityPubActorFromEntity without public key should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActorFromEntity error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorFromEntityWithEmptyPublicKey(t *testing.T) {
service := makeFakeServiceWithEmptyPublicKey()
_, err := NewActivityPubActorFromEntity(service)
if err == nil {
t.Error("NewActivityPubActorFromEntity with empty public key should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActorFromEntity error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorFromEntityWithoutId(t *testing.T) {
service := makeFakeServiceWithoutId()
_, err := NewActivityPubActorFromEntity(service)
if err == nil {
t.Error("NewActivityPubActorFromEntity without ID should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActorFromEntity error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorFromEntityWithoutInbox(t *testing.T) {
service := makeFakeServiceWithoutInbox()
_, err := NewActivityPubActorFromEntity(service)
if err == nil {
t.Error("NewActivityPubActorFromEntity without inbox should return error")
}
if !errors.Is(err, ErrActorMissingRequiredField) {
t.Errorf("NewActivityPubActorFromEntity error = %v, want ErrActorMissingRequiredField", err)
}
}
func TestNewActivityPubActorFromEntityWithValidEntity(t *testing.T) {
service := makeFakeService()
actor, err := NewActivityPubActorFromEntity(service)
if err != nil {
t.Errorf("NewActivityPubActorFromEntity with valid entity should not return error, got %v", err)
}
if actor == nil {
t.Fatal("NewActivityPubActorFromEntity with valid entity should return non-nil actor")
}
// Verify required fields are non-nil
if actor.ActorIri == nil {
t.Error("actor.ActorIri should not be nil")
}
if actor.Inbox == nil {
t.Error("actor.Inbox should not be nil")
}
// Verify extracted values match the fake service data
expectedIri := "https://fake.fediverse.server/user/mrfoo"
if actor.ActorIriString() != expectedIri {
t.Errorf("actor.ActorIri = %v, want %v", actor.ActorIriString(), expectedIri)
}
expectedInbox := "https://fake.fediverse.server/user/mrfoo/inbox"
if actor.InboxString() != expectedInbox {
t.Errorf("actor.Inbox = %v, want %v", actor.InboxString(), expectedInbox)
}
expectedName := "Mr Foo"
if actor.Name != expectedName {
t.Errorf("actor.Name = %v, want %v", actor.Name, expectedName)
}
expectedUsername := "foodawg"
if actor.Username != expectedUsername {
t.Errorf("actor.Username = %v, want %v", actor.Username, expectedUsername)
}
expectedImage := "https://fake.fediverse.server/user/mrfoo/avatar.png"
if actor.ImageString() != expectedImage {
t.Errorf("actor.Image = %v, want %v", actor.ImageString(), expectedImage)
}
}
// Test that safe accessors don't panic on zero-value struct
func TestSafeAccessorsOnZeroValueStruct(t *testing.T) {
var actor ActivityPubActor
// These should not panic
_ = actor.ActorIriString()
_ = actor.InboxString()
_ = actor.ImageString()
_ = actor.FollowRequestIriString()
_ = actor.ActorIriHostname()
_ = actor.Validate()
_ = actor.IsValid()
}
// Test that safe accessors work correctly with optional nil fields on otherwise valid actor
func TestSafeAccessorsWithOptionalNilFields(t *testing.T) {
iri, _ := url.Parse("https://example.com/user/test")
inbox, _ := url.Parse("https://example.com/inbox")
actor := ActivityPubActor{
ActorIri: iri,
Inbox: inbox,
// Image and FollowRequestIri are intentionally nil
}
// Required fields should return values
if actor.ActorIriString() != "https://example.com/user/test" {
t.Errorf("ActorIriString() = %v, want non-empty", actor.ActorIriString())
}
if actor.InboxString() != "https://example.com/inbox" {
t.Errorf("InboxString() = %v, want non-empty", actor.InboxString())
}
// Optional nil fields should return empty strings without panicking
if actor.ImageString() != "" {
t.Errorf("ImageString() = %v, want empty string", actor.ImageString())
}
if actor.FollowRequestIriString() != "" {
t.Errorf("FollowRequestIriString() = %v, want empty string", actor.FollowRequestIriString())
}
// Actor should still be valid (only ActorIri and Inbox are required)
if !actor.IsValid() {
t.Error("Actor with ActorIri and Inbox should be valid even with nil optional fields")
}
}
+1 -1
View File
@@ -9,7 +9,7 @@ import (
// MakeHashtag will create and return a mastodon toot hashtag object with the provided name.
func MakeHashtag(name string) vocab.TootHashtag {
u, _ := url.Parse("https://owncast.directory/tags/" + name)
u, _ := url.Parse("https://directory.owncast.online/tags/" + name)
hashtag := streams.NewTootHashtag()
hashtagName := streams.NewActivityStreamsNameProperty()
+6 -163
View File
@@ -2,14 +2,13 @@ package apmodels
import (
"encoding/json"
"fmt"
"net/url"
"path"
"path/filepath"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
@@ -28,9 +27,7 @@ func MakeRemoteIRIForResource(resourcePath string, host string) (*url.URL, error
// MakeLocalIRIForResource will create an IRI for the local server.
func MakeLocalIRIForResource(resourcePath string) *url.URL {
configRepository := configrepository.Get()
host := configRepository.GetServerURL()
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI url", host, err)
@@ -44,9 +41,7 @@ func MakeLocalIRIForResource(resourcePath string) *url.URL {
// MakeLocalIRIForAccount will return a full IRI for the local server account username.
func MakeLocalIRIForAccount(account string) *url.URL {
configRepository := configrepository.Get()
host := configRepository.GetServerURL()
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI account server url", err)
@@ -69,9 +64,7 @@ func Serialize(obj vocab.Type) ([]byte, error) {
// MakeLocalIRIForStreamURL will return a full IRI for the local server stream url.
func MakeLocalIRIForStreamURL() *url.URL {
configRepository := configrepository.Get()
host := configRepository.GetServerURL()
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI stream url", err)
@@ -85,9 +78,7 @@ func MakeLocalIRIForStreamURL() *url.URL {
// MakeLocalIRIforLogo will return a full IRI for the local server logo.
func MakeLocalIRIforLogo() *url.URL {
configRepository := configrepository.Get()
host := configRepository.GetServerURL()
host := data.GetServerURL()
u, err := url.Parse(host)
if err != nil {
log.Errorln("unable to parse local IRI stream url", err)
@@ -102,9 +93,7 @@ func MakeLocalIRIforLogo() *url.URL {
// GetLogoType will return the rel value for the webfinger response and
// the default static image is of type png.
func GetLogoType() string {
configRepository := configrepository.Get()
imageFilename := configRepository.GetLogoPath()
imageFilename := data.GetLogoPath()
if imageFilename == "" {
return "image/png"
}
@@ -119,149 +108,3 @@ func GetLogoType() string {
}
return logoType
}
// ErrMissingIRI is returned when an IRI cannot be extracted from an ActivityStreams property.
var ErrMissingIRI = fmt.Errorf("missing IRI")
// GetIRIFromActorProperty safely extracts the IRI from an ActivityStreamsActorProperty.
// Returns the IRI and nil error on success, or nil and an error if the IRI cannot be extracted.
func GetIRIFromActorProperty(actor vocab.ActivityStreamsActorProperty) (*url.URL, error) {
if actor == nil || actor.Empty() || actor.Len() == 0 {
return nil, fmt.Errorf("%w: actor property is empty or nil", ErrMissingIRI)
}
first := actor.At(0)
if first == nil {
return nil, fmt.Errorf("%w: actor property first element is nil", ErrMissingIRI)
}
iri := first.GetIRI()
if iri == nil {
return nil, fmt.Errorf("%w: actor IRI is nil", ErrMissingIRI)
}
return iri, nil
}
// GetIRIStringFromActorProperty safely extracts the IRI string from an ActivityStreamsActorProperty.
// Returns the IRI string and nil error on success, or empty string and an error if extraction fails.
func GetIRIStringFromActorProperty(actor vocab.ActivityStreamsActorProperty) (string, error) {
iri, err := GetIRIFromActorProperty(actor)
if err != nil {
return "", err
}
return iri.String(), nil
}
// GetIRIFromObjectProperty safely extracts the IRI from an ActivityStreamsObjectProperty.
// Returns the IRI and nil error on success, or nil and an error if the IRI cannot be extracted.
func GetIRIFromObjectProperty(object vocab.ActivityStreamsObjectProperty) (*url.URL, error) {
if object == nil || object.Len() == 0 {
return nil, fmt.Errorf("%w: object property is empty or nil", ErrMissingIRI)
}
first := object.At(0)
if first == nil {
return nil, fmt.Errorf("%w: object property first element is nil", ErrMissingIRI)
}
iri := first.GetIRI()
if iri == nil {
return nil, fmt.Errorf("%w: object IRI is nil", ErrMissingIRI)
}
return iri, nil
}
// GetIRIStringFromObjectProperty safely extracts the IRI string from an ActivityStreamsObjectProperty.
// Returns the IRI string and nil error on success, or empty string and an error if extraction fails.
func GetIRIStringFromObjectProperty(object vocab.ActivityStreamsObjectProperty) (string, error) {
iri, err := GetIRIFromObjectProperty(object)
if err != nil {
return "", err
}
return iri.String(), nil
}
// GetIRIFromJSONLDIdProperty safely extracts the IRI from a JSONLDIdProperty.
// Returns the IRI and nil error on success, or nil and an error if the IRI cannot be extracted.
func GetIRIFromJSONLDIdProperty(id vocab.JSONLDIdProperty) (*url.URL, error) {
if id == nil {
return nil, fmt.Errorf("%w: JSONLD id property is nil", ErrMissingIRI)
}
iri := id.GetIRI()
if iri == nil {
return nil, fmt.Errorf("%w: JSONLD id IRI is nil", ErrMissingIRI)
}
return iri, nil
}
// GetIRIStringFromJSONLDIdProperty safely extracts the IRI string from a JSONLDIdProperty.
// Returns the IRI string and nil error on success, or empty string and an error if extraction fails.
func GetIRIStringFromJSONLDIdProperty(id vocab.JSONLDIdProperty) (string, error) {
iri, err := GetIRIFromJSONLDIdProperty(id)
if err != nil {
return "", err
}
return iri.String(), nil
}
// GetPublicKeyPem safely extracts the public key PEM from a W3IDSecurityV1PublicKey.
// Returns the PEM string and nil error on success, or empty string and an error if extraction fails.
func GetPublicKeyPem(publicKey vocab.W3IDSecurityV1PublicKey) (string, error) {
if publicKey == nil {
return "", fmt.Errorf("public key is nil")
}
pemProp := publicKey.GetW3IDSecurityV1PublicKeyPem()
if pemProp == nil {
return "", fmt.Errorf("public key PEM property is nil")
}
return pemProp.Get(), nil
}
// IsFirstObjectActivityStreamsPerson safely checks if the first element of an
// ActivityStreamsObjectProperty is an ActivityStreamsPerson.
// Returns false if the object is nil, empty, or the first element is not a Person.
func IsFirstObjectActivityStreamsPerson(object vocab.ActivityStreamsObjectProperty) bool {
if object == nil || object.Len() == 0 {
return false
}
first := object.At(0)
if first == nil {
return false
}
return first.IsActivityStreamsPerson()
}
// GetImageFromIcon safely extracts the image URL from an ActivityStreamsIconProperty.
// Returns the URL and nil error on success, or nil and nil if the icon is not present or invalid.
// This handles the common pattern of icon -> image -> url -> iri.
func GetImageFromIcon(icon vocab.ActivityStreamsIconProperty) *url.URL {
if icon == nil || icon.Empty() {
return nil
}
first := icon.At(0)
if first == nil {
return nil
}
image := first.GetActivityStreamsImage()
if image == nil {
return nil
}
urlProp := image.GetActivityStreamsUrl()
if urlProp == nil {
return nil
}
begin := urlProp.Begin()
if begin == nil {
return nil
}
return begin.GetIRI()
}
// GetHostnameFromJSONLDId safely extracts the hostname from a JSONLDIdProperty.
// Returns the hostname string, or empty string if extraction fails.
func GetHostnameFromJSONLDId(id vocab.JSONLDIdProperty) string {
if id == nil {
return ""
}
iri := id.GetIRI()
if iri == nil {
return ""
}
return iri.Hostname()
}
-302
View File
@@ -1,302 +0,0 @@
package apmodels
import (
"errors"
"net/url"
"testing"
"github.com/go-fed/activity/streams"
)
func TestGetIRIFromActorPropertyWithNil(t *testing.T) {
_, err := GetIRIFromActorProperty(nil)
if err == nil {
t.Error("GetIRIFromActorProperty(nil) should return error")
}
if !errors.Is(err, ErrMissingIRI) {
t.Errorf("GetIRIFromActorProperty(nil) error = %v, want ErrMissingIRI", err)
}
}
func TestGetIRIFromActorPropertyWithEmpty(t *testing.T) {
actor := streams.NewActivityStreamsActorProperty()
_, err := GetIRIFromActorProperty(actor)
if err == nil {
t.Error("GetIRIFromActorProperty with empty actor should return error")
}
if !errors.Is(err, ErrMissingIRI) {
t.Errorf("GetIRIFromActorProperty error = %v, want ErrMissingIRI", err)
}
}
func TestGetIRIFromActorPropertyWithValidIRI(t *testing.T) {
actor := streams.NewActivityStreamsActorProperty()
iri, _ := url.Parse("https://example.com/user/test")
actor.AppendIRI(iri)
result, err := GetIRIFromActorProperty(actor)
if err != nil {
t.Errorf("GetIRIFromActorProperty with valid IRI should not return error, got %v", err)
}
if result.String() != "https://example.com/user/test" {
t.Errorf("GetIRIFromActorProperty = %v, want %v", result.String(), "https://example.com/user/test")
}
}
func TestGetIRIStringFromActorPropertyWithValidIRI(t *testing.T) {
actor := streams.NewActivityStreamsActorProperty()
iri, _ := url.Parse("https://example.com/user/test")
actor.AppendIRI(iri)
result, err := GetIRIStringFromActorProperty(actor)
if err != nil {
t.Errorf("GetIRIStringFromActorProperty with valid IRI should not return error, got %v", err)
}
if result != "https://example.com/user/test" {
t.Errorf("GetIRIStringFromActorProperty = %v, want %v", result, "https://example.com/user/test")
}
}
func TestGetIRIFromObjectPropertyWithNil(t *testing.T) {
_, err := GetIRIFromObjectProperty(nil)
if err == nil {
t.Error("GetIRIFromObjectProperty(nil) should return error")
}
if !errors.Is(err, ErrMissingIRI) {
t.Errorf("GetIRIFromObjectProperty(nil) error = %v, want ErrMissingIRI", err)
}
}
func TestGetIRIFromObjectPropertyWithEmpty(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
_, err := GetIRIFromObjectProperty(object)
if err == nil {
t.Error("GetIRIFromObjectProperty with empty object should return error")
}
if !errors.Is(err, ErrMissingIRI) {
t.Errorf("GetIRIFromObjectProperty error = %v, want ErrMissingIRI", err)
}
}
func TestGetIRIFromObjectPropertyWithValidIRI(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
iri, _ := url.Parse("https://example.com/post/123")
object.AppendIRI(iri)
result, err := GetIRIFromObjectProperty(object)
if err != nil {
t.Errorf("GetIRIFromObjectProperty with valid IRI should not return error, got %v", err)
}
if result.String() != "https://example.com/post/123" {
t.Errorf("GetIRIFromObjectProperty = %v, want %v", result.String(), "https://example.com/post/123")
}
}
func TestGetIRIStringFromObjectPropertyWithValidIRI(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
iri, _ := url.Parse("https://example.com/post/123")
object.AppendIRI(iri)
result, err := GetIRIStringFromObjectProperty(object)
if err != nil {
t.Errorf("GetIRIStringFromObjectProperty with valid IRI should not return error, got %v", err)
}
if result != "https://example.com/post/123" {
t.Errorf("GetIRIStringFromObjectProperty = %v, want %v", result, "https://example.com/post/123")
}
}
func TestGetIRIFromJSONLDIdPropertyWithNil(t *testing.T) {
_, err := GetIRIFromJSONLDIdProperty(nil)
if err == nil {
t.Error("GetIRIFromJSONLDIdProperty(nil) should return error")
}
if !errors.Is(err, ErrMissingIRI) {
t.Errorf("GetIRIFromJSONLDIdProperty(nil) error = %v, want ErrMissingIRI", err)
}
}
func TestGetIRIFromJSONLDIdPropertyWithValidIRI(t *testing.T) {
id := streams.NewJSONLDIdProperty()
iri, _ := url.Parse("https://example.com/activity/456")
id.SetIRI(iri)
result, err := GetIRIFromJSONLDIdProperty(id)
if err != nil {
t.Errorf("GetIRIFromJSONLDIdProperty with valid IRI should not return error, got %v", err)
}
if result.String() != "https://example.com/activity/456" {
t.Errorf("GetIRIFromJSONLDIdProperty = %v, want %v", result.String(), "https://example.com/activity/456")
}
}
func TestGetIRIStringFromJSONLDIdPropertyWithValidIRI(t *testing.T) {
id := streams.NewJSONLDIdProperty()
iri, _ := url.Parse("https://example.com/activity/456")
id.SetIRI(iri)
result, err := GetIRIStringFromJSONLDIdProperty(id)
if err != nil {
t.Errorf("GetIRIStringFromJSONLDIdProperty with valid IRI should not return error, got %v", err)
}
if result != "https://example.com/activity/456" {
t.Errorf("GetIRIStringFromJSONLDIdProperty = %v, want %v", result, "https://example.com/activity/456")
}
}
func TestGetIRIFromJSONLDIdPropertyWithNoIRI(t *testing.T) {
id := streams.NewJSONLDIdProperty()
// Set a non-IRI value (using Set instead of SetIRI)
iri, _ := url.Parse("https://example.com/activity/456")
id.Set(iri)
// When using Set() the IRI should still be retrievable via GetIRI()
result, err := GetIRIFromJSONLDIdProperty(id)
if err != nil {
t.Errorf("GetIRIFromJSONLDIdProperty should work with Set(), got error %v", err)
}
if result == nil {
t.Error("GetIRIFromJSONLDIdProperty result should not be nil")
}
}
func TestGetPublicKeyPemWithNil(t *testing.T) {
_, err := GetPublicKeyPem(nil)
if err == nil {
t.Error("GetPublicKeyPem(nil) should return error")
}
}
func TestGetPublicKeyPemWithValidKey(t *testing.T) {
publicKey := streams.NewW3IDSecurityV1PublicKey()
pemProp := streams.NewW3IDSecurityV1PublicKeyPemProperty()
pemProp.Set("-----BEGIN PUBLIC KEY-----\ntest\n-----END PUBLIC KEY-----")
publicKey.SetW3IDSecurityV1PublicKeyPem(pemProp)
result, err := GetPublicKeyPem(publicKey)
if err != nil {
t.Errorf("GetPublicKeyPem with valid key should not return error, got %v", err)
}
if result != "-----BEGIN PUBLIC KEY-----\ntest\n-----END PUBLIC KEY-----" {
t.Errorf("GetPublicKeyPem = %v, want PEM string", result)
}
}
func TestGetPublicKeyPemWithNoPem(t *testing.T) {
publicKey := streams.NewW3IDSecurityV1PublicKey()
// Don't set PEM property
_, err := GetPublicKeyPem(publicKey)
if err == nil {
t.Error("GetPublicKeyPem with no PEM should return error")
}
}
// Test that safe accessors don't panic on nil/empty inputs
func TestSafeAccessorsNoPanic(t *testing.T) {
// These should not panic
_, _ = GetIRIFromActorProperty(nil)
_, _ = GetIRIStringFromActorProperty(nil)
_, _ = GetIRIFromObjectProperty(nil)
_, _ = GetIRIStringFromObjectProperty(nil)
_, _ = GetIRIFromJSONLDIdProperty(nil)
_, _ = GetIRIStringFromJSONLDIdProperty(nil)
_, _ = GetPublicKeyPem(nil)
_ = GetImageFromIcon(nil)
_ = GetHostnameFromJSONLDId(nil)
_ = IsFirstObjectActivityStreamsPerson(nil)
// Empty properties should also not panic
_, _ = GetIRIFromActorProperty(streams.NewActivityStreamsActorProperty())
_, _ = GetIRIFromObjectProperty(streams.NewActivityStreamsObjectProperty())
_ = GetImageFromIcon(streams.NewActivityStreamsIconProperty())
_ = IsFirstObjectActivityStreamsPerson(streams.NewActivityStreamsObjectProperty())
}
func TestGetImageFromIconWithNil(t *testing.T) {
result := GetImageFromIcon(nil)
if result != nil {
t.Error("GetImageFromIcon(nil) should return nil")
}
}
func TestGetImageFromIconWithEmpty(t *testing.T) {
icon := streams.NewActivityStreamsIconProperty()
result := GetImageFromIcon(icon)
if result != nil {
t.Error("GetImageFromIcon with empty icon should return nil")
}
}
func TestGetImageFromIconWithValidImage(t *testing.T) {
icon := streams.NewActivityStreamsIconProperty()
image := streams.NewActivityStreamsImage()
urlProp := streams.NewActivityStreamsUrlProperty()
imageURL, _ := url.Parse("https://example.com/avatar.png")
urlProp.AppendIRI(imageURL)
image.SetActivityStreamsUrl(urlProp)
icon.AppendActivityStreamsImage(image)
result := GetImageFromIcon(icon)
if result == nil {
t.Error("GetImageFromIcon with valid image should not return nil")
}
if result.String() != "https://example.com/avatar.png" {
t.Errorf("GetImageFromIcon = %v, want %v", result.String(), "https://example.com/avatar.png")
}
}
func TestGetHostnameFromJSONLDIdWithNil(t *testing.T) {
result := GetHostnameFromJSONLDId(nil)
if result != "" {
t.Errorf("GetHostnameFromJSONLDId(nil) = %v, want empty string", result)
}
}
func TestGetHostnameFromJSONLDIdWithValidId(t *testing.T) {
id := streams.NewJSONLDIdProperty()
iri, _ := url.Parse("https://example.com/user/test")
id.SetIRI(iri)
result := GetHostnameFromJSONLDId(id)
if result != "example.com" {
t.Errorf("GetHostnameFromJSONLDId = %v, want %v", result, "example.com")
}
}
func TestIsFirstObjectActivityStreamsPersonWithNil(t *testing.T) {
result := IsFirstObjectActivityStreamsPerson(nil)
if result {
t.Error("IsFirstObjectActivityStreamsPerson(nil) should return false")
}
}
func TestIsFirstObjectActivityStreamsPersonWithEmpty(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
result := IsFirstObjectActivityStreamsPerson(object)
if result {
t.Error("IsFirstObjectActivityStreamsPerson with empty object should return false")
}
}
func TestIsFirstObjectActivityStreamsPersonWithPerson(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
person := streams.NewActivityStreamsPerson()
object.AppendActivityStreamsPerson(person)
result := IsFirstObjectActivityStreamsPerson(object)
if !result {
t.Error("IsFirstObjectActivityStreamsPerson with person should return true")
}
}
func TestIsFirstObjectActivityStreamsPersonWithNonPerson(t *testing.T) {
object := streams.NewActivityStreamsObjectProperty()
note := streams.NewActivityStreamsNote()
object.AppendActivityStreamsNote(note)
result := IsFirstObjectActivityStreamsPerson(object)
if result {
t.Error("IsFirstObjectActivityStreamsPerson with note should return false")
}
}
+3 -5
View File
@@ -9,14 +9,12 @@ import (
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
)
// ActorHandler handles requests for a single actor.
func ActorHandler(w http.ResponseWriter, r *http.Request) {
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
@@ -24,7 +22,7 @@ func ActorHandler(w http.ResponseWriter, r *http.Request) {
pathComponents := strings.Split(r.URL.Path, "/")
accountName := pathComponents[3]
if _, valid := configRepository.GetFederatedInboxMap()[accountName]; !valid {
if _, valid := data.GetFederatedInboxMap()[accountName]; !valid {
// User is not valid
w.WriteHeader(http.StatusNotFound)
return
+7 -11
View File
@@ -14,9 +14,9 @@ import (
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
)
const (
@@ -44,8 +44,8 @@ func FollowersHandler(w http.ResponseWriter, r *http.Request) {
}
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte("internal server error"))
return
}
@@ -60,8 +60,7 @@ func FollowersHandler(w http.ResponseWriter, r *http.Request) {
}
func getInitialFollowersRequest(r *http.Request) (vocab.ActivityStreamsOrderedCollection, error) {
followersRepo := followersrepository.Get()
followerCount, _ := followersRepo.GetCount()
followerCount, _ := persistence.GetFollowerCount()
collection := streams.NewActivityStreamsOrderedCollection()
idProperty := streams.NewJSONLDIdProperty()
id, err := createPageURL(r, nil)
@@ -94,13 +93,12 @@ func getFollowersPage(page string, r *http.Request) (vocab.ActivityStreamsOrdere
return nil, errors.Wrap(err, "unable to parse page number")
}
followersRepo := followersrepository.Get()
followerCount, err := followersRepo.GetCount()
followerCount, err := persistence.GetFollowerCount()
if err != nil {
return nil, errors.Wrap(err, "unable to get follower count")
}
followers, _, err := followersRepo.GetFollowers(followersPageSize, (pageInt-1)*followersPageSize)
followers, _, err := persistence.GetFederationFollowers(followersPageSize, (pageInt-1)*followersPageSize)
if err != nil {
return nil, errors.Wrap(err, "unable to get federation followers")
}
@@ -147,9 +145,7 @@ func getFollowersPage(page string, r *http.Request) (vocab.ActivityStreamsOrdere
}
func createPageURL(r *http.Request, page *string) (*url.URL, error) {
configRepository := configrepository.Get()
domain := configRepository.GetServerURL()
domain := data.GetServerURL()
if domain == "" {
return nil, errors.New("unable to get server URL")
}
+3 -5
View File
@@ -7,7 +7,7 @@ import (
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/inbox"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
@@ -22,9 +22,7 @@ func InboxHandler(w http.ResponseWriter, r *http.Request) {
}
func acceptInboxRequest(w http.ResponseWriter, r *http.Request) {
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
@@ -41,7 +39,7 @@ func acceptInboxRequest(w http.ResponseWriter, r *http.Request) {
// The account this request is for must match the account name we have set
// for federation.
if forLocalAccount != configRepository.GetFederationUsername() {
if forLocalAccount != data.GetFederationUsername() {
w.WriteHeader(http.StatusNotFound)
return
}
+15 -33
View File
@@ -10,7 +10,7 @@ import (
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
@@ -25,14 +25,12 @@ func NodeInfoController(w http.ResponseWriter, r *http.Request) {
Links []links `json:"links"`
}
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
serverURL := configRepository.GetServerURL()
serverURL := data.GetServerURL()
if serverURL == "" {
w.WriteHeader(http.StatusNotFound)
return
@@ -91,9 +89,7 @@ func NodeInfoV2Controller(w http.ResponseWriter, r *http.Request) {
Metadata metadata `json:"metadata"`
}
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
@@ -121,7 +117,7 @@ func NodeInfoV2Controller(w http.ResponseWriter, r *http.Request) {
OpenRegistrations: false,
Protocols: []string{"activitypub"},
Metadata: metadata{
ChatEnabled: !configRepository.GetChatDisabled(),
ChatEnabled: !data.GetChatDisabled(),
},
}
@@ -167,14 +163,12 @@ func XNodeInfo2Controller(w http.ResponseWriter, r *http.Request) {
OpenRegistrations bool `json:"openRegistrations"`
}
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
serverURL := configRepository.GetServerURL()
serverURL := data.GetServerURL()
if serverURL == "" {
w.WriteHeader(http.StatusNotFound)
return
@@ -184,7 +178,7 @@ func XNodeInfo2Controller(w http.ResponseWriter, r *http.Request) {
res := &response{
Organization: Organization{
Name: configRepository.GetServerName(),
Name: data.GetServerName(),
Contact: serverURL,
},
Server: Server{
@@ -238,14 +232,12 @@ func InstanceV1Controller(w http.ResponseWriter, r *http.Request) {
InvitesEnabled bool `json:"invites_enabled"`
}
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
serverURL := configRepository.GetServerURL()
serverURL := data.GetServerURL()
if serverURL == "" {
w.WriteHeader(http.StatusNotFound)
return
@@ -262,9 +254,9 @@ func InstanceV1Controller(w http.ResponseWriter, r *http.Request) {
res := response{
URI: serverURL,
Title: configRepository.GetServerName(),
ShortDescription: configRepository.GetServerSummary(),
Description: configRepository.GetServerSummary(),
Title: data.GetServerName(),
ShortDescription: data.GetServerSummary(),
Description: data.GetServerSummary(),
Version: config.GetReleaseString(),
Stats: Stats{
UserCount: 1,
@@ -283,9 +275,7 @@ func InstanceV1Controller(w http.ResponseWriter, r *http.Request) {
}
func writeResponse(payload interface{}, w http.ResponseWriter) error {
configRepository := configrepository.Get()
accountName := configRepository.GetDefaultFederationUsername()
accountName := data.GetDefaultFederationUsername()
actorIRI := apmodels.MakeLocalIRIForAccount(accountName)
publicKey := crypto.GetPublicKey(actorIRI)
@@ -294,15 +284,7 @@ func writeResponse(payload interface{}, w http.ResponseWriter) error {
// HostMetaController points to webfinger.
func HostMetaController(w http.ResponseWriter, r *http.Request) {
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
log.Debugln("host meta request rejected! Federation is not enabled")
return
}
serverURL := configRepository.GetServerURL()
serverURL := data.GetServerURL()
if serverURL == "" {
w.WriteHeader(http.StatusNotFound)
return
+5 -7
View File
@@ -8,33 +8,31 @@ import (
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
// ObjectHandler handles requests for a single federated ActivityPub object.
func ObjectHandler(w http.ResponseWriter, r *http.Request) {
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
// If private federation mode is enabled do not allow access to objects.
if configRepository.GetFederationIsPrivate() {
if data.GetFederationIsPrivate() {
w.WriteHeader(http.StatusNotFound)
return
}
iri := strings.Join([]string{strings.TrimSuffix(configRepository.GetServerURL(), "/"), r.URL.Path}, "")
iri := strings.Join([]string{strings.TrimSuffix(data.GetServerURL(), "/"), r.URL.Path}, "")
object, _, _, err := persistence.GetObjectByIRI(iri)
if err != nil {
w.WriteHeader(http.StatusNotFound)
return
}
accountName := configRepository.GetDefaultFederationUsername()
accountName := data.GetDefaultFederationUsername()
actorIRI := apmodels.MakeLocalIRIForAccount(accountName)
publicKey := crypto.GetPublicKey(actorIRI)
+2 -3
View File
@@ -41,8 +41,8 @@ func OutboxHandler(w http.ResponseWriter, r *http.Request) {
}
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte("internal server error"))
return
}
@@ -65,8 +65,7 @@ func ActorObjectHandler(w http.ResponseWriter, r *http.Request) {
// controllers.WriteSimpleResponse(w, false, err.Error())
}
w.Header().Set("Content-Type", "application/activity+json")
if _, err := w.Write([]byte(object)); err != nil { //nolint:gosec
if _, err := w.Write([]byte(object)); err != nil {
log.Errorln(err)
}
}
+5 -7
View File
@@ -6,22 +6,20 @@ import (
"strings"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
)
// WebfingerHandler will handle webfinger lookup requests.
func WebfingerHandler(w http.ResponseWriter, r *http.Request) {
configRepository := configrepository.Get()
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
w.WriteHeader(http.StatusMethodNotAllowed)
log.Debugln("webfinger request rejected! Federation is not enabled")
return
}
instanceHostURL := configRepository.GetServerURL()
instanceHostURL := data.GetServerURL()
if instanceHostURL == "" {
w.WriteHeader(http.StatusNotFound)
log.Warnln("webfinger request rejected! Federation is enabled but server URL is empty.")
@@ -31,7 +29,7 @@ func WebfingerHandler(w http.ResponseWriter, r *http.Request) {
instanceHostString := utils.GetHostnameFromURLString(instanceHostURL)
if instanceHostString == "" {
w.WriteHeader(http.StatusNotFound)
log.Warnln("webfinger request rejected! Federation is enabled but server URL is not set properly. data.GetServerURL(): " + configRepository.GetServerURL())
log.Warnln("webfinger request rejected! Federation is enabled but server URL is not set properly. data.GetServerURL(): " + data.GetServerURL())
return
}
@@ -53,7 +51,7 @@ func WebfingerHandler(w http.ResponseWriter, r *http.Request) {
host := userComponents[1]
user := userComponents[0]
if _, valid := configRepository.GetFederatedInboxMap()[user]; !valid {
if _, valid := data.GetFederatedInboxMap()[user]; !valid {
w.WriteHeader(http.StatusNotFound)
log.Debugln("webfinger request rejected! Invalid user: " + user)
return
+3 -7
View File
@@ -8,15 +8,13 @@ import (
"errors"
"net/url"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
// GetPublicKey will return the public key for the provided actor.
func GetPublicKey(actorIRI *url.URL) PublicKey {
configRepository := configrepository.Get()
key := configRepository.GetPublicKey()
key := data.GetPublicKey()
idURL, err := url.Parse(actorIRI.String() + "#main-key")
if err != nil {
log.Errorln("unable to parse actor iri string", idURL, err)
@@ -31,9 +29,7 @@ func GetPublicKey(actorIRI *url.URL) PublicKey {
// GetPrivateKey will return the internal server private key.
func GetPrivateKey() *rsa.PrivateKey {
configRepository := configrepository.Get()
key := configRepository.GetPrivateKey()
key := data.GetPrivateKey()
block, _ := pem.Decode([]byte(key))
if block == nil {
-19
View File
@@ -1,19 +0,0 @@
package events
import (
"time"
"github.com/teris-io/shortid"
)
// Event is any kind of event.
type Event struct {
Timestamp time.Time `json:"timestamp"`
ID string `json:"id"`
}
// SetDefaults will set default properties of all inbound events.
func (e *Event) SetDefaults() {
e.ID = shortid.MustGenerate()
e.Timestamp = time.Now()
}
@@ -1,8 +0,0 @@
package events
type FediverseEngagementFollowEvent struct {
Event
Name string `json:"name"`
Username string `json:"username"`
Image string `json:"image"`
}
+3 -11
View File
@@ -5,24 +5,16 @@ import (
"time"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/core/chat/events"
"github.com/pkg/errors"
)
func handleAnnounceRequest(c context.Context, activity vocab.ActivityStreamsAnnounce) error {
objectIRI, err := apmodels.GetIRIStringFromObjectProperty(activity.GetActivityStreamsObject())
if err != nil {
return errors.Wrap(err, "announce activity is missing object IRI")
}
actorIRI, err := apmodels.GetIRIStringFromActorProperty(activity.GetActivityStreamsActor())
if err != nil {
return errors.Wrap(err, "announce activity is missing actor IRI")
}
object := activity.GetActivityStreamsObject()
actorReference := activity.GetActivityStreamsActor()
objectIRI := object.At(0).GetIRI().String()
actorIRI := actorReference.At(0).GetIRI().String()
if hasPreviouslyhandled, err := persistence.HasPreviouslyHandledInboundActivity(objectIRI, actorIRI, events.FediverseEngagementRepost); hasPreviouslyhandled || err != nil {
return errors.Wrap(err, "inbound activity of share/re-post has already been handled")
+14 -27
View File
@@ -4,57 +4,43 @@ import (
"fmt"
"github.com/go-fed/activity/streams/vocab"
"github.com/microcosm-cc/bluemonday"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/core/chat"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
)
// sanitizeActorName strips HTML tags from the ActivityPub actor display name.
// Falls back to the username if the display name is empty or entirely HTML.
func sanitizeActorName(displayName, username string) string {
strict := bluemonday.StrictPolicy()
name := strict.Sanitize(displayName)
if name == "" {
name = strict.Sanitize(username)
}
return name
}
func handleEngagementActivity(eventType events.EventType, isLiveNotification bool, actorReference vocab.ActivityStreamsActorProperty, action string) error {
configRepository := configrepository.Get()
// Do nothing if displaying engagement actions has been turned off.
if !configRepository.GetFederationShowEngagement() {
if !data.GetFederationShowEngagement() {
return nil
}
// Do nothing if chat is disabled
if configRepository.GetChatDisabled() {
if data.GetChatDisabled() {
return nil
}
// Get actor of the action
actor, err := resolvers.GetResolvedActorFromActorProperty(actorReference)
if err != nil {
return fmt.Errorf("unable to resolve actor for engagement activity: %w", err)
}
actor, _ := resolvers.GetResolvedActorFromActorProperty(actorReference)
// Send chat message
actorName := sanitizeActorName(actor.Name, actor.Username)
actorIRI := actor.ActorIriString()
actorName := actor.Name
if actorName == "" {
actorName = actor.Username
}
actorIRI := actorReference.Begin().GetIRI().String()
userPrefix := fmt.Sprintf("%s ", actorName)
var suffix string
if isLiveNotification && action == events.FediverseEngagementLike {
suffix = "liked that this stream went live."
} else if action == events.FediverseEngagementLike {
suffix = fmt.Sprintf("liked a post from %s.", configRepository.GetServerName())
suffix = fmt.Sprintf("liked a post from %s.", data.GetServerName())
} else if isLiveNotification && action == events.FediverseEngagementRepost {
suffix = "shared this stream with their followers."
} else if action == events.FediverseEngagementRepost {
suffix = fmt.Sprintf("shared a post from %s.", configRepository.GetServerName())
suffix = fmt.Sprintf("shared a post from %s.", data.GetServerName())
} else if action == events.FediverseEngagementFollow {
suffix = "followed this stream."
} else {
@@ -63,8 +49,9 @@ func handleEngagementActivity(eventType events.EventType, isLiveNotification boo
body := fmt.Sprintf("%s %s", userPrefix, suffix)
var image *string
if imageStr := actor.ImageString(); imageStr != "" {
image = &imageStr
if actor.Image != nil {
s := actor.Image.String()
image = &s
}
if err := chat.SendFediverseAction(eventType, actor.FullUsername, image, body, actorIRI); err != nil {
-120
View File
@@ -1,120 +0,0 @@
package inbox
import (
"testing"
)
func TestSanitizeActorName(t *testing.T) {
tests := []struct {
name string
displayName string
username string
expected string
}{
{
name: "plain display name",
displayName: "Alice",
username: "alice",
expected: "Alice",
},
{
name: "display name with emoji",
displayName: "Alice 🦊",
username: "alice",
expected: "Alice 🦊",
},
{
name: "display name with unicode",
displayName: "Ålice Böb",
username: "alice",
expected: "Ålice Böb",
},
{
name: "empty display name falls back to username",
displayName: "",
username: "alice",
expected: "alice",
},
{
name: "script tag in display name",
displayName: `<script>alert("xss")</script>`,
username: "alice",
expected: "alice",
},
{
name: "iframe injection in display name",
displayName: `<iframe src="https://evil.com" style="position:fixed;top:0;left:0;width:100%;height:100%"></iframe>`,
username: "alice",
expected: "alice",
},
{
name: "img tag in display name",
displayName: `<img src="https://evil.com/track.png">`,
username: "alice",
expected: "alice",
},
{
name: "form injection in display name",
displayName: `<form action="https://evil.com/steal"><input name="pw" type="password"></form>`,
username: "alice",
expected: "alice",
},
{
name: "meta refresh in display name",
displayName: `<meta http-equiv="refresh" content="0;url=https://evil.com">`,
username: "alice",
expected: "alice",
},
{
name: "mixed text and HTML in display name",
displayName: `Alice <script>alert(1)</script> Bob`,
username: "alice",
expected: "Alice Bob",
},
{
name: "custom emoji HTML in display name",
displayName: `Alice :blobcat: <img src="https://instance.com/emoji/blobcat.png" class="custom-emoji">`,
username: "alice",
expected: "Alice :blobcat: ",
},
{
name: "HTML in both display name and username",
displayName: `<script>alert(1)</script>`,
username: `<b>alice</b>`,
expected: "alice",
},
{
name: "entirely HTML display name falls back to username",
displayName: `<div></div>`,
username: "alice",
expected: "alice",
},
{
name: "style tag in display name",
displayName: `<style>body{display:none}</style>Alice`,
username: "alice",
expected: "Alice",
},
{
name: "nested HTML tags",
displayName: `<div><span><a href="https://evil.com">Click me</a></span></div>`,
username: "alice",
expected: "Click me",
},
{
name: "event handler attributes",
displayName: `<img src=x onerror="alert(1)">Alice`,
username: "alice",
expected: "Alice",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := sanitizeActorName(tt.displayName, tt.username)
if result != tt.expected {
t.Errorf("sanitizeActorName(%q, %q) = %q, want %q", tt.displayName, tt.username, result, tt.expected)
}
})
}
}
+1 -5
View File
@@ -4,14 +4,10 @@ import (
"context"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/pkg/errors"
)
func handleCreateRequest(c context.Context, activity vocab.ActivityStreamsCreate) error {
iri, err := apmodels.GetIRIStringFromJSONLDIdProperty(activity.GetJSONLDId())
if err != nil {
return errors.Wrap(err, "create activity is missing IRI")
}
iri := activity.GetJSONLDId().GetIRI().String()
return errors.New("not handling create request of: " + iri)
}
+11 -25
View File
@@ -6,23 +6,17 @@ import (
"time"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/core/webhooks"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
func handleFollowInboxRequest(c context.Context, activity vocab.ActivityStreamsFollow) error {
configRepository := configrepository.Get()
followersRepo := followersrepository.Get()
follow, err := resolvers.MakeFollowRequest(c, activity)
if err != nil {
log.Errorln("unable to create follow inbox request", err)
@@ -33,37 +27,30 @@ func handleFollowInboxRequest(c context.Context, activity vocab.ActivityStreamsF
return fmt.Errorf("unable to handle request")
}
approved := !configRepository.GetFederationIsPrivate()
approved := !data.GetFederationIsPrivate()
followRequest := *follow
if err := followersRepo.Add(followRequest, approved); err != nil {
if err := persistence.AddFollow(followRequest, approved); err != nil {
log.Errorln("unable to save follow request", err)
return err
}
localAccountName := configRepository.GetDefaultFederationUsername()
objectIRI, err := apmodels.GetIRIStringFromObjectProperty(activity.GetActivityStreamsObject())
if err != nil {
return errors.Wrap(err, "follow activity is missing object IRI")
}
actorIRI, err := apmodels.GetIRIStringFromActorProperty(activity.GetActivityStreamsActor())
if err != nil {
return errors.Wrap(err, "follow activity is missing actor IRI")
}
actorReference := activity.GetActivityStreamsActor()
localAccountName := data.GetDefaultFederationUsername()
if approved {
if err := requests.SendFollowAccept(follow.Inbox, activity, localAccountName); err != nil {
log.Errorln("unable to send follow accept", err)
return err
}
go webhooks.SendFediverseEngagementFollowEvent(actorIRI)
}
// Save as an accepted activity
actorReference := activity.GetActivityStreamsActor()
object := activity.GetActivityStreamsObject()
objectIRI := object.At(0).GetIRI().String()
actorIRI := actorReference.At(0).GetIRI().String()
// If this request is approved and we have not previously sent an action to
// chat due to a previous follow request, then do so.
hasPreviouslyhandled := true // Default so we don't send anything if it fails.
@@ -97,6 +84,5 @@ func handleUnfollowRequest(c context.Context, activity vocab.ActivityStreamsUndo
unfollowRequest := *request
log.Traceln("unfollow request:", unfollowRequest)
followersRepo := followersrepository.Get()
return followersRepo.Remove(unfollowRequest)
return persistence.RemoveFollow(unfollowRequest)
}
+11 -11
View File
@@ -5,24 +5,24 @@ import (
"time"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/core/chat/events"
"github.com/pkg/errors"
)
func handleLikeRequest(c context.Context, activity vocab.ActivityStreamsLike) error {
objectIRI, err := apmodels.GetIRIStringFromObjectProperty(activity.GetActivityStreamsObject())
if err != nil {
return errors.Wrap(err, "like activity is missing object IRI")
}
actorIRI, err := apmodels.GetIRIStringFromActorProperty(activity.GetActivityStreamsActor())
if err != nil {
return errors.Wrap(err, "like activity is missing actor IRI")
}
object := activity.GetActivityStreamsObject()
actorReference := activity.GetActivityStreamsActor()
if object.Len() < 1 {
return errors.New("like activity is missing object")
}
if actorReference.Len() < 1 {
return errors.New("like activity is missing actor")
}
objectIRI := object.At(0).GetIRI().String()
actorIRI := actorReference.At(0).GetIRI().String()
if hasPreviouslyhandled, err := persistence.HasPreviouslyHandledInboundActivity(objectIRI, actorIRI, events.FediverseEngagementLike); hasPreviouslyhandled || err != nil {
return errors.Wrap(err, "inbound activity of like has already been handled")
-251
View File
@@ -1,251 +0,0 @@
package inbox
import (
"context"
"net/url"
"testing"
"github.com/go-fed/activity/streams"
)
func mustParseURL(s string) *url.URL {
u, err := url.Parse(s)
if err != nil {
panic(err)
}
return u
}
// These tests verify that handler functions don't panic when given
// ActivityPub activities with nil or missing properties that could
// cause nil pointer dereferences.
func TestHandleFollowWithNilObject(t *testing.T) {
activity := streams.NewActivityStreamsFollow()
// Don't set object or actor - they will be nil
// This should return an error, not panic
err := handleFollowInboxRequest(context.Background(), activity)
if err == nil {
t.Error("handleFollowInboxRequest with nil object should return error")
}
}
func TestHandleFollowWithEmptyObject(t *testing.T) {
activity := streams.NewActivityStreamsFollow()
object := streams.NewActivityStreamsObjectProperty()
activity.SetActivityStreamsObject(object)
// Object is set but empty (no items)
err := handleFollowInboxRequest(context.Background(), activity)
if err == nil {
t.Error("handleFollowInboxRequest with empty object should return error")
}
}
func TestHandleFollowWithNilActorIRI(t *testing.T) {
activity := streams.NewActivityStreamsFollow()
// Set a valid object with IRI
object := streams.NewActivityStreamsObjectProperty()
objectNote := streams.NewActivityStreamsNote()
objectID := streams.NewJSONLDIdProperty()
objectID.SetIRI(mustParseURL("https://example.com/note/1"))
objectNote.SetJSONLDId(objectID)
object.AppendActivityStreamsNote(objectNote)
activity.SetActivityStreamsObject(object)
// Actor is nil
err := handleFollowInboxRequest(context.Background(), activity)
if err == nil {
t.Error("handleFollowInboxRequest with nil actor should return error")
}
}
func TestHandleAnnounceWithNilObject(t *testing.T) {
activity := streams.NewActivityStreamsAnnounce()
// Don't set object or actor
err := handleAnnounceRequest(context.Background(), activity)
if err == nil {
t.Error("handleAnnounceRequest with nil object should return error")
}
}
func TestHandleAnnounceWithEmptyObject(t *testing.T) {
activity := streams.NewActivityStreamsAnnounce()
object := streams.NewActivityStreamsObjectProperty()
activity.SetActivityStreamsObject(object)
err := handleAnnounceRequest(context.Background(), activity)
if err == nil {
t.Error("handleAnnounceRequest with empty object should return error")
}
}
func TestHandleAnnounceWithNilActorIRI(t *testing.T) {
activity := streams.NewActivityStreamsAnnounce()
// Set object with IRI
object := streams.NewActivityStreamsObjectProperty()
object.AppendIRI(mustParseURL("https://example.com/note/1"))
activity.SetActivityStreamsObject(object)
// Actor is nil
err := handleAnnounceRequest(context.Background(), activity)
if err == nil {
t.Error("handleAnnounceRequest with nil actor should return error")
}
}
func TestHandleLikeWithNilObject(t *testing.T) {
activity := streams.NewActivityStreamsLike()
// Don't set object or actor
err := handleLikeRequest(context.Background(), activity)
if err == nil {
t.Error("handleLikeRequest with nil object should return error")
}
}
func TestHandleLikeWithEmptyObject(t *testing.T) {
activity := streams.NewActivityStreamsLike()
object := streams.NewActivityStreamsObjectProperty()
activity.SetActivityStreamsObject(object)
err := handleLikeRequest(context.Background(), activity)
if err == nil {
t.Error("handleLikeRequest with empty object should return error")
}
}
func TestHandleLikeWithNilActorIRI(t *testing.T) {
activity := streams.NewActivityStreamsLike()
// Set object with IRI
object := streams.NewActivityStreamsObjectProperty()
object.AppendIRI(mustParseURL("https://example.com/note/1"))
activity.SetActivityStreamsObject(object)
// Actor is nil
err := handleLikeRequest(context.Background(), activity)
if err == nil {
t.Error("handleLikeRequest with nil actor should return error")
}
}
func TestHandleCreateWithNilId(t *testing.T) {
activity := streams.NewActivityStreamsCreate()
// Don't set JSONLD ID
err := handleCreateRequest(context.Background(), activity)
if err == nil {
t.Error("handleCreateRequest with nil ID should return error")
}
}
func TestHandleCreateWithIdButNilIRI(t *testing.T) {
activity := streams.NewActivityStreamsCreate()
id := streams.NewJSONLDIdProperty()
// Set the ID property but don't set an IRI on it
activity.SetJSONLDId(id)
err := handleCreateRequest(context.Background(), activity)
if err == nil {
t.Error("handleCreateRequest with ID but nil IRI should return error")
}
}
func TestHandleUpdateWithNilObject(t *testing.T) {
activity := streams.NewActivityStreamsUpdate()
// Don't set object - should return nil (not an error, just skip)
// This should not panic and should return nil since we only care about Person updates
err := handleUpdateRequest(context.Background(), activity)
if err != nil {
t.Errorf("handleUpdateRequest with nil object should return nil (skip), got %v", err)
}
}
func TestHandleUpdateWithEmptyObject(t *testing.T) {
activity := streams.NewActivityStreamsUpdate()
object := streams.NewActivityStreamsObjectProperty()
activity.SetActivityStreamsObject(object)
// Should return nil since empty object means it's not a Person update
err := handleUpdateRequest(context.Background(), activity)
if err != nil {
t.Errorf("handleUpdateRequest with empty object should return nil (skip), got %v", err)
}
}
func TestHandleUpdateWithNonPersonObject(t *testing.T) {
activity := streams.NewActivityStreamsUpdate()
object := streams.NewActivityStreamsObjectProperty()
note := streams.NewActivityStreamsNote()
object.AppendActivityStreamsNote(note)
activity.SetActivityStreamsObject(object)
// Should return nil since it's not a Person update
err := handleUpdateRequest(context.Background(), activity)
if err != nil {
t.Errorf("handleUpdateRequest with non-Person object should return nil (skip), got %v", err)
}
}
// TestNilSafetyNoPanic verifies that none of the handlers panic when given
// completely empty activities. This is the most important test - we want to
// ensure that malformed ActivityPub payloads don't crash the server.
func TestNilSafetyNoPanic(t *testing.T) {
ctx := context.Background()
t.Run("Follow with nil properties", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("handleFollowInboxRequest panicked: %v", r)
}
}()
activity := streams.NewActivityStreamsFollow()
_ = handleFollowInboxRequest(ctx, activity)
})
t.Run("Announce with nil properties", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("handleAnnounceRequest panicked: %v", r)
}
}()
activity := streams.NewActivityStreamsAnnounce()
_ = handleAnnounceRequest(ctx, activity)
})
t.Run("Like with nil properties", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("handleLikeRequest panicked: %v", r)
}
}()
activity := streams.NewActivityStreamsLike()
_ = handleLikeRequest(ctx, activity)
})
t.Run("Create with nil properties", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("handleCreateRequest panicked: %v", r)
}
}()
activity := streams.NewActivityStreamsCreate()
_ = handleCreateRequest(ctx, activity)
})
t.Run("Update with nil properties", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("handleUpdateRequest panicked: %v", r)
}
}()
activity := streams.NewActivityStreamsUpdate()
_ = handleUpdateRequest(ctx, activity)
})
}
+3 -5
View File
@@ -4,15 +4,14 @@ import (
"context"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/resolvers"
log "github.com/sirupsen/logrus"
)
func handleUpdateRequest(c context.Context, activity vocab.ActivityStreamsUpdate) error {
// We only care about update events to followers.
if !apmodels.IsFirstObjectActivityStreamsPerson(activity.GetActivityStreamsObject()) {
if !activity.GetActivityStreamsObject().At(0).IsActivityStreamsPerson() {
return nil
}
@@ -22,6 +21,5 @@ func handleUpdateRequest(c context.Context, activity vocab.ActivityStreamsUpdate
return err
}
followersRepo := followersrepository.Get()
return followersRepo.Update(actor.ActorIriString(), actor.InboxString(), actor.SharedInboxString(), actor.Name, actor.FullUsername, actor.ImageString())
return persistence.UpdateFollower(actor.ActorIri.String(), actor.Inbox.String(), actor.Name, actor.FullUsername, actor.Image.String())
}
+5 -11
View File
@@ -13,9 +13,9 @@ import (
"github.com/go-fed/httpsig"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
log "github.com/sirupsen/logrus"
)
@@ -96,10 +96,7 @@ func Verify(request *http.Request) (bool, error) {
return false, err
}
key, err := apmodels.GetPublicKeyPem(publicKey)
if err != nil {
return false, errors.Wrap(err, "failed to get public key PEM")
}
key := publicKey.GetW3IDSecurityV1PublicKeyPem().Get()
block, _ := pem.Decode([]byte(key))
if block == nil {
log.Errorln("failed to parse PEM block containing the public key")
@@ -134,9 +131,7 @@ func Verify(request *http.Request) (bool, error) {
}
func isBlockedDomain(domain string) bool {
configRepository := configrepository.Get()
blockedDomains := configRepository.GetBlockedFederatedDomains()
blockedDomains := data.GetBlockedFederatedDomains()
for _, blockedDomain := range blockedDomains {
if strings.Contains(domain, blockedDomain) {
@@ -148,8 +143,7 @@ func isBlockedDomain(domain string) bool {
}
func isBlockedActor(actorIRI *url.URL) (bool, error) {
followersRepo := followersrepository.Get()
blockedactor, err := followersRepo.GetByIRI(actorIRI.String())
blockedactor, err := persistence.GetFollower(actorIRI.String())
if blockedactor != nil && blockedactor.DisabledAt != nil {
return true, errors.Wrap(err, "remote actor is blocked")
+6 -12
View File
@@ -8,9 +8,7 @@ import (
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/persistence/configrepository"
)
func makeFakePerson() vocab.ActivityStreamsPerson {
@@ -51,24 +49,21 @@ func makeFakePerson() vocab.ActivityStreamsPerson {
func TestMain(m *testing.M) {
data.SetupPersistence(":memory:")
configRepository := configrepository.Get()
configRepository.SetServerURL("https://my.cool.site.biz")
data.SetServerURL("https://my.cool.site.biz")
persistence.Setup(data.GetDatastore())
m.Run()
}
func TestBlockedDomains(t *testing.T) {
configRepository := configrepository.Get()
person := makeFakePerson()
configRepository.SetBlockedFederatedDomains([]string{"freedom.eagle", "guns.life"})
data.SetBlockedFederatedDomains([]string{"freedom.eagle", "guns.life"})
if len(configRepository.GetBlockedFederatedDomains()) != 2 {
if len(data.GetBlockedFederatedDomains()) != 2 {
t.Error("Blocked federated domains is not set correctly")
}
for _, domain := range configRepository.GetBlockedFederatedDomains() {
for _, domain := range data.GetBlockedFederatedDomains() {
if domain == person.GetJSONLDId().GetIRI().Host {
return
}
@@ -80,14 +75,13 @@ func TestBlockedDomains(t *testing.T) {
func TestBlockedActors(t *testing.T) {
person := makeFakePerson()
fakeRequest := streams.NewActivityStreamsFollow()
followersRepo := followersrepository.Get()
followersRepo.Add(apmodels.ActivityPubActor{
persistence.AddFollow(apmodels.ActivityPubActor{
ActorIri: person.GetJSONLDId().GetIRI(),
Inbox: person.GetJSONLDId().GetIRI(),
FollowRequestIri: person.GetJSONLDId().GetIRI(),
RequestObject: fakeRequest,
}, false)
followersRepo.BlockOrReject(person.GetJSONLDId().GetIRI().String())
persistence.BlockOrRejectFollower(person.GetJSONLDId().GetIRI().String())
blocked, err := isBlockedActor(person.GetJSONLDId().GetIRI())
if err != nil {
-7
View File
@@ -37,14 +37,7 @@ func worker(workerID int, queue <-chan Job) {
log.Debugf("Started ActivityPub worker %d", workerID)
for job := range queue {
func() {
defer func() {
if r := recover(); r != nil {
log.Errorf("Recovered from panic in ActivityPub worker %d: %v", workerID, r)
}
}()
handle(job.request)
}()
log.Tracef("Done with ActivityPub inbox handler using worker %d", workerID)
}
-117
View File
@@ -1,117 +0,0 @@
package jobs
import (
"time"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/persistence/configrepository"
log "github.com/sirupsen/logrus"
)
const (
// ValidationInterval is how often the validation job runs.
ValidationInterval = 1 * time.Hour
// FollowersPerRun is how many followers to validate per job run.
FollowersPerRun = 5
// FailureDurationThreshold is how long a follower must be unreachable before removal.
FailureDurationThreshold = 7 * 24 * time.Hour // 7 days
// DelayBetweenFollowers is the delay between validating individual followers.
DelayBetweenFollowers = 2 * time.Second
)
// GetValidationInterval returns the configured validation interval, or the default.
func GetValidationInterval() time.Duration {
if config.FollowerValidationInterval > 0 {
return config.FollowerValidationInterval
}
return ValidationInterval
}
// StartFollowerValidationJob starts the background job that periodically validates followers.
func StartFollowerValidationJob() {
interval := GetValidationInterval()
ticker := time.NewTicker(interval)
go func() {
for range ticker.C {
runFollowerValidation()
}
}()
log.Debugf("Follower validation job scheduled with interval %v", interval)
}
func runFollowerValidation() {
configRepo := configrepository.Get()
if !configRepo.GetFederationEnabled() {
return
}
followersRepo := followersrepository.Get()
followers, err := followersRepo.GetFollowersToValidate(FollowersPerRun)
if err != nil {
log.Errorln("Failed to get followers for validation:", err)
return
}
for _, follower := range followers {
validateAndUpdateFollower(followersRepo, follower)
time.Sleep(DelayBetweenFollowers)
}
}
func validateAndUpdateFollower(repo followersrepository.FollowersRepository, follower models.Follower) {
resolvedActor, err := resolvers.GetResolvedActorFromIRI(follower.ActorIRI)
if err != nil {
handleValidationFailure(repo, follower, err)
return
}
// Success - clear failure timestamp and update data
if err := repo.UpdateFollowerValidationSuccess(follower.ActorIRI); err != nil {
log.Errorln("Failed to update validation success:", err)
}
// Update follower data
if err := repo.Update(
resolvedActor.ActorIriString(),
resolvedActor.InboxString(),
resolvedActor.SharedInboxString(),
resolvedActor.Name,
resolvedActor.FullUsername,
resolvedActor.ImageString(),
); err != nil {
log.Errorln("Failed to update follower data:", err)
}
}
func handleValidationFailure(repo followersrepository.FollowersRepository, follower models.Follower, resolveErr error) {
log.Debugf("Follower validation failed for %s: %v", follower.ActorIRI, resolveErr)
// Check removal eligibility BEFORE updating failure timestamp.
// We use the existing FirstValidationFailureAt from the database to determine
// if this follower has been failing long enough to be removed.
// If FirstValidationFailureAt is not set, this is a new failure and we just record it.
shouldRemove := false
if follower.FirstValidationFailureAt.Valid {
failureDuration := time.Since(follower.FirstValidationFailureAt.Time)
shouldRemove = failureDuration >= FailureDurationThreshold
}
// Update failure timestamp (sets first_validation_failure_at if not already set)
if err := repo.UpdateFollowerValidationFailure(follower.ActorIRI); err != nil {
log.Errorln("Failed to update validation failure:", err)
return
}
// Remove follower if they've exceeded the failure threshold
if shouldRemove {
failureDuration := time.Since(follower.FirstValidationFailureAt.Time)
log.Infof("Removing follower %s after %v of consecutive failures",
follower.ActorIRI, failureDuration.Round(time.Hour))
if err := repo.RemoveByIRI(follower.ActorIRI); err != nil {
log.Errorln("Failed to remove invalid follower:", err)
}
}
}
-351
View File
@@ -1,351 +0,0 @@
package jobs
import (
"net/url"
"os"
"testing"
"time"
"github.com/go-fed/activity/streams"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/utils"
)
func TestMain(m *testing.M) {
setup()
code := m.Run()
os.Exit(code)
}
var datastore *data.Datastore
func setup() {
resetTestDatabase()
}
// resetTestDatabase initializes a fresh in-memory database for testing.
func resetTestDatabase() {
data.SetupPersistence(":memory:")
datastore = data.GetDatastore()
persistence.Setup(datastore)
}
// setupTestWithRepo resets the database and returns a new repository instance.
func setupTestWithRepo(t *testing.T) followersrepository.FollowersRepository {
t.Helper()
resetTestDatabase()
return followersrepository.New(datastore)
}
func createTestFollower(repo followersrepository.FollowersRepository, iri, inbox, name, username string) {
actorIRI, _ := url.Parse(iri)
inboxURL, _ := url.Parse(inbox)
requestIRI, _ := url.Parse("https://fake.server/follow/request")
fakeRequest := streams.NewActivityStreamsFollow()
repo.Add(apmodels.ActivityPubActor{
ActorIri: actorIRI,
Inbox: inboxURL,
Name: name,
Username: username,
FullUsername: username + "@fake.server",
FollowRequestIri: requestIRI,
RequestObject: fakeRequest,
}, true)
}
func TestGetFollowersToValidate(t *testing.T) {
repo := setupTestWithRepo(t)
// Create some test followers
for i := 0; i < 10; i++ {
user, _ := utils.GenerateRandomString(10)
createTestFollower(repo, "https://fake.server/user/"+user, "https://fake.server/user/"+user+"/inbox", user, user)
}
// Get followers to validate
followers, err := repo.GetFollowersToValidate(5)
if err != nil {
t.Fatalf("Error getting followers to validate: %s", err)
}
if len(followers) != 5 {
t.Errorf("Expected 5 followers to validate, got %d", len(followers))
}
}
func TestUpdateFollowerValidationSuccess(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testuser"
createTestFollower(repo, testIRI, "https://fake.server/user/testuser/inbox", "Test User", "testuser")
// Mark validation as successful
err := repo.UpdateFollowerValidationSuccess(testIRI)
if err != nil {
t.Fatalf("Error updating follower validation success: %s", err)
}
// Verify the follower was updated
followers, err := repo.GetFollowersToValidate(10)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
// After validation success, FirstValidationFailureAt should be NULL/invalid
for _, f := range followers {
if f.ActorIRI == testIRI {
if f.FirstValidationFailureAt.Valid {
t.Error("Expected FirstValidationFailureAt to be NULL after success")
}
}
}
}
func TestUpdateFollowerValidationFailure(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testuser"
createTestFollower(repo, testIRI, "https://fake.server/user/testuser/inbox", "Test User", "testuser")
// Mark validation as failed
err := repo.UpdateFollowerValidationFailure(testIRI)
if err != nil {
t.Fatalf("Error updating follower validation failure: %s", err)
}
// Verify the FirstValidationFailureAt is set
followers, err := repo.GetFollowersToValidate(10)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
found := false
for _, f := range followers {
if f.ActorIRI == testIRI {
found = true
if !f.FirstValidationFailureAt.Valid {
t.Error("Expected FirstValidationFailureAt to be set after failure")
}
}
}
if !found {
t.Error("Test follower not found in results")
}
}
func TestValidationFailureClearedOnSuccess(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testuser"
createTestFollower(repo, testIRI, "https://fake.server/user/testuser/inbox", "Test User", "testuser")
// Mark validation as failed first
err := repo.UpdateFollowerValidationFailure(testIRI)
if err != nil {
t.Fatalf("Error updating follower validation failure: %s", err)
}
// Then mark as successful
err = repo.UpdateFollowerValidationSuccess(testIRI)
if err != nil {
t.Fatalf("Error updating follower validation success: %s", err)
}
// Verify FirstValidationFailureAt is cleared
followers, err := repo.GetFollowersToValidate(10)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
for _, f := range followers {
if f.ActorIRI == testIRI {
if f.FirstValidationFailureAt.Valid {
t.Error("Expected FirstValidationFailureAt to be NULL after success")
}
}
}
}
func TestRemoveByIRI(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testuser"
createTestFollower(repo, testIRI, "https://fake.server/user/testuser/inbox", "Test User", "testuser")
// Verify follower exists
count, err := repo.GetCount()
if err != nil {
t.Fatalf("Error getting count: %s", err)
}
if count != 1 {
t.Errorf("Expected 1 follower, got %d", count)
}
// Remove the follower
err = repo.RemoveByIRI(testIRI)
if err != nil {
t.Fatalf("Error removing follower: %s", err)
}
// Verify follower is removed
count, err = repo.GetCount()
if err != nil {
t.Fatalf("Error getting count: %s", err)
}
if count != 0 {
t.Errorf("Expected 0 followers after removal, got %d", count)
}
}
func TestFailureDurationThresholdLogic(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testfailure"
createTestFollower(repo, testIRI, "https://fake.server/user/testfailure/inbox", "Test User", "testfailure")
// Set first_validation_failure_at to 8 days ago (past threshold)
eightDaysAgo := time.Now().Add(-8 * 24 * time.Hour)
_, err := datastore.DB.Exec(
"UPDATE ap_followers SET first_validation_failure_at = ? WHERE iri = ?",
eightDaysAgo, testIRI,
)
if err != nil {
t.Fatalf("Error setting first_validation_failure_at: %s", err)
}
// Fetch the follower
followers, err := repo.GetFollowersToValidate(1)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
if len(followers) != 1 {
t.Fatalf("Expected 1 follower, got %d", len(followers))
}
// Verify the failure duration is calculated correctly
failureDuration := time.Since(followers[0].FirstValidationFailureAt.Time)
if failureDuration < FailureDurationThreshold {
t.Errorf("Expected failure duration (%v) to be >= threshold (%v)", failureDuration, FailureDurationThreshold)
}
// Test removal - this directly tests the removal logic without network calls
err = repo.RemoveByIRI(testIRI)
if err != nil {
t.Fatalf("Error removing follower: %s", err)
}
// Verify the follower was removed
count, err := repo.GetCount()
if err != nil {
t.Fatalf("Error getting count: %s", err)
}
if count != 0 {
t.Errorf("Expected follower to be removed, but count is %d", count)
}
}
func TestFailureNotRemovedBeforeThreshold(t *testing.T) {
repo := setupTestWithRepo(t)
// Create a test follower
testIRI := "https://fake.server/user/testnotremoved"
createTestFollower(repo, testIRI, "https://fake.server/user/testnotremoved/inbox", "Test User", "testnotremoved")
// Set first_validation_failure_at to 1 day ago (not past threshold)
oneDayAgo := time.Now().Add(-1 * 24 * time.Hour)
_, err := datastore.DB.Exec(
"UPDATE ap_followers SET first_validation_failure_at = ? WHERE iri = ?",
oneDayAgo, testIRI,
)
if err != nil {
t.Fatalf("Error setting first_validation_failure_at: %s", err)
}
// Fetch the follower
followers, err := repo.GetFollowersToValidate(1)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
if len(followers) != 1 {
t.Fatalf("Expected 1 follower, got %d", len(followers))
}
// Verify the failure duration is below threshold
failureDuration := time.Since(followers[0].FirstValidationFailureAt.Time)
if failureDuration >= FailureDurationThreshold {
t.Errorf("Expected failure duration (%v) to be < threshold (%v)", failureDuration, FailureDurationThreshold)
}
// Follower should NOT be removed yet
count, err := repo.GetCount()
if err != nil {
t.Fatalf("Error getting count: %s", err)
}
if count != 1 {
t.Errorf("Expected follower to NOT be removed yet, but count is %d", count)
}
}
func TestFollowersOrderedByOldestValidatedFirst(t *testing.T) {
repo := setupTestWithRepo(t)
// Create followers
iri1 := "https://fake.server/user/first"
iri2 := "https://fake.server/user/second"
iri3 := "https://fake.server/user/third"
createTestFollower(repo, iri1, "https://fake.server/user/first/inbox", "First", "first")
createTestFollower(repo, iri2, "https://fake.server/user/second/inbox", "Second", "second")
createTestFollower(repo, iri3, "https://fake.server/user/third/inbox", "Third", "third")
// Set different last_validated_at times
now := time.Now()
_, err := datastore.DB.Exec("UPDATE ap_followers SET last_validated_at = ? WHERE iri = ?",
now.Add(-3*time.Hour), iri1)
if err != nil {
t.Fatalf("Error updating: %s", err)
}
_, err = datastore.DB.Exec("UPDATE ap_followers SET last_validated_at = ? WHERE iri = ?",
now.Add(-1*time.Hour), iri2)
if err != nil {
t.Fatalf("Error updating: %s", err)
}
// iri3 has NULL last_validated_at (never validated)
// Get followers - should return in order: iri3 (NULL), iri1 (oldest), iri2 (newest)
followers, err := repo.GetFollowersToValidate(3)
if err != nil {
t.Fatalf("Error getting followers: %s", err)
}
if len(followers) != 3 {
t.Fatalf("Expected 3 followers, got %d", len(followers))
}
// NULL values should come first (NULLS FIRST in query)
if followers[0].ActorIRI != iri3 {
t.Errorf("Expected first follower to be %s (never validated), got %s", iri3, followers[0].ActorIRI)
}
// Then oldest validated
if followers[1].ActorIRI != iri1 {
t.Errorf("Expected second follower to be %s (oldest validated), got %s", iri1, followers[1].ActorIRI)
}
// Then newest validated
if followers[2].ActorIRI != iri2 {
t.Errorf("Expected third follower to be %s (newest validated), got %s", iri2, followers[2].ActorIRI)
}
}
-79
View File
@@ -1,79 +0,0 @@
package outbox
import (
"testing"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
)
// TestAddWithNilId verifies that Add doesn't panic when given an item with nil JSONLD ID.
func TestAddWithNilId(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("Add panicked with nil ID: %v", r)
}
}()
note := streams.NewActivityStreamsNote()
// Don't set JSONLD ID
err := Add(note, "test-id", false)
if err == nil {
t.Error("Add with nil ID should return error")
}
}
// TestAddWithIdButNilIRI verifies that Add doesn't panic when given an item
// with a JSONLD ID property that has no IRI set.
func TestAddWithIdButNilIRI(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("Add panicked with ID but nil IRI: %v", r)
}
}()
note := streams.NewActivityStreamsNote()
id := streams.NewJSONLDIdProperty()
// Set the ID property but don't set an IRI on it
note.SetJSONLDId(id)
err := Add(note, "test-id", false)
if err == nil {
t.Error("Add with ID but nil IRI should return error")
}
}
// TestAddNoPanic verifies that Add doesn't panic with various nil/empty inputs.
func TestAddNoPanic(t *testing.T) {
testCases := []struct {
name string
item func() vocab.ActivityStreamsNote
}{
{
name: "note with no properties",
item: func() vocab.ActivityStreamsNote {
return streams.NewActivityStreamsNote()
},
},
{
name: "note with empty ID property",
item: func() vocab.ActivityStreamsNote {
note := streams.NewActivityStreamsNote()
note.SetJSONLDId(streams.NewJSONLDIdProperty())
return note
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("Add panicked: %v", r)
}
}()
_ = Add(tc.item(), "test-id", false)
})
}
}
+35 -113
View File
@@ -6,22 +6,20 @@ import (
"path/filepath"
"regexp"
"strings"
"time"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/persistence"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/activitypub/requests"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/activitypub/webfinger"
"github.com/owncast/owncast/activitypub/workerpool"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/pkg/errors"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
"github.com/teris-io/shortid"
@@ -29,9 +27,7 @@ import (
// SendLive will send all followers the message saying you started a live stream.
func SendLive() error {
configRepository := configrepository.Get()
textContent := configRepository.GetFederationGoLiveMessage()
textContent := data.GetFederationGoLiveMessage()
// If the message is empty then do not send it.
if textContent == "" {
@@ -42,7 +38,7 @@ func SendLive() error {
reg := regexp.MustCompile("[^a-zA-Z0-9]+")
tagProp := streams.NewActivityStreamsTagProperty()
for _, tagString := range configRepository.GetServerMetadataTags() {
for _, tagString := range data.GetServerMetadataTags() {
tagWithoutSpecialCharacters := reg.ReplaceAllString(tagString, "")
hashtag := apmodels.MakeHashtag(tagWithoutSpecialCharacters)
tagProp.AppendTootHashtag(hashtag)
@@ -61,23 +57,23 @@ func SendLive() error {
tagsString := strings.Join(tagStrings, " ")
var streamTitle string
if title := configRepository.GetStreamTitle(); title != "" {
if title := data.GetStreamTitle(); title != "" {
streamTitle = fmt.Sprintf("<p>%s</p>", title)
}
textContent = fmt.Sprintf("<p>%s</p>%s<p>%s</p><p><a href=\"%s\">%s</a></p>", textContent, streamTitle, tagsString, configRepository.GetServerURL(), configRepository.GetServerURL())
textContent = fmt.Sprintf("<p>%s</p>%s<p>%s</p><p><a href=\"%s\">%s</a></p>", textContent, streamTitle, tagsString, data.GetServerURL(), data.GetServerURL())
activity, _, note, noteID := createBaseOutboundMessage(textContent)
to, cc := getAddressingToFollowers()
note.SetActivityStreamsTo(to)
note.SetActivityStreamsCc(cc)
activity.SetActivityStreamsTo(to)
activity.SetActivityStreamsCc(cc)
// To the public if we're not treating ActivityPub as "private".
if !data.GetFederationIsPrivate() {
note = apmodels.MakeNotePublic(note)
activity = apmodels.MakeActivityPublic(activity)
}
note.SetActivityStreamsTag(tagProp)
// Attach an image along with the Federated message.
previewURL, err := url.Parse(configRepository.GetServerURL())
previewURL, err := url.Parse(data.GetServerURL())
if err == nil {
var imageToAttach string
var mediaType string
@@ -98,7 +94,7 @@ func SendLive() error {
}
}
if configRepository.GetNSFW() {
if data.GetNSFW() {
// Mark content as sensitive.
sensitive := streams.NewActivityStreamsSensitiveProperty()
sensitive.AppendXMLSchemaBoolean(true)
@@ -177,11 +173,10 @@ func SendPublicMessage(textContent string) error {
activity, _, note, noteID := createBaseOutboundMessage(textContent)
note.SetActivityStreamsTag(tagProp)
to, cc := getAddressingToFollowers()
note.SetActivityStreamsTo(to)
note.SetActivityStreamsCc(cc)
activity.SetActivityStreamsTo(to)
activity.SetActivityStreamsCc(cc)
if !data.GetFederationIsPrivate() {
note = apmodels.MakeNotePublic(note)
activity = apmodels.MakeActivityPublic(activity)
}
b, err := apmodels.Serialize(activity)
if err != nil {
@@ -200,22 +195,9 @@ func SendPublicMessage(textContent string) error {
return nil
}
// if public, cc the followers and to the Public uri, else private, address followers directly.
func getAddressingToFollowers() (vocab.ActivityStreamsToProperty, vocab.ActivityStreamsCcProperty) {
configRepository := configrepository.Get()
server_url := configRepository.GetServerURL()
followers_iri, _ := url.Parse(server_url)
username := configRepository.GetDefaultFederationUsername()
followers_iri = followers_iri.JoinPath("federation", "user", username, "followers")
return apmodels.MakeAddressingToFollowers(followers_iri, !configRepository.GetFederationIsPrivate())
}
// nolint: unparam
func createBaseOutboundMessage(textContent string) (vocab.ActivityStreamsCreate, string, vocab.ActivityStreamsNote, string) {
configRepository := configrepository.Get()
localActor := apmodels.MakeLocalIRIForAccount(configRepository.GetDefaultFederationUsername())
localActor := apmodels.MakeLocalIRIForAccount(data.GetDefaultFederationUsername())
noteID := shortid.MustGenerate()
noteIRI := apmodels.MakeLocalIRIForResource(noteID)
id := shortid.MustGenerate()
@@ -231,94 +213,35 @@ func createBaseOutboundMessage(textContent string) (vocab.ActivityStreamsCreate,
// Get Hashtag HTML link for a given tag (without # prefix).
func getHashtagLinkHTMLFromTagString(baseHashtag string) string {
return fmt.Sprintf("<a class=\"hashtag\" href=\"https://owncast.directory/tags/%s\">#%s</a>", baseHashtag, baseHashtag)
return fmt.Sprintf("<a class=\"hashtag\" href=\"https://directory.owncast.online/tags/%s\">#%s</a>", baseHashtag, baseHashtag)
}
// SendToFollowers will send an arbitrary payload to all follower inboxes.
// It uses shared inboxes when available to reduce the number of outbound requests.
func SendToFollowers(payload []byte) error {
configRepository := configrepository.Get()
followersRepo := followersrepository.Get()
localActor := apmodels.MakeLocalIRIForAccount(configRepository.GetDefaultFederationUsername())
localActor := apmodels.MakeLocalIRIForAccount(data.GetDefaultFederationUsername())
// Get unique delivery inboxes (prefers shared inboxes over individual inboxes)
inboxes, err := followersRepo.GetUniqueDeliveryInboxes()
followers, _, err := persistence.GetFederationFollowers(-1, 0)
if err != nil {
log.Errorln("unable to fetch delivery inboxes", err)
return errors.New("unable to fetch delivery inboxes to send payload to")
}
// Batch size and delay to prevent resource exhaustion during delivery.
// This spreads CPU load from cryptographic signing over time.
const batchSize = 50
const batchDelay = 100 * time.Millisecond
queued := 0
skipped := 0
for i, inboxURL := range inboxes {
inbox, err := url.Parse(inboxURL)
if err != nil {
log.Warnln("unable to parse inbox URL", inboxURL, err)
continue
}
// SSRF protection: reject non-HTTPS schemes and internal/loopback hosts.
// A malicious remote actor could set their inbox to an internal address
// to trick this server into making requests to internal services.
if inbox.Scheme != "https" {
log.Warnln("rejecting non-HTTPS inbox URL for SSRF protection:", inboxURL)
continue
}
if utils.IsHostnameInternal(inbox.Hostname()) {
log.Warnln("rejecting internal/loopback inbox URL for SSRF protection:", inboxURL)
continue
}
// Pre-check circuit breaker BEFORE expensive cryptographic signing.
// This saves CPU cycles for domains we know are failing.
if workerpool.ShouldSkipDomain(inbox.Host) {
skipped++
continue
log.Errorln("unable to fetch followers to send to", err)
return errors.New("unable to fetch followers to send payload to")
}
for _, follower := range followers {
inbox, _ := url.Parse(follower.Inbox)
req, err := crypto.CreateSignedRequest(payload, inbox, localActor)
if err != nil {
log.Errorln("unable to create outbox request", inboxURL, err)
continue
log.Errorln("unable to create outbox request", follower.Inbox, err)
return errors.New("unable to create outbox request: " + follower.Inbox)
}
workerpool.AddToOutboundQueue(req)
queued++
// Add a small delay between batches to spread out CPU and network load.
// This helps prevent ActivityPub delivery from competing with video encoding.
// Use queued count (not loop index) to ensure consistent rate limiting
// even when followers are skipped due to circuit breaker or parse errors.
if queued%batchSize == 0 && i+1 < len(inboxes) {
time.Sleep(batchDelay)
}
}
if skipped > 0 {
log.Debugf("Skipped %d followers due to circuit breaker, queued %d", skipped, queued)
}
return nil
}
// SendToUser will send a payload to a single specific inbox.
func SendToUser(inbox *url.URL, payload []byte) error {
// SSRF protection: reject non-HTTPS schemes and internal/loopback hosts.
if inbox.Scheme != "https" {
return errors.Errorf("rejecting non-HTTPS inbox URL for SSRF protection: %s", inbox.String())
}
if utils.IsHostnameInternal(inbox.Hostname()) {
return errors.Errorf("rejecting internal/loopback inbox URL for SSRF protection: %s", inbox.String())
}
configRepository := configrepository.Get()
localActor := apmodels.MakeLocalIRIForAccount(configRepository.GetDefaultFederationUsername())
localActor := apmodels.MakeLocalIRIForAccount(data.GetDefaultFederationUsername())
req, err := requests.CreateSignedRequest(payload, inbox, localActor)
if err != nil {
@@ -332,10 +255,8 @@ func SendToUser(inbox *url.URL, payload []byte) error {
// UpdateFollowersWithAccountUpdates will send an update to all followers alerting of a profile update.
func UpdateFollowersWithAccountUpdates() error {
configRepository := configrepository.Get()
// Don't do anything if federation is disabled.
if !configRepository.GetFederationEnabled() {
if !data.GetFederationEnabled() {
return nil
}
@@ -344,7 +265,7 @@ func UpdateFollowersWithAccountUpdates() error {
activity := apmodels.MakeUpdateActivity(objectID)
actor := streams.NewActivityStreamsPerson()
actorID := apmodels.MakeLocalIRIForAccount(configRepository.GetDefaultFederationUsername())
actorID := apmodels.MakeLocalIRIForAccount(data.GetDefaultFederationUsername())
actorIDProperty := streams.NewJSONLDIdProperty()
actorIDProperty.Set(actorID)
actor.SetJSONLDId(actorIDProperty)
@@ -367,13 +288,14 @@ func UpdateFollowersWithAccountUpdates() error {
// Add will save an ActivityPub object to the datastore.
func Add(item vocab.Type, id string, isLiveNotification bool) error {
iri, err := apmodels.GetIRIStringFromJSONLDIdProperty(item.GetJSONLDId())
if err != nil {
log.Errorln("Unable to get iri from item:", err)
return errors.Wrap(err, "unable to get iri from item "+id)
}
iri := item.GetJSONLDId().GetIRI().String()
typeString := item.GetTypeName()
if iri == "" {
log.Errorln("Unable to get iri from item")
return errors.New("Unable to get iri from item " + id)
}
b, err := apmodels.Serialize(item)
if err != nil {
log.Errorln("unable to serialize model when saving to outbox", err)
+94 -3
View File
@@ -1,6 +1,12 @@
package persistence
import (
"context"
"github.com/owncast/owncast/db"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
@@ -10,7 +16,6 @@ func createFederationFollowersTable() {
createTableSQL := `CREATE TABLE IF NOT EXISTS ap_followers (
"iri" TEXT NOT NULL,
"inbox" TEXT NOT NULL,
"shared_inbox" TEXT,
"name" TEXT,
"username" TEXT NOT NULL,
"image" TEXT,
@@ -19,10 +24,96 @@ func createFederationFollowersTable() {
"approved_at" TIMESTAMP,
"disabled_at" TIMESTAMP,
"request_object" BLOB,
"last_validated_at" TIMESTAMP,
"first_validation_failure_at" TIMESTAMP,
PRIMARY KEY (iri));`
_datastore.MustExec(createTableSQL)
_datastore.MustExec(`CREATE INDEX IF NOT EXISTS idx_iri ON ap_followers (iri);`)
_datastore.MustExec(`CREATE INDEX IF NOT EXISTS idx_approved_at ON ap_followers (approved_at);`)
}
// GetFollowerCount will return the number of followers we're keeping track of.
func GetFollowerCount() (int64, error) {
ctx := context.Background()
return _datastore.GetQueries().GetFollowerCount(ctx)
}
// GetFederationFollowers will return a slice of the followers we keep track of locally.
func GetFederationFollowers(limit int, offset int) ([]models.Follower, int, error) {
ctx := context.Background()
total, err := _datastore.GetQueries().GetFollowerCount(ctx)
if err != nil {
return nil, 0, errors.Wrap(err, "unable to fetch total number of followers")
}
followersResult, err := _datastore.GetQueries().GetFederationFollowersWithOffset(ctx, db.GetFederationFollowersWithOffsetParams{
Limit: int32(limit),
Offset: int32(offset),
})
if err != nil {
return nil, 0, err
}
followers := make([]models.Follower, 0)
for _, row := range followersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
Inbox: row.Inbox,
Timestamp: utils.NullTime(row.CreatedAt),
}
followers = append(followers, singleFollower)
}
return followers, int(total), nil
}
// GetPendingFollowRequests will return pending follow requests.
func GetPendingFollowRequests() ([]models.Follower, error) {
pendingFollowersResult, err := _datastore.GetQueries().GetFederationFollowerApprovalRequests(context.Background())
if err != nil {
return nil, err
}
followers := make([]models.Follower, 0)
for _, row := range pendingFollowersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
Inbox: row.Inbox,
Timestamp: utils.NullTime{Time: row.CreatedAt.Time, Valid: true},
}
followers = append(followers, singleFollower)
}
return followers, nil
}
// GetBlockedAndRejectedFollowers will return blocked and rejected followers.
func GetBlockedAndRejectedFollowers() ([]models.Follower, error) {
pendingFollowersResult, err := _datastore.GetQueries().GetRejectedAndBlockedFollowers(context.Background())
if err != nil {
return nil, err
}
followers := make([]models.Follower, 0)
for _, row := range pendingFollowersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
DisabledAt: utils.NullTime{Time: row.DisabledAt.Time, Valid: true},
Timestamp: utils.NullTime{Time: row.CreatedAt.Time, Valid: true},
}
followers = append(followers, singleFollower)
}
return followers, nil
}
+5 -218
View File
@@ -1,14 +1,9 @@
package persistence
import (
"net/url"
"os"
"strings"
"testing"
"github.com/go-fed/activity/streams"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/persistence/followersrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
@@ -27,30 +22,16 @@ func setup() {
_datastore = data.GetDatastore()
createFederationFollowersTable()
followersRepo := followersrepository.New(_datastore)
number := 100
for i := 0; i < number; i++ {
u := createFakeFollower()
actorIRI, _ := url.Parse(u.ActorIRI)
inboxURL, _ := url.Parse(u.Inbox)
requestIRI, _ := url.Parse("https://fake.fediverse.server/some/request")
fakeRequest := streams.NewActivityStreamsFollow()
followersRepo.Add(apmodels.ActivityPubActor{
ActorIri: actorIRI,
Inbox: inboxURL,
Name: u.Name,
Username: u.Username,
FollowRequestIri: requestIRI,
RequestObject: fakeRequest,
}, true)
createFollow(u.ActorIRI, u.Inbox, "https://fake.fediverse.server/some/request", u.Name, u.Username, u.Image, nil, true)
followers = append(followers, u)
}
}
func TestQueryFollowers(t *testing.T) {
followersRepo := followersrepository.New(_datastore)
f, total, err := followersRepo.GetFollowers(10, 0)
f, total, err := GetFederationFollowers(10, 0)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
@@ -65,8 +46,7 @@ func TestQueryFollowers(t *testing.T) {
}
func TestQueryFollowersWithOffset(t *testing.T) {
followersRepo := followersrepository.New(_datastore)
f, total, err := followersRepo.GetFollowers(10, 10)
f, total, err := GetFederationFollowers(10, 10)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
@@ -81,8 +61,7 @@ func TestQueryFollowersWithOffset(t *testing.T) {
}
func TestQueryFollowersWithOffsetAndLimit(t *testing.T) {
followersRepo := followersrepository.New(_datastore)
f, total, err := followersRepo.GetFollowers(10, 90)
f, total, err := GetFederationFollowers(10, 90)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
@@ -97,8 +76,7 @@ func TestQueryFollowersWithOffsetAndLimit(t *testing.T) {
}
func TestQueryFollowersWithPagination(t *testing.T) {
followersRepo := followersrepository.New(_datastore)
f, _, err := followersRepo.GetFollowers(15, 10)
f, _, err := GetFederationFollowers(15, 10)
if err != nil {
t.Errorf("Error querying followers: %s", err)
}
@@ -127,194 +105,3 @@ func createFakeFollower() models.Follower {
Timestamp: utils.NullTime{},
}
}
func createTestFollower(followersRepo followersrepository.FollowersRepository, actor, inbox, sharedInbox, request, name, username string) {
actorIRI, _ := url.Parse(actor)
inboxURL, _ := url.Parse(inbox)
var sharedInboxURL *url.URL
if sharedInbox != "" {
sharedInboxURL, _ = url.Parse(sharedInbox)
}
requestIRI, _ := url.Parse(request)
fakeRequest := streams.NewActivityStreamsFollow()
followersRepo.Add(apmodels.ActivityPubActor{
ActorIri: actorIRI,
Inbox: inboxURL,
SharedInbox: sharedInboxURL,
Name: name,
Username: username,
FollowRequestIri: requestIRI,
RequestObject: fakeRequest,
}, true)
}
func TestGetUniqueDeliveryInboxes(t *testing.T) {
// Set up a fresh database for this test
data.SetupPersistence(":memory:")
ds := data.GetDatastore()
_datastore = ds
createFederationFollowersTable()
followersRepo := followersrepository.New(ds)
// Create followers from server1 with a shared inbox (3 users, 1 shared inbox)
server1SharedInbox := "https://server1.example.com/inbox"
for i := 0; i < 3; i++ {
user, _ := utils.GenerateRandomString(10)
createTestFollower(
followersRepo,
"https://server1.example.com/user/"+user,
"https://server1.example.com/user/"+user+"/inbox",
server1SharedInbox,
"https://server1.example.com/follow/"+user,
user,
user,
)
}
// Create followers from server2 with a shared inbox (2 users, 1 shared inbox)
server2SharedInbox := "https://server2.example.com/inbox"
for i := 0; i < 2; i++ {
user, _ := utils.GenerateRandomString(10)
createTestFollower(
followersRepo,
"https://server2.example.com/user/"+user,
"https://server2.example.com/user/"+user+"/inbox",
server2SharedInbox,
"https://server2.example.com/follow/"+user,
user,
user,
)
}
// Create followers from server3 WITHOUT a shared inbox (2 users, 2 individual inboxes)
for i := 0; i < 2; i++ {
user, _ := utils.GenerateRandomString(10)
createTestFollower(
followersRepo,
"https://server3.example.com/user/"+user,
"https://server3.example.com/user/"+user+"/inbox",
"",
"https://server3.example.com/follow/"+user,
user,
user,
)
}
// Total: 7 followers, but should result in 4 unique delivery inboxes:
// - 1 shared inbox for server1
// - 1 shared inbox for server2
// - 2 individual inboxes for server3
inboxes, err := followersRepo.GetUniqueDeliveryInboxes()
if err != nil {
t.Fatalf("Error getting unique delivery inboxes: %s", err)
}
if len(inboxes) != 4 {
t.Errorf("Expected 4 unique delivery inboxes, got %d: %v", len(inboxes), inboxes)
}
// Verify the shared inboxes are included
hasServer1Shared := false
hasServer2Shared := false
server3IndividualCount := 0
for _, inbox := range inboxes {
if inbox == server1SharedInbox {
hasServer1Shared = true
}
if inbox == server2SharedInbox {
hasServer2Shared = true
}
if len(inbox) > 0 && inbox != server1SharedInbox && inbox != server2SharedInbox {
// Should be one of server3's individual inboxes
if !strings.Contains(inbox, "server3.example.com") {
t.Errorf("Unexpected inbox in results: %s", inbox)
}
server3IndividualCount++
}
}
if !hasServer1Shared {
t.Error("Expected server1 shared inbox to be in results")
}
if !hasServer2Shared {
t.Error("Expected server2 shared inbox to be in results")
}
if server3IndividualCount != 2 {
t.Errorf("Expected 2 individual inboxes from server3, got %d", server3IndividualCount)
}
}
func TestSharedInboxPreferredOverIndividual(t *testing.T) {
// Set up a fresh database for this test
data.SetupPersistence(":memory:")
ds := data.GetDatastore()
_datastore = ds
createFederationFollowersTable()
followersRepo := followersrepository.New(ds)
// Create a single follower with both individual and shared inbox
sharedInbox := "https://mastodon.social/inbox"
individualInbox := "https://mastodon.social/users/testuser/inbox"
createTestFollower(
followersRepo,
"https://mastodon.social/users/testuser",
individualInbox,
sharedInbox,
"https://mastodon.social/follow/123",
"Test User",
"testuser",
)
inboxes, err := followersRepo.GetUniqueDeliveryInboxes()
if err != nil {
t.Fatalf("Error getting unique delivery inboxes: %s", err)
}
if len(inboxes) != 1 {
t.Errorf("Expected 1 unique delivery inbox, got %d", len(inboxes))
}
// The shared inbox should be returned, not the individual inbox
if inboxes[0] != sharedInbox {
t.Errorf("Expected shared inbox %s, got %s", sharedInbox, inboxes[0])
}
}
func TestIndividualInboxUsedWhenNoSharedInbox(t *testing.T) {
// Set up a fresh database for this test
data.SetupPersistence(":memory:")
ds := data.GetDatastore()
_datastore = ds
createFederationFollowersTable()
followersRepo := followersrepository.New(ds)
// Create a follower without a shared inbox
individualInbox := "https://pleroma.example.com/users/testuser/inbox"
createTestFollower(
followersRepo,
"https://pleroma.example.com/users/testuser",
individualInbox,
"",
"https://pleroma.example.com/follow/123",
"Test User",
"testuser",
)
inboxes, err := followersRepo.GetUniqueDeliveryInboxes()
if err != nil {
t.Fatalf("Error getting unique delivery inboxes: %s", err)
}
if len(inboxes) != 1 {
t.Errorf("Expected 1 unique delivery inbox, got %d", len(inboxes))
}
// The individual inbox should be returned when no shared inbox exists
if inboxes[0] != individualInbox {
t.Errorf("Expected individual inbox %s, got %s", individualInbox, inboxes[0])
}
}
@@ -1,472 +0,0 @@
package followersrepository
import (
"context"
"database/sql"
"encoding/json"
"net/url"
"time"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/db"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
// FollowersRepository handles persistence of ActivityPub followers.
type FollowersRepository interface {
// GetCount returns the number of followers.
GetCount() (int64, error)
// GetFollowers returns a paginated list of followers.
GetFollowers(limit int, offset int) ([]models.Follower, int, error)
// GetPendingFollowRequests returns pending follow requests.
GetPendingFollowRequests() ([]models.Follower, error)
// GetBlockedAndRejected returns blocked and rejected followers.
GetBlockedAndRejected() ([]models.Follower, error)
// GetUniqueDeliveryInboxes returns unique inbox URLs for delivery.
GetUniqueDeliveryInboxes() ([]string, error)
// GetByIRI returns a single follower by IRI.
GetByIRI(iri string) (*apmodels.ActivityPubActor, error)
// Add saves a new follow to the datastore.
Add(follow apmodels.ActivityPubActor, approved bool) error
// Remove removes a follow from the datastore.
Remove(unfollow apmodels.ActivityPubActor) error
// ApprovePreviousRequest approves a pending follow request.
ApprovePreviousRequest(iri string) error
// BlockOrReject blocks an existing follower or rejects a follow request.
BlockOrReject(iri string) error
// Update updates the details of a stored follower.
Update(actorIRI string, inbox string, sharedInbox string, name string, username string, image string) error
// GetFollowersToValidate returns followers needing validation, ordered by oldest validated first.
GetFollowersToValidate(limit int) ([]models.Follower, error)
// UpdateFollowerValidationSuccess marks a follower as successfully validated and clears failure timestamp.
UpdateFollowerValidationSuccess(iri string) error
// UpdateFollowerValidationFailure marks a validation failure, setting first failure time if not already set.
UpdateFollowerValidationFailure(iri string) error
// RemoveByIRI removes a follower directly by IRI string.
RemoveByIRI(iri string) error
}
// SqlFollowersRepository is the SQL-based implementation of FollowersRepository.
type SqlFollowersRepository struct {
datastore *data.Datastore
}
// NOTE: This is temporary during the transition period.
var temporaryGlobalInstance FollowersRepository
// Get returns the followers repository singleton.
func Get() FollowersRepository {
if temporaryGlobalInstance == nil {
i := New(data.GetDatastore())
temporaryGlobalInstance = i
}
return temporaryGlobalInstance
}
// New creates a new instance of the FollowersRepository.
func New(datastore *data.Datastore) FollowersRepository {
r := SqlFollowersRepository{
datastore: datastore,
}
return &r
}
// GetCount returns the number of followers.
func (r *SqlFollowersRepository) GetCount() (int64, error) {
ctx := context.Background()
return r.datastore.GetQueries().GetFollowerCount(ctx)
}
// GetFollowers returns a paginated list of followers.
func (r *SqlFollowersRepository) GetFollowers(limit int, offset int) ([]models.Follower, int, error) {
ctx := context.Background()
total, err := r.datastore.GetQueries().GetFollowerCount(ctx)
if err != nil {
return nil, 0, errors.Wrap(err, "unable to fetch total number of followers")
}
followersResult, err := r.datastore.GetQueries().GetFederationFollowersWithOffset(ctx, db.GetFederationFollowersWithOffsetParams{
Limit: utils.SafeIntToInt32(limit),
Offset: utils.SafeIntToInt32(offset),
})
if err != nil {
return nil, 0, err
}
followers := make([]models.Follower, 0)
for _, row := range followersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
Inbox: row.Inbox,
SharedInbox: row.SharedInbox.String,
Timestamp: utils.NullTime(row.CreatedAt),
}
followers = append(followers, singleFollower)
}
return followers, int(total), nil
}
// GetPendingFollowRequests returns pending follow requests.
func (r *SqlFollowersRepository) GetPendingFollowRequests() ([]models.Follower, error) {
pendingFollowersResult, err := r.datastore.GetQueries().GetFederationFollowerApprovalRequests(context.Background())
if err != nil {
return nil, err
}
followers := make([]models.Follower, 0)
for _, row := range pendingFollowersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
Inbox: row.Inbox,
SharedInbox: row.SharedInbox.String,
Timestamp: utils.NullTime{Time: row.CreatedAt.Time, Valid: true},
}
followers = append(followers, singleFollower)
}
return followers, nil
}
// GetBlockedAndRejected returns blocked and rejected followers.
func (r *SqlFollowersRepository) GetBlockedAndRejected() ([]models.Follower, error) {
pendingFollowersResult, err := r.datastore.GetQueries().GetRejectedAndBlockedFollowers(context.Background())
if err != nil {
return nil, err
}
followers := make([]models.Follower, 0)
for _, row := range pendingFollowersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
DisabledAt: utils.NullTime{Time: row.DisabledAt.Time, Valid: true},
Timestamp: utils.NullTime{Time: row.CreatedAt.Time, Valid: true},
}
followers = append(followers, singleFollower)
}
return followers, nil
}
// GetUniqueDeliveryInboxes returns unique inbox URLs for delivery.
func (r *SqlFollowersRepository) GetUniqueDeliveryInboxes() ([]string, error) {
ctx := context.Background()
return r.datastore.GetQueries().GetUniqueDeliveryInboxes(ctx)
}
// GetByIRI returns a single follower by IRI.
func (r *SqlFollowersRepository) GetByIRI(iri string) (*apmodels.ActivityPubActor, error) {
result, err := r.datastore.GetQueries().GetFollowerByIRI(context.Background(), iri)
if err != nil {
return nil, err
}
followIRI, err := url.Parse(result.Request)
if err != nil {
return nil, errors.Wrap(err, "error parsing follow request IRI")
}
iriURL, err := url.Parse(result.Iri)
if err != nil {
return nil, errors.Wrap(err, "error parsing actor IRI")
}
inbox, err := url.Parse(result.Inbox)
if err != nil {
return nil, errors.Wrap(err, "error parsing acting inbox")
}
var sharedInbox *url.URL
if result.SharedInbox.Valid && result.SharedInbox.String != "" {
sharedInbox, err = url.Parse(result.SharedInbox.String)
if err != nil {
log.Warnln("error parsing shared inbox, ignoring:", err)
}
}
requestObjectBytes := result.RequestObject
var followRequestObject vocab.ActivityStreamsFollow
resolver, err := streams.NewJSONResolver(func(c context.Context, followObject vocab.ActivityStreamsFollow) error {
followRequestObject = followObject
return nil
})
if err != nil {
return nil, errors.Wrap(err, "error creating JSON resolver")
}
jsonMap := make(map[string]interface{})
err = json.Unmarshal(requestObjectBytes, &jsonMap)
if err != nil {
return nil, errors.Wrap(err, "error unmarshaling follow request object")
}
err = resolver.Resolve(context.Background(), jsonMap)
if err != nil {
return nil, errors.Wrap(err, "error resolving follow request object")
}
image, _ := url.Parse(result.Image.String)
var disabledAt *time.Time
if result.DisabledAt.Valid {
disabledAt = &result.DisabledAt.Time
}
follower := apmodels.ActivityPubActor{
ActorIri: iriURL,
Inbox: inbox,
SharedInbox: sharedInbox,
Name: result.Name.String,
Username: result.Username,
Image: image,
FollowRequestIri: followIRI,
DisabledAt: disabledAt,
RequestObject: followRequestObject,
}
return &follower, nil
}
// Add saves a new follow to the datastore.
func (r *SqlFollowersRepository) Add(follow apmodels.ActivityPubActor, approved bool) error {
if err := follow.Validate(); err != nil {
return errors.Wrap(err, "cannot add invalid follow")
}
log.Traceln("Saving", follow.ActorIriString(), "as a follower.")
followRequestObject, err := apmodels.Serialize(follow.RequestObject)
if err != nil {
return errors.Wrap(err, "error serializing follow request object")
}
return r.createFollow(follow.ActorIriString(), follow.InboxString(), follow.SharedInboxString(), follow.FollowRequestIriString(), follow.Name, follow.Username, follow.ImageString(), followRequestObject, approved)
}
// Remove removes a follow from the datastore.
func (r *SqlFollowersRepository) Remove(unfollow apmodels.ActivityPubActor) error {
if err := unfollow.Validate(); err != nil {
return errors.Wrap(err, "cannot remove invalid follow")
}
log.Traceln("Removing", unfollow.ActorIriString(), "as a follower.")
return r.removeFollow(unfollow.ActorIri)
}
// ApprovePreviousRequest approves a pending follow request.
func (r *SqlFollowersRepository) ApprovePreviousRequest(iri string) error {
return r.datastore.GetQueries().ApproveFederationFollower(context.Background(), db.ApproveFederationFollowerParams{
Iri: iri,
ApprovedAt: sql.NullTime{
Time: time.Now(),
Valid: true,
},
})
}
// BlockOrReject blocks an existing follower or rejects a follow request.
func (r *SqlFollowersRepository) BlockOrReject(iri string) error {
return r.datastore.GetQueries().RejectFederationFollower(context.Background(), db.RejectFederationFollowerParams{
Iri: iri,
DisabledAt: sql.NullTime{
Time: time.Now(),
Valid: true,
},
})
}
// Update updates the details of a stored follower.
func (r *SqlFollowersRepository) Update(actorIRI string, inbox string, sharedInbox string, name string, username string, image string) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
tx, err := r.datastore.DB.Begin()
if err != nil {
return errors.Wrap(err, "error beginning transaction")
}
defer func() {
_ = tx.Rollback()
}()
if err = r.datastore.GetQueries().WithTx(tx).UpdateFollowerByIRI(context.Background(), db.UpdateFollowerByIRIParams{
Inbox: inbox,
SharedInbox: sql.NullString{String: sharedInbox, Valid: sharedInbox != ""},
Name: sql.NullString{String: name, Valid: true},
Username: username,
Image: sql.NullString{String: image, Valid: true},
Iri: actorIRI,
}); err != nil {
return errors.Wrap(err, "error updating follower "+actorIRI)
}
return tx.Commit()
}
func (r *SqlFollowersRepository) createFollow(actor, inbox, sharedInbox, request, name, username, image string, requestObject []byte, approved bool) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
tx, err := r.datastore.DB.Begin()
if err != nil {
return errors.Wrap(err, "error beginning transaction")
}
defer func() {
_ = tx.Rollback()
}()
var approvedAt sql.NullTime
if approved {
approvedAt = sql.NullTime{
Time: time.Now(),
Valid: true,
}
}
if err = r.datastore.GetQueries().WithTx(tx).AddFollower(context.Background(), db.AddFollowerParams{
Iri: actor,
Inbox: inbox,
SharedInbox: sql.NullString{String: sharedInbox, Valid: sharedInbox != ""},
Name: sql.NullString{String: name, Valid: true},
Username: username,
Image: sql.NullString{String: image, Valid: true},
ApprovedAt: approvedAt,
Request: request,
RequestObject: requestObject,
}); err != nil {
return errors.Wrap(err, "error creating new federation follow")
}
return tx.Commit()
}
func (r *SqlFollowersRepository) removeFollow(actor *url.URL) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
tx, err := r.datastore.DB.Begin()
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
if err := r.datastore.GetQueries().WithTx(tx).RemoveFollowerByIRI(context.Background(), actor.String()); err != nil {
return err
}
return tx.Commit()
}
// GetFollowersToValidate returns followers needing validation, ordered by oldest validated first.
func (r *SqlFollowersRepository) GetFollowersToValidate(limit int) ([]models.Follower, error) {
ctx := context.Background()
followersResult, err := r.datastore.GetQueries().GetFollowersToValidate(ctx, utils.SafeIntToInt32(limit))
if err != nil {
return nil, err
}
followers := make([]models.Follower, 0)
for _, row := range followersResult {
singleFollower := models.Follower{
Name: row.Name.String,
Username: row.Username,
Image: row.Image.String,
ActorIRI: row.Iri,
Inbox: row.Inbox,
SharedInbox: row.SharedInbox.String,
FirstValidationFailureAt: utils.NullTime(row.FirstValidationFailureAt),
}
followers = append(followers, singleFollower)
}
return followers, nil
}
// UpdateFollowerValidationSuccess marks a follower as successfully validated and clears failure timestamp.
func (r *SqlFollowersRepository) UpdateFollowerValidationSuccess(iri string) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
ctx := context.Background()
tx, err := r.datastore.DB.Begin()
if err != nil {
return errors.Wrap(err, "error beginning transaction")
}
defer func() {
_ = tx.Rollback()
}()
if err := r.datastore.GetQueries().WithTx(tx).UpdateFollowerValidationSuccess(ctx, db.UpdateFollowerValidationSuccessParams{
LastValidatedAt: sql.NullTime{Time: time.Now(), Valid: true},
Iri: iri,
}); err != nil {
return errors.Wrap(err, "error updating follower validation success")
}
return tx.Commit()
}
// UpdateFollowerValidationFailure marks a validation failure, setting first failure time if not already set.
func (r *SqlFollowersRepository) UpdateFollowerValidationFailure(iri string) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
ctx := context.Background()
tx, err := r.datastore.DB.Begin()
if err != nil {
return errors.Wrap(err, "error beginning transaction")
}
defer func() {
_ = tx.Rollback()
}()
if err := r.datastore.GetQueries().WithTx(tx).UpdateFollowerValidationFailure(ctx, db.UpdateFollowerValidationFailureParams{
LastValidatedAt: sql.NullTime{Time: time.Now(), Valid: true},
Iri: iri,
}); err != nil {
return errors.Wrap(err, "error updating follower validation failure")
}
return tx.Commit()
}
// RemoveByIRI removes a follower directly by IRI string.
func (r *SqlFollowersRepository) RemoveByIRI(iri string) error {
r.datastore.DbLock.Lock()
defer r.datastore.DbLock.Unlock()
tx, err := r.datastore.DB.Begin()
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
if err := r.datastore.GetQueries().WithTx(tx).RemoveFollowerByIRI(context.Background(), iri); err != nil {
return err
}
return tx.Commit()
}
+167 -7
View File
@@ -4,15 +4,16 @@ import (
"context"
"database/sql"
"fmt"
"net/url"
"time"
"github.com/go-fed/activity/streams"
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/resolvers"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/db"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
@@ -29,9 +30,168 @@ func Setup(datastore *data.Datastore) {
addFollowersFixtureData()
}
// GetDatastore returns the datastore instance for use by sub-repositories.
func GetDatastore() *data.Datastore {
return _datastore
// AddFollow will save a follow to the datastore.
func AddFollow(follow apmodels.ActivityPubActor, approved bool) error {
log.Traceln("Saving", follow.ActorIri, "as a follower.")
var image string
if follow.Image != nil {
image = follow.Image.String()
}
followRequestObject, err := apmodels.Serialize(follow.RequestObject)
if err != nil {
return errors.Wrap(err, "error serializing follow request object")
}
return createFollow(follow.ActorIri.String(), follow.Inbox.String(), follow.FollowRequestIri.String(), follow.Name, follow.Username, image, followRequestObject, approved)
}
// RemoveFollow will remove a follow from the datastore.
func RemoveFollow(unfollow apmodels.ActivityPubActor) error {
log.Traceln("Removing", unfollow.ActorIri, "as a follower.")
return removeFollow(unfollow.ActorIri)
}
// GetFollower will return a single follower/request given an IRI.
func GetFollower(iri string) (*apmodels.ActivityPubActor, error) {
result, err := _datastore.GetQueries().GetFollowerByIRI(context.Background(), iri)
if err != nil {
return nil, err
}
followIRI, err := url.Parse(result.Request)
if err != nil {
return nil, errors.Wrap(err, "error parsing follow request IRI")
}
iriURL, err := url.Parse(result.Iri)
if err != nil {
return nil, errors.Wrap(err, "error parsing actor IRI")
}
inbox, err := url.Parse(result.Inbox)
if err != nil {
return nil, errors.Wrap(err, "error parsing acting inbox")
}
image, _ := url.Parse(result.Image.String)
var disabledAt *time.Time
if result.DisabledAt.Valid {
disabledAt = &result.DisabledAt.Time
}
follower := apmodels.ActivityPubActor{
ActorIri: iriURL,
Inbox: inbox,
Name: result.Name.String,
Username: result.Username,
Image: image,
FollowRequestIri: followIRI,
DisabledAt: disabledAt,
}
return &follower, nil
}
// ApprovePreviousFollowRequest will approve a follow request.
func ApprovePreviousFollowRequest(iri string) error {
return _datastore.GetQueries().ApproveFederationFollower(context.Background(), db.ApproveFederationFollowerParams{
Iri: iri,
ApprovedAt: sql.NullTime{
Time: time.Now(),
Valid: true,
},
})
}
// BlockOrRejectFollower will block an existing follower or reject a follow request.
func BlockOrRejectFollower(iri string) error {
return _datastore.GetQueries().RejectFederationFollower(context.Background(), db.RejectFederationFollowerParams{
Iri: iri,
DisabledAt: sql.NullTime{
Time: time.Now(),
Valid: true,
},
})
}
func createFollow(actor, inbox, request, name, username, image string, requestObject []byte, approved bool) error {
tx, err := _datastore.DB.Begin()
if err != nil {
log.Debugln(err)
}
defer func() {
_ = tx.Rollback()
}()
var approvedAt sql.NullTime
if approved {
approvedAt = sql.NullTime{
Time: time.Now(),
Valid: true,
}
}
if err = _datastore.GetQueries().WithTx(tx).AddFollower(context.Background(), db.AddFollowerParams{
Iri: actor,
Inbox: inbox,
Name: sql.NullString{String: name, Valid: true},
Username: username,
Image: sql.NullString{String: image, Valid: true},
ApprovedAt: approvedAt,
Request: request,
RequestObject: requestObject,
}); err != nil {
log.Errorln("error creating new federation follow: ", err)
}
return tx.Commit()
}
// UpdateFollower will update the details of a stored follower given an IRI.
func UpdateFollower(actorIRI string, inbox string, name string, username string, image string) error {
_datastore.DbLock.Lock()
defer _datastore.DbLock.Unlock()
tx, err := _datastore.DB.Begin()
if err != nil {
log.Debugln(err)
}
defer func() {
_ = tx.Rollback()
}()
if err = _datastore.GetQueries().WithTx(tx).UpdateFollowerByIRI(context.Background(), db.UpdateFollowerByIRIParams{
Inbox: inbox,
Name: sql.NullString{String: name, Valid: true},
Username: username,
Image: sql.NullString{String: image, Valid: true},
Iri: actorIRI,
}); err != nil {
return fmt.Errorf("error updating follower %s %s", actorIRI, err)
}
return tx.Commit()
}
func removeFollow(actor *url.URL) error {
_datastore.DbLock.Lock()
defer _datastore.DbLock.Unlock()
tx, err := _datastore.DB.Begin()
if err != nil {
return err
}
defer func() {
_ = tx.Rollback()
}()
if err := _datastore.GetQueries().WithTx(tx).RemoveFollowerByIRI(context.Background(), actor.String()); err != nil {
return err
}
return tx.Commit()
}
// createFederatedActivitiesTable will create the accepted
@@ -77,7 +237,7 @@ func GetOutbox(limit int, offset int) (vocab.ActivityStreamsOrderedCollection, e
orderedItems := streams.NewActivityStreamsOrderedItemsProperty()
rows, err := _datastore.GetQueries().GetOutboxWithOffset(
context.Background(),
db.GetOutboxWithOffsetParams{Limit: utils.SafeIntToInt32(limit), Offset: utils.SafeIntToInt32(offset)},
db.GetOutboxWithOffsetParams{Limit: int32(limit), Offset: int32(offset)},
)
if err != nil {
return collection, err
@@ -149,8 +309,8 @@ func SaveInboundFediverseActivity(objectIRI string, actorIRI string, eventType s
func GetInboundActivities(limit int, offset int) ([]models.FederatedActivity, int, error) {
ctx := context.Background()
rows, err := _datastore.GetQueries().GetInboundActivitiesWithOffset(ctx, db.GetInboundActivitiesWithOffsetParams{
Limit: utils.SafeIntToInt32(limit),
Offset: utils.SafeIntToInt32(offset),
Limit: int32(limit),
Offset: int32(offset),
})
if err != nil {
return nil, 0, err
-10
View File
@@ -9,22 +9,12 @@ import (
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/activitypub/workerpool"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
"github.com/teris-io/shortid"
)
// SendFollowAccept will send an accept activity to a follow request from a specified local user.
func SendFollowAccept(inbox *url.URL, originalFollowActivity vocab.ActivityStreamsFollow, fromLocalAccountName string) error {
// SSRF protection: reject non-HTTPS schemes and internal/loopback hosts.
if inbox.Scheme != "https" {
return errors.Errorf("rejecting non-HTTPS inbox URL for SSRF protection: %s", inbox.String())
}
if utils.IsHostnameInternal(inbox.Hostname()) {
return errors.Errorf("rejecting internal/loopback inbox URL for SSRF protection: %s", inbox.String())
}
followAccept := makeAcceptFollow(originalFollowActivity, fromLocalAccountName)
localAccountIRI := apmodels.MakeLocalIRIForAccount(fromLocalAccountName)
+2 -3
View File
@@ -18,10 +18,10 @@ func getPersonFromFollow(activity vocab.ActivityStreamsFollow) (apmodels.Activit
func MakeFollowRequest(c context.Context, activity vocab.ActivityStreamsFollow) (*apmodels.ActivityPubActor, error) {
person, err := getPersonFromFollow(activity)
if err != nil {
return nil, errors.Wrap(err, "unable to resolve person from follow request")
return nil, errors.New("unable to resolve person from follow request: " + err.Error())
}
hostname := person.ActorIriHostname()
hostname := person.ActorIri.Hostname()
username := person.Username
fullUsername := fmt.Sprintf("%s@%s", username, hostname)
@@ -29,7 +29,6 @@ func MakeFollowRequest(c context.Context, activity vocab.ActivityStreamsFollow)
ActorIri: person.ActorIri,
FollowRequestIri: activity.GetJSONLDId().Get(),
Inbox: person.Inbox,
SharedInbox: person.SharedInbox,
Name: person.Name,
Username: fullUsername,
Image: person.Image,
-171
View File
@@ -1,171 +0,0 @@
package resolvers
import (
"net/url"
"testing"
"github.com/go-fed/activity/streams"
)
func mustParseURL(s string) *url.URL {
u, err := url.Parse(s)
if err != nil {
panic(err)
}
return u
}
// TestGetResolvedActorFromActorPropertyWithNil verifies that the function
// doesn't panic when given a nil actor property.
func TestGetResolvedActorFromActorPropertyWithNil(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("GetResolvedActorFromActorProperty panicked with nil: %v", r)
}
}()
_, err := GetResolvedActorFromActorProperty(nil)
if err == nil {
t.Error("GetResolvedActorFromActorProperty(nil) should return error")
}
}
// TestGetResolvedActorFromActorPropertyWithEmpty verifies that the function
// doesn't panic when given an empty actor property.
func TestGetResolvedActorFromActorPropertyWithEmpty(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("GetResolvedActorFromActorProperty panicked with empty: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
_, err := GetResolvedActorFromActorProperty(actor)
if err == nil {
t.Error("GetResolvedActorFromActorProperty with empty actor should return error")
}
}
// TestGetResolvedActorFromActorPropertyWithIRI verifies that the function
// handles the case where an IRI needs to be resolved.
// Note: This test involves network calls and config repository access which may
// panic in a test environment. This is expected since we're testing nil safety
// of ActivityPub property handling, not network behavior.
func TestGetResolvedActorFromActorPropertyWithIRI(t *testing.T) {
t.Skip("Skipping IRI resolution test - requires network access and proper config setup")
}
// TestGetResolvedActorFromActorPropertyWithPersonButMissingFields verifies that
// the function handles Person objects that are missing required fields.
func TestGetResolvedActorFromActorPropertyWithPersonMissingFields(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("GetResolvedActorFromActorProperty panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
person := streams.NewActivityStreamsPerson()
// Person has no ID, inbox, username, or public key
actor.AppendActivityStreamsPerson(person)
_, err := GetResolvedActorFromActorProperty(actor)
if err == nil {
t.Error("GetResolvedActorFromActorProperty with incomplete Person should return error")
}
}
// TestGetResolvedActorFromActorPropertyWithServiceMissingFields verifies that
// the function handles Service objects that are missing required fields.
func TestGetResolvedActorFromActorPropertyWithServiceMissingFields(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("GetResolvedActorFromActorProperty panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
service := streams.NewActivityStreamsService()
// Service has no ID, inbox, username, or public key
actor.AppendActivityStreamsService(service)
_, err := GetResolvedActorFromActorProperty(actor)
if err == nil {
t.Error("GetResolvedActorFromActorProperty with incomplete Service should return error")
}
}
// TestGetResolvedActorFromActorPropertyWithApplicationMissingFields verifies that
// the function handles Application objects that are missing required fields.
func TestGetResolvedActorFromActorPropertyWithApplicationMissingFields(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("GetResolvedActorFromActorProperty panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
app := streams.NewActivityStreamsApplication()
// Application has no ID, inbox, username, or public key
actor.AppendActivityStreamsApplication(app)
_, err := GetResolvedActorFromActorProperty(actor)
if err == nil {
t.Error("GetResolvedActorFromActorProperty with incomplete Application should return error")
}
}
// TestNilSafetyNoPanic is a comprehensive test that ensures none of the
// resolver functions panic with nil/empty inputs.
func TestNilSafetyNoPanic(t *testing.T) {
t.Run("GetResolvedActorFromActorProperty with nil", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("panicked: %v", r)
}
}()
_, _ = GetResolvedActorFromActorProperty(nil)
})
t.Run("GetResolvedActorFromActorProperty with empty", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("panicked: %v", r)
}
}()
_, _ = GetResolvedActorFromActorProperty(streams.NewActivityStreamsActorProperty())
})
t.Run("GetResolvedActorFromActorProperty with Person no fields", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
actor.AppendActivityStreamsPerson(streams.NewActivityStreamsPerson())
_, _ = GetResolvedActorFromActorProperty(actor)
})
t.Run("GetResolvedActorFromActorProperty with Service no fields", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
actor.AppendActivityStreamsService(streams.NewActivityStreamsService())
_, _ = GetResolvedActorFromActorProperty(actor)
})
t.Run("GetResolvedActorFromActorProperty with Application no fields", func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("panicked: %v", r)
}
}()
actor := streams.NewActivityStreamsActorProperty()
actor.AppendActivityStreamsApplication(streams.NewActivityStreamsApplication())
_, _ = GetResolvedActorFromActorProperty(actor)
})
}
+49 -51
View File
@@ -10,8 +10,7 @@ import (
"github.com/go-fed/activity/streams/vocab"
"github.com/owncast/owncast/activitypub/apmodels"
"github.com/owncast/owncast/activitypub/crypto"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/utils"
"github.com/owncast/owncast/core/data"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
@@ -46,61 +45,49 @@ func Resolve(c context.Context, data []byte, callbacks ...interface{}) error {
return nil
}
// ResolveIRI will resolve an IRI and call the correct callback for the resolved type.
// Uses a retryable HTTP client for resilience against transient failures.
// ResolveIRI will resolve an IRI ahd call the correct callback for the resolved type.
func ResolveIRI(c context.Context, iri string, callbacks ...interface{}) error {
configRepository := configrepository.Get()
log.Debugln("Resolving", iri)
req, _ := http.NewRequest(http.MethodGet, iri, nil)
req.Header.Set("Accept", "application/activity+json, application/ld+json")
actor := apmodels.MakeLocalIRIForAccount(configRepository.GetDefaultFederationUsername())
actor := apmodels.MakeLocalIRIForAccount(data.GetDefaultFederationUsername())
if err := crypto.SignRequest(req, nil, actor); err != nil {
return err
}
client := utils.GetRetryableHTTPClient()
response, err := client.Do(req)
response, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode >= 400 {
return errors.New("request failed with status: " + response.Status)
}
data, err := io.ReadAll(response.Body)
if err != nil {
return err
}
// fmt.Println(string(data))
return Resolve(c, data, callbacks...)
}
// GetResolvedActorFromActorProperty resolve an external actor property to a
// fully populated internal actor representation.
// Returns an error if the actor cannot be resolved or is missing required fields.
func GetResolvedActorFromActorProperty(actor vocab.ActivityStreamsActorProperty) (apmodels.ActivityPubActor, error) {
var err error
var apActor apmodels.ActivityPubActor
resolved := false
if actor == nil || actor.Empty() || actor.Len() == 0 || actor.At(0) == nil {
return apActor, errors.New("actor property is empty or nil")
}
if !actor.Empty() && actor.Len() > 0 && actor.At(0) != nil {
// Explicitly use only the first actor that might be listed.
actorObjectOrIRI := actor.At(0)
var actorEntity apmodels.ExternalEntity
// If the actor is an unresolved IRI then we need to resolve it.
if actorObjectOrIRI.IsIRI() {
iri := actorObjectOrIRI.GetIRI()
if iri == nil {
return apActor, errors.New("actor IRI is nil despite IsIRI() returning true")
}
return GetResolvedActorFromIRI(iri.String())
iri := actorObjectOrIRI.GetIRI().String()
return GetResolvedActorFromIRI(iri)
}
if actorObjectOrIRI.IsActivityStreamsPerson() {
@@ -110,17 +97,29 @@ func GetResolvedActorFromActorProperty(actor vocab.ActivityStreamsActorProperty)
} else if actorObjectOrIRI.IsActivityStreamsApplication() {
actorEntity = actorObjectOrIRI.GetActivityStreamsApplication()
} else {
return apActor, errors.New("unrecognized external ActivityPub type: " + actorObjectOrIRI.Name())
err = errors.New("unrecognized external ActivityPub type: " + actorObjectOrIRI.Name())
return apActor, err
}
// If any of the resolution or population failed then return the error.
if err != nil {
return apActor, err
}
// Convert the external AP entity into an internal actor representation.
// NewActivityPubActorFromEntity validates that required fields (ActorIri, Inbox) are present.
apa, err := apmodels.NewActivityPubActorFromEntity(actorEntity)
if err != nil {
return apActor, errors.Wrap(err, "failed to create actor from entity")
apa, e := apmodels.MakeActorFromExernalAPEntity(actorEntity)
if apa != nil {
apActor = *apa
resolved = true
}
err = e
}
return *apa, nil
if !resolved && err == nil {
err = errors.New("unknown error resolving actor from property value")
}
return apActor, err
}
// GetResolvedPublicKeyFromIRI will resolve a publicKey IRI string to a vocab.W3IDSecurityV1PublicKey.
@@ -190,49 +189,48 @@ func GetResolvedPublicKeyFromIRI(publicKeyIRI string) (vocab.W3IDSecurityV1Publi
}
// GetResolvedActorFromIRI will resolve an IRI string to a fully populated actor.
// Returns an error if the actor cannot be resolved or is missing required fields.
func GetResolvedActorFromIRI(personOrServiceIRI string) (apmodels.ActivityPubActor, error) {
var resolveErr error
var err error
var apActor apmodels.ActivityPubActor
resolved := false
personCallback := func(c context.Context, person vocab.ActivityStreamsPerson) error {
apa, e := apmodels.NewActivityPubActorFromEntity(person)
if e != nil {
return e
}
apa, e := apmodels.MakeActorFromExernalAPEntity(person)
if apa != nil {
apActor = *apa
resolved = true
return nil
}
return e
}
serviceCallback := func(c context.Context, service vocab.ActivityStreamsService) error {
apa, e := apmodels.NewActivityPubActorFromEntity(service)
if e != nil {
return e
}
apa, e := apmodels.MakeActorFromExernalAPEntity(service)
if apa != nil {
apActor = *apa
resolved = true
return nil
}
return e
}
applicationCallback := func(c context.Context, app vocab.ActivityStreamsApplication) error {
apa, e := apmodels.NewActivityPubActorFromEntity(app)
if e != nil {
return e
}
apa, e := apmodels.MakeActorFromExernalAPEntity(app)
if apa != nil {
apActor = *apa
resolved = true
return nil
}
return e
}
if e := ResolveIRI(context.Background(), personOrServiceIRI, personCallback, serviceCallback, applicationCallback); e != nil {
resolveErr = errors.Wrap(e, "error resolving actor from IRI")
err = e
}
if !resolved && resolveErr == nil {
resolveErr = errors.New("failed to resolve actor from IRI: " + personOrServiceIRI)
if err != nil {
err = errors.Wrap(err, "error resolving actor from property value")
}
return apActor, resolveErr
if !resolved {
err = errors.New("error resolving actor from property value")
}
return apActor, err
}
+2 -12
View File
@@ -44,27 +44,17 @@ func GetWebfingerLinks(account string) ([]map[string]interface{}, error) {
},
}
req, err := http.NewRequest("GET", requestURL.String(), nil)
response, err := client.Get(requestURL.String())
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
response, err := client.Do(req)
if err != nil {
return nil, err
}
if response.StatusCode != http.StatusOK {
return nil, errors.New("webfinger request returned bad status code: " + http.StatusText(response.StatusCode) + ", check account details")
}
defer response.Body.Close()
var links webfingerResponse
decoder := json.NewDecoder(response.Body)
if err := decoder.Decode(&links); err != nil {
return nil, fmt.Errorf("error decoding webfinger response: %s", err)
return nil, err
}
return links.Links, nil
+10 -124
View File
@@ -2,13 +2,14 @@ package workerpool
import (
"net/http"
"sync"
"time"
"runtime"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
)
// workerPoolSize defines the number of concurrent HTTP ActivityPub requests.
var workerPoolSize = runtime.GOMAXPROCS(0)
// Job struct bundling the ActivityPub and the payload in one struct.
type Job struct {
request *http.Request
@@ -16,44 +17,9 @@ type Job struct {
var queue chan Job
// Circuit breaker backoff durations for exponential backoff.
var circuitBreakerBackoffDurations = []time.Duration{
1 * time.Minute, // 1st failure: 1 minute
5 * time.Minute, // 2nd failure: 5 minutes
15 * time.Minute, // 3rd failure: 15 minutes
30 * time.Minute, // 4th failure: 30 minutes
60 * time.Minute, // 5+ failures: 1 hour (max)
}
// httpClient is a configured HTTP client with timeouts and connection limits.
var httpClient *http.Client
// failedDomains tracks domains that are consistently failing with their failure count and backoff time.
var (
failedDomains = make(map[string]*domainFailure)
failedDomainsMutex sync.RWMutex
)
type domainFailure struct {
count int
lastFailed time.Time
backoffUntil time.Time
}
// InitOutboundWorkerPool starts n go routines that await ActivityPub jobs.
func InitOutboundWorkerPool(workerPoolSize int) {
// Use a larger buffer to decouple request creation from processing.
// This prevents SendToFollowers from blocking when many followers need updates.
const minQueueBuffer = 500
queueBuffer := workerPoolSize * 10
if queueBuffer < minQueueBuffer {
queueBuffer = minQueueBuffer
}
queue = make(chan Job, queueBuffer)
// Initialize HTTP client with retry logic for transient failures
// The retryable client handles 502/503/504 errors automatically
httpClient = utils.GetRetryableHTTPClient()
func InitOutboundWorkerPool() {
queue = make(chan Job)
// start workers
for i := 1; i <= workerPoolSize; i++ {
@@ -63,19 +29,8 @@ func InitOutboundWorkerPool(workerPoolSize int) {
// AddToOutboundQueue will queue up an outbound http request.
func AddToOutboundQueue(req *http.Request) {
// Check if domain should be skipped due to circuit breaker
if ShouldSkipDomain(req.URL.Host) {
log.Debugf("Skipping request to %s due to circuit breaker", req.URL.Host)
return
}
select {
case queue <- Job{req}:
default:
log.Debugln("Outbound ActivityPub job queue is full")
queue <- Job{req} // will block until received by a worker at this point
}
log.Tracef("Queued request for ActivityPub destination %s", req.RequestURI)
queue <- Job{req}
}
func worker(workerID int, queue <-chan Job) {
@@ -84,89 +39,20 @@ func worker(workerID int, queue <-chan Job) {
for job := range queue {
if err := sendActivityPubMessageToInbox(job); err != nil {
log.Errorf("ActivityPub destination %s failed to send Error: %s", job.request.RequestURI, err)
recordDomainFailure(job.request.URL.Host)
} else {
// Reset domain failure count on success
resetDomainFailure(job.request.URL.Host)
}
log.Tracef("Done with ActivityPub destination %s using worker %d", job.request.RequestURI, workerID)
}
}
func sendActivityPubMessageToInbox(job Job) error {
resp, err := httpClient.Do(job.request)
client := &http.Client{}
resp, err := client.Do(job.request)
if err != nil {
return err
}
defer resp.Body.Close()
// Consider HTTP 4xx and 5xx as failures for circuit breaker purposes
if resp.StatusCode >= 400 {
return &httpError{statusCode: resp.StatusCode, message: resp.Status}
}
return nil
}
// httpError represents an HTTP error response.
type httpError struct {
statusCode int
message string
}
func (e *httpError) Error() string {
return e.message
}
// ShouldSkipDomain checks if a domain should be skipped due to circuit breaker.
// This is exported so callers can check before expensive operations like request signing.
func ShouldSkipDomain(domain string) bool {
failedDomainsMutex.RLock()
defer failedDomainsMutex.RUnlock()
failure, exists := failedDomains[domain]
if !exists {
return false
}
// If we're still in backoff period, skip this domain
return time.Now().Before(failure.backoffUntil)
}
// recordDomainFailure records a failure for a domain and implements exponential backoff.
func recordDomainFailure(domain string) {
failedDomainsMutex.Lock()
defer failedDomainsMutex.Unlock()
failure, exists := failedDomains[domain]
if !exists {
failure = &domainFailure{}
failedDomains[domain] = failure
}
failure.count++
failure.lastFailed = time.Now()
// Use exponential backoff with pre-defined durations
backoffIndex := failure.count - 1
if backoffIndex >= len(circuitBreakerBackoffDurations) {
backoffIndex = len(circuitBreakerBackoffDurations) - 1
}
backoffDuration := circuitBreakerBackoffDurations[backoffIndex]
failure.backoffUntil = time.Now().Add(backoffDuration)
log.Warnf("Domain %s failed %d times, backing off for %v", domain, failure.count, backoffDuration)
}
// resetDomainFailure resets the failure count for a domain on successful delivery.
func resetDomainFailure(domain string) {
failedDomainsMutex.Lock()
defer failedDomainsMutex.Unlock()
if failure, exists := failedDomains[domain]; exists && failure.count > 0 {
log.Debugf("Resetting failure count for domain %s after successful delivery", domain)
delete(failedDomains, domain)
}
}
-133
View File
@@ -1,133 +0,0 @@
package workerpool
import (
"testing"
"time"
)
// resetCircuitBreakerForTesting clears all failed domain state for testing purposes.
// This function provides test isolation by resetting the global circuit breaker state.
func resetCircuitBreakerForTesting() {
failedDomainsMutex.Lock()
defer failedDomainsMutex.Unlock()
failedDomains = make(map[string]*domainFailure)
}
func TestCircuitBreaker(t *testing.T) {
// Ensure clean state before test
resetCircuitBreakerForTesting()
defer resetCircuitBreakerForTesting() // Clean up after test
testDomain := "failing.example.com"
// Initially, domain should not be skipped
if ShouldSkipDomain(testDomain) {
t.Error("Domain should not be skipped initially")
}
// Record failures
recordDomainFailure(testDomain)
recordDomainFailure(testDomain)
recordDomainFailure(testDomain)
// Domain should now be skipped
if !ShouldSkipDomain(testDomain) {
t.Error("Domain should be skipped after failures")
}
// After successful delivery, domain should be reset
resetDomainFailure(testDomain)
if ShouldSkipDomain(testDomain) {
t.Error("Domain should not be skipped after reset")
}
}
func TestHTTPTimeouts(t *testing.T) {
// Ensure clean state before test
resetCircuitBreakerForTesting()
defer resetCircuitBreakerForTesting() // Clean up after test
// Initialize HTTP client
InitOutboundWorkerPool(1)
if httpClient == nil {
t.Error("HTTP client should be initialized")
}
if httpClient.Timeout != 8*time.Second {
t.Errorf("HTTP client should have 8 second timeout, got %v", httpClient.Timeout)
}
}
func TestWorkerPoolSizing(t *testing.T) {
// Ensure clean state before test
resetCircuitBreakerForTesting()
defer resetCircuitBreakerForTesting() // Clean up after test
// Test that queue buffer is at least the minimum (500) even for small worker pools
InitOutboundWorkerPool(5)
if cap(queue) < 500 {
t.Errorf("Queue capacity should be at least 500, got %d", cap(queue))
}
// Test that larger worker pools get proportionally larger buffers
InitOutboundWorkerPool(100)
if cap(queue) != 1000 {
t.Errorf("Queue capacity should be 1000 for 100 workers, got %d", cap(queue))
}
}
func TestBackoffDurations(t *testing.T) {
// Test that backoff durations are properly configured
expectedDurations := []time.Duration{
1 * time.Minute,
5 * time.Minute,
15 * time.Minute,
30 * time.Minute,
60 * time.Minute,
}
if len(circuitBreakerBackoffDurations) != len(expectedDurations) {
t.Errorf("Expected %d backoff durations, got %d", len(expectedDurations), len(circuitBreakerBackoffDurations))
}
for i, expected := range expectedDurations {
if circuitBreakerBackoffDurations[i] != expected {
t.Errorf("Backoff duration at index %d: expected %v, got %v", i, expected, circuitBreakerBackoffDurations[i])
}
}
}
func TestCircuitBreakerIsolation(t *testing.T) {
// Test that multiple tests don't interfere with each other
resetCircuitBreakerForTesting()
defer resetCircuitBreakerForTesting()
domain1 := "test1.example.com"
domain2 := "test2.example.com"
// Neither domain should be blocked initially
if ShouldSkipDomain(domain1) || ShouldSkipDomain(domain2) {
t.Error("Domains should not be blocked initially")
}
// Record failures for domain1 only
recordDomainFailure(domain1)
recordDomainFailure(domain1)
recordDomainFailure(domain1)
// Only domain1 should be blocked
if !ShouldSkipDomain(domain1) {
t.Error("Domain1 should be blocked after failures")
}
if ShouldSkipDomain(domain2) {
t.Error("Domain2 should not be blocked")
}
// Reset and verify clean state
resetCircuitBreakerForTesting()
if ShouldSkipDomain(domain1) || ShouldSkipDomain(domain2) {
t.Error("Both domains should be unblocked after reset")
}
}
+1 -1
View File
@@ -68,7 +68,7 @@ func RegisterFediverseOTP(accessToken, userID, userDisplayName, account string)
defer lock.Unlock()
if len(pendingAuthRequests)+1 > maxPendingRequests {
return request, false, errors.New("too many pending requests, please try again later")
return request, false, errors.New("Please try again later. Too many pending requests.")
}
code, _ := createCode()
+2 -4
View File
@@ -11,7 +11,7 @@ import (
"sync"
"time"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/utils"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
@@ -47,8 +47,6 @@ func setupExpiredRequestPruner() {
// StartAuthFlow will begin the IndieAuth flow by generating an auth request.
func StartAuthFlow(authHost, userID, accessToken, displayName string) (*url.URL, error) {
configRepository := configrepository.Get()
// Limit the number of pending requests
if len(pendingAuthRequests) >= maxPendingRequests {
return nil, errors.New("Please try again later. Too many pending requests.")
@@ -70,7 +68,7 @@ func StartAuthFlow(authHost, userID, accessToken, displayName string) (*url.URL,
return nil, errors.New("only servers secured with https are supported")
}
serverURL := configRepository.GetServerURL()
serverURL := data.GetServerURL()
if serverURL == "" {
return nil, errors.New("Owncast server URL must be set when using auth")
}
+5 -7
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"time"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/core/data"
"github.com/pkg/errors"
"github.com/teris-io/shortid"
)
@@ -70,8 +70,6 @@ func StartServerAuth(clientID, redirectURI, codeChallenge, state, me string) (*S
// CompleteServerAuth will verify that the values provided in the final step
// of the IndieAuth flow are correct, and return some basic profile info.
func CompleteServerAuth(code, redirectURI, clientID string, codeVerifier string) (*ServerProfileResponse, error) {
configRepository := configrepository.Get()
request, pending := pendingServerAuthRequests[code]
if !pending {
return nil, errors.New("no pending authentication request")
@@ -91,11 +89,11 @@ func CompleteServerAuth(code, redirectURI, clientID string, codeVerifier string)
}
response := ServerProfileResponse{
Me: configRepository.GetServerURL(),
Me: data.GetServerURL(),
Profile: ServerProfile{
Name: configRepository.GetServerName(),
URL: configRepository.GetServerURL(),
Photo: fmt.Sprintf("%s/%s", configRepository.GetServerURL(), configRepository.GetLogoPath()),
Name: data.GetServerName(),
URL: data.GetServerURL(),
Photo: fmt.Sprintf("%s/%s", data.GetServerURL(), data.GetLogoPath()),
},
}
+1 -1
View File
@@ -21,4 +21,4 @@ if [ -n "${EARTHLY_BUILD_BRANCH}" ]; then
git checkout "${EARTHLY_BUILD_BRANCH}" || exit
fi
earthly -P --ci +docker-all --images="ghcr.io/owncast/${EARTHLY_IMAGE_NAME}:${BUILD_TAG}" --version="${VERSION}"
earthly --ci +docker-all --images="ghcr.io/owncast/${EARTHLY_IMAGE_NAME}:${BUILD_TAG}" --version="${VERSION}"
+12 -9
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# Tools are managed in tools/go.mod and installed to ./bin
# go install github.com/deepmap/oapi-codegen/v2/cmd/oapi-codegen@latest
# setup
package="generated"
@@ -8,13 +8,19 @@ folderPath="webserver/handlers/generated"
specPath="openapi.yaml"
# validate scripts are installed
if ! command -v redocly &>/dev/null; then
echo "Please install \`redocly cli\` before running this script: npm install -g @redocly/cli"
if ! command -v swagger-cli &>/dev/null; then
echo "Please install \`swagger-cli\` before running this script"
exit 1
fi
if ! command -v oapi-codegen &>/dev/null; then
echo "Please install \`oapi-codegen\` before running this script"
echo "Hint: run \`go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@latest\` to install"
exit 1
fi
# validate schema
npx redocly lint $specPath
swagger-cli validate $specPath
if [ $? -ne 0 ]; then
echo "Open API specification is not valid"
exit 1
@@ -24,12 +30,9 @@ fi
rm -r $folderPath
mkdir -p $folderPath
# install oapi-codegen
GOBIN=$(pwd)/bin go install -C tools github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen
# codegen
./bin/oapi-codegen -generate types -o $folderPath/$package-types.gen.go -package $package $specPath
./bin/oapi-codegen -generate "chi-server" -o $folderPath/$package.gen.go -package $package $specPath
oapi-codegen -generate types -o $folderPath/$package-types.gen.go -package $package $specPath
oapi-codegen -generate "chi-server" -o $folderPath/$package.gen.go -package $package $specPath
# go
go mod tidy
-4
View File
@@ -43,10 +43,6 @@ var EnableAutoUpdate = false
// A temporary stream key that can be set via the command line.
var TemporaryStreamKey = ""
// FollowerValidationInterval is how often the follower validation job runs.
// Defaults to 0 which means use the default (1 hour).
var FollowerValidationInterval time.Duration = 0
// GetCommit will return an identifier used for identifying the point in time this build took place.
func GetCommit() string {
if GitCommit == "" {
+1 -1
View File
@@ -4,7 +4,7 @@ import "path/filepath"
const (
// StaticVersionNumber is the version of Owncast that is used when it's not overwritten via build-time settings.
StaticVersionNumber = "0.2.4" // Shown when you build from develop
StaticVersionNumber = "0.2.0" // Shown when you build from develop
// FfmpegSuggestedVersion is the version of ffmpeg we suggest.
FfmpegSuggestedVersion = "v4.1.5" // Requires the v
// DataDirectory is the directory we save data to.
+5 -8
View File
@@ -4,7 +4,6 @@ import (
"time"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/webserver/handlers/generated"
)
// Defaults will hold default configuration values.
@@ -26,7 +25,7 @@ type Defaults struct {
WebServerIP string
Name string
AdminPassword string
StreamKeys []generated.StreamKey
StreamKeys []models.StreamKey
StreamVariants []models.StreamOutputVariant
@@ -44,16 +43,14 @@ type Defaults struct {
// GetDefaults will return default configuration values.
func GetDefaults() Defaults {
defaultStreamKey := "abc123"
defaultStreamKeyComment := "Default stream key"
return Defaults{
Name: "New Owncast Server",
Summary: "This is a new live video streaming server powered by Owncast.",
ServerWelcomeMessage: "",
Logo: "logo.svg",
AdminPassword: "abc123",
StreamKeys: []generated.StreamKey{
{Key: &defaultStreamKey, Comment: &defaultStreamKeyComment},
StreamKeys: []models.StreamKey{
{Key: "abc123", Comment: "Default stream key"},
},
Tags: []string{
"owncast",
@@ -65,7 +62,7 @@ func GetDefaults() Defaults {
- This is a live stream powered by [Owncast](https://owncast.online), a free and open source live streaming server.
- To discover more examples of streams, visit [Owncast's directory](https://owncast.directory).
- To discover more examples of streams, visit [Owncast's directory](https://directory.owncast.online).
- If you're the owner of this server you should visit the admin and customize the content on this page.
@@ -79,7 +76,7 @@ func GetDefaults() Defaults {
DatabaseFilePath: "data/owncast.db",
YPEnabled: false,
YPServer: "https://owncast.directory",
YPServer: "https://directory.owncast.online",
WebServerPort: 8080,
WebServerIP: "0.0.0.0",
-36
View File
@@ -1,36 +0,0 @@
@echo off
setlocal enabledelayedexpansion
REM This script will make your local Owncast server available at a public URL.
REM It's particularly useful for testing on mobile devices or want to test
REM activitypub integration.
REM Pass a custom domain as an argument if you have previously set it up at
REM localhost.run. Otherwise, a random hostname will be generated.
REM SET DOMAIN=me.example.com && test\test-local.bat
REM Pass a port number as an argument if you are running Owncast on a different port.
REM By default, it will use port 8080.
REM SET PORT=8080 && test\test-local.bat
REM Set default port if not provided
if "%PORT%"=="" set PORT=8080
set HOST=localhost
echo Checking if web server is running on port %PORT%...
REM Using PowerShell to check if the port is open (equivalent to nc -zv)
powershell -Command "try { $tcpConnection = New-Object System.Net.Sockets.TcpClient; $tcpConnection.Connect('%HOST%', %PORT%); $tcpConnection.Close(); exit 0 } catch { exit 1 }"
if %ERRORLEVEL% equ 0 (
echo Your web server is running on port %PORT%. Good.
) else (
echo Please make sure your Owncast server is running on port %PORT%.
exit /b 1
)
if not "%DOMAIN%"=="" (
echo Attempting to use custom domain: %DOMAIN%
ssh -R "%DOMAIN%":80:localhost:%PORT% localhost.run
) else (
echo Using auto-generated hostname for tunnel.
ssh -R 80:localhost:%PORT% localhost.run
)
-350
View File
@@ -1,350 +0,0 @@
#!/usr/bin/env node
/**
* ocTestStream.js - RTMP test stream utility
*
* Requirements:
* - Node.js
* - ffmpeg (a recent version with loop video support)
* - a Sans family font (for overlay text)
*
* Example: node ocTestStream.js ~/Downloads/*.mp4 rtmp://127.0.0.1/live/abc123
*/
const fs = require('fs');
const path = require('path');
const { spawn, exec } = require('child_process');
const os = require('os');
class OcTestStream {
constructor() {
this.ffmpegExec = null;
this.destinationHost = 'rtmp://127.0.0.1/live/abc123';
this.videoFiles = [];
this.listFile = 'list.txt';
}
/**
* Find ffmpeg executable across different platforms
*/
async findFFmpeg() {
const ffmpegExecs = ['ffmpeg', 'ffmpeg.exe'];
const ffmpegPaths = ['./', '../', ''];
// Try local paths first
for (const execName of ffmpegExecs) {
for (const execPath of ffmpegPaths) {
const fullPath = path.join(execPath, execName);
try {
await this.checkExecutable(fullPath);
this.ffmpegExec = fullPath;
return true;
} catch (e) {
// Continue searching
}
}
}
// Try system PATH
for (const execName of ffmpegExecs) {
try {
await this.checkExecutable(execName);
this.ffmpegExec = execName;
return true;
} catch (e) {
// Continue searching
}
}
return false;
}
/**
* Check if an executable exists and is accessible
*/
checkExecutable(execPath) {
return new Promise((resolve, reject) => {
exec(`"${execPath}" -version`, (error, stdout, stderr) => {
if (error) {
reject(error);
} else {
resolve(stdout);
}
});
});
}
/**
* Get ffmpeg version information
*/
async getFFmpegVersion() {
try {
const output = await this.checkExecutable(this.ffmpegExec);
const versionMatch = output.match(/ffmpeg version ([^\s]+)/);
return versionMatch ? versionMatch[1] : 'unknown';
} catch (e) {
return 'unknown';
}
}
/**
* Get ffmpeg executable path
*/
async getFFmpegPath() {
return new Promise((resolve) => {
const command = os.platform() === 'win32' ? `where "${this.ffmpegExec}"` : `which "${this.ffmpegExec}"`;
exec(command, (error, stdout, stderr) => {
if (error) {
resolve('unknown');
} else {
resolve(stdout.trim().split('\n')[0]);
}
});
});
}
/**
* Parse command line arguments
*/
parseArguments(args) {
if (args.includes('--help')) {
this.showHelp();
return false;
}
// Check if last argument is RTMP URL
const lastArg = args[args.length - 1];
if (lastArg && lastArg.includes('rtmp://')) {
console.log('RTMP server is specified');
this.destinationHost = lastArg;
this.videoFiles = args.slice(0, -1);
} else {
console.log('RTMP server is not specified');
this.videoFiles = args;
}
return true;
}
/**
* Show help information
*/
showHelp() {
console.log('ocTestStream is used for sending pre-recorded or internal test content to an RTMP server.');
console.log('Usage: node ocTestStream.js [VIDEO_FILES] [RTMP_DESTINATION]');
console.log('VIDEO_FILES: path to one or multiple videos for sending to the RTMP server (optional)');
console.log('RTMP_DESTINATION: URL of RTMP server with key (optional; default: rtmp://127.0.0.1/live/abc123)');
}
/**
* Get system font path for overlay text
*/
getSystemFont() {
const platform = os.platform();
if (platform === 'win32') {
// Workaround lack of font config on Windows.
const windir = process.env.WINDIR || 'C:\\Windows';
return path.join(windir, 'fonts', 'arial.ttf').replace(/\\/g, '/').replace(/:/g, '\\\\:');
} else {
// Use default font.
return '';
}
}
/**
* Stream internal test video pattern
*/
streamTestPattern() {
console.log(`Streaming internal test video loop to ${this.destinationHost}`);
console.log('...press ctrl+c to exit');
const font = this.getSystemFont();
const fontParam = font ? `:fontfile=${font}` : '';
const args = [
'-hide_banner', '-loglevel', 'panic', '-nostdin', '-re', '-f', 'lavfi',
'-i', 'testsrc=size=1280x720:rate=60[out0];sine=frequency=400:sample_rate=48000[out1]',
'-vf', `[in]drawtext=fontsize=96:box=1:boxcolor=black@0.75:boxborderw=5${fontParam}:fontcolor=white:x=(w-text_w)/2:y=((h-text_h)/2)+((h-text_h)/-2):text='Owncast Test Stream',drawtext=fontsize=96:box=1:boxcolor=black@0.75:boxborderw=5${fontParam}:fontcolor=white:x=(w-text_w)/2:y=((h-text_h)/2)+((h-text_h)/2):text='%{gmtime\\:%H\\\\\\:%M\\\\\\:%S} UTC'[out]`,
'-nal-hrd', 'cbr',
'-metadata:s:v', 'encoder=test',
'-vcodec', 'libx264',
'-acodec', 'aac',
'-preset', 'veryfast',
'-profile:v', 'baseline',
'-tune', 'zerolatency',
'-bf', '0',
'-g', '0',
'-b:v', '6320k',
'-b:a', '160k',
'-ac', '2',
'-ar', '48000',
'-minrate', '6320k',
'-maxrate', '6320k',
'-bufsize', '6320k',
'-muxrate', '6320k',
'-r', '60',
'-pix_fmt', 'yuv420p',
'-color_range', '1', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1',
'-flags:v', '+global_header',
'-bsf:v', 'dump_extra',
'-x264-params', 'nal-hrd=cbr:min-keyint=2:keyint=2:scenecut=0:bframes=0',
'-f', 'flv', this.destinationHost
];
this.streamWithArgs(args);
}
/**
* Stream with arguments
*/
streamWithArgs(args) {
const ffmpeg = spawn(this.ffmpegExec, args, { detached: false, stdio: 'inherit' });
ffmpeg.on('error', (err) => {
console.error('Error starting ffmpeg:', err);
process.exit(1);
});
process.on('SIGINT', () => {
if (os.platform() === 'win32') {
// Because ffmpeg can spawn children, on windows we need to use taskkill.
// Using ffmpeg.kill on Windows may result in orphaned processes.
exec(`taskkill /T /PID ${ffmpeg.pid}`);
} else {
ffmpeg.kill('SIGINT');
}
});
process.on('exit', () => {
if (os.platform() === 'win32') {
exec(`taskkill /F /T /PID ${ffmpeg.pid}`);
} else {
ffmpeg.kill('SIGINT');
}
});
}
/**
* Create playlist file for video files
*/
createPlaylist() {
try {
const content = this.videoFiles.map(file => `file '${file}'`).join('\n');
fs.writeFileSync(this.listFile, content);
return true;
} catch (err) {
console.error('Error creating playlist file:', err);
return false;
}
}
/**
* Validate video files exist
*/
validateFiles() {
for (const file of this.videoFiles) {
if (!fs.existsSync(file)) {
console.error(`ERROR: File not found: ${file}`);
return false;
}
}
return true;
}
/**
* Stream video files
*/
streamVideoFiles() {
if (!this.validateFiles()) {
process.exit(1);
}
if (!this.createPlaylist()) {
process.exit(1);
}
console.log(`Streaming a loop of ${this.videoFiles.length} video(s) to ${this.destinationHost}`);
if (this.videoFiles.length > 1) {
console.log('Warning: If these files differ greatly in formats, transitioning from one to another may not always work correctly.');
}
console.log(this.videoFiles.join(' '));
console.log('...press ctrl+c to exit');
const font = this.getSystemFont();
const fontParam = font ? `:fontfile=${font}` : '';
const args = [
'-hide_banner', '-loglevel', 'panic', '-nostdin', '-stream_loop', '-1', '-re', '-f', 'concat',
'-safe', '0',
'-i', this.listFile,
'-vcodec', 'libx264',
'-profile:v', 'high',
'-g', '48',
'-r', '24',
'-sc_threshold', '0',
'-b:v', '1300k',
'-preset', 'veryfast',
'-acodec', 'copy',
'-vf', `drawtext=fontsize=96:box=1:boxcolor=black@0.75:boxborderw=5${fontParam}:fontcolor=white:x=(w-text_w)/2:y=((h-text_h)/2)+((h-text_h)/4):text='%{gmtime\\:%H\\\\\\:%M\\\\\\:%S}'`,
'-f', 'flv', this.destinationHost
];
this.streamWithArgs(args);
}
/**
* Cleanup temporary files
*/
cleanup() {
try {
if (fs.existsSync(this.listFile)) {
fs.unlinkSync(this.listFile);
}
} catch (err) {
console.error('Error during cleanup:', err);
}
}
/**
* Main execution function
*/
async run() {
// Parse command line arguments (skip node and script name)
const args = process.argv.slice(2);
if (!this.parseArguments(args)) {
return;
}
// Find ffmpeg executable
if (!(await this.findFFmpeg())) {
console.error('ERROR: ffmpeg was not found in path or in the current directory! Please install ffmpeg before using this script.');
process.exit(1);
}
// Show ffmpeg information
const version = await this.getFFmpegVersion();
const execPath = await this.getFFmpegPath();
console.log(`ffmpeg executable: ${this.ffmpegExec} (${version})`);
console.log(`ffmpeg path: ${execPath}`);
// Stream based on whether files were provided
if (this.videoFiles.length === 0) {
this.streamTestPattern();
} else {
this.streamVideoFiles();
}
}
}
// Run if called directly
if (require.main === module) {
const stream = new OcTestStream();
stream.run().catch(err => {
console.error('Error:', err);
process.exit(1);
});
}
module.exports = OcTestStream;
+6 -11
View File
@@ -7,9 +7,8 @@ import (
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/persistence/chatmessagerepository"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
log "github.com/sirupsen/logrus"
@@ -24,8 +23,6 @@ var (
func Start(getStatusFunc func() models.Status) error {
setupPersistence()
configRepository := configrepository.Get()
getStatus = getStatusFunc
_server = NewChat()
@@ -38,7 +35,7 @@ func Start(getStatusFunc func() models.Status) error {
Help: "The number of chat messages incremented over time.",
ConstLabels: map[string]string{
"version": config.VersionNumber,
"host": configRepository.GetServerURL(),
"host": data.GetServerURL(),
},
})
@@ -104,8 +101,7 @@ func SendSystemMessage(text string, ephemeral bool) error {
}
if !ephemeral {
chatMessageRepository := chatmessagerepository.Get()
chatMessageRepository.SaveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
saveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
}
return nil
@@ -126,14 +122,14 @@ func SendFediverseAction(eventType string, userAccountName string, image *string
}
message.SetDefaults()
message.RenderBody()
if err := Broadcast(&message); err != nil {
log.Errorln("error sending system message", err)
return err
}
chatMessageRepository := chatmessagerepository.Get()
chatMessageRepository.SaveFederatedAction(message)
saveFederatedAction(message)
return nil
}
@@ -154,8 +150,7 @@ func SendSystemAction(text string, ephemeral bool) error {
}
if !ephemeral {
chatMessageRepository := chatmessagerepository.Get()
chatMessageRepository.SaveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
saveEvent(message.ID, nil, message.Body, message.GetMessageType(), nil, message.Timestamp, nil, nil, nil, nil)
}
return nil
+3 -7
View File
@@ -13,8 +13,8 @@ import (
"github.com/gorilla/websocket"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/services/geoip"
)
@@ -133,9 +133,7 @@ func (c *Client) readPump() {
}
// Check if this message passes the optional language filter
configRepository := configrepository.Get()
if configRepository.GetChatSlurFilterEnabled() && !c.messageFilter.Allow(string(message)) {
if data.GetChatSlurFilterEnabled() && !c.messageFilter.Allow(string(message)) {
c.sendAction("Sorry, that message contained language that is not allowed in this chat.")
continue
}
@@ -211,11 +209,9 @@ func (c *Client) close() {
}
func (c *Client) passesRateLimit() bool {
configRepository := configrepository.Get()
// If spam rate limiting is disabled, or the user is a moderator, always
// allow the message.
if !configRepository.GetChatSpamProtectionEnabled() || c.User.IsModerator() {
if !data.GetChatSpamProtectionEnabled() || c.User.IsModerator() {
return true
}
+2 -2
View File
@@ -10,14 +10,14 @@ import (
log "github.com/sirupsen/logrus"
)
func setSystemConcurrentConnectionLimit(limit uint64) {
func setSystemConcurrentConnectionLimit(limit int64) {
var rLimit syscall.Rlimit
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit); err != nil {
log.Fatalln(err)
}
originalLimit := rLimit.Cur
rLimit.Cur = limit
rLimit.Cur = uint64(limit)
if err := syscall.Setrlimit(syscall.RLIMIT_NOFILE, &rLimit); err != nil {
log.Fatalln(err)
}
+1 -1
View File
@@ -9,7 +9,7 @@ import (
log "github.com/sirupsen/logrus"
)
func setSystemConcurrentConnectionLimit(limit uint64) {
func setSystemConcurrentConnectionLimit(limit int64) {
var rLimit syscall.Rlimit
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit); err != nil {
log.Fatalln(err)
+1 -1
View File
@@ -3,4 +3,4 @@
package chat
func setSystemConcurrentConnectionLimit(limit uint64) {}
func setSystemConcurrentConnectionLimit(limit int64) {}
+4 -19
View File
@@ -8,9 +8,8 @@ import (
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/webhooks"
"github.com/owncast/owncast/persistence/chatmessagerepository"
"github.com/owncast/owncast/persistence/configrepository"
"github.com/owncast/owncast/persistence/userrepository"
"github.com/owncast/owncast/utils"
log "github.com/sirupsen/logrus"
@@ -23,28 +22,14 @@ func (s *Server) userNameChanged(eventData chatClientEvent) {
return
}
configRepository := configrepository.Get()
proposedUsername := receivedEvent.NewName
// Check if name is on the blocklist
blocklist := configRepository.GetForbiddenUsernameList()
blocklist := data.GetForbiddenUsernameList()
// Names have a max length
proposedUsername = utils.MakeSafeStringOfLength(proposedUsername, config.MaxChatDisplayNameLength)
// Check if the sanitized name is empty or just whitespace
if strings.TrimSpace(proposedUsername) == "" {
log.Debugln(logSanitize(eventData.client.User.DisplayName), "attempted to change name to empty or whitespace-only name")
message := "Display name cannot be empty or contain only whitespace."
s.sendActionToClient(eventData.client, message)
// Resend the client's user so their username is in sync.
eventData.client.sendConnectedClientInfo()
return
}
for _, blockedName := range blocklist {
normalizedName := strings.TrimSpace(blockedName)
normalizedName = strings.ToLower(normalizedName)
@@ -185,8 +170,8 @@ func (s *Server) userMessageSent(eventData chatClientEvent) {
// Send chat message sent webhook
webhooks.SendChatEvent(&event)
chatMessagesSentCounter.Inc()
chatMessageRepository := chatmessagerepository.Get()
chatMessageRepository.SaveUserMessage(event)
SaveUserMessage(event)
eventData.client.MessageCount++
}
+15 -15
View File
@@ -17,7 +17,7 @@ import (
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/renderer/html"
"github.com/yuin/goldmark/util"
"mvdan.cc/xurls/v2"
"mvdan.cc/xurls"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/models"
@@ -100,13 +100,13 @@ func newEmojis(emotes ...emojiDef.Emoji) emojiDef.Emojis {
return self
}
func (e *emojis) Get(shortName string) (*emojiDef.Emoji, bool) {
v, ok := e.names[strings.ToLower(shortName)]
func (self *emojis) Get(shortName string) (*emojiDef.Emoji, bool) {
v, ok := self.names[strings.ToLower(shortName)]
if ok {
return v, ok
}
for _, child := range e.children {
for _, child := range self.children {
v, ok := child.Get(shortName)
if ok {
return v, ok
@@ -116,18 +116,18 @@ func (e *emojis) Get(shortName string) (*emojiDef.Emoji, bool) {
return nil, false
}
func (e *emojis) Add(emotes emojiDef.Emojis) {
e.children = append(e.children, emotes)
func (self *emojis) Add(emotes emojiDef.Emojis) {
self.children = append(self.children, emotes)
}
func (e *emojis) Clone() emojiDef.Emojis {
func (self *emojis) Clone() emojiDef.Emojis {
clone := &emojis{
list: e.list,
names: e.names,
children: make([]emojiDef.Emojis, len(e.children)),
list: self.list,
names: self.names,
children: make([]emojiDef.Emojis, len(self.children)),
}
copy(clone.children, e.children)
copy(clone.children, self.children)
return clone
}
@@ -137,7 +137,7 @@ var (
emojiDefs = newEmojis()
emojiHTML = make(map[string]string)
emojiModTime time.Time
emojiHTMLFormat = `<img src="{{ .Url }}" class="emoji" alt=":{{ .Name }}:" title=":{{ .Name }}:">`
emojiHTMLFormat = `<img src="{{ .URL }}" class="emoji" alt=":{{ .Name }}:" title=":{{ .Name }}:">`
emojiHTMLTemplate = template.Must(template.New("emojiHTML").Parse(emojiHTMLFormat))
)
@@ -162,9 +162,9 @@ func loadEmoji() {
if err != nil {
return
}
emojiHTML[strings.ToLower(*emojiList[i].Name)] = buf.String()
emojiHTML[strings.ToLower(emojiList[i].Name)] = buf.String()
emoji := emojiDef.NewEmoji(*emojiList[i].Name, nil, strings.ToLower(*emojiList[i].Name))
emoji := emojiDef.NewEmoji(emojiList[i].Name, nil, strings.ToLower(emojiList[i].Name))
emojiArr = append(emojiArr, emoji)
}
@@ -220,7 +220,7 @@ func RenderMarkdown(raw string) string {
[]byte("https:"),
}),
extension.WithLinkifyURLRegexp(
xurls.Strict(),
xurls.Strict,
),
),
emoji.New(

Some files were not shown because too many files have changed in this diff Show More