Platform credential health check #95

New issue

Closed

opened 2026-03-09 00:14:36 +01:00 by myrmidex · 1 comment

myrmidex commented

2026-03-09 00:14:36 +01:00

Owner

Summary

Periodically verify that platform account credentials are still valid and notify if they expire.

Context

Split from #82. Depends on #92 (notification infrastructure). If Lemmy credentials expire, publishing silently fails. An automated check prevents this.

Tasks

Scheduled job to test login for each active platform account
Create notification on auth failure
Mark account as unhealthy (surface in UI)
Tests

## Summary Periodically verify that platform account credentials are still valid and notify if they expire. ## Context Split from #82. Depends on #92 (notification infrastructure). If Lemmy credentials expire, publishing silently fails. An automated check prevents this. ## Tasks - [ ] Scheduled job to test login for each active platform account - [ ] Create notification on auth failure - [ ] Mark account as unhealthy (surface in UI) - [ ] Tests

myrmidex added this to the v1.3.0 milestone 2026-03-09 00:14:36 +01:00

myrmidex referenced this issue

2026-03-09 00:14:52 +01:00

Feed health monitoring and notifications #82

myrmidex added a new dependency 2026-03-09 00:16:23 +01:00

#92 In-app notification infrastructure

myrmidex added the

enhancement

label 2026-03-09 00:24:28 +01:00

myrmidex modified the milestone from v1.3.0 to v1.4.0

2026-03-18 18:12:17 +01:00

myrmidex self-assigned this 2026-08-11 00:03:56 +02:00

myrmidex referenced this issue

2026-08-11 00:39:07 +02:00

Intermittent failure in ArticlesTest::test_expanding_a_feed_shows_all_its_articles_beyond_one_page #134

myrmidex commented

2026-08-11 00:45:21 +02:00

Author

Owner

Implemented in c40e167.

Delivered

AccountStatusEnum — untested / healthy / unhealthy, cast on PlatformAccount
Migration adding consecutive_failures, plus a data update mapping the legacy active → healthy
PlatformAccount::recordCredentialCheckPassed() / recordCredentialCheckFailed() / isUnhealthy(), threshold in FAILURES_BEFORE_UNHEALTHY
CheckPlatformCredentialsJob, scheduled daily
CREDENTIAL_EXPIRED notification on the healthy→unhealthy transition
"Credentials failing" badge on the Channels page
9 new tests

Design decisions

Daily, not hourly. LemmyApiService::login() posts real credentials to a live instance, already carries rate-limit detection, and LemmyRequest has a 30s timeout. Hourly checks across every active account risk throttling for little benefit — credentials do not expire on an hourly timescale.

Unhealthy only after 3 consecutive failures. A network blip, a 502, or an instance restart makes login() return null in exactly the same way expired credentials do. Alerting on the first failure would fire every time the instance restarts. Worst case is ~3 days to detect a genuine expiry, which is the right trade.

Found along the way

The status column had five undocumented values in circulation: untested (db default + factory), active (CreatePlatformAccountAction), and working / failed (factory states). Only the first two were known when this started — working and failed surfaced as ValueError when the enum cast met the factory. Verified that nothing in app/ ever wrote working or failed, so no production row can hold them and no backfill was needed for those.

Known limitations

The migration's down() is not a true inverse — it rewrites every healthy row to active, including accounts the health check legitimately marked healthy. Acceptable because releases go out by tag rather than migrate:rollback. Noted in a comment in the migration.
LemmyApiService::login() logs the response body on failure. Lemmy is not believed to echo submitted credentials there, but that is inference, not verified against Lemmy's source. Pre-existing behaviour shared with the manual credential-test flow; this job does not change it.
The unhealthy badge has not been seen in a browser — it only renders after 3 consecutive real failures, which is impractical to stage without breaking live credentials. The state transition is covered by tests; the badge markup is not.

Verification

977 tests / 2553 assertions passing (968 before), Pint clean (308 files), PHPStan clean. Migration run against the dev database — the one existing active row converted to healthy correctly.

An intermittent, unrelated test failure observed during this work is tracked separately as #134.

Implemented in `c40e167`. ## Delivered - `AccountStatusEnum` — `untested` / `healthy` / `unhealthy`, cast on `PlatformAccount` - Migration adding `consecutive_failures`, plus a data update mapping the legacy `active` → `healthy` - `PlatformAccount::recordCredentialCheckPassed()` / `recordCredentialCheckFailed()` / `isUnhealthy()`, threshold in `FAILURES_BEFORE_UNHEALTHY` - `CheckPlatformCredentialsJob`, scheduled **daily** - `CREDENTIAL_EXPIRED` notification on the healthy→unhealthy transition - "Credentials failing" badge on the Channels page - 9 new tests ## Design decisions **Daily, not hourly.** `LemmyApiService::login()` posts real credentials to a live instance, already carries rate-limit detection, and `LemmyRequest` has a 30s timeout. Hourly checks across every active account risk throttling for little benefit — credentials do not expire on an hourly timescale. **Unhealthy only after 3 consecutive failures.** A network blip, a 502, or an instance restart makes `login()` return `null` in exactly the same way expired credentials do. Alerting on the first failure would fire every time the instance restarts. Worst case is ~3 days to detect a genuine expiry, which is the right trade. ## Found along the way The `status` column had **five** undocumented values in circulation: `untested` (db default + factory), `active` (`CreatePlatformAccountAction`), and `working` / `failed` (factory states). Only the first two were known when this started — `working` and `failed` surfaced as `ValueError` when the enum cast met the factory. Verified that nothing in `app/` ever wrote `working` or `failed`, so no production row can hold them and no backfill was needed for those. ## Known limitations - The migration's `down()` is **not a true inverse** — it rewrites every `healthy` row to `active`, including accounts the health check legitimately marked healthy. Acceptable because releases go out by tag rather than `migrate:rollback`. Noted in a comment in the migration. - `LemmyApiService::login()` logs the response body on failure. Lemmy is not believed to echo submitted credentials there, but that is **inference, not verified** against Lemmy's source. Pre-existing behaviour shared with the manual credential-test flow; this job does not change it. - The unhealthy badge has **not been seen in a browser** — it only renders after 3 consecutive real failures, which is impractical to stage without breaking live credentials. The state transition is covered by tests; the badge markup is not. ## Verification 977 tests / 2553 assertions passing (968 before), Pint clean (308 files), PHPStan clean. Migration run against the dev database — the one existing `active` row converted to `healthy` correctly. An intermittent, unrelated test failure observed during this work is tracked separately as #134.

myrmidex closed this issue

2026-08-11 00:45:30 +02:00

myrmidex referenced this issue

2026-08-15 00:02:58 +02:00

Release v1.4.0 #146