Detect and warn about feeds that fetch successfully but return no articles #116

New issue

Closed

opened 2026-08-01 11:28:31 +02:00 by myrmidex · 0 comments

myrmidex commented

2026-08-01 11:28:31 +02:00

Owner

Summary

A feed whose URL has broken can fetch "successfully" and produce zero articles indefinitely, with nothing surfaced to the user. This is how #115 went unnoticed — the Belga RSS feed was retired upstream and the app kept reporting healthy fetches.

The gap

ArticleDiscoveryForFeedJob::handle() updates last_fetched_at unconditionally after calling the fetcher:

$articles = $articleFetcher->getArticlesFromFeed($this->feed);
$logSaver->info('Feed article fetch completed', null, [..., 'articles_count' => $articles->count()]);
$this->feed->update(['last_fetched_at' => now()]);

ArticleFetcher returns an empty collection rather than throwing, in every failure path:

Unsupported feed type → logs warning, returns collect()
RSS XML fails to parse → logs warning, returns collect()
(In #115: a 404 HTML error page was passed to simplexml_load_string(), which failed exactly this way)

Meanwhile CheckFeedStalenessJob detects staleness purely via Feed::stale():

$query->whereNull('last_fetched_at')
      ->orWhere('last_fetched_at', '<', now()->subHours($thresholdHours));

Because the broken feed keeps updating last_fetched_at, it never looks stale. The warning exists in the log but nothing reaches the UI, and no notification fires.

Proposed

Track article-yield per feed and notify when a feed fetches repeatedly without producing anything.

Sketch — details open to discussion:

Add something like last_article_at (or a consecutive-empty-fetch counter) to feeds
ArticleDiscoveryForFeedJob updates it when $articles->isNotEmpty()
Extend CheckFeedStalenessJob, or add a sibling job, to flag feeds where last_fetched_at is recent but no articles have arrived within a threshold
New NotificationTypeEnum case (e.g. FEED_EMPTY) — FEED_STALE means something different and reusing it would muddy both
Reuse the existing dedupe pattern (skip if an unread notification of that type already exists for the feed)
Threshold as a Setting, following feed_staleness_threshold

Design questions

Counter vs. timestamp — a consecutive-empty counter is simpler to reason about but resets awkwardly; a last_article_at timestamp composes better with the existing threshold pattern. Leaning timestamp.
What counts as "no articles" — zero fetched, or zero new ones? A feed legitimately returning only already-seen articles is healthy; a feed returning nothing at all is not. These differ and the distinction matters.
Low-volume feeds — a genuinely quiet source could trip the warning. The threshold needs to be generous, or configurable per feed.

Why this matters beyond Belga

The same failure mode applies to every feed. VRT and Guardian could break identically — an upstream URL change, a markup change breaking a parser, a retired feed — and the app would report healthy fetches indefinitely. Discovering it depends on someone noticing missing articles.

Acceptance criteria

A feed fetching successfully but yielding no articles over the threshold produces a notification
Notification appears in the UI (notification bell), not just the log
Deduplicated — one unread notification per feed, matching FEED_STALE behaviour
Threshold configurable via Setting, with a sensible default; 0 disables
Existing FEED_STALE behaviour unchanged
Tests cover: empty-yield feed warns, healthy feed does not, dedupe holds, disabled threshold is a no-op

#115 — the Belga failure this would have caught

## Summary A feed whose URL has broken can fetch "successfully" and produce zero articles indefinitely, with nothing surfaced to the user. This is how #115 went unnoticed — the Belga RSS feed was retired upstream and the app kept reporting healthy fetches. ## The gap `ArticleDiscoveryForFeedJob::handle()` updates `last_fetched_at` unconditionally after calling the fetcher: ```php $articles = $articleFetcher->getArticlesFromFeed($this->feed); $logSaver->info('Feed article fetch completed', null, [..., 'articles_count' => $articles->count()]); $this->feed->update(['last_fetched_at' => now()]); ``` `ArticleFetcher` returns an **empty collection** rather than throwing, in every failure path: - Unsupported feed type → logs warning, returns `collect()` - RSS XML fails to parse → logs warning, returns `collect()` - (In #115: a 404 HTML error page was passed to `simplexml_load_string()`, which failed exactly this way) Meanwhile `CheckFeedStalenessJob` detects staleness purely via `Feed::stale()`: ```php $query->whereNull('last_fetched_at') ->orWhere('last_fetched_at', '<', now()->subHours($thresholdHours)); ``` Because the broken feed keeps updating `last_fetched_at`, **it never looks stale.** The warning exists in the log but nothing reaches the UI, and no notification fires. ## Proposed Track article-yield per feed and notify when a feed fetches repeatedly without producing anything. Sketch — details open to discussion: - Add something like `last_article_at` (or a consecutive-empty-fetch counter) to `feeds` - `ArticleDiscoveryForFeedJob` updates it when `$articles->isNotEmpty()` - Extend `CheckFeedStalenessJob`, or add a sibling job, to flag feeds where `last_fetched_at` is recent but no articles have arrived within a threshold - New `NotificationTypeEnum` case (e.g. `FEED_EMPTY`) — `FEED_STALE` means something different and reusing it would muddy both - Reuse the existing dedupe pattern (skip if an unread notification of that type already exists for the feed) - Threshold as a `Setting`, following `feed_staleness_threshold` ### Design questions - **Counter vs. timestamp** — a consecutive-empty counter is simpler to reason about but resets awkwardly; a `last_article_at` timestamp composes better with the existing threshold pattern. Leaning timestamp. - **What counts as "no articles"** — zero *fetched*, or zero *new* ones? A feed legitimately returning only already-seen articles is healthy; a feed returning nothing at all is not. These differ and the distinction matters. - **Low-volume feeds** — a genuinely quiet source could trip the warning. The threshold needs to be generous, or configurable per feed. ## Why this matters beyond Belga The same failure mode applies to every feed. VRT and Guardian could break identically — an upstream URL change, a markup change breaking a parser, a retired feed — and the app would report healthy fetches indefinitely. Discovering it depends on someone noticing missing articles. ## Acceptance criteria - [ ] A feed fetching successfully but yielding no articles over the threshold produces a notification - [ ] Notification appears in the UI (notification bell), not just the log - [ ] Deduplicated — one unread notification per feed, matching `FEED_STALE` behaviour - [ ] Threshold configurable via `Setting`, with a sensible default; `0` disables - [ ] Existing `FEED_STALE` behaviour unchanged - [ ] Tests cover: empty-yield feed warns, healthy feed does not, dedupe holds, disabled threshold is a no-op ## Related - #115 — the Belga failure this would have caught