Thumbnail URLs are not HTML-entity decoded, so custom_thumbnail fails #117

New issue

Closed

opened 2026-08-01 16:34:23 +02:00 by myrmidex · 0 comments

myrmidex commented

2026-08-01 16:34:23 +02:00

Owner

Summary

Published posts have no thumbnail. The thumbnail is extracted and is passed to Lemmy, but the URL contains raw HTML entities (& instead of &), so it is malformed by the time it reaches custom_thumbnail.

Found while verifying the Belga fix (#115) — a Belga article published successfully, but without its image.

Root cause

extractThumbnail() returns the regex capture verbatim, with no entity decoding:

public static function extractThumbnail(string $html): ?string
{
    if (preg_match('/<meta property="og:image" content="([^"]+)"/i', $html, $matches)) {
        return $matches[1];   // <- raw, still entity-encoded
    }

    if (preg_match('/<img[^>]+src="([^"]+)"/i', $html, $matches)) {
        return $matches[1];   // <- same
    }
    ...
}

Verified against a live Belga article page (/press-releases/35285/):

thumbnail: 'https://picturepackcdn-h33aaywmsq-ew.a.run.app/belgapicturepack:181511382:full?v=6a6c828f&amp;m=cdodnjha'

That & should be &. The URL as returned does not resolve.

The inconsistency

The same files do decode entities for other fields. In BelgaArticlePageParser:

extractTitle() → html_entity_decode($matches[1], ENT_QUOTES, 'UTF-8')
extractDescription() → html_entity_decode(...)
extractThumbnail() → no decoding

So this looks like an oversight rather than a deliberate choice.

Affects all three parsers

Identical un-decoded return $matches[1]; in:

app/Services/Parsers/BelgaArticlePageParser.php:108-119
app/Services/Parsers/VrtArticlePageParser.php:67-75
app/Services/Parsers/GuardianArticlePageParser.php:70

It only bites when the image URL contains query parameters (hence &). Belga's CDN URLs always do, which is why it surfaced there first. VRT and Guardian may be silently affected depending on their image hosts.

The plumbing is otherwise fine

No new wiring needed — the path already exists end to end:

*ArticlePageParser::extractData() returns a thumbnail key
LemmyPublisher.php:65 passes $extractedData['thumbnail'] ?? null
LemmyApiService::createPost() sets $postData['custom_thumbnail'] = $thumbnail

Only the decoding is missing.

Note on `Article::image_url`

Separately, all six Belga articles currently have image_url = NULL in the database even though extraction succeeds at publish time. Worth checking during implementation whether image_url is ever populated from the extracted thumbnail, or whether the thumbnail is only ever read transiently during publishing. If the column is meant to be populated, that is a second gap — flagging rather than assuming, since it may be intentional.

Acceptance criteria

extractThumbnail() decodes HTML entities in all three parsers
A Belga article publishes to Lemmy with its thumbnail visible
VRT and Guardian thumbnails still work (regression check)
Unit tests cover an og:image URL containing &, asserting the decoded form
Decide and document whether Article::image_url should be populated

#115 — Belga discovery, where this was found

## Summary Published posts have no thumbnail. The thumbnail *is* extracted and *is* passed to Lemmy, but the URL contains raw HTML entities (`&` instead of `&`), so it is malformed by the time it reaches `custom_thumbnail`. Found while verifying the Belga fix (#115) — a Belga article published successfully, but without its image. ## Root cause `extractThumbnail()` returns the regex capture verbatim, with no entity decoding: ```php public static function extractThumbnail(string $html): ?string { if (preg_match('/<meta property="og:image" content="([^"]+)"/i', $html, $matches)) { return $matches[1]; // <- raw, still entity-encoded } if (preg_match('/<img[^>]+src="([^"]+)"/i', $html, $matches)) { return $matches[1]; // <- same } ... } ``` Verified against a live Belga article page (`/press-releases/35285/`): ``` thumbnail: 'https://picturepackcdn-h33aaywmsq-ew.a.run.app/belgapicturepack:181511382:full?v=6a6c828f&m=cdodnjha' ``` That `&` should be `&`. The URL as returned does not resolve. ## The inconsistency The same files **do** decode entities for other fields. In `BelgaArticlePageParser`: - `extractTitle()` → `html_entity_decode($matches[1], ENT_QUOTES, 'UTF-8')` - `extractDescription()` → `html_entity_decode(...)` - `extractThumbnail()` → **no decoding** So this looks like an oversight rather than a deliberate choice. ## Affects all three parsers Identical un-decoded `return $matches[1];` in: - `app/Services/Parsers/BelgaArticlePageParser.php:108-119` - `app/Services/Parsers/VrtArticlePageParser.php:67-75` - `app/Services/Parsers/GuardianArticlePageParser.php:70` It only bites when the image URL contains query parameters (hence `&`). Belga's CDN URLs always do, which is why it surfaced there first. VRT and Guardian may be silently affected depending on their image hosts. ## The plumbing is otherwise fine No new wiring needed — the path already exists end to end: - `*ArticlePageParser::extractData()` returns a `thumbnail` key - `LemmyPublisher.php:65` passes `$extractedData['thumbnail'] ?? null` - `LemmyApiService::createPost()` sets `$postData['custom_thumbnail'] = $thumbnail` Only the decoding is missing. ## Note on `Article::image_url` Separately, all six Belga articles currently have `image_url = NULL` in the database even though extraction succeeds at publish time. Worth checking during implementation whether `image_url` is ever populated from the extracted thumbnail, or whether the thumbnail is only ever read transiently during publishing. If the column is meant to be populated, that is a second gap — flagging rather than assuming, since it may be intentional. ## Acceptance criteria - [ ] `extractThumbnail()` decodes HTML entities in all three parsers - [ ] A Belga article publishes to Lemmy with its thumbnail visible - [ ] VRT and Guardian thumbnails still work (regression check) - [ ] Unit tests cover an `og:image` URL containing `&`, asserting the decoded form - [ ] Decide and document whether `Article::image_url` should be populated ## Related - #115 — Belga discovery, where this was found