BidaAngMamayan: public supporter tribute news desk for Senator BAM Aquino
Forked from Forthena (PERSONAL/forthena) and retargeted: - NO login, NO LLM: the edition is public and the ranking is deterministic (stories that name Sen. BAM lead; freshest first) — Daniel: no like/dislike or store profile for visitors. - Crawler scoped to BAM: Bing news keyword queries + Inquirer/Rappler fallback wires, with a two-tier relevance gate (named-BAM always; Aquino+legislative-context only from fallback) so op-eds and non-Bam senators are dropped. - Desks: BAM (lead), Gaming, Education, Politics, Energy, PH, Business. - Community board /board: 'Thank you Senator BAM' — nickname-only posts, HTML-stripped, URL-only rejected, IP rate-limited (3/10min, 20/day). - Yellow-gamer theme (Orbitron/Rajdhani/JetBrains Mono, dark + gold, HUD corners), supporter disclaimer in header and footer. - Tests: category/date/relevance/text suites + Mongo board suite + full HTTP e2e (all passing). Infra: 66.181.46.132 (bidaangmamamayan.com), local Mongo 7.0, nginx + certbot TLS, PM2 bama (:3000) + 15-min crawl cron.
This commit is contained in:
Ryan
commit
6b06e08c08
5
.gitignore
vendored
Normal file
5
.gitignore
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
node_modules
|
||||
.next
|
||||
.env*
|
||||
*.log
|
||||
.DS_Store
|
||||
97
README.md
Normal file
97
README.md
Normal file
@ -0,0 +1,97 @@
|
||||
# BidaAngMamayan
|
||||
|
||||
**BIDA ANG MAMAMAYAN** — a public supporter tribute news desk for
|
||||
**Senator Bam Aquino**, the gamer of the Philippine Senate and champion of
|
||||
college students, scholarships, and fair electricity.
|
||||
|
||||
> Forked from the Forthena architecture (Next.js + Mongo + an RSS crawler),
|
||||
> retargeted to follow ONE subject and stripped of everything a private paper
|
||||
> needs but a public tribute does not: **no login, no accounts, no
|
||||
> like/dislike, no per-reader AI.**
|
||||
|
||||
---
|
||||
|
||||
## What it is
|
||||
|
||||
- **A live news crawl** (every 15 min, PM2 cron) that follows every story on
|
||||
the Philippine wire that *names* Sen. Bam Aquino, plus a quiet-day fallback
|
||||
to general PH wires. A **relevance gate** drops the senator's name from
|
||||
unrelated copy — only genuine BAM stories print.
|
||||
- **A yellow/gamer visual motif** (Orbitron + Rajdhani + JetBrains Mono, CRT
|
||||
scanlines, an XP-bar edition meter, a scrolling BAM-wire ticker, a neon
|
||||
"LIVE DESK" HUD) — because the senator is a gamer.
|
||||
- **A public community board — "Thank you, Senator BAM"** — visitors post
|
||||
with a **nickname only** (no account, no email). Light abuse controls:
|
||||
per-IP rate limit, hard content caps, markup stripping, URL-only rejection.
|
||||
- **An in-page reader drawer** that lifts the full article body server-side
|
||||
(Readability) so a story can be read without leaving the site, with an
|
||||
honest "read at source" fallback when a publisher blocks us.
|
||||
- **A disclaimer, always visible** (masthead + footer): the site is run by
|
||||
*supporters* and is **not affiliated with, endorsed by, or operated by**
|
||||
the senator or his office.
|
||||
|
||||
## The seven desks
|
||||
|
||||
`BAM Front` (lead, named-BAM) · `Education` · `Energy & Electricity` ·
|
||||
`Gamer's Corner` · `The Senate Floor` · `The Philippines` · `World`.
|
||||
|
||||
## Stack
|
||||
|
||||
Next.js 15 (React 19, App Router, plain CSS — no Tailwind in the UI),
|
||||
MongoDB 7.0, PM2, nginx + Let's Encrypt. **No LLM in the runtime path** —
|
||||
ranking is deterministic (`named` → fresh → photo). The Forthena llmrouter
|
||||
dependency was removed entirely.
|
||||
|
||||
## Run locally
|
||||
|
||||
```bash
|
||||
npm install
|
||||
# point at a Mongo (defaults to mongodb://127.0.0.1:27017/bama)
|
||||
npm run build
|
||||
npm start # :3000
|
||||
```
|
||||
|
||||
Tests:
|
||||
|
||||
```bash
|
||||
bash tests/run-all.sh # self-contained (no Mongo)
|
||||
MONGODB_URI=mongodb://127.0.0.1:27099/bama node tests/board-test.mjs
|
||||
node tests/e2e-run.mjs # boots the built app on :4003
|
||||
```
|
||||
|
||||
## Deploy (manual — no CI, by owner order)
|
||||
|
||||
Target box: **66.181.46.132**, domain **bidaangmamamayan.com** (apex + www).
|
||||
|
||||
1. On the box, `/var/www/bama` holds the source. `.env.production` lives
|
||||
**only on the box** (never rsync it): `MONGODB_URI`, `CRON_SECRET`.
|
||||
2. `rsync -az --delete --exclude node_modules --exclude .next --exclude .git --exclude '.env*' ./ root@66.181.46.132:/var/www/bama/`
|
||||
3. On the box: `cd /var/www/bama && npm ci && npm run build && pm2 reload bama`
|
||||
4. Crawl tick is PM2 app `bama-crawl` (`*/15 * * * *` → `POST /api/crawl`
|
||||
with `x-bama-cron=$CRON_SECRET`). First edition appears within ~15 min;
|
||||
fire one crawl by hand to seed it immediately.
|
||||
5. TLS: `certbot --nginx -d bidaangmamamayan.com -d www.bidaangmamamayan.com`
|
||||
(nginx is already configured for both; the 443 block is live once the
|
||||
cert exists). Auto-renewal timer is armed by certbot.
|
||||
|
||||
Mongo note: the box runs kernel 7.0 (Ubuntu 26.04) where Mongo 8.x is broken
|
||||
(8.0 refuses the kernel, 8.2 SIGSEGVs). **MongoDB 7.0 from the jammy channel
|
||||
is the working version** — installed at provision time.
|
||||
|
||||
## Files that matter
|
||||
|
||||
| path | role |
|
||||
| --- | --- |
|
||||
| `lib/sources.js` | the BAM query registry + Bing feed helpers |
|
||||
| `lib/crawl.js` | crawler + **relevance gate** (`bamRelevance`) |
|
||||
| `lib/categories.js` | the 7-desk registry (single source of truth) |
|
||||
| `lib/feed.js` | public edition: deterministic rank, desk headlines, stats |
|
||||
| `lib/article.js` | reader extraction (Readability + honest fallbacks) |
|
||||
| `lib/board.js` | community board: caps, sanitise, rate limit |
|
||||
| `app/page.js` | the whole front-end (client SPA: views + reader drawer) |
|
||||
| `app/globals.css` | the yellow/gamer theme |
|
||||
| `ecosystem.config.cjs` | PM2 apps (`bama`, `bama-crawl`) |
|
||||
|
||||
---
|
||||
|
||||
*A fan project. Bida ang Mamamayan.* 🎮
|
||||
23
app/api/article/route.js
Normal file
23
app/api/article/route.js
Normal file
@ -0,0 +1,23 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// The article body for the reader pane. Public (no auth), zero LLM.
|
||||
// Mongo first (the crawler warms it on the tick), publisher fetch on a miss.
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
export const maxDuration = 60;
|
||||
|
||||
export async function GET(req) {
|
||||
const q = new URL(req.url).searchParams;
|
||||
const url = q.get("url") || "";
|
||||
if (!/^https?:\/\//i.test(url)) return NextResponse.json({ ok: false, error: "url required" }, { status: 400 });
|
||||
|
||||
try {
|
||||
const { getArticle } = await import("../../../lib/article");
|
||||
const a = await getArticle({ url, force: q.get("force") === "1" });
|
||||
const { story, realUrl, ...article } = a;
|
||||
return NextResponse.json({ ok: true, url, realUrl: realUrl || url, article, story: story || null });
|
||||
} catch (e) {
|
||||
console.warn("[bama] /api/article failed:", String((e && e.message) || e).slice(0, 160));
|
||||
return NextResponse.json({ ok: false, error: "article unavailable" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
37
app/api/board/route.js
Normal file
37
app/api/board/route.js
Normal file
@ -0,0 +1,37 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// The community board — "Thank you Senator BAM".
|
||||
// GET -> the last 200 posts, newest first.
|
||||
// POST -> { nick, message } (nickname only, no account — by design).
|
||||
//
|
||||
// Public by nature; the controls live in lib/board.js (rate limit per IP,
|
||||
// content caps, markup stripping, URL-only rejection).
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
export async function GET() {
|
||||
try {
|
||||
const { getBoard, boardCount } = await import("../../../lib/board");
|
||||
const [posts, count] = await Promise.all([getBoard(), boardCount()]);
|
||||
return NextResponse.json({ ok: true, posts, count });
|
||||
} catch (e) {
|
||||
return NextResponse.json({ ok: false, error: "board unavailable" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
export async function POST(req) {
|
||||
let body;
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return NextResponse.json({ ok: false, error: "expected json" }, { status: 400 });
|
||||
}
|
||||
try {
|
||||
const { postBoard } = await import("../../../lib/board");
|
||||
const r = await postBoard(req, body);
|
||||
return NextResponse.json(r, { status: r.ok ? 200 : r.status });
|
||||
} catch (e) {
|
||||
console.warn("[bama] board post failed:", String((e && e.message) || e).slice(0, 160));
|
||||
return NextResponse.json({ ok: false, error: "could not post right now" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
21
app/api/crawl/route.js
Normal file
21
app/api/crawl/route.js
Normal file
@ -0,0 +1,21 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// The crawl tick, fired by the PM2 cron (scripts/cron-digest.cjs) every 15 min.
|
||||
// Protected by the CRON_SECRET header so a random visitor cannot trigger a
|
||||
// crawl on demand (it costs feed fetches + og:image rescues + article
|
||||
// prefetches).
|
||||
export const maxDuration = 290;
|
||||
|
||||
export async function POST(req) {
|
||||
const secret = req.headers.get("x-bama-cron");
|
||||
if (secret !== (process.env.CRON_SECRET || "bama-cron")) {
|
||||
return NextResponse.json({ error: "unauthorized" }, { status: 401 });
|
||||
}
|
||||
try {
|
||||
const { crawlOnce } = await import("../../../lib/crawl");
|
||||
const crawl = await crawlOnce();
|
||||
return NextResponse.json({ ok: true, crawl });
|
||||
} catch (e) {
|
||||
return NextResponse.json({ ok: false, error: String((e && e.message) || e) }, { status: 500 });
|
||||
}
|
||||
}
|
||||
20
app/api/feed/route.js
Normal file
20
app/api/feed/route.js
Normal file
@ -0,0 +1,20 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// The public edition. NO auth — this site has no login by design.
|
||||
// No LLM on a page load either: the ranking is deterministic (lib/feed.js).
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
export async function GET(req) {
|
||||
try {
|
||||
const { getFeed, deskHeadlines, editionStats } = await import("../../../lib/feed");
|
||||
const [feed, desks, stats] = await Promise.all([
|
||||
getFeed({ limit: 60 }),
|
||||
deskHeadlines(),
|
||||
editionStats(),
|
||||
]);
|
||||
return NextResponse.json({ ok: true, feed, desks, stats });
|
||||
} catch (e) {
|
||||
return NextResponse.json({ ok: false, error: "edition unavailable", detail: String((e && e.message) || e).slice(0, 200) }, { status: 500 });
|
||||
}
|
||||
}
|
||||
23
app/api/health/route.js
Normal file
23
app/api/health/route.js
Normal file
@ -0,0 +1,23 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// Liveness probe (nginx/monitoring). Public. Reports the process is up and
|
||||
// pings Mongo so a DB outage is visible here, not hidden behind a 200 feed.
|
||||
//
|
||||
// NOTE the double await is REQUIRED: getDb() is async (returns Promise<Db>).
|
||||
// `await getDb().command(...)` would call .command on the PROMISE and throw
|
||||
// "command is not a function" (caught in e2e 2026-09-15).
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
export async function GET() {
|
||||
let mongo = "unknown";
|
||||
try {
|
||||
const { getDb } = await import("../../../lib/db");
|
||||
const db = await getDb();
|
||||
await db.command({ ping: 1 });
|
||||
mongo = "ok";
|
||||
} catch {
|
||||
mongo = "down";
|
||||
}
|
||||
return NextResponse.json({ ok: true, service: "bidaangmamamayan", mongo, time: new Date().toISOString() });
|
||||
}
|
||||
67
app/api/img/route.js
Normal file
67
app/api/img/route.js
Normal file
@ -0,0 +1,67 @@
|
||||
import { NextResponse } from "next/server";
|
||||
|
||||
// Image proxy. News CDN images hotlink-block on Referer and some are http-only
|
||||
// (mixed content against our https site), so the paper loads them through here.
|
||||
// No auth needed: images are public anyway, and it keeps the <img> tags dumb.
|
||||
|
||||
const MAX_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
export async function GET(req) {
|
||||
const raw = new URL(req.url).searchParams.get("u") || "";
|
||||
let target;
|
||||
try {
|
||||
target = new URL(raw);
|
||||
} catch {
|
||||
return new NextResponse("bad url", { status: 400 });
|
||||
}
|
||||
if (!/^https?:$/.test(target.protocol)) return new NextResponse("bad scheme", { status: 400 });
|
||||
|
||||
const cacheKey = raw;
|
||||
const cached = globalThis.__ft_img?.get(cacheKey);
|
||||
if (cached && Date.now() - cached.t < 1000 * 60 * 60 * 12) {
|
||||
return new NextResponse(cached.body, {
|
||||
headers: imgHeaders(cached.type, "public, max-age=43200"),
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const res = await fetch(target.href, {
|
||||
headers: {
|
||||
// A browser-looking Referer is what unblocks most PH/tech CDNs.
|
||||
Referer: `${target.protocol}//${target.host}/`,
|
||||
"User-Agent":
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
|
||||
Accept: "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
|
||||
},
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(12000),
|
||||
});
|
||||
if (!res.ok) return new NextResponse("upstream " + res.status, { status: 502 });
|
||||
|
||||
const len = Number(res.headers.get("content-length") || 0);
|
||||
if (len > MAX_BYTES) return new NextResponse("too big", { status: 413 });
|
||||
|
||||
const type = res.headers.get("content-type") || "image/jpeg";
|
||||
if (!/^image\//i.test(type)) return new NextResponse("not an image", { status: 415 });
|
||||
|
||||
const buf = await res.arrayBuffer();
|
||||
if (buf.byteLength > MAX_BYTES) return new NextResponse("too big", { status: 413 });
|
||||
|
||||
if (!globalThis.__ft_img) globalThis.__ft_img = new Map();
|
||||
const map = globalThis.__ft_img;
|
||||
if (map.size > 400) for (const k of map.keys()) { map.delete(k); if (map.size < 300) break; }
|
||||
map.set(cacheKey, { t: Date.now(), type, body: buf });
|
||||
|
||||
return new NextResponse(buf, { headers: imgHeaders(type, "public, max-age=43200") });
|
||||
} catch (e) {
|
||||
return new NextResponse("error", { status: 502 });
|
||||
}
|
||||
}
|
||||
|
||||
function imgHeaders(type, cache) {
|
||||
return {
|
||||
"Content-Type": type,
|
||||
"Cache-Control": cache,
|
||||
"X-Content-Type-Options": "nosniff",
|
||||
};
|
||||
}
|
||||
647
app/globals.css
Normal file
647
app/globals.css
Normal file
@ -0,0 +1,647 @@
|
||||
/* BidaAngMamamayan — dark gamer theme with a gold/yellow motif.
|
||||
Fonts (loaded in layout.js): Orbitron (display/masthead), Rajdhani (body),
|
||||
JetBrains Mono (HUD labels, timestamps, counters). */
|
||||
|
||||
:root {
|
||||
--bg: #0c0b09;
|
||||
--bg-2: #12100c;
|
||||
--panel: #171410;
|
||||
--panel-2: #1d1913;
|
||||
--line: #3a3226;
|
||||
--line-soft: #2a241a;
|
||||
--gold: #ffd426;
|
||||
--gold-2: #ffbf00;
|
||||
--gold-dim: #8a7422;
|
||||
--neon: #2bff88;
|
||||
--red: #ff4d5e;
|
||||
--text: #efe9dc;
|
||||
--muted: #a89e8a;
|
||||
--faint: #6f6754;
|
||||
--glow: 0 0 18px rgba(255, 212, 38, 0.35);
|
||||
--glow-soft: 0 0 10px rgba(255, 212, 38, 0.18);
|
||||
--font-display: "Orbitron", sans-serif;
|
||||
--font-body: "Rajdhani", sans-serif;
|
||||
--font-mono: "JetBrains Mono", monospace;
|
||||
}
|
||||
|
||||
* { box-sizing: border-box; }
|
||||
html, body { margin: 0; padding: 0; }
|
||||
body {
|
||||
background:
|
||||
radial-gradient(1200px 500px at 50% -100px, rgba(255, 212, 38, 0.07), transparent 60%),
|
||||
var(--bg);
|
||||
color: var(--text);
|
||||
font-family: var(--font-body);
|
||||
font-size: 17px;
|
||||
line-height: 1.45;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
min-height: 100vh;
|
||||
}
|
||||
|
||||
/* subtle CRT scanline + vignette, very low, pure CSS (no images) */
|
||||
body::before {
|
||||
content: "";
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
pointer-events: none;
|
||||
z-index: 0;
|
||||
background: repeating-linear-gradient(
|
||||
to bottom,
|
||||
rgba(255, 255, 255, 0.012) 0px,
|
||||
rgba(255, 255, 255, 0.012) 1px,
|
||||
transparent 1px,
|
||||
transparent 3px
|
||||
);
|
||||
}
|
||||
body::after {
|
||||
content: "";
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
pointer-events: none;
|
||||
z-index: 0;
|
||||
background: radial-gradient(120% 90% at 50% 10%, transparent 60%, rgba(0, 0, 0, 0.5) 100%);
|
||||
}
|
||||
main, header, footer, .ticker { position: relative; z-index: 1; }
|
||||
|
||||
a { color: var(--gold); text-decoration: none; }
|
||||
a:hover { text-decoration: underline; }
|
||||
button { font-family: inherit; cursor: pointer; }
|
||||
input, textarea { font-family: inherit; }
|
||||
|
||||
/* ── masthead ─────────────────────────────────────────────────────────── */
|
||||
.masthead {
|
||||
padding: 26px 16px 14px;
|
||||
text-align: center;
|
||||
border-bottom: 2px solid var(--line);
|
||||
background: linear-gradient(to bottom, rgba(255, 212, 38, 0.05), transparent);
|
||||
}
|
||||
.brand {
|
||||
font-family: var(--font-display);
|
||||
font-weight: 900;
|
||||
letter-spacing: 0.02em;
|
||||
font-size: clamp(30px, 7vw, 58px);
|
||||
line-height: 1;
|
||||
color: var(--text);
|
||||
text-transform: uppercase;
|
||||
user-select: none;
|
||||
}
|
||||
.brand .bam {
|
||||
color: var(--gold);
|
||||
font-weight: 900;
|
||||
text-shadow: var(--glow);
|
||||
-webkit-text-stroke: 1px rgba(255, 212, 38, 0.4);
|
||||
}
|
||||
.brand .bam-a { -webkit-text-stroke: 0; }
|
||||
.masthead .slogan {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 12px;
|
||||
letter-spacing: 0.42em;
|
||||
color: var(--gold);
|
||||
margin-top: 10px;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.masthead .who {
|
||||
margin: 10px auto 0;
|
||||
max-width: 720px;
|
||||
color: var(--muted);
|
||||
font-size: 16px;
|
||||
font-weight: 500;
|
||||
}
|
||||
.masthead .who b { color: var(--text); font-weight: 700; }
|
||||
|
||||
/* disclaimer bar — the site's legal heart, always visible on the front page */
|
||||
.disclaimer {
|
||||
margin: 14px auto 0;
|
||||
max-width: 860px;
|
||||
border: 1px solid var(--line);
|
||||
background: rgba(255, 77, 94, 0.05);
|
||||
color: var(--muted);
|
||||
font-size: 13.5px;
|
||||
font-weight: 500;
|
||||
padding: 8px 14px;
|
||||
border-radius: 6px;
|
||||
text-align: center;
|
||||
}
|
||||
.disclaimer b { color: var(--red); font-weight: 700; }
|
||||
|
||||
/* nav / desk strip */
|
||||
.navstrip {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 40;
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
overflow-x: auto;
|
||||
padding: 10px 14px;
|
||||
background: rgba(12, 11, 9, 0.92);
|
||||
backdrop-filter: blur(6px);
|
||||
border-bottom: 1px solid var(--line-soft);
|
||||
scrollbar-width: none;
|
||||
}
|
||||
.navstrip::-webkit-scrollbar { display: none; }
|
||||
.navbtn {
|
||||
flex: 0 0 auto;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 12px;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
color: var(--muted);
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
padding: 7px 12px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.navbtn:hover { color: var(--gold); border-color: var(--gold-dim); text-decoration: none; }
|
||||
.navbtn.active {
|
||||
color: #12100c;
|
||||
background: var(--gold);
|
||||
border-color: var(--gold);
|
||||
font-weight: 700;
|
||||
box-shadow: var(--glow-soft);
|
||||
}
|
||||
.navbtn .dot {
|
||||
display: inline-block;
|
||||
width: 6px; height: 6px;
|
||||
background: var(--neon);
|
||||
border-radius: 50%;
|
||||
margin-right: 6px;
|
||||
box-shadow: 0 0 6px var(--neon);
|
||||
}
|
||||
|
||||
/* ── HUD stat bar (gamer XP/HP vibe) ──────────────────────────────────── */
|
||||
.hud {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 10px;
|
||||
justify-content: center;
|
||||
padding: 14px 14px 4px;
|
||||
}
|
||||
.hud .chip {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 11.5px;
|
||||
letter-spacing: 0.06em;
|
||||
color: var(--muted);
|
||||
border: 1px solid var(--line);
|
||||
background: var(--panel);
|
||||
border-radius: 4px;
|
||||
padding: 6px 10px;
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
}
|
||||
.hud .chip b { color: var(--gold); font-weight: 700; }
|
||||
.hud .chip .live {
|
||||
width: 7px; height: 7px; border-radius: 50%;
|
||||
background: var(--neon);
|
||||
box-shadow: 0 0 8px var(--neon);
|
||||
animation: pulse 1.6s infinite;
|
||||
}
|
||||
@keyframes pulse { 0%,100% { opacity: 1; } 50% { opacity: 0.35; } }
|
||||
|
||||
/* XP bar: how full the edition is */
|
||||
.xpbar {
|
||||
width: min(560px, 92vw);
|
||||
margin: 10px auto 0;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 10.5px;
|
||||
letter-spacing: 0.14em;
|
||||
color: var(--faint);
|
||||
}
|
||||
.xpbar .track {
|
||||
height: 8px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 4px;
|
||||
background: var(--bg-2);
|
||||
overflow: hidden;
|
||||
margin-top: 4px;
|
||||
}
|
||||
.xpbar .fill {
|
||||
height: 100%;
|
||||
background: linear-gradient(90deg, var(--gold-2), var(--gold));
|
||||
box-shadow: var(--glow-soft);
|
||||
transition: width 0.6s ease;
|
||||
}
|
||||
|
||||
/* ── news ticker ──────────────────────────────────────────────────────── */
|
||||
.ticker {
|
||||
overflow: hidden;
|
||||
border-top: 1px solid var(--line-soft);
|
||||
border-bottom: 1px solid var(--line);
|
||||
background: var(--bg-2);
|
||||
white-space: nowrap;
|
||||
padding: 7px 0;
|
||||
position: relative;
|
||||
}
|
||||
.ticker .label {
|
||||
position: absolute;
|
||||
left: 0; top: 0; bottom: 0;
|
||||
z-index: 2;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
background: var(--gold);
|
||||
color: #12100c;
|
||||
font-family: var(--font-mono);
|
||||
font-weight: 700;
|
||||
font-size: 11px;
|
||||
letter-spacing: 0.1em;
|
||||
padding: 0 10px;
|
||||
}
|
||||
.ticker .rail {
|
||||
display: inline-block;
|
||||
padding-left: 110px;
|
||||
animation: ticker 60s linear infinite;
|
||||
}
|
||||
.ticker:hover .rail { animation-play-state: paused; }
|
||||
.ticker .item {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 12.5px;
|
||||
color: var(--muted);
|
||||
margin-right: 42px;
|
||||
}
|
||||
.ticker .item b { color: var(--gold); font-weight: 700; margin-right: 8px; }
|
||||
.ticker .item a { color: var(--text); }
|
||||
.ticker .item a:hover { color: var(--gold); }
|
||||
@keyframes ticker { from { transform: translateX(0); } to { transform: translateX(-50%); } }
|
||||
|
||||
/* ── layout: lead + rail ─────────────────────────────────────────────── */
|
||||
.wrap { max-width: 1160px; margin: 0 auto; padding: 18px 14px 60px; }
|
||||
.cols {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 330px;
|
||||
gap: 20px;
|
||||
align-items: start;
|
||||
}
|
||||
@media (max-width: 900px) { .cols { grid-template-columns: 1fr; } }
|
||||
|
||||
.panel {
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.panel .ph {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 10px 14px;
|
||||
border-bottom: 1px solid var(--line-soft);
|
||||
font-family: var(--font-mono);
|
||||
font-size: 11.5px;
|
||||
letter-spacing: 0.18em;
|
||||
color: var(--gold);
|
||||
text-transform: uppercase;
|
||||
background: linear-gradient(to bottom, rgba(255, 212, 38, 0.06), transparent);
|
||||
}
|
||||
.panel .ph .count { margin-left: auto; color: var(--faint); letter-spacing: 0.04em; }
|
||||
|
||||
/* lead story */
|
||||
.lead { position: relative; cursor: pointer; }
|
||||
.lead .hero {
|
||||
position: relative;
|
||||
aspect-ratio: 16/8.5;
|
||||
background: var(--bg-2) center/cover no-repeat;
|
||||
border-bottom: 1px solid var(--line);
|
||||
overflow: hidden;
|
||||
}
|
||||
.lead .hero::after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
background: linear-gradient(to top, rgba(12, 11, 9, 0.95) 12%, rgba(12, 11, 9, 0.25) 55%, transparent);
|
||||
}
|
||||
.lead .hero .ph-tag {
|
||||
position: absolute;
|
||||
left: 12px; bottom: 12px;
|
||||
z-index: 2;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
}
|
||||
.tag {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 10.5px;
|
||||
letter-spacing: 0.14em;
|
||||
text-transform: uppercase;
|
||||
padding: 4px 9px;
|
||||
border-radius: 3px;
|
||||
width: fit-content;
|
||||
font-weight: 700;
|
||||
}
|
||||
.tag.named { background: var(--gold); color: #12100c; box-shadow: var(--glow-soft); }
|
||||
.tag.desk { background: rgba(12, 11, 9, 0.8); color: var(--gold); border: 1px solid var(--gold-dim); }
|
||||
.tag.src { background: rgba(12, 11, 9, 0.8); color: var(--muted); border: 1px solid var(--line); font-weight: 400; }
|
||||
.lead h1 {
|
||||
margin: 16px 18px 6px;
|
||||
font-family: var(--font-display);
|
||||
font-weight: 800;
|
||||
font-size: clamp(21px, 3.4vw, 30px);
|
||||
line-height: 1.18;
|
||||
letter-spacing: 0.005em;
|
||||
color: var(--text);
|
||||
text-transform: none;
|
||||
}
|
||||
.lead .dek { margin: 0 18px 14px; color: var(--muted); font-size: 17.5px; font-weight: 500; }
|
||||
.lead .meta { padding: 0 18px 16px; font-family: var(--font-mono); font-size: 11.5px; color: var(--faint); letter-spacing: 0.06em; }
|
||||
.lead .meta .ts { color: var(--neon); }
|
||||
|
||||
/* story rows */
|
||||
.story {
|
||||
display: grid;
|
||||
grid-template-columns: 96px 1fr;
|
||||
gap: 12px;
|
||||
padding: 12px 14px;
|
||||
border-bottom: 1px solid var(--line-soft);
|
||||
cursor: pointer;
|
||||
transition: background 0.15s ease;
|
||||
}
|
||||
.story:last-child { border-bottom: none; }
|
||||
.story:hover { background: var(--panel-2); }
|
||||
.story .thumb {
|
||||
width: 96px; height: 72px;
|
||||
border-radius: 5px;
|
||||
background: var(--bg-2) center/cover no-repeat;
|
||||
border: 1px solid var(--line-soft);
|
||||
position: relative;
|
||||
}
|
||||
.story .thumb .tag { position: absolute; left: 5px; top: 5px; font-size: 9px; padding: 2px 6px; }
|
||||
.story .t { font-weight: 700; font-size: 16.5px; line-height: 1.25; color: var(--text); margin: 0 0 4px; }
|
||||
.story .s { color: var(--muted); font-size: 14px; line-height: 1.35; margin: 0 0 6px; }
|
||||
.story .m { font-family: var(--font-mono); font-size: 10.5px; color: var(--faint); letter-spacing: 0.05em; }
|
||||
.story .m b { color: var(--gold); font-weight: 700; }
|
||||
@media (max-width: 560px) {
|
||||
.story { grid-template-columns: 1fr; }
|
||||
.story .thumb { width: 100%; height: 150px; }
|
||||
}
|
||||
|
||||
/* desk grid */
|
||||
.deskgrid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(215px, 1fr));
|
||||
gap: 12px;
|
||||
}
|
||||
.deskcard {
|
||||
border: 1px solid var(--line);
|
||||
background: var(--panel);
|
||||
border-radius: 8px;
|
||||
padding: 12px 14px;
|
||||
cursor: pointer;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
transition: border-color 0.15s ease, transform 0.15s ease;
|
||||
}
|
||||
.deskcard:hover { border-color: var(--gold-dim); transform: translateY(-2px); }
|
||||
.deskcard .dh {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 10.5px;
|
||||
letter-spacing: 0.16em;
|
||||
text-transform: uppercase;
|
||||
color: var(--gold);
|
||||
font-weight: 700;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 7px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.deskcard .dh .n { margin-left: auto; color: var(--faint); letter-spacing: 0; }
|
||||
.deskcard .dh .pix {
|
||||
width: 8px; height: 8px;
|
||||
background: var(--neon);
|
||||
box-shadow: 0 0 7px var(--neon);
|
||||
clip-path: polygon(50% 0, 100% 50%, 50% 100%, 0 50%);
|
||||
}
|
||||
.deskcard .dt { font-weight: 700; font-size: 15px; line-height: 1.3; color: var(--text); margin: 0; }
|
||||
.deskcard .dm { font-family: var(--font-mono); font-size: 10.5px; color: var(--faint); margin-top: 8px; letter-spacing: 0.05em; }
|
||||
.deskcard .db { color: var(--muted); font-size: 13px; margin-top: 2px; }
|
||||
|
||||
/* rail cards */
|
||||
.railcard { margin-bottom: 16px; }
|
||||
.board-teaser .post { padding: 10px 14px; border-bottom: 1px solid var(--line-soft); }
|
||||
.board-teaser .post:last-of-type { border-bottom: none; }
|
||||
.board-teaser .nick { font-family: var(--font-mono); font-size: 12px; color: var(--neon); font-weight: 700; letter-spacing: 0.05em; }
|
||||
.board-teaser .msg { color: var(--text); font-size: 14.5px; margin-top: 3px; line-height: 1.4; }
|
||||
.board-teaser .ts { font-family: var(--font-mono); font-size: 10px; color: var(--faint); margin-top: 5px; }
|
||||
.board-teaser .more { display: block; text-align: center; padding: 10px; font-family: var(--font-mono); font-size: 12px; letter-spacing: 0.14em; color: var(--gold); border-top: 1px solid var(--line-soft); }
|
||||
.board-teaser .more:hover { background: rgba(255, 212, 38, 0.06); text-decoration: none; }
|
||||
|
||||
/* ── reader drawer ────────────────────────────────────────────────────── */
|
||||
.drawer {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 100;
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
background: rgba(5, 4, 3, 0.7);
|
||||
backdrop-filter: blur(3px);
|
||||
}
|
||||
.drawer .sheet {
|
||||
width: min(720px, 100vw);
|
||||
background: var(--bg-2);
|
||||
border-left: 2px solid var(--gold-dim);
|
||||
overflow-y: auto;
|
||||
animation: slidein 0.22s ease;
|
||||
box-shadow: -18px 0 40px rgba(0, 0, 0, 0.5);
|
||||
}
|
||||
@keyframes slidein { from { transform: translateX(40px); opacity: 0; } to { transform: translateX(0); opacity: 1; } }
|
||||
.sheet .top {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 2;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
padding: 12px 16px;
|
||||
background: var(--bg-2);
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.sheet .top .k {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 10.5px;
|
||||
letter-spacing: 0.2em;
|
||||
color: var(--gold);
|
||||
}
|
||||
.sheet .close {
|
||||
margin-left: auto;
|
||||
background: var(--panel);
|
||||
color: var(--text);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 4px;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 12px;
|
||||
padding: 6px 12px;
|
||||
}
|
||||
.sheet .close:hover { border-color: var(--red); color: var(--red); }
|
||||
.sheet .img { width: 100%; max-height: 340px; object-fit: cover; border-bottom: 1px solid var(--line); }
|
||||
.sheet .body { padding: 18px 20px 40px; }
|
||||
.sheet h2 {
|
||||
font-family: var(--font-display);
|
||||
font-size: clamp(19px, 3vw, 25px);
|
||||
font-weight: 800;
|
||||
line-height: 1.22;
|
||||
margin: 0 0 10px;
|
||||
}
|
||||
.sheet .byline { font-family: var(--font-mono); font-size: 11.5px; color: var(--faint); letter-spacing: 0.06em; margin-bottom: 16px; }
|
||||
.sheet .byline b { color: var(--gold); }
|
||||
.sheet .para { font-size: 17px; line-height: 1.65; margin: 0 0 13px; color: var(--text); font-weight: 500; }
|
||||
.sheet .para.head { font-weight: 700; color: var(--gold); font-size: 16px; margin-top: 20px; letter-spacing: 0.02em; }
|
||||
.sheet .srcbtn {
|
||||
display: inline-block;
|
||||
margin-top: 8px;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 12.5px;
|
||||
letter-spacing: 0.08em;
|
||||
color: #12100c;
|
||||
background: var(--gold);
|
||||
padding: 10px 18px;
|
||||
border-radius: 4px;
|
||||
font-weight: 700;
|
||||
}
|
||||
.sheet .srcbtn:hover { background: var(--gold-2); text-decoration: none; box-shadow: var(--glow-soft); }
|
||||
.sheet .blocked {
|
||||
border: 1px dashed var(--line);
|
||||
background: var(--panel);
|
||||
padding: 18px;
|
||||
color: var(--muted);
|
||||
font-size: 15px;
|
||||
border-radius: 6px;
|
||||
}
|
||||
.sheet .loading { padding: 30px 16px; text-align: center; font-family: var(--font-mono); color: var(--faint); font-size: 13px; letter-spacing: 0.14em; }
|
||||
|
||||
/* ── board page ───────────────────────────────────────────────────────── */
|
||||
.boardhead { text-align: center; padding: 34px 16px 10px; }
|
||||
.boardhead .h {
|
||||
font-family: var(--font-display);
|
||||
font-weight: 900;
|
||||
font-size: clamp(24px, 5vw, 40px);
|
||||
color: var(--gold);
|
||||
text-shadow: var(--glow-soft);
|
||||
letter-spacing: 0.02em;
|
||||
}
|
||||
.boardhead .sub { color: var(--muted); margin-top: 8px; font-size: 16.5px; font-weight: 500; }
|
||||
.postform {
|
||||
max-width: 640px;
|
||||
margin: 22px auto 8px;
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 10px;
|
||||
padding: 18px;
|
||||
}
|
||||
.postform label {
|
||||
display: block;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 11px;
|
||||
letter-spacing: 0.16em;
|
||||
color: var(--gold);
|
||||
margin: 0 0 6px;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.postform input, .postform textarea {
|
||||
width: 100%;
|
||||
background: var(--bg-2);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
color: var(--text);
|
||||
font-size: 16px;
|
||||
padding: 11px 13px;
|
||||
margin-bottom: 14px;
|
||||
outline: none;
|
||||
font-family: var(--font-body);
|
||||
font-weight: 500;
|
||||
}
|
||||
.postform input:focus, .postform textarea:focus { border-color: var(--gold-dim); box-shadow: var(--glow-soft); }
|
||||
.postform .go {
|
||||
width: 100%;
|
||||
background: var(--gold);
|
||||
color: #12100c;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-family: var(--font-display);
|
||||
font-weight: 800;
|
||||
font-size: 15px;
|
||||
letter-spacing: 0.1em;
|
||||
padding: 13px;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.postform .go:hover { background: var(--gold-2); box-shadow: var(--glow); }
|
||||
.postform .go:disabled { opacity: 0.5; cursor: wait; }
|
||||
.postform .err { color: var(--red); font-size: 14px; margin-top: 10px; font-weight: 600; }
|
||||
.postform .okmsg { color: var(--neon); font-size: 14px; margin-top: 10px; font-weight: 600; }
|
||||
|
||||
.postlist { max-width: 640px; margin: 18px auto 60px; }
|
||||
.post {
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
padding: 13px 16px;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
.post .toprow { display: flex; align-items: baseline; gap: 10px; }
|
||||
.post .nick {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 13.5px;
|
||||
font-weight: 700;
|
||||
color: var(--neon);
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
.post .nick::before { content: "▸ "; color: var(--gold-dim); }
|
||||
.post .ts { margin-left: auto; font-family: var(--font-mono); font-size: 10.5px; color: var(--faint); letter-spacing: 0.05em; }
|
||||
.post .msg { margin-top: 6px; color: var(--text); font-size: 16px; font-weight: 500; line-height: 1.5; overflow-wrap: anywhere; }
|
||||
.post.fresh { border-color: var(--gold-dim); box-shadow: var(--glow-soft); animation: fresh 2.4s ease; }
|
||||
@keyframes fresh { from { background: rgba(255, 212, 38, 0.14); } to { background: var(--panel); } }
|
||||
.empty {
|
||||
text-align: center;
|
||||
color: var(--faint);
|
||||
font-family: var(--font-mono);
|
||||
font-size: 13px;
|
||||
letter-spacing: 0.1em;
|
||||
padding: 40px 10px;
|
||||
}
|
||||
|
||||
/* toast */
|
||||
.toast {
|
||||
position: fixed;
|
||||
bottom: 22px;
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
z-index: 200;
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--gold-dim);
|
||||
color: var(--text);
|
||||
font-family: var(--font-mono);
|
||||
font-size: 13px;
|
||||
padding: 11px 18px;
|
||||
border-radius: 6px;
|
||||
box-shadow: 0 8px 30px rgba(0, 0, 0, 0.5);
|
||||
animation: toastin 0.2s ease;
|
||||
max-width: 90vw;
|
||||
}
|
||||
@keyframes toastin { from { opacity: 0; transform: translate(-50%, 10px); } to { opacity: 1; transform: translate(-50%, 0); } }
|
||||
|
||||
/* footer */
|
||||
footer.site {
|
||||
border-top: 1px solid var(--line);
|
||||
margin-top: 30px;
|
||||
padding: 26px 16px 40px;
|
||||
text-align: center;
|
||||
color: var(--faint);
|
||||
font-size: 13.5px;
|
||||
font-weight: 500;
|
||||
}
|
||||
footer.site .fb {
|
||||
font-family: var(--font-display);
|
||||
font-weight: 800;
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
letter-spacing: 0.18em;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
footer.site .fb .bam { color: var(--gold); }
|
||||
footer.site .disc { max-width: 640px; margin: 0 auto 10px; line-height: 1.5; }
|
||||
|
||||
/* skeleton shimmer while the edition loads */
|
||||
.sk {
|
||||
background: linear-gradient(90deg, var(--panel) 25%, var(--panel-2) 50%, var(--panel) 75%);
|
||||
background-size: 200% 100%;
|
||||
animation: shimmer 1.4s infinite;
|
||||
border-radius: 6px;
|
||||
}
|
||||
@keyframes shimmer { from { background-position: 200% 0; } to { background-position: -200% 0; } }
|
||||
42
app/layout.js
Normal file
42
app/layout.js
Normal file
@ -0,0 +1,42 @@
|
||||
import "./globals.css";
|
||||
|
||||
export const metadata = {
|
||||
title: "BidaAngMamamayan — BIDA ANG MAMAMAYAN",
|
||||
description:
|
||||
"A tribute news desk for Senator BAM Aquino — the gamer senator, champion of college students and education. Every story that names him, crawled live from the Philippine wire.",
|
||||
keywords:
|
||||
"Bam Aquino, Bida ang Mamamayan, senator, education, college, esports, gaming, Philippines Senate, supporter tribute",
|
||||
openGraph: {
|
||||
title: "BidaAngMamamayan — BIDA ANG MAMAMAYAN",
|
||||
description:
|
||||
"A supporter tribute news desk for Senator BAM Aquino: the latest news on the gamer senator and his fight for college students and education.",
|
||||
siteName: "BidaAngMamamayan",
|
||||
type: "website",
|
||||
},
|
||||
};
|
||||
|
||||
export const viewport = {
|
||||
width: "device-width",
|
||||
initialScale: 1,
|
||||
themeColor: "#0c0b09",
|
||||
};
|
||||
|
||||
export default function RootLayout({ children }) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<head>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossOrigin="anonymous" />
|
||||
<link
|
||||
href="https://fonts.googleapis.com/css2?family=Orbitron:wght@600;800;900&family=Rajdhani:wght@400;500;600;700&family=JetBrains+Mono:wght@400;700&display=swap"
|
||||
rel="stylesheet"
|
||||
/>
|
||||
<link
|
||||
rel="icon"
|
||||
href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Crect width='64' height='64' rx='12' fill='%230c0b09'/%3E%3Ctext x='32' y='45' font-family='Arial Black, sans-serif' font-size='34' font-weight='900' text-anchor='middle' fill='%23ffd426'%3EB%3C/text%3E%3Crect x='40' y='14' width='14' height='14' rx='3' fill='%232bff88'/%3E%3C/svg%3E"
|
||||
/>
|
||||
</head>
|
||||
<body>{children}</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
501
app/page.js
Normal file
501
app/page.js
Normal file
@ -0,0 +1,501 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { CATEGORIES, sectionName } from "../lib/categories.js";
|
||||
import { clipText, dek } from "../lib/text.js";
|
||||
|
||||
// BidaAngMamamayan — a PUBLIC tribute news desk for Senator BAM Aquino.
|
||||
// No login, no accounts, no "I like this" — the site's whole identity is the
|
||||
// subject. Views: front, each desk, and the community board. The reader is a
|
||||
// side drawer over the current view.
|
||||
|
||||
const DAY_NAMES = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
|
||||
const MONTHS = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
|
||||
|
||||
function fmtDate(d) {
|
||||
return `${DAY_NAMES[d.getDay()]}, ${MONTHS[d.getMonth()]} ${d.getDate()}, ${d.getFullYear()}`;
|
||||
}
|
||||
function ago(iso) {
|
||||
if (!iso) return "";
|
||||
const h = (Date.now() - new Date(iso).getTime()) / 3600e3;
|
||||
if (!Number.isFinite(h) || h < 0) return "";
|
||||
if (h < 1) return `${Math.max(1, Math.round(h * 60))}m`;
|
||||
if (h < 24) return `${Math.round(h)}h`;
|
||||
return `${Math.round(h / 24)}d`;
|
||||
}
|
||||
function imgSrc(u) {
|
||||
if (!u) return null;
|
||||
return `/api/img?u=${encodeURIComponent(u)}`;
|
||||
}
|
||||
|
||||
// The masthead wordmark: BIDA ANG MAMAMAYAN, with the B-A-M letters carried in
|
||||
// gold (bold) as the tribute's highlight.
|
||||
function Brand() {
|
||||
return (
|
||||
<div className="brand">
|
||||
<span className="bam">B</span>ida<span className="bam">A</span>ng<span className="bam">M</span>amamayan
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function Home() {
|
||||
const [view, setView] = useState("front"); // front | <catKey> | board
|
||||
const [edition, setEdition] = useState(null);
|
||||
const [desks, setDesks] = useState([]);
|
||||
const [stats, setStats] = useState(null);
|
||||
const [boardPosts, setBoardPosts] = useState(null);
|
||||
const [boardCount, setBoardCount] = useState(0);
|
||||
const [reading, setReading] = useState(null); // story in the drawer
|
||||
const [toast, setToast] = useState("");
|
||||
const toastTimer = useRef(null);
|
||||
const [loadErr, setLoadErr] = useState("");
|
||||
|
||||
const say = useCallback((msg) => {
|
||||
setToast(msg);
|
||||
clearTimeout(toastTimer.current);
|
||||
toastTimer.current = setTimeout(() => setToast(""), 3200);
|
||||
}, []);
|
||||
|
||||
async function loadFeed() {
|
||||
try {
|
||||
const r = await fetch("/api/feed", { cache: "no-store", signal: AbortSignal.timeout(20000) });
|
||||
const j = await r.json();
|
||||
if (j.ok) {
|
||||
setEdition(j.feed || null);
|
||||
setDesks(j.desks || []);
|
||||
setStats(j.stats || null);
|
||||
setLoadErr("");
|
||||
} else setLoadErr("The desk is still warming up — check back in a minute.");
|
||||
} catch {
|
||||
setLoadErr("Could not reach the desk. The site may be mid-deploy — try again shortly.");
|
||||
}
|
||||
}
|
||||
|
||||
async function loadBoard() {
|
||||
try {
|
||||
const r = await fetch("/api/board", { cache: "no-store", signal: AbortSignal.timeout(15000) });
|
||||
const j = await r.json();
|
||||
if (j.ok) {
|
||||
setBoardPosts(j.posts || []);
|
||||
setBoardCount(j.count || 0);
|
||||
}
|
||||
} catch {
|
||||
/* board is optional chrome on the front view */
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
loadFeed();
|
||||
loadBoard();
|
||||
return () => clearTimeout(toastTimer.current);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
// Refresh the edition every 5 min so a long-open tab keeps ticking.
|
||||
useEffect(() => {
|
||||
const t = setInterval(loadFeed, 5 * 60 * 1000);
|
||||
return () => clearInterval(t);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
const items = useMemo(() => (edition ? edition.items || [] : []), [edition]);
|
||||
|
||||
const namedItems = useMemo(() => items.filter((i) => i.named).sort((a, b) => b.score - a.score), [items]);
|
||||
|
||||
const lead = useMemo(() => {
|
||||
if (!items.length) return null;
|
||||
const n = namedItems[0];
|
||||
if (n) return n;
|
||||
return items[0];
|
||||
}, [items, namedItems]);
|
||||
|
||||
const deskByKey = useMemo(() => new Map(desks.map((d) => [d.key, d])), [desks]);
|
||||
|
||||
// The view's story list: the whole edition on front; a desk's stories on a
|
||||
// desk view. The desk grid itself feeds from the pool (deskHeadlines), which
|
||||
// is a superset of the ranked edition — use whichever has more.
|
||||
const viewItems = useMemo(() => {
|
||||
if (view === "front") return items;
|
||||
if (view === "board") return [];
|
||||
const d = deskByKey.get(view);
|
||||
const fromPool = d ? d.stories || [] : [];
|
||||
const fromEdition = items.filter((i) => i.category === view);
|
||||
const map = new Map();
|
||||
for (const s of [...fromEdition, ...fromPool]) map.set(s.url, s);
|
||||
return [...map.values()].sort((a, b) => b.score - a.score || new Date(b.published || 0) - new Date(a.published || 0));
|
||||
}, [view, items, deskByKey]);
|
||||
|
||||
function openStory(s) {
|
||||
if (!s) return;
|
||||
setReading({ story: s, state: "loading", article: null, realUrl: s.url });
|
||||
}
|
||||
function closeReading() {
|
||||
setReading(null);
|
||||
}
|
||||
|
||||
// ── ticker: named-BAM headlines scroll, pause on hover ─────────────────
|
||||
const tickerItems = useMemo(() => namedItems.slice(0, 12), [namedItems]);
|
||||
|
||||
// ── board posting ──────────────────────────────────────────────────────
|
||||
const [nick, setNick] = useState("");
|
||||
const [msg, setMsg] = useState("");
|
||||
const [posting, setPosting] = useState(false);
|
||||
const [postErr, setPostErr] = useState("");
|
||||
const [freshId, setFreshId] = useState(null);
|
||||
|
||||
async function doPost(e) {
|
||||
e.preventDefault();
|
||||
setPostErr("");
|
||||
setPosting(true);
|
||||
try {
|
||||
const r = await fetch("/api/board", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ nick, message: msg }),
|
||||
signal: AbortSignal.timeout(15000),
|
||||
});
|
||||
const j = await r.json();
|
||||
if (j.ok && j.post) {
|
||||
setBoardPosts((p) => (p ? [j.post, ...p] : [j.post]));
|
||||
setBoardCount((c) => c + 1);
|
||||
setMsg("");
|
||||
setFreshId(j.post.id);
|
||||
say("Posted. Thank you — the board is full of you.");
|
||||
} else {
|
||||
setPostErr(j.error || "Could not post right now.");
|
||||
}
|
||||
} catch {
|
||||
setPostErr("Network hiccup — try again.");
|
||||
} finally {
|
||||
setPosting(false);
|
||||
}
|
||||
}
|
||||
|
||||
const deskLabel = view === "front" ? "Front Desk" : view === "board" ? "The Board" : sectionName(view);
|
||||
|
||||
return (
|
||||
<>
|
||||
<header className="masthead">
|
||||
<Brand />
|
||||
<div className="slogan">Bida ang Mamamayan</div>
|
||||
<div className="who">
|
||||
The supporter tribute desk for <b>Senator Bam Aquino</b> — the gamer of the Philippine
|
||||
Senate, champion of college students, scholarships, and the people's power bill. Every
|
||||
story that names him, crawled live from the wire.
|
||||
</div>
|
||||
<div className="disclaimer">
|
||||
<b>DISCLAIMER</b> — BidaAngMamamayan is built and run by <b>supporters</b> of Senator Bam
|
||||
Aquino. It is <b>not affiliated with, endorsed by, or operated by</b> the senator or his
|
||||
office. News items belong to their original publishers; this site aggregates them for
|
||||
tribute purposes only.
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<nav className="navstrip">
|
||||
<a className={`navbtn ${view === "front" ? "active" : ""}`} href="#front" onClick={(e) => { e.preventDefault(); setView("front"); window.scrollTo({ top: 0 }); }}>
|
||||
<span className="dot" />Front
|
||||
</a>
|
||||
{CATEGORIES.filter((c) => c.key !== "bam").map((c) => (
|
||||
<a key={c.key} className={`navbtn ${view === c.key ? "active" : ""}`} href={`#${c.key}`} onClick={(e) => { e.preventDefault(); setView(c.key); window.scrollTo({ top: 0 }); }}>
|
||||
{c.label}
|
||||
</a>
|
||||
))}
|
||||
<a className={`navbtn ${view === "board" ? "active" : ""}`} href="#board" onClick={(e) => { e.preventDefault(); setView("board"); window.scrollTo({ top: 0 }); }}>
|
||||
{boardCount > 0 && <span className="dot" />}Thank You Board
|
||||
</a>
|
||||
</nav>
|
||||
|
||||
{stats && (
|
||||
<div className="hud">
|
||||
<div className="chip"><span className="live" />LIVE DESK</div>
|
||||
<div className="chip">STORIES <b>{edition ? edition.total : "—"}</b></div>
|
||||
<div className="chip">LAST 24H <b>{stats.today ?? "—"}</b></div>
|
||||
<div className="chip">NAMED BAM <b>{edition ? edition.named : "—"}</b></div>
|
||||
<div className="chip">BOARD <b>{boardCount}</b></div>
|
||||
<div className="chip">EDITION <b>{fmtDate(new Date())}</b></div>
|
||||
</div>
|
||||
)}
|
||||
{edition && (
|
||||
<div className="xpbar">
|
||||
<span>EDITION STRENGTH — {edition.total} WIRE STORIES LOADED</span>
|
||||
<div className="track"><div className="fill" style={{ width: `${Math.min(100, Math.round((edition.total / 60) * 100))}%` }} /></div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{tickerItems.length > 1 && (
|
||||
<div className="ticker">
|
||||
<span className="label">BAM WIRE</span>
|
||||
<div className="rail">
|
||||
{[...tickerItems, ...tickerItems].map((s, i) => (
|
||||
<span className="item" key={i}>
|
||||
<b>{ago(s.published) || s.source}</b>
|
||||
<a href={s.url} target="_blank" rel="noopener noreferrer" onClick={() => openStory(s)}>{s.title}</a>
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<main className="wrap">
|
||||
{loadErr && !edition && (
|
||||
<div className="panel" style={{ padding: 40, textAlign: "center" }}>
|
||||
<div className="empty">{loadErr}</div>
|
||||
<button className="srcbtn" onClick={loadFeed}>RETRY</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{view === "front" && (
|
||||
<>
|
||||
<div className="cols">
|
||||
<div>
|
||||
{!edition && (
|
||||
<>
|
||||
<div className="sk" style={{ aspectRatio: "16/8.5", marginBottom: 12 }} />
|
||||
<div className="sk" style={{ height: 72, marginBottom: 8 }} />
|
||||
<div className="sk" style={{ height: 72, marginBottom: 8 }} />
|
||||
<div className="sk" style={{ height: 72 }} />
|
||||
</>
|
||||
)}
|
||||
{lead && (
|
||||
<article className="panel lead" onClick={() => openStory(lead)}>
|
||||
<div className="hero" style={lead.image ? { backgroundImage: `url(${imgSrc(lead.image)})` } : null}>
|
||||
<div className="ph-tag">
|
||||
{lead.named && <span className="tag named">★ NAMED BAM</span>}
|
||||
<span className="tag desk">{sectionName(lead.category)}</span>
|
||||
<span className="tag src">{lead.source}</span>
|
||||
</div>
|
||||
</div>
|
||||
<h1>{lead.title}</h1>
|
||||
{lead.summary && <div className="dek">{dek(lead.summary, 220)}</div>}
|
||||
<div className="meta"><span className="ts">{ago(lead.published)}</span> · {lead.desk || sectionName(lead.category)} · READ IN THE DESK →</div>
|
||||
</article>
|
||||
)}
|
||||
|
||||
<div className="panel" style={{ marginTop: 16 }}>
|
||||
<div className="ph">On the wire <span className="count">{viewItems.length} stories</span></div>
|
||||
{viewItems.slice(0, 14).map((s) => (
|
||||
<StoryRow key={s.url} s={s} onOpen={() => openStory(s)} />
|
||||
))}
|
||||
{viewItems.length === 0 && <div className="empty">The wire is quiet — the next crawl tick is on the way.</div>}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<div className="panel railcard">
|
||||
<div className="ph">In this edition <span className="count">{desks.length} desks</span></div>
|
||||
<div className="deskgrid" style={{ padding: 12 }}>
|
||||
{desks.map((d) => (
|
||||
<div key={d.key} className="deskcard" onClick={() => { setView(d.key); window.scrollTo({ top: 0 }); }}>
|
||||
<div className="dh"><span className="pix" />{d.label}<span className="n">{d.count}</span></div>
|
||||
<div className="dt">{clipText(d.stories[0].title, 90)}</div>
|
||||
<div className="dm">{ago(d.stories[0].published)} · {d.stories[0].source}</div>
|
||||
</div>
|
||||
))}
|
||||
{desks.length === 0 && <div className="empty" style={{ gridColumn: "1/-1" }}>Desks staff as the wire fills up.</div>}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="panel railcard board-teaser">
|
||||
<div className="ph">Thank you, Senator BAM <span className="count">{boardCount} posted</span></div>
|
||||
{(boardPosts || []).slice(0, 4).map((p) => (
|
||||
<div className="post" key={p.id}>
|
||||
<div className="nick">{p.nick}</div>
|
||||
<div className="msg">{clipText(p.message, 120)}</div>
|
||||
<div className="ts">{ago(p.createdAt)}</div>
|
||||
</div>
|
||||
))}
|
||||
{(boardPosts || []).length === 0 && <div className="empty">No messages yet — be the first to say thank you.</div>}
|
||||
<a className="more" href="#board" onClick={(e) => { e.preventDefault(); setView("board"); window.scrollTo({ top: 0 }); }}>
|
||||
SIGN THE BOARD →
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<div className="panel railcard">
|
||||
<div className="ph">Who he is</div>
|
||||
<div style={{ padding: 14, color: "var(--muted)", fontSize: 15, fontWeight: 500, lineHeight: 1.5 }}>
|
||||
<b style={{ color: "var(--text)" }}>Francis "Bam" Tan Aquino IV</b> — Senator for
|
||||
Pampanga. The first gamer to reach the Philippine Senate: a runner-up in the
|
||||
2025 <b style={{ color: "var(--gold)" }}>PalaPinas</b> esports championship, and
|
||||
a legislator who has made the fight for <b style={{ color: "var(--gold)" }}>college
|
||||
students, scholarships, and fair electricity</b> his signature. This desk follows
|
||||
the wire for every story that bears his name.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{view !== "front" && view !== "board" && (
|
||||
<div className="panel">
|
||||
<div className="ph">{deskLabel} <span className="count">{viewItems.length} stories</span></div>
|
||||
{viewItems.map((s) => (
|
||||
<StoryRow key={s.url} s={s} onOpen={() => openStory(s)} />
|
||||
))}
|
||||
{viewItems.length === 0 && (
|
||||
<div className="empty">
|
||||
Nothing on this desk right now — the wire is a slow burn for a single senator. The
|
||||
next crawl tick (every 15 min) may staff it.
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{view === "board" && (
|
||||
<div>
|
||||
<div className="boardhead">
|
||||
<div className="h">Thank you, Senator BAM</div>
|
||||
<div className="sub">
|
||||
The community board. No account, no email — just your nickname and a word for the
|
||||
senator. Plain words only: no links, no markup.
|
||||
</div>
|
||||
</div>
|
||||
<form className="postform" onSubmit={doPost}>
|
||||
<label htmlFor="nick">Your nickname</label>
|
||||
<input id="nick" value={nick} onChange={(e) => setNick(e.target.value)} maxLength={24} placeholder="e.g. PampangaProud, GMAmateur, BamGang13" required />
|
||||
<label htmlFor="msg">Your message</label>
|
||||
<textarea id="msg" value={msg} onChange={(e) => setMsg(e.target.value)} maxLength={600} rows={4} placeholder="Thank you, Senator BAM — ..." required />
|
||||
<button className="go" type="submit" disabled={posting || !nick.trim() || !msg.trim()}>
|
||||
{posting ? "Sending..." : "Post to the board"}
|
||||
</button>
|
||||
{postErr && <div className="err">{postErr}</div>}
|
||||
</form>
|
||||
<div className="postlist">
|
||||
{(boardPosts || []).map((p) => (
|
||||
<div key={p.id} className={`post ${p.id === freshId ? "fresh" : ""}`}>
|
||||
<div className="toprow">
|
||||
<span className="nick">{p.nick}</span>
|
||||
<span className="ts">{ago(p.createdAt)}</span>
|
||||
</div>
|
||||
<div className="msg">{p.message}</div>
|
||||
</div>
|
||||
))}
|
||||
{boardPosts && boardPosts.length === 0 && (
|
||||
<div className="empty">The board is empty. Say the first thank you.</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</main>
|
||||
|
||||
<footer className="site">
|
||||
<div className="fb">
|
||||
<span className="bam">B</span>ida<span className="bam">A</span>ng<span className="bam">M</span>amamayan
|
||||
</div>
|
||||
<div className="disc">
|
||||
A fan-made tribute run by supporters of Senator Bam Aquino. <b>Not affiliated with,
|
||||
endorsed by, or operated by</b> the senator or his office. All news content belongs to its
|
||||
original publishers.
|
||||
</div>
|
||||
<div>
|
||||
Bida ang Mamamayan · built with love (and one very loud speaker) · {fmtDate(new Date()).split(",")[1].trim()}
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
{reading && (
|
||||
<Reader
|
||||
state={reading}
|
||||
onClose={closeReading}
|
||||
onLoaded={(article, realUrl) => setReading((r) => (r ? { ...r, article, realUrl, state: "ready" } : r))}
|
||||
onToast={say}
|
||||
/>
|
||||
)}
|
||||
{toast && <div className="toast">{toast}</div>}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function StoryRow({ s, onOpen }) {
|
||||
return (
|
||||
<div className="story" onClick={onOpen}>
|
||||
<div className="thumb" style={s.image ? { backgroundImage: `url(${imgSrc(s.image)})` } : null}>
|
||||
{s.named && <span className="tag named">★</span>}
|
||||
</div>
|
||||
<div>
|
||||
<div className="t">{clipText(s.title, 120)}</div>
|
||||
{s.summary && <div className="s">{clipText(s.summary, 110)}</div>}
|
||||
<div className="m"><b>{s.source}</b> · {ago(s.published)} · {sectionName(s.category)}</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// The reader drawer: pulls the article body from /api/article (Mongo cache
|
||||
// first, publisher fetch on a miss — warmed by the crawl tick).
|
||||
function Reader({ state, onClose, onLoaded, onToast }) {
|
||||
const { story, article, realUrl } = state;
|
||||
const [failed, setFailed] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
let alive = true;
|
||||
(async () => {
|
||||
try {
|
||||
const r = await fetch(`/api/article?url=${encodeURIComponent(story.url)}`, {
|
||||
cache: "no-store",
|
||||
signal: AbortSignal.timeout(60000),
|
||||
});
|
||||
const j = await r.json();
|
||||
if (!alive) return;
|
||||
if (j.ok) onLoaded(j.article, j.realUrl);
|
||||
else { setFailed(true); onToast("Article unavailable — read at the source instead."); }
|
||||
} catch {
|
||||
if (alive) { setFailed(true); onToast("Could not load the article."); }
|
||||
}
|
||||
})();
|
||||
return () => { alive = false; };
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [story.url]);
|
||||
|
||||
// Close on ESC
|
||||
useEffect(() => {
|
||||
function k(e) { if (e.key === "Escape") onClose(); }
|
||||
window.addEventListener("keydown", k);
|
||||
return () => window.removeEventListener("keydown", k);
|
||||
}, [onClose]);
|
||||
|
||||
return (
|
||||
<div className="drawer" onClick={(e) => { if (e.target.classList.contains("drawer")) onClose(); }}>
|
||||
<div className="sheet">
|
||||
<div className="top">
|
||||
<span className="k">READING — {story.source?.toUpperCase()}</span>
|
||||
<button className="close" onClick={onClose}>ESC ✕</button>
|
||||
</div>
|
||||
{story.image && !failed && <img className="img" src={imgSrc(story.image)} alt="" />}
|
||||
<div className="body">
|
||||
<h2>{story.title}</h2>
|
||||
<div className="byline">
|
||||
<b>{story.source}</b> · {ago(story.published)} ago · {sectionName(story.category)}
|
||||
{story.named ? " · ★ NAMED BAM" : ""}
|
||||
</div>
|
||||
{state.state === "loading" && <div className="loading">EXTRACTING ARTICLE...</div>}
|
||||
{failed && (
|
||||
<div className="blocked">
|
||||
The publisher is blocking the extractor (paywall or bot wall). The story is still real —
|
||||
read it at the source below.
|
||||
{story.summary && <div style={{ marginTop: 12 }}>{story.summary}</div>}
|
||||
</div>
|
||||
)}
|
||||
{!failed && state.state === "ready" && article && (
|
||||
<>
|
||||
{(article.paras || []).map((p, i) =>
|
||||
p.text ? (
|
||||
<p key={i} className={`para ${p.tag && p.tag !== "p" ? "head" : ""}`}>{p.text}</p>
|
||||
) : (
|
||||
<div key={i} className="blocked" style={{ marginTop: 16 }}>
|
||||
Continue at the source — the full story lives on {story.source}.
|
||||
</div>
|
||||
)
|
||||
)}
|
||||
{(article.paras || []).length === 0 && article.metaDesc && (
|
||||
<p className="para">{article.metaDesc}</p>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
<a className="srcbtn" href={realUrl || story.url} target="_blank" rel="noopener noreferrer">
|
||||
READ AT {story.source?.toUpperCase()} ↗
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
111
app/ui.js
Normal file
111
app/ui.js
Normal file
@ -0,0 +1,111 @@
|
||||
// Shared UI atoms for the edition — ink icons only.
|
||||
// The raw ♥ codepoint renders as a broken/arrow-ish glyph on some Android fonts,
|
||||
// so the heart, cross and arrow are inline SVG paths filled with currentColor.
|
||||
//
|
||||
// Reading-pass atoms (2026-08-31): ReadStamp, ReadAtButton and Headline live here
|
||||
// because the front page, the section pages and the reader pane all draw them and
|
||||
// they must not drift apart.
|
||||
|
||||
export function HeartIcon({ size = 11 }) {
|
||||
return (
|
||||
<svg viewBox="0 0 24 24" width={size} height={size} aria-hidden="true" style={{ display: "inline", verticalAlign: "-1px" }}>
|
||||
<path d="M12 21.35l-1.45-1.32C5.4 15.36 2 12.28 2 8.5 2 5.42 4.42 3 7.5 3c1.74 0 3.41.81 4.5 2.09C13.09 3.81 14.76 3 16.5 3 19.58 3 22 5.42 22 8.5c0 3.78-3.4 6.86-8.55 11.54L12 21.35z" fill="currentColor" />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
export function CrossIcon({ size = 9 }) {
|
||||
return (
|
||||
<svg viewBox="0 0 24 24" width={size} height={size} aria-hidden="true" style={{ display: "inline", verticalAlign: "-1px" }}>
|
||||
<path d="M19 6.41L17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z" fill="currentColor" />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
// North-east arrow — the "leaves the paper" glyph. Also SVG for the same font reason.
|
||||
export function ArrowIcon({ size = 10 }) {
|
||||
return (
|
||||
<svg viewBox="0 0 24 24" width={size} height={size} aria-hidden="true" style={{ display: "inline", verticalAlign: "-1px" }}>
|
||||
<path d="M6 18L18 6M18 6H9.2M18 6v8.8" fill="none" stroke="currentColor" strokeWidth="2.6" strokeLinecap="square" />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
const imgSrc = (u) => (u ? `/api/img?u=${encodeURIComponent(u)}` : null);
|
||||
|
||||
// Photo sized by its context (.ft-sec img, .ft-brief-item img, .ft-story img).
|
||||
// No image -> ink hatching placeholder of the same box.
|
||||
export function Thumb({ story, eager = false }) {
|
||||
if (!story.image) return <span className="ft-noimg" aria-hidden="true" />;
|
||||
return (
|
||||
<img
|
||||
src={imgSrc(story.image)}
|
||||
alt=""
|
||||
loading={eager ? "eager" : "lazy"}
|
||||
referrerPolicy="no-referrer"
|
||||
onError={(e) => { e.currentTarget.style.display = "none"; }}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// Proof the story was read, drawn as a rubber stamp rather than a badge: rotated,
|
||||
// letter-spaced, double-ruled. `opened` gets a dot — weaker evidence, weaker mark.
|
||||
export function ReadStamp({ state }) {
|
||||
if (!state) return null;
|
||||
if (state.read) {
|
||||
return (
|
||||
<span
|
||||
className="ft-readstamp"
|
||||
title={state.readAt ? `Read ${new Date(state.readAt).toLocaleDateString()}` : "Read"}
|
||||
>
|
||||
READ
|
||||
</span>
|
||||
);
|
||||
}
|
||||
if (state.opened) return <span className="ft-opendot" title="Opened — not finished" aria-label="Opened, not finished" />;
|
||||
return null;
|
||||
}
|
||||
|
||||
// "Read at <source>" — the only way out of the paper, so it is a button, not a
|
||||
// whispered underlined link (Daniel, 2026-08-31). Outlined red + arrow; the one
|
||||
// control allowed to open a new tab.
|
||||
export function ReadAtButton({ story, size = "sm", as = "div", label }) {
|
||||
const Tag = as === "span" ? "span" : "div";
|
||||
return (
|
||||
<Tag className={as === "span" ? "ft-readat ft-readat-inline" : "ft-readat"}>
|
||||
<a
|
||||
className={`ft-btn ft-btn--outred${size === "lg" ? " big" : ""}`}
|
||||
href={story.url}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
title={`Open this story on ${story.source || "the publisher's site"}`}
|
||||
>
|
||||
Read at {story.source || "source"} <ArrowIcon />
|
||||
</a>
|
||||
{label && <span className="ft-readat-note">{label}</span>}
|
||||
</Tag>
|
||||
);
|
||||
}
|
||||
|
||||
// Headline as a real link that stays inside the paper: the pane, not a new tab.
|
||||
// The browser never follows it (we have no <base href>) — href keeps middle-click,
|
||||
// copy-link-address and right-click alive for people who want the URL.
|
||||
export function Headline({ story, onOpen, as = "h3", className, style, children }) {
|
||||
const Tag = as;
|
||||
return (
|
||||
<Tag className={className} style={style}>
|
||||
<a
|
||||
href={story.url}
|
||||
onClick={(e) => {
|
||||
if (e.metaKey || e.ctrlKey || e.shiftKey || e.button !== 0 || e.altKey) return;
|
||||
e.preventDefault();
|
||||
onOpen(story);
|
||||
}}
|
||||
>
|
||||
{children || story.title}
|
||||
</a>
|
||||
</Tag>
|
||||
);
|
||||
}
|
||||
|
||||
export { imgSrc };
|
||||
24
ecosystem.config.cjs
Normal file
24
ecosystem.config.cjs
Normal file
@ -0,0 +1,24 @@
|
||||
module.exports = {
|
||||
apps: [
|
||||
{
|
||||
name: "bama",
|
||||
script: "node_modules/next/dist/bin/next",
|
||||
args: "start -p 3000",
|
||||
cwd: "/var/www/bama",
|
||||
instances: 1,
|
||||
autorestart: true,
|
||||
max_memory_restart: "1G",
|
||||
env: { NODE_ENV: "production" },
|
||||
},
|
||||
{
|
||||
name: "bama-crawl",
|
||||
script: "scripts/cron-crawl.cjs",
|
||||
// Every 15 min: crawl the BAM queries + (quiet days) the associated wires,
|
||||
// warm the reader cache. No LLM anywhere in this stack.
|
||||
cron_restart: "*/15 * * * *",
|
||||
autorestart: false,
|
||||
cwd: "/var/www/bama",
|
||||
env: { NODE_ENV: "production", PORT: "3000" },
|
||||
},
|
||||
],
|
||||
};
|
||||
469
lib/article.js
Normal file
469
lib/article.js
Normal file
@ -0,0 +1,469 @@
|
||||
// Article fetcher — the reader pane's supply line.
|
||||
//
|
||||
// Goal (Daniel, 2026-08-31): reading a story must not require leaving BAM-tribute. We fetch the
|
||||
// publisher page server-side, run it through Readability to get the article body (the same
|
||||
// extraction engine Firefox's reader mode ships), and cache it on the story doc.
|
||||
//
|
||||
// Discipline:
|
||||
// * Server-only (imports fs-free DOM code). Never pulled into a client component.
|
||||
// * Returns PLAIN PARAGRAPHS, never raw HTML — the reader renders text nodes, so a publisher
|
||||
// page can't inject markup, scripts, or trackers into our paper.
|
||||
// * Budgeted at crawl time + on demand, with an in-flight lock: one URL is fetched once,
|
||||
// no matter how many tickers/cards point at it.
|
||||
// * Failure is a state, not an exception: status "blocked"/"error" is stored and the UI
|
||||
// falls back to "read at <source>" honestly. No fake article text, ever.
|
||||
|
||||
import { Readability } from "@mozilla/readability";
|
||||
import { parseHTML } from "linkedom";
|
||||
import { getCollections } from "./db.js";
|
||||
|
||||
const UA =
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36";
|
||||
|
||||
const MAX_BYTES = 2_500_000; // stop reading a page past 2.5 MB
|
||||
const MAX_PARAS = 80;
|
||||
const MAX_CHARS = 24_000; // stored body cap — a long-read stays readable, Mongo stays sane;
|
||||
// the overflow gets an honest "continue at the source" link
|
||||
const MIN_WORDS = 90; // below this Readability chewed off a shell/teaser, not an article
|
||||
const FETCH_TIMEOUT_MS = 12_000;
|
||||
const RETRY_AFTER_MS = 1000 * 60 * 60 * 12; // re-try a blocked/error page twice a day at most
|
||||
|
||||
const inflight = new Map(); // url -> promise
|
||||
|
||||
// Hosts that refuse us, with a cooling-off window. Discovered the hard way on
|
||||
// 2026-08-31: newsinfo.inquirer.net answers EVERY article with 403 (every UA and header
|
||||
// combination we tried), and because its feed sits early in the crawl order it ate 10 of
|
||||
// the 14 prefetch slots in one tick — the reader came out warm for only 4 stories, all
|
||||
// SunStar. Remembering the wall means we pay for it once per window instead of per story,
|
||||
// and it frees the budget for publishers who actually answer.
|
||||
const blockedHosts = new Map(); // host -> { until, reason }
|
||||
const BLOCK_TTL_MS = 1000 * 60 * 60 * 6;
|
||||
|
||||
function hostOf(url) {
|
||||
try { return new URL(url).hostname.toLowerCase(); } catch { return null; }
|
||||
}
|
||||
|
||||
function blockedUntil(host) {
|
||||
const e = host && blockedHosts.get(host);
|
||||
if (!e) return 0;
|
||||
if (Date.now() > e.until) { blockedHosts.delete(host); return 0; }
|
||||
return e.until;
|
||||
}
|
||||
|
||||
function noteBlock(host, reason) {
|
||||
if (!host) return;
|
||||
blockedHosts.set(host, { until: Date.now() + BLOCK_TTL_MS, reason });
|
||||
}
|
||||
|
||||
// Google News "articles/CBMi..." links are JS redirect wrappers; the publisher URL is
|
||||
// packed in the base64url article id. Same decoder lib/crawl.js uses for og:image.
|
||||
export function unwrapGoogleNews(link) {
|
||||
const m = /news\.google\.com\/(?:rss\/)?articles\/([A-Za-z0-9_-]+)/.exec(String(link || ""));
|
||||
if (!m) return link;
|
||||
try {
|
||||
const raw = Buffer.from(m[1], "base64url").toString("latin1");
|
||||
const urls = raw.match(/https?:\/\/[^\x00-\x1f\x7f"]+/g) || [];
|
||||
const real = urls.map((u) => u.replace(/[\x00-\x1f]+.*$/, "")).filter((u) => !/google|gstatic/i.test(u));
|
||||
if (real.length) return real.sort((a, b) => b.length - a.length)[0];
|
||||
} catch {
|
||||
/* keep the wrapper */
|
||||
}
|
||||
return link;
|
||||
}
|
||||
|
||||
function absolutise(src, base) {
|
||||
try {
|
||||
return new URL(src, base).href;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// Readability hands back a fragment of markup. We re-parse it and pull text out of the
|
||||
// block elements — deliberately losing all formatting so nothing hostile survives.
|
||||
function parasFrom(article, baseHref) {
|
||||
// article.textContent is the authority Readability itself computed (and the
|
||||
// same value `words` is counted from). linkedom's document.body can parse to an EMPTY
|
||||
// textContent on a re-parsed fragment — verified on scmp.com 2026-08-31: a 633-word
|
||||
// story where body.textContent was "" while 6 <p> blocks existed — so the plain-text
|
||||
// fallback below must never re-read the DOM for its source.
|
||||
const plainText = String(article?.textContent || "").replace(/\r/g, "");
|
||||
const frag = String(article?.content || "");
|
||||
if (!frag) return [];
|
||||
const { document } = parseHTML(`<body>${frag}</body>`);
|
||||
const out = [];
|
||||
let chars = 0;
|
||||
const blocks = [...document.querySelectorAll("h2, h3, h4, p, blockquote, li, pre, figcaption")];
|
||||
for (const node of blocks) {
|
||||
const tag = node.nodeName ? String(node.nodeName).toLowerCase() : "p";
|
||||
const text = String(node.textContent || "").replace(/\s+/g, " ").trim();
|
||||
if (!text || text.length < 3) continue;
|
||||
chars += text.length;
|
||||
if (chars > MAX_CHARS || out.length > MAX_PARAS) {
|
||||
out.push({ tag: "p", text: "", truncated: true });
|
||||
break;
|
||||
}
|
||||
out.push({ tag, text });
|
||||
}
|
||||
const images = [];
|
||||
for (const img of document.querySelectorAll("img")) {
|
||||
const u = absolutise(img.getAttribute("src") || img.getAttribute("data-src") || "", baseHref);
|
||||
if (u && /^https?:/i.test(u) && !/1x1|spacer|pixel|blank|logo|sprite|tracking/i.test(u)) images.push(u);
|
||||
if (images.length >= 12) break;
|
||||
}
|
||||
|
||||
// Safety net: if the block walk produced nothing usable, the copy is sitting in
|
||||
// non-block elements (a bare <div>, or <p> wrappers that were all empty). SCMP returned
|
||||
// 1107 words but 0 surviving paragraphs on 2026-08-31 — a blank pane: status ok, words
|
||||
// populated, nothing to render. This net runs whenever NO usable paragraph survived, so
|
||||
// words>0 can never again ship an empty page.
|
||||
if (!out.some((p) => p.text)) {
|
||||
// Split the Readability text into paragraphs: prefer blank-line breaks, then single
|
||||
// newlines, and as a last resort (some publishers return one flat line) break long
|
||||
// runs on sentence boundaries so the pane never shows a wall of unbroken text.
|
||||
const splitInto = (src, re) => src.split(re).map((x) => x.replace(/\s+/g, " ").trim()).filter((t) => t.length > 2);
|
||||
let chunks = splitInto(plainText, /\n{2,}/);
|
||||
if (chunks.length <= 1) chunks = splitInto(plainText, /\n/);
|
||||
if (chunks.length <= 1 && plainText.replace(/\s+/g, " ").trim().length > 300) {
|
||||
chunks = splitInto(plainText.replace(/\s+/g, " "), /(?<=[.!?])\s+(?=["A-Z\u201c])/);
|
||||
}
|
||||
out.length = 0;
|
||||
chars = 0;
|
||||
for (const t of chunks) {
|
||||
chars += t.length;
|
||||
if (chars > MAX_CHARS || out.length > MAX_PARAS) { out.push({ tag: "p", text: "", truncated: true }); break; }
|
||||
out.push({ tag: "p", text: t });
|
||||
}
|
||||
}
|
||||
return { paras: out, images };
|
||||
}
|
||||
|
||||
// Some outlets ship the article as JSON in the page and render it client-side, so
|
||||
// Readability sees a shell (ABS-CBN is the case that forced this: 60 words extracted
|
||||
// out of a real 500-word story). Pull the body out of the structured data instead.
|
||||
// Order: JSON-LD articleBody -> Next.js pageProps body_html -> any big body-ish field.
|
||||
function structuredBody(doc, html) {
|
||||
// 1. schema.org NewsArticle / articleBody
|
||||
for (const s of doc.querySelectorAll('script[type="application/ld+json"]')) {
|
||||
const raw = String(s.textContent || "").trim();
|
||||
if (!raw) continue;
|
||||
try {
|
||||
const j = JSON.parse(raw);
|
||||
const seen = [];
|
||||
const walk = (n, d) => {
|
||||
if (!n || d > 6 || seen.length > 3) return;
|
||||
if (Array.isArray(n)) { for (const x of n) walk(x, d + 1); return; }
|
||||
if (typeof n === "object") {
|
||||
for (const [k, v] of Object.entries(n)) {
|
||||
if (typeof v === "string" && /^(articlebody|articleBody|text)$/i.test(k) && v.length > 240) seen.push(v);
|
||||
else walk(v, d + 1);
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(j, 0);
|
||||
if (seen.length) return { html: seen.sort((a, b) => b.length - a.length)[0], from: "jsonld" };
|
||||
} catch { /* malformed ld+json is normal on news sites */ }
|
||||
}
|
||||
|
||||
// 2. Next.js / Nuxt payload with a body_html (EV/Superdesk "content" object)
|
||||
const nd = /<script[^>]*id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/.exec(html);
|
||||
if (nd) {
|
||||
try {
|
||||
const j = JSON.parse(nd[1]);
|
||||
const hits = [];
|
||||
const walk = (n, d) => {
|
||||
if (!n || d > 8 || hits.length > 4) return;
|
||||
if (Array.isArray(n)) { for (const x of n) walk(x, d + 1); return; }
|
||||
if (typeof n === "object") {
|
||||
for (const [k, v] of Object.entries(n)) {
|
||||
if (typeof v === "string" && /^(body_html|bodyHtml|articleBody|content_html)$/i.test(k) && v.length > 240) hits.push(v);
|
||||
else walk(v, d + 1);
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(j?.props?.pageProps ?? j, 0);
|
||||
if (hits.length) return { html: hits.sort((a, b) => b.length - a.length)[0], from: "nextdata" };
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Readability leaks photo credits at the top ("DOST-PAGASA FB PAGE", "Abhijeet Mishra /
|
||||
// SamMobile", "… | INQUIRER File Photo") and newsletter/cookie boilerplate at the tail.
|
||||
// Trim both — never reorder or rewrite what is left.
|
||||
const CREDIT = /(?:file photo|courtesy|photo by|image:|getty|facebook page|^by \p{Lu}|\p{Lu}\/(?:inquirer|rappler|cnn|gmanetwork|abs-cbn|philstar|bworld|pna)\b)/iu;
|
||||
const ALLCAPS = /^[\p{Lu}\d\s.,'’&()\/-]+$/u;
|
||||
const JUNK_TAIL = /^(?:cookie|subscribe|sign ?in|log ?in|advertisement|sponsored content|all rights reserved|read more\b|follow us|join the (?:conversation|discussion)|related stories|comments?\b|tags?\b|this article first appeared|want news alerts|email this article|print this article|also read|read next|most read|trending|popular\b|newsletter\b|share this\b)/i;
|
||||
|
||||
// Structured bodies arrive as HTML; parasFrom's fallback wants a text form of it.
|
||||
function stripTagsOf(html) {
|
||||
return String(html || "").replace(/<\/(p|div|section|article|li|h[1-6]|blockquote)>/gi, "\n\n").replace(/<[^>]*>/g, " ");
|
||||
}
|
||||
|
||||
function cleanParas(paras) {
|
||||
const list = paras.filter((p) => p && p.text);
|
||||
// Leading credit lines are always short — a genuine lede is not. That length guard is
|
||||
// what keeps "KATHMANDU, Nepal – Three months before…" (mixed case, long) intact.
|
||||
let start = 0;
|
||||
while (start < list.length - 1 && list[start].text.length < 90) {
|
||||
const t = list[start].text.trim();
|
||||
const isCredit = CREDIT.test(t) || ALLCAPS.test(t) || /^(?:updated|published|manila|quezon city|cebucity|cebuphoto)\b/i.test(t);
|
||||
if (!isCredit) break;
|
||||
start++;
|
||||
}
|
||||
let end = list.length - 1;
|
||||
while (end > start && (JUNK_TAIL.test(list[end].text.trim()) || (list[end].text.length < 24 && list[end].tag === "p"))) end--;
|
||||
return list.slice(start, end + 1);
|
||||
}
|
||||
|
||||
async function fetchArticle(url) {
|
||||
const res = await fetch(url, {
|
||||
headers: {
|
||||
"User-Agent": UA,
|
||||
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
||||
"Accept-Language": "en-PH,en;q=0.9",
|
||||
Referer: new URL(url).origin + "/",
|
||||
},
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||
});
|
||||
|
||||
// 403/451 is a bot wall, not a bug — the reader shows the honest "read at source"
|
||||
// state instead of retrying forever (newsinfo.inquirer.net hard-403s every UA we tried).
|
||||
if (!res.ok) return { status: res.status === 403 || res.status === 451 || res.status === 429 ? "blocked" : "error", reason: `upstream ${res.status}` };
|
||||
|
||||
// Read with a byte ceiling so a runaway page cannot eat the box's memory.
|
||||
const chunks = [];
|
||||
let total = 0;
|
||||
if (res.body) {
|
||||
for await (const c of res.body) {
|
||||
total += c.length || 0;
|
||||
chunks.push(c);
|
||||
if (total > MAX_BYTES) break;
|
||||
}
|
||||
} else {
|
||||
const buf = await res.arrayBuffer();
|
||||
chunks.push(new Uint8Array(buf));
|
||||
total = buf.byteLength;
|
||||
}
|
||||
const html = Buffer.concat(chunks.map((c) => Buffer.from(c))).toString("utf8");
|
||||
if (/<(script[^>]*>\s*window\.__sveltekit|js-circleci|recaptcha)/i.test(html) && total < 4000) {
|
||||
return { status: "blocked", reason: "bot shell" };
|
||||
}
|
||||
|
||||
const { window } = parseHTML(html);
|
||||
const doc = window.document;
|
||||
const canonical =
|
||||
absolutise(doc.querySelector('link[rel="canonical"]')?.getAttribute("href") || "", url) || url;
|
||||
const baseHref = canonical || url;
|
||||
|
||||
// Structured body must be read BEFORE the script strip — it lives inside <script> tags.
|
||||
// ABS-CBN and other SPA newsrooms render the copy client-side, so Readability alone
|
||||
// returns a 60-word shell while the real article sits in __NEXT_DATA__.body_html.
|
||||
const sb = structuredBody(doc, html);
|
||||
|
||||
// Strip the usual non-content noise before Readability sees it (script/style/ads/nav).
|
||||
for (const sel of ["script", "style", "noscript", "iframe", "form", "svg", "[id*=advert]", "[class*=advert]", "[class*=newsletter]", "[id*=paywall]"]) {
|
||||
for (const n of doc.querySelectorAll(sel)) n.remove?.();
|
||||
}
|
||||
|
||||
let article = null;
|
||||
try {
|
||||
article = new Readability(doc.cloneNode(true), { charThreshold: MIN_WORDS }).parse();
|
||||
} catch (e) {
|
||||
if (!sb) return { status: "error", reason: `readability: ${String(e && e.message || e).slice(0, 80)}` };
|
||||
}
|
||||
|
||||
const body = article?.textContent ? String(article.textContent).replace(/\s+/g, " ").trim() : "";
|
||||
let words = body ? body.split(" ").length : 0;
|
||||
|
||||
// Prefer whichever source actually has the story.
|
||||
let useStructured = false;
|
||||
if (sb) {
|
||||
const sbWords = String(sb.html).replace(/<[^>]*>/g, " ").replace(/\s+/g, " ").trim().split(" ").filter(Boolean).length;
|
||||
if (sbWords > words) { useStructured = true; words = sbWords; }
|
||||
}
|
||||
|
||||
if (!useStructured && (!body || words < MIN_WORDS)) {
|
||||
const metaDesc = String(doc.querySelector('meta[property="og:description"], meta[name="description"]')?.getAttribute("content") || "").trim();
|
||||
return {
|
||||
status: "blocked",
|
||||
reason: words ? `thin extract (${words}w)` : "no article body",
|
||||
metaDesc: metaDesc.slice(0, 600),
|
||||
siteName: article?.siteName || null,
|
||||
};
|
||||
}
|
||||
|
||||
const parsed = useStructured
|
||||
? parasFrom({ content: String(sb.html).replace(/<\/?(?:html|body|head)[^>]*>/gi, " "),
|
||||
textContent: stripTagsOf(sb.html) }, baseHref)
|
||||
: parasFrom(article, baseHref);
|
||||
let paras = Array.isArray(parsed) ? parsed : parsed.paras || [];
|
||||
// Readability sometimes yields a single giant <div>; fall back to split paragraphs.
|
||||
if (paras.length === 1 && paras[0].text.length > 600) {
|
||||
paras = paras[0].text.split(/\n{2,}/).map((t) => ({ tag: "p", text: t })).filter((p) => p.text.length > 2);
|
||||
}
|
||||
paras = paras.filter((p) => p.text).slice(0, MAX_PARAS);
|
||||
|
||||
// Words but no surviving paragraphs is a blank pane, not an article. Reporting it honestly
|
||||
// as "blocked" means the reader shows the "read at source" path AND the 12h retry window
|
||||
// re-attempts it — whereas status:"ok" with an empty body is cached FOREVER. That is how
|
||||
// every SCMP Tech story got stored as words:407 / paras:0 on 2026-08-31.
|
||||
if (!paras.length) {
|
||||
const metaDesc = String(doc.querySelector('meta[property="og:description"], meta[name="description"]')?.getAttribute("content") || "").trim();
|
||||
return {
|
||||
status: "blocked",
|
||||
reason: `no paragraphs (${words}w extracted)`,
|
||||
metaDesc: metaDesc.slice(0, 600),
|
||||
siteName: article?.siteName || null,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
status: "ok",
|
||||
title: (article?.title && String(article.title).trim().slice(0, 300)) || null,
|
||||
byline: (article?.byline && String(article.byline).trim().slice(0, 160)) || null,
|
||||
siteName: (article?.siteName && String(article.siteName).trim().slice(0, 80)) || null,
|
||||
excerpt: (article?.excerpt && String(article.excerpt).trim().slice(0, 600)) || null,
|
||||
published: article?.publishedTime || null,
|
||||
lang: article?.lang || null,
|
||||
canonical: baseHref,
|
||||
words,
|
||||
truncated: total > MAX_BYTES || paras.some((p) => p.truncated),
|
||||
images: (Array.isArray(parsed) ? [] : parsed.images) || [],
|
||||
paras,
|
||||
};
|
||||
}
|
||||
|
||||
// Public entry: the stored body if we have one, otherwise fetch + cache. Never throws.
|
||||
export async function getArticle({ url, force = false } = {}) {
|
||||
if (!url || !/^https?:\/\//i.test(url)) return { status: "error", reason: "bad url" };
|
||||
const real = unwrapGoogleNews(url);
|
||||
|
||||
// Known wall: return the honest blocked state immediately rather than spending another
|
||||
// 12s upstream on a page that has already refused us this hour.
|
||||
const h = hostOf(real);
|
||||
if (blockedUntil(h)) {
|
||||
const e = blockedHosts.get(h);
|
||||
return { status: "blocked", reason: e.reason || "publisher blocks us", paras: [], words: 0, images: [], hostBlocked: true };
|
||||
}
|
||||
|
||||
const { stories } = await getCollections();
|
||||
const st = await stories.findOne({ url }, { projection: { article: 1, title: 1, source: 1, summary: 1, url: 1, category: 1, image: 1, published: 1 } });
|
||||
if (!st) return { status: "error", reason: "story not found" };
|
||||
|
||||
const have = st.article;
|
||||
// An "ok" body with no paragraphs IS the blank pane, not a cache hit. Stories stored before
|
||||
// the 2026-08-31 parasFrom fix (every SCMP Tech row) look perfectly fresh and would pin a
|
||||
// white page for the life of the row, so emptiness is treated as stale and re-fetched.
|
||||
const usable = have && have.status === "ok" && Array.isArray(have.paras) && have.paras.some((p) => p && p.text);
|
||||
const fresh =
|
||||
have &&
|
||||
(usable || (have.fetchedAt && Date.now() - new Date(have.fetchedAt).getTime() < RETRY_AFTER_MS));
|
||||
if (have && have.status === "ok" && usable && fresh && !force) {
|
||||
return { ...have, story: shapeStory(st), realUrl: have.canonical || real };
|
||||
}
|
||||
|
||||
if (inflight.has(real)) {
|
||||
const r = await inflight.get(real);
|
||||
return { ...r, story: shapeStory(st), realUrl: real };
|
||||
}
|
||||
|
||||
const job = (async () => {
|
||||
try {
|
||||
const a = await fetchArticle(real);
|
||||
const doc = {
|
||||
status: a.status,
|
||||
fetchedAt: new Date(),
|
||||
reason: a.reason || null,
|
||||
...(a.status === "ok"
|
||||
? {
|
||||
title: a.title, byline: a.byline, siteName: a.siteName, excerpt: a.excerpt,
|
||||
published: a.published, lang: a.lang, canonical: a.canonical, words: a.words,
|
||||
truncated: !!a.truncated, images: a.images, paras: a.paras,
|
||||
}
|
||||
: { metaDesc: a.metaDesc || null, siteName: a.siteName || null }),
|
||||
};
|
||||
if (a.status === "blocked" && /upstream 4\d\d/i.test(a.reason || "")) noteBlock(hostOf(real), a.reason);
|
||||
else if (a.status === "ok" && h) blockedHosts.delete(h);
|
||||
await stories.updateOne({ url }, { $set: { article: doc } });
|
||||
// A page we could read deserves its picture too — backfill the card thumb.
|
||||
if (a.status === "ok" && !st.image && a.images?.[0]) {
|
||||
await stories.updateOne({ url }, { $set: { image: a.images[0] } });
|
||||
}
|
||||
return doc;
|
||||
} catch (e) {
|
||||
const msg = String((e && e.message) || e).slice(0, 120);
|
||||
const doc = { status: /timeout|abort/i.test(msg) ? "error" : "error", reason: msg, fetchedAt: new Date() };
|
||||
await stories.updateOne({ url }, { $set: { article: doc } }).catch(() => {});
|
||||
return doc;
|
||||
} finally {
|
||||
inflight.delete(real);
|
||||
}
|
||||
})();
|
||||
inflight.set(real, job);
|
||||
|
||||
const result = await job;
|
||||
return { ...result, story: shapeStory(st), realUrl: real };
|
||||
}
|
||||
|
||||
// Prefetch at crawl time: the reader must feel instant, and we must not be fetching a
|
||||
// publisher page while the reader is waiting. Bounded budget, low concurrency, and only
|
||||
// for the stories the paper actually prints.
|
||||
export async function prefetchArticles(urls, { budget = 12, concurrency = 3, deadlineMs = 35000 } = {}) {
|
||||
// Diversity cap: without it, one host at the top of the crawl order consumes the whole
|
||||
// budget (that is exactly how Inquirer took 10 of 14 slots on 2026-08-31). Two stories
|
||||
// per outlet, and outlets we already know refuse us are skipped entirely.
|
||||
const MAX_PER_HOST = 2;
|
||||
const perHost = new Map();
|
||||
const list = [];
|
||||
for (const u of new Set((urls || []).filter(Boolean))) {
|
||||
const hh = hostOf(u);
|
||||
if (blockedUntil(hh)) continue;
|
||||
const n = perHost.get(hh) || 0;
|
||||
if (n >= MAX_PER_HOST) continue;
|
||||
perHost.set(hh, n + 1);
|
||||
list.push(u);
|
||||
if (list.length >= budget) break;
|
||||
}
|
||||
if (!list.length) return { fetched: 0, ok: 0 };
|
||||
// A hard wall clock, not just a per-item count: the digest trigger aborts the whole
|
||||
// /api/crawl request at 160s, and a slow newsroom (BusinessWorld has been measured at
|
||||
// 10s for one page) could otherwise push the AI pass off the end of the tick and freeze
|
||||
// the paper at last hour's front page. Warm bodies are a luxury; the edition is not.
|
||||
const deadline = Date.now() + deadlineMs;
|
||||
const { stories } = await getCollections();
|
||||
let fetched = 0;
|
||||
let ok = 0;
|
||||
let i = 0;
|
||||
const workers = Array.from({ length: Math.min(concurrency, list.length) }, async () => {
|
||||
while (i < list.length) {
|
||||
if (Date.now() > deadline) break;
|
||||
const url = list[i++];
|
||||
try {
|
||||
const existing = await stories.findOne({ url }, { projection: { "article.status": 1 } });
|
||||
if (existing?.article?.status === "ok") continue;
|
||||
const a = await getArticle({ url });
|
||||
fetched++;
|
||||
if (a.status === "ok") ok++;
|
||||
} catch {
|
||||
/* best effort — a failed prefetch only means the reader fetches on demand */
|
||||
}
|
||||
}
|
||||
});
|
||||
await Promise.all(workers);
|
||||
return { fetched, ok };
|
||||
}
|
||||
|
||||
// What the reader pane needs to draw its own masthead even when extraction failed.
|
||||
function shapeStory(st) {
|
||||
return {
|
||||
url: st.url,
|
||||
title: st.title,
|
||||
source: st.source,
|
||||
category: st.category,
|
||||
summary: st.summary,
|
||||
image: st.image || null,
|
||||
published: st.published,
|
||||
};
|
||||
}
|
||||
131
lib/board.js
Normal file
131
lib/board.js
Normal file
@ -0,0 +1,131 @@
|
||||
// The community board — "Thank you Senator BAM".
|
||||
//
|
||||
// Visitors post with a nickname only (no accounts, no email, by design —
|
||||
// Daniel: the site is public and the barrier to saying thanks must be a
|
||||
// text field, not a login). The board is an open guestbook: the abuse
|
||||
// controls are deliberately light but real:
|
||||
//
|
||||
// * per-IP rate limit: 3 posts / 10 min, 12 / day (lib/board.js, Mongo
|
||||
// rate collection, no Redis needed).
|
||||
// * hard content caps (nickname 24 chars, message 600 chars) so a drive-by
|
||||
// can't park a wall of text.
|
||||
// * message sanitisation: HTML is stripped to text. The UI renders plain
|
||||
// text nodes only — there is no markdown, no links, no images. A post
|
||||
// that is ONLY a URL is rejected (the classic guestbook spam shape).
|
||||
// * nothing is executed, fetched, or rendered as markup. Ever.
|
||||
//
|
||||
// The default title of the board is "Thank you Senator BAM" and every post
|
||||
// is a thank-you / message to the senator — that is the page's social
|
||||
// contract, stated in the copy, enforced lightly (tone is not machine-
|
||||
// judged).
|
||||
|
||||
import { getCollections } from "./db.js";
|
||||
|
||||
const NICK_MAX = 24;
|
||||
const MSG_MAX = 600;
|
||||
const RATE_WINDOW_MS = 10 * 60 * 1000;
|
||||
const RATE_LIMIT_WINDOW = 3; // per 10 min
|
||||
const RATE_LIMIT_DAY = 12; // per 24h
|
||||
const LIST_LIMIT = 200; // the board prints newest-first, capped
|
||||
|
||||
function cleanNick(s) {
|
||||
return String(s || "")
|
||||
.replace(/<[^>]*>/g, " ")
|
||||
.replace(/[\x00-\x1f]/g, "")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim()
|
||||
.slice(0, NICK_MAX);
|
||||
}
|
||||
function cleanMsg(s) {
|
||||
// Strip markup/controls, collapse whitespace, cap length.
|
||||
const t = String(s || "")
|
||||
.replace(/<[^>]*>/g, " ")
|
||||
.replace(/[\x00-\x1f]/g, "")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
return t.slice(0, MSG_MAX);
|
||||
}
|
||||
// A message that is one URL (or a URL plus a couple of words) is spam-shaped.
|
||||
// \b(?<=^|\s)https?:// = a URL at the start or after whitespace — the classic
|
||||
// guestbook spam shape. A URL embedded in a real sentence ("read more at
|
||||
// https://...") is left alone; plain words remain the rule, not the exception.
|
||||
function looksLikeUrlOnly(t) {
|
||||
const withUrl = /(^|\s)(https?:\/\/|www\.)\S+/i.test(t);
|
||||
if (!withUrl) return false;
|
||||
const stripped = t.replace(/(^|\s)(https?:\/\/|www\.)\S+/gi, " ").replace(/\s+/g, " ").trim();
|
||||
return stripped.length < 12;
|
||||
}
|
||||
|
||||
function ipOf(req) {
|
||||
const fwd = req.headers.get("x-forwarded-for");
|
||||
if (fwd) return fwd.split(",")[0].trim().slice(0, 64);
|
||||
return req.headers.get("x-real-ip") || "unknown";
|
||||
}
|
||||
|
||||
export async function getBoard({ limit = LIST_LIMIT } = {}) {
|
||||
const { board } = await getCollections();
|
||||
const rows = await board
|
||||
.find({})
|
||||
.sort({ createdAt: -1 })
|
||||
.limit(Math.min(Math.max(1, limit), LIST_LIMIT))
|
||||
.toArray();
|
||||
return rows.map((r) => ({
|
||||
id: String(r._id),
|
||||
nick: r.nick,
|
||||
message: r.message,
|
||||
createdAt: r.createdAt,
|
||||
}));
|
||||
}
|
||||
|
||||
export async function postBoard(req, body) {
|
||||
const ip = ipOf(req);
|
||||
const now = Date.now();
|
||||
const { rate, board } = await getCollections();
|
||||
|
||||
// Rate limit: count this IP's posts in the 10-min and 24h windows.
|
||||
// createdAt is stored as a BSON Date, so the comparison MUST be a Date too —
|
||||
// a numeric epoch silently matches nothing (verified: the counter stayed 0).
|
||||
const dayAgo = now - 24 * 3600e3;
|
||||
const [inWindow, inDay] = await Promise.all([
|
||||
rate.countDocuments({ ip, createdAt: { $gte: new Date(now - RATE_WINDOW_MS) } }),
|
||||
rate.countDocuments({ ip, createdAt: { $gte: new Date(dayAgo) } }),
|
||||
]);
|
||||
if (inWindow >= RATE_LIMIT_WINDOW) {
|
||||
return { ok: false, error: "Slow down — a few thanks at a time. Try again in a few minutes.", status: 429 };
|
||||
}
|
||||
if (inDay >= RATE_LIMIT_DAY) {
|
||||
return { ok: false, error: "That's plenty of thanks for today. Come back tomorrow.", status: 429 };
|
||||
}
|
||||
|
||||
const nick = cleanNick(body && body.nick);
|
||||
const message = cleanMsg(body && body.message);
|
||||
if (nick.length < 2) return { ok: false, error: "Give us a nickname (2+ characters) so we know who to thank.", status: 400 };
|
||||
if (message.length < 3) return { ok: false, error: "Your message is empty.", status: 400 };
|
||||
if (looksLikeUrlOnly(message)) return { ok: false, error: "Plain words only, please — no links.", status: 400 };
|
||||
|
||||
const doc = {
|
||||
nick,
|
||||
message,
|
||||
ip: ip.slice(0, 64),
|
||||
createdAt: new Date(now),
|
||||
};
|
||||
const res = await board.insertOne(doc);
|
||||
// Bump the rate row (upsert keeps one doc per ip+day, but we just count
|
||||
// recent createdAt values, so insert a lightweight marker per post).
|
||||
await rate.insertOne({ ip, createdAt: new Date(now) });
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
post: { id: String(res.insertedId), nick, message, createdAt: doc.createdAt },
|
||||
};
|
||||
}
|
||||
|
||||
// A friendly, non-authoritative total for the masthead.
|
||||
export async function boardCount() {
|
||||
try {
|
||||
const { board } = await getCollections();
|
||||
return await board.countDocuments({});
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
51
lib/categories.js
Normal file
51
lib/categories.js
Normal file
@ -0,0 +1,51 @@
|
||||
// BidaAngMamamayan desk registry — single source of truth for categories.
|
||||
//
|
||||
// A tribute site has fewer desks than a general paper, but each one is
|
||||
// deliberate: the "BAM" lead desk, his signature beats (education/college,
|
||||
// energy/electricity, gaming/esports, the Senate floor), and the PH + world
|
||||
// context he works inside.
|
||||
|
||||
export const CATEGORIES = [
|
||||
{ key: "bam", label: "BAM Front", blurb: "Every story that names him" },
|
||||
{ key: "education", label: "Education", blurb: "College, scholarship, the classroom" },
|
||||
{ key: "energy", label: "Energy & Electricity", blurb: "The power bill fight" },
|
||||
{ key: "gaming", label: "Gamer's Corner", blurb: "Esports, gaming, the 2025 runner-up" },
|
||||
{ key: "politics", label: "The Senate Floor", blurb: "Bills, sessions, the chamber" },
|
||||
{ key: "ph", label: "The Philippines", blurb: "Home context, region by region" },
|
||||
{ key: "world", label: "World", blurb: "The rest of the wire" },
|
||||
];
|
||||
|
||||
export const CATEGORY_KEYS = new Set(CATEGORIES.map((c) => c.key));
|
||||
export const CATEGORY_LABELS = Object.fromEntries(CATEGORIES.map((c) => [c.key, c.label]));
|
||||
|
||||
// Anything that isn't a valid desk key falls back to the lead desk. The gate
|
||||
// (lib/crawl.js) already decided the story belongs on this site, so "bam" is
|
||||
// the honest default, not "world".
|
||||
export function normalizeCategory(c) {
|
||||
if (typeof c === "string" && CATEGORY_KEYS.has(c)) return c;
|
||||
return "bam";
|
||||
}
|
||||
|
||||
export function sectionName(c) {
|
||||
return CATEGORY_LABELS[normalizeCategory(c)] || "BAM Front";
|
||||
}
|
||||
|
||||
// The relevance gate lives in lib/crawl.js (it needs the description text);
|
||||
// this is just the desk classifier for rows that pass the gate but arrive
|
||||
// without a query tag.
|
||||
const CLASSIFIERS = [
|
||||
["gaming", /\besports?\b|\bgaming\b|\bvideo games?\b|\bsteam\b|\bplaystation\b|\bnintendo\b|\bxbox\b|\bconsole\b|\bgame (studio|developer|release)\b|\briot\b|\bvalorant\b|\blol\b|\bdota\b/i],
|
||||
["education", /\beducation\b|\bschool(s|ing)?\b|\bcollege\b|\buniversity\b|\bscholarship\b|\bstudents?\b|\bclassroom\b|\bteachers?\b|\blearning\b|\bched\b|\bdeped\b/i],
|
||||
["energy", /\bpower (outage|bill|rates?|grid)\b|\belectricity\b|\bbrownout\b|\bblackout\b|\bvat.*\bpower\b|\benergy\b|\bsystem loss\b|\belectric rate\b/i],
|
||||
["politics", /\bsenate\b|\bbill\b|\bcongress\b|\bpresidenc?\b|\belection\b|\bparty\b|\bpolitical\b|\bimpeach\b|\bcommitte\w*\b/i],
|
||||
["ph", /\bphilippines?\b|\bmanila\b|\bmarikina\b|\bquezon city\b|\bpampanga\b|\bc[eé]bu city\b|\bdavao\b|\bbaguio\b|\bpalawan\b|\bmindanao\b|\bvisayas\b|\bluzon\b|\bcalabarzon\b|\bbarangay\b|\bsenior citizen\b|\bprovincial\b|\bpeso\b/i],
|
||||
];
|
||||
|
||||
export function classifyStory({ title, summary, source } = {}) {
|
||||
const text = `${title || ""} ${summary || ""} ${source || ""}`;
|
||||
if (!text.trim()) return "bam";
|
||||
for (const [key, re] of CLASSIFIERS) {
|
||||
if (re.test(text)) return key;
|
||||
}
|
||||
return "bam";
|
||||
}
|
||||
373
lib/crawl.js
Normal file
373
lib/crawl.js
Normal file
@ -0,0 +1,373 @@
|
||||
// BidaAngMamamayan crawler — the tribute edition's supply line.
|
||||
//
|
||||
// Forked from forthena's lib/crawl.js, retargeted for ONE subject: Sen. BAM
|
||||
// Aquino. The differences that matter:
|
||||
//
|
||||
// * Every job is a BAM query (lib/sources.js) — nothing general ever runs
|
||||
// on a normal tick. The PH native wires run ONLY when a tick produced
|
||||
// zero named-BAM stories (quiet-day fallback).
|
||||
// * A RELEVANCE GATE, not just a freshness window. Bing's keyword search
|
||||
// matches on article content, so "Bam Aquino power electricity" returns
|
||||
// VAT-op-eds and committee gossip that never name him. Only stories whose
|
||||
// title OR description names him (or a senate context with an Aquino
|
||||
// surname) are admitted from the query jobs.
|
||||
// * maxAge is 10 days, not 48h. A senator's mention volume is low
|
||||
// (measured: ~12 unique stories/week in Sept 2026); a named-BAM story
|
||||
// from last Thursday is still front-page material this weekend.
|
||||
// * No per-reader profiles, no learning, no feedback. One public pool,
|
||||
// one public edition.
|
||||
|
||||
import Parser from "rss-parser";
|
||||
import { getCollections } from "./db.js";
|
||||
import { sanitize, extractImage } from "./llm.js";
|
||||
import { BAM_QUERIES, PH_NATIVE, queryFeed, unwrapFeedLink, stripPublisherSuffix, publisherFrom, publisherFromUrl } from "./sources.js";
|
||||
import { normalizeCategory } from "./categories.js";
|
||||
import { prefetchArticles } from "./article.js";
|
||||
|
||||
const UA = {
|
||||
"User-Agent":
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
|
||||
Accept: "application/rss+xml,application/atom+xml,application/xml,text/xml,*/*",
|
||||
};
|
||||
|
||||
const parser = new Parser({
|
||||
timeout: 20000,
|
||||
headers: UA,
|
||||
customFields: {
|
||||
item: [
|
||||
["media:content", "mediaContent", { keepArray: true }],
|
||||
["media:thumbnail", "mediaThumbnail", { keepArray: true }],
|
||||
["content:encoded", "contentEncoded"],
|
||||
["News:Source", "newsSource"],
|
||||
["News:Image", "newsImage"],
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
// ── the relevance gate ──────────────────────────────────────────────────
|
||||
// Strong: the story says his name (in any of the ways the wire does).
|
||||
const BAM_NAMED = /\bBam\s*(?:A\.)?\s*Aquino\b|\bSen(?:ator|ador|\.?)\s+Bam\b|\bSen\s*Bam\b|\bBam\b(?=.{0,80}\bAquino\b)/i;
|
||||
// Context: a legislative-chamber story that may be about him without naming
|
||||
// him in the snippet. Admitted ONLY from the fallback native wires, where the
|
||||
// story also has to carry an "Aquino" surname. (The 2026-09-15 PNA headline
|
||||
// for the VisMin power probe reads "Aquino echoes House call..." — "house"
|
||||
// is a required context word; a bare surname is not, but a bare "senate bill"
|
||||
// with no Aquino in sight is dropped too.)
|
||||
const AQUINO_CTX = /\b(senate|senator|sen\.|congress|house|congressional|legislature|floor|vote[d]?|bill|inquiry|resolution|probe|lawmakers?|committee)\b/i;
|
||||
|
||||
export function bamRelevance(title, description, { requireNamed = true } = {}) {
|
||||
const t = `${title || ""} ${description || ""}`;
|
||||
if (BAM_NAMED.test(t)) return { ok: true, named: true };
|
||||
if (!requireNamed && /\bAquino\b/i.test(t) && AQUINO_CTX.test(t)) return { ok: true, named: false };
|
||||
return { ok: false, named: false };
|
||||
}
|
||||
|
||||
// Bing's own thumbnail host — a re-encoded cache copy. Fine as a fallback.
|
||||
const BING_IMG = /bing\.com\/th\?id=ONUT/i;
|
||||
|
||||
function pickImage(item) {
|
||||
const real = extractImage(item, null);
|
||||
if (real && !BING_IMG.test(real)) return { image: real, fromBing: false };
|
||||
const b = String(item.newsImage || "").trim();
|
||||
if (/^https?:\/\//i.test(b)) return { image: b, fromBing: true };
|
||||
return { image: real || null, fromBing: false };
|
||||
}
|
||||
|
||||
async function mapWithLimit(items, size, fn) {
|
||||
const out = [];
|
||||
let i = 0;
|
||||
const workers = Array.from({ length: Math.min(size, items.length) }, async () => {
|
||||
while (i < items.length) {
|
||||
const idx = i++;
|
||||
try {
|
||||
out.push(await fn(items[idx]));
|
||||
} catch (e) {
|
||||
out.push({ error: String(e && e.message ? e.message : e).slice(0, 120) });
|
||||
}
|
||||
}
|
||||
});
|
||||
await Promise.all(workers);
|
||||
return out;
|
||||
}
|
||||
|
||||
// Syndication mirrors: drop them at ingest so the reader never lands in
|
||||
// MSN/Yahoo chrome.
|
||||
const MIRROR_HOSTS = ["msn.com", "yahoo.com", "microsoft.com", "bing.com", "msftcdn.net"];
|
||||
function isMirrorHost(url) {
|
||||
let host;
|
||||
try {
|
||||
host = new URL(url).hostname.replace(/^www\./i, "").toLowerCase();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
return MIRROR_HOSTS.some((h) => host === h || host.endsWith("." + h));
|
||||
}
|
||||
|
||||
// Title fingerprint for cross-query dedupe (the same story arrives from
|
||||
// "Bam Aquino" and "Bam Aquino power electricity").
|
||||
function titleKey(t) {
|
||||
return String(t || "")
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, " ")
|
||||
.trim()
|
||||
.split(" ")
|
||||
.filter((w) => w.length > 3)
|
||||
.slice(0, 6)
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
function buildJobs({ useFallback = false } = {}) {
|
||||
const jobs = [];
|
||||
for (const q of BAM_QUERIES) jobs.push({ url: queryFeed(q.query), kind: "bam", name: q.name, category: q.category });
|
||||
if (useFallback) {
|
||||
for (const src of PH_NATIVE) for (const url of src.feeds) jobs.push({ url, kind: "native", name: src.name, category: src.category });
|
||||
}
|
||||
return jobs;
|
||||
}
|
||||
|
||||
// og:image (or first large <img>) from the real article page. Best-effort.
|
||||
async function resolveOgImage(link) {
|
||||
if (!/^https?:\/\//i.test(link)) return null;
|
||||
try {
|
||||
const res = await fetch(link, {
|
||||
headers: { "User-Agent": UA["User-Agent"], Accept: "text/html,*/*" },
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(8000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const head = (await res.text()).slice(0, 120000);
|
||||
const m =
|
||||
/<meta[^>]+property=["']og:image["'][^>]+content=["']([^"']+)["']/i.exec(head) ||
|
||||
/<meta[^>]+content=["']([^"']+)["'][^>]+property=["']og:image["']/i.exec(head) ||
|
||||
/<meta[^>]+name=["']twitter:image["'][^>]+content=["']([^"']+)["']/i.exec(head);
|
||||
let u = m && m[1] ? m[1].trim() : null;
|
||||
if (!u) return null;
|
||||
if (u.startsWith("//")) u = "https:" + u;
|
||||
if (!/^https?:\/\//i.test(u)) return null;
|
||||
if (/1x1|spacer|pixel|blank|logo|sprite\.svg/i.test(u)) return null;
|
||||
return u.slice(0, 500);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// Headline-date veto (kept from forthena, learned the hard way): an aggregator
|
||||
// can re-deliver a week-old story with a fresh pubDate. Scan the text for
|
||||
// explicit dates and refuse anything older than the window.
|
||||
const MONTHS = { january: 0, feb: 1, february: 1, mar: 2, march: 2, apr: 3, april: 3, may: 4, jun: 5, june: 5, jul: 6, july: 6, aug: 7, august: 7, sep: 8, sept: 8, september: 8, oct: 9, october: 9, nov: 10, november: 10, dec: 11, december: 11 };
|
||||
function headlineDateAge(text, nowMs) {
|
||||
const t = String(text || "").toLowerCase();
|
||||
let newest = -1;
|
||||
const re = /\b(jan(?:uary)?|feb(?:ruary)?|mar(?:ch)?|apr(?:il)?|may|jun(?:e)?|jul(?:y)?|aug(?:ust)?|sep(?:t(?:ember)?)?|oct(?:ober)?|nov(?:ember)?|dec(?:ember)?)\s+(\d{1,2})(?:st|nd|rd|th)?(?:,\s*(\d{4}))?|\b(\d{1,2})\s+(jan(?:uary)?|feb(?:ruary)?|mar(?:ch)?|apr(?:il)?|may|jun(?:e)?|jul(?:y)?|aug(?:ust)?|sep(?:t(?:ember)?)?|oct(?:ober)?|nov(?:ember)?|dec(?:ember)?)(?:,\s*(\d{4}))?/gi;
|
||||
let m;
|
||||
while ((m = re.exec(t))) {
|
||||
const monName = (m[1] || m[5] || "").replace(/\.$/, "");
|
||||
const mon = MONTHS[monName];
|
||||
const day = parseInt(m[2] || m[4] || "1", 10);
|
||||
let year = parseInt(m[3] || m[6] || "", 10);
|
||||
if (mon === undefined || Number.isNaN(mon)) continue;
|
||||
if (!Number.isFinite(year)) {
|
||||
const nowY = new Date(nowMs).getFullYear();
|
||||
year = new Date(nowY, mon, day) > nowMs ? nowY - 1 : nowY;
|
||||
}
|
||||
const d = new Date(year, mon, day).getTime();
|
||||
if (Number.isFinite(d) && d <= nowMs + 86400e3) newest = Math.max(newest, d);
|
||||
}
|
||||
if (newest < 0) return null;
|
||||
return (nowMs - newest) / 86400e3;
|
||||
}
|
||||
|
||||
export async function crawlOnce({ perFeed = 20, maxStories = 300, concurrency = 6, ogBudget = 24, maxAgeH = 240 } = {}) {
|
||||
const { stories } = await getCollections();
|
||||
const now = new Date();
|
||||
const maxAgeMs = maxAgeH * 3600e3;
|
||||
const newestAllowed = now.getTime() - maxAgeMs;
|
||||
|
||||
// First pass: the BAM queries only.
|
||||
let jobs = buildJobs({ useFallback: false });
|
||||
let results = await mapWithLimit(jobs, concurrency, async (job) => {
|
||||
const feed = await parser.parseURL(job.url);
|
||||
return { job, feed };
|
||||
});
|
||||
|
||||
// Normalise + gate (Pass 1).
|
||||
let candidates = [];
|
||||
const errors = [];
|
||||
let skippedStale = 0, skippedUndated = 0, skippedSynd = 0, gatedOut = 0;
|
||||
for (const r of results) {
|
||||
if (!r || r.error || !r.feed) {
|
||||
if (r && r.error) errors.push(`${r.job?.name || "job"}: ${r.error}`);
|
||||
continue;
|
||||
}
|
||||
const { job, feed } = r;
|
||||
const items = (feed.items || []).slice(0, perFeed).map((item) => {
|
||||
const rawTitle = sanitize(item.title || "");
|
||||
const wrapped = (item.link || "").trim();
|
||||
const link = job.kind === "native" ? wrapped : unwrapFeedLink(wrapped);
|
||||
if (!link) return null;
|
||||
const desc = sanitize(item.contentSnippet || item.contentEncoded || item.content || item.summary || "").slice(0, 900);
|
||||
const hadDate = !!item.isoDate;
|
||||
const pub = hadDate ? new Date(item.isoDate) : now;
|
||||
const img = job.kind === "native"
|
||||
? { image: extractImage(item, feed), fromBing: false }
|
||||
: pickImage(item);
|
||||
const source = job.kind === "native"
|
||||
? sanitize(job.name || feed.title).slice(0, 80)
|
||||
: (sanitize(item.newsSource || "").slice(0, 80)
|
||||
|| publisherFromUrl(link)
|
||||
|| publisherFrom(rawTitle, job.name).slice(0, 80));
|
||||
return { job, rawTitle, link, feedUrl: job.kind === "native" ? null : wrapped, hadDate, title: job.kind === "native" ? rawTitle : stripPublisherSuffix(rawTitle), desc, source, summary: desc, image: img.image, bingImage: img.fromBing, published: pub, author: sanitize(item.creator || "").slice(0, 120) };
|
||||
}).filter(Boolean);
|
||||
// relevance gate: query jobs must NAME him; native fallback must carry
|
||||
// an Aquino surname in a senate context.
|
||||
candidates.push(...items.filter((it) => {
|
||||
const gate = bamRelevance(it.title, it.desc, { requireNamed: it.job.kind !== "native" });
|
||||
if (!gate.ok) { gatedOut++; return false; }
|
||||
it.named = gate.named;
|
||||
return true;
|
||||
}).filter((it) => {
|
||||
if (isNaN(it.published.getTime())) { skippedUndated++; return false; }
|
||||
if (isMirrorHost(it.link)) { skippedSynd++; return false; }
|
||||
if (it.published.getTime() < newestAllowed) { skippedStale++; return false; }
|
||||
return true;
|
||||
}));
|
||||
}
|
||||
|
||||
// Quiet day: zero named-BAM stories this tick -> run the associated wires.
|
||||
let usedFallback = false;
|
||||
const namedCount = candidates.filter((c) => c.named).length;
|
||||
if (namedCount === 0) {
|
||||
usedFallback = true;
|
||||
const fbJobs = buildJobs({ useFallback: true }).filter((j) => j.kind === "native");
|
||||
const fbResults = await mapWithLimit(fbJobs, concurrency, async (job) => {
|
||||
const feed = await parser.parseURL(job.url);
|
||||
return { job, feed };
|
||||
});
|
||||
for (const r of fbResults) {
|
||||
if (!r || r.error || !r.feed) {
|
||||
if (r && r.error) errors.push(`${r.job?.name || "job"}: ${r.error}`);
|
||||
continue;
|
||||
}
|
||||
const { job, feed } = r;
|
||||
const items = (feed.items || []).slice(0, perFeed).map((item) => {
|
||||
const rawTitle = sanitize(item.title || "");
|
||||
const wrapped = (item.link || "").trim();
|
||||
const link = wrapped;
|
||||
if (!link) return null;
|
||||
const desc = sanitize(item.contentSnippet || item.contentEncoded || item.content || item.summary || "").slice(0, 900);
|
||||
const hadDate = !!item.isoDate;
|
||||
const pub = hadDate ? new Date(item.isoDate) : now;
|
||||
const img = { image: extractImage(item, feed), fromBing: false };
|
||||
return { job, rawTitle, link, feedUrl: null, hadDate, title: rawTitle, desc, source: sanitize(job.name || feed.title).slice(0, 80), summary: desc, image: img.image, bingImage: false, published: pub, author: sanitize(item.creator || "").slice(0, 120) };
|
||||
}).filter(Boolean);
|
||||
candidates.push(...items.filter((it) => {
|
||||
const gate = bamRelevance(it.title, it.desc, { requireNamed: false });
|
||||
if (!gate.ok) { gatedOut++; return false; }
|
||||
it.named = gate.named;
|
||||
if (isNaN(it.published.getTime())) { skippedUndated++; return false; }
|
||||
if (isMirrorHost(it.link)) { skippedSynd++; return false; }
|
||||
if (it.published.getTime() < newestAllowed) { skippedStale++; return false; }
|
||||
return true;
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
// Global image-reuse guard (kept from forthena): one URL on >=5 distinct
|
||||
// stories = boilerplate.
|
||||
const globalFreq = new Map();
|
||||
for (const it of candidates) {
|
||||
if (!it.image) continue;
|
||||
const s = globalFreq.get(it.image) || new Set();
|
||||
s.add(titleKey(it.title) || it.link);
|
||||
globalFreq.set(it.image, s);
|
||||
}
|
||||
const boiler = new Set([...globalFreq.entries()].filter(([, set]) => set.size >= 5).map(([u]) => u));
|
||||
if (boiler.size) for (const it of candidates) if (boiler.has(it.image)) it.image = null;
|
||||
|
||||
// Cross-query dedupe: URL exact, then title fingerprint. Named-BAM stories
|
||||
// beat context stories; the earlier query wins ties (BAM Wire is first).
|
||||
const urlKeys = new Set();
|
||||
const titleKeys = new Map();
|
||||
const picked = [];
|
||||
const ordered = [...candidates].sort((a, b) => (b.named ? 1 : 0) - (a.named ? 1 : 0));
|
||||
for (const it of ordered) {
|
||||
if (!it.rawTitle || !it.link || !/^https?:\/\//i.test(it.link)) continue;
|
||||
if (urlKeys.has(it.link)) continue;
|
||||
// Headline-date veto (a named-BAM story is worth 10 days, so the window
|
||||
// here is the whole crawl window, not half of it).
|
||||
const txtAge = headlineDateAge(it.title + " " + it.summary, now.getTime());
|
||||
if (txtAge !== null && txtAge > maxAgeH / 24 + 2) { skippedStale++; continue; }
|
||||
const tk = titleKey(it.title);
|
||||
if (tk) {
|
||||
const prev = titleKeys.get(tk);
|
||||
if (prev) {
|
||||
// only replace when the new one is stronger (named over context)
|
||||
if (!(it.named && !prev.named)) continue;
|
||||
}
|
||||
titleKeys.set(tk, { named: it.named });
|
||||
}
|
||||
urlKeys.add(it.link);
|
||||
picked.push(it);
|
||||
if (picked.length >= maxStories) break;
|
||||
}
|
||||
|
||||
// og:image rescue for photo-less stories, budget-capped.
|
||||
const needOg = picked.filter((it) => !it.image).slice(0, ogBudget);
|
||||
await mapWithLimit(needOg, 5, async (it) => {
|
||||
const og = await resolveOgImage(it.link);
|
||||
if (og) it.image = og;
|
||||
});
|
||||
|
||||
// Insert (upsert by url). `named` and `fallback` are stored: the front page
|
||||
// leads with named-BAM stories and the masthead can count them.
|
||||
let added = 0, withImage = 0, reingested = 0;
|
||||
for (const it of picked) {
|
||||
const res = await stories.updateOne(
|
||||
{ url: it.link },
|
||||
{
|
||||
$setOnInsert: {
|
||||
url: it.link,
|
||||
feedUrl: it.feedUrl || null,
|
||||
title: it.title.slice(0, 300),
|
||||
summary: it.summary,
|
||||
image: it.image,
|
||||
imageSource: it.image ? (it.bingImage ? "bing" : "feed") : null,
|
||||
author: it.author || it.source,
|
||||
source: it.source,
|
||||
category: normalizeCategory(it.job ? it.job.category : "ph"),
|
||||
desk: it.job ? it.job.name : "Associated",
|
||||
feedKind: it.job ? it.job.kind : "native",
|
||||
named: !!it.named,
|
||||
fallback: it.job ? false : true,
|
||||
published: it.published,
|
||||
discoveredAt: now,
|
||||
ingestedAt: now,
|
||||
},
|
||||
},
|
||||
{ upsert: true }
|
||||
);
|
||||
if (res.upsertedCount) { added++; if (it.image) withImage++; }
|
||||
else {
|
||||
// refresh the image on an existing row when we newly rescued one
|
||||
reingested++;
|
||||
if (it.image && !it.bingImage) {
|
||||
const cur = await stories.findOne({ url: it.link }, { projection: { image: 1 } });
|
||||
if (!cur || !cur.image) await stories.updateOne({ url: it.link }, { $set: { image: it.image, imageSource: "og" } });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Warm the reader (kept from forthena): fetch the article bodies NOW so
|
||||
// opening a story is a cache hit.
|
||||
let articles = { fetched: 0, ok: 0 };
|
||||
try {
|
||||
articles = await prefetchArticles(
|
||||
picked.map((it) => it.link),
|
||||
{ budget: 12, concurrency: 3, deadlineMs: 40000 }
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn("[bama] article prefetch failed:", String((e && e.message) || e).slice(0, 120));
|
||||
}
|
||||
|
||||
return { added, reingested, withImage, gatedOut, skippedStale, skippedUndated, skippedSynd, named: namedCount, feeds: jobs.length, usedFallback, articles, errors: errors.slice(0, 12), collectedAt: now };
|
||||
}
|
||||
52
lib/db.js
Normal file
52
lib/db.js
Normal file
@ -0,0 +1,52 @@
|
||||
import { MongoClient } from "mongodb";
|
||||
|
||||
// BidaAngMamamayan — local Mongo on the site's own box, db "bama".
|
||||
// No auth: the site is public and the data is a public news cache + an
|
||||
// open community board. bindIp 127.0.0.1 keeps it off the wire.
|
||||
const uri =
|
||||
process.env.MONGODB_URI ||
|
||||
"mongodb://127.0.0.1:27017/bama";
|
||||
|
||||
const client = new MongoClient(uri, {
|
||||
serverSelectionTimeoutMS: 8000,
|
||||
});
|
||||
|
||||
let cached = global.__bama_mongo;
|
||||
if (!cached) {
|
||||
cached = global.__bama_mongo = { client, promise: null };
|
||||
}
|
||||
|
||||
export const DB_NAME = process.env.BAMA_DB_NAME || "bama";
|
||||
|
||||
export async function getDb() {
|
||||
if (!cached.promise) {
|
||||
cached.promise = cached.client.connect();
|
||||
}
|
||||
const c = await cached.promise;
|
||||
return c.db(DB_NAME);
|
||||
}
|
||||
|
||||
// One-shot scripts (db-reset, the digest trigger) need the client to close so
|
||||
// the process actually exits.
|
||||
export async function closeDb() {
|
||||
if (!cached.client) return;
|
||||
try {
|
||||
await cached.client.close();
|
||||
} finally {
|
||||
cached.client = null;
|
||||
cached.promise = null;
|
||||
if (global.__bama_mongo === cached) delete global.__bama_mongo;
|
||||
}
|
||||
}
|
||||
|
||||
export async function getCollections() {
|
||||
const db = await getDb();
|
||||
return {
|
||||
// Crawled news (the shared pool — one site, one public edition).
|
||||
stories: db.collection("stories"),
|
||||
// Community board ("Thank you Senator BAM").
|
||||
board: db.collection("board"),
|
||||
// Rate-limit counters (board spam control).
|
||||
rate: db.collection("rate"),
|
||||
};
|
||||
}
|
||||
126
lib/feed.js
Normal file
126
lib/feed.js
Normal file
@ -0,0 +1,126 @@
|
||||
// BidaAngMamamayan feed read path — the public edition.
|
||||
//
|
||||
// No profiles, no learning, no LLM. The ranking is deliberate and
|
||||
// deterministic: stories that NAME Sen. Bam lead; within each tier, freshness
|
||||
// decides. This is a tribute site, not a taste engine — the "taste" is baked
|
||||
// in by the crawler's relevance gate.
|
||||
import { getCollections } from "./db.js";
|
||||
import { normalizeCategory, sectionName, CATEGORY_LABELS, CATEGORIES } from "./categories.js";
|
||||
|
||||
export { sectionName };
|
||||
|
||||
const POOL_HOURS = 24 * 14; // the site's memory window for the edition
|
||||
|
||||
// Simple, stable score: named-BAM beats context, fresh beats old, image
|
||||
// presence nudges a card up (a photoless tribute edition looks broken).
|
||||
function rank(s, nowMs) {
|
||||
let score = 0;
|
||||
if (s.named) score += 1000;
|
||||
if (s.image) score += 50;
|
||||
const ageH = (nowMs - new Date(s.published || s.discoveredAt || 0).getTime()) / 3600e3;
|
||||
if (Number.isFinite(ageH)) score += Math.max(0, 100 - ageH); // 0..100, decaying
|
||||
return score;
|
||||
}
|
||||
|
||||
function shape(s) {
|
||||
return {
|
||||
url: s.url,
|
||||
title: s.title,
|
||||
summary: s.summary,
|
||||
image: s.image || null,
|
||||
source: s.source,
|
||||
category: normalizeCategory(s.category),
|
||||
desk: s.desk || null,
|
||||
published: s.published,
|
||||
discoveredAt: s.discoveredAt,
|
||||
named: !!s.named,
|
||||
fallback: !!s.fallback,
|
||||
score: 0,
|
||||
feedKind: s.feedKind || "native",
|
||||
};
|
||||
}
|
||||
|
||||
export async function getFeed({ limit = 60 } = {}) {
|
||||
const { stories } = await getCollections();
|
||||
const now = new Date();
|
||||
const rows = await stories
|
||||
.find({
|
||||
discoveredAt: { $gte: new Date(now.getTime() - POOL_HOURS * 3600e3) },
|
||||
desk: { $ne: "system" },
|
||||
})
|
||||
.sort({ discoveredAt: -1 })
|
||||
.limit(1000)
|
||||
.toArray();
|
||||
|
||||
const nowMs = now.getTime();
|
||||
const items = rows
|
||||
.map((s) => {
|
||||
const it = shape(s);
|
||||
it.score = rank(s, nowMs);
|
||||
return it;
|
||||
})
|
||||
.sort((a, b) => b.score - a.score || new Date(b.published || b.discoveredAt || 0) - new Date(a.published || a.discoveredAt || 0))
|
||||
.slice(0, limit);
|
||||
|
||||
return {
|
||||
items,
|
||||
generatedAt: now,
|
||||
mode: "public",
|
||||
ranked: true,
|
||||
// masthead counts
|
||||
named: rows.filter((s) => s.named).length,
|
||||
total: rows.length,
|
||||
};
|
||||
}
|
||||
|
||||
// One (or a few) headline(s) per desk, straight from the pool — the front
|
||||
// page's desk grid is built from this, exactly as forthena's was. Zero LLM.
|
||||
export async function deskHeadlines() {
|
||||
try {
|
||||
const { stories } = await getCollections();
|
||||
const nowMs = Date.now();
|
||||
const rows = await stories
|
||||
.find({
|
||||
discoveredAt: { $gte: new Date(nowMs - POOL_HOURS * 3600e3) },
|
||||
desk: { $ne: "system" },
|
||||
})
|
||||
.limit(2000)
|
||||
.toArray();
|
||||
const per = new Map();
|
||||
const count = new Map();
|
||||
for (const s of rows) {
|
||||
const cat = normalizeCategory(s.category);
|
||||
count.set(cat, (count.get(cat) || 0) + 1);
|
||||
const it = shape(s);
|
||||
it.score = rank(s, nowMs);
|
||||
const arr = per.get(cat) || [];
|
||||
arr.push(it);
|
||||
per.set(cat, arr);
|
||||
}
|
||||
return CATEGORIES
|
||||
.map((c) => {
|
||||
const arr = (per.get(c.key) || []).sort((a, b) => b.score - a.score);
|
||||
if (!arr.length) return null;
|
||||
return { key: c.key, label: sectionName(c.key), blurb: c.blurb || "", count: count.get(c.key) || arr.length, stories: arr.slice(0, 12) };
|
||||
})
|
||||
.filter(Boolean);
|
||||
} catch (e) {
|
||||
console.warn("[bama] desk headlines unavailable:", String((e && e.message) || e).slice(0, 140));
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
// Masthead line: how much of the wire the desk has scoured lately.
|
||||
export async function editionStats() {
|
||||
try {
|
||||
const { stories } = await getCollections();
|
||||
const [total, last24] = await Promise.all([
|
||||
stories.countDocuments({ discoveredAt: { $gte: new Date(Date.now() - 14 * 86400e3) } }),
|
||||
stories.countDocuments({ discoveredAt: { $gte: new Date(Date.now() - 24 * 3600e3) } }),
|
||||
]);
|
||||
const board = await (await getCollections()).board.countDocuments({});
|
||||
return { stories: total, today: last24, boardPosts: board };
|
||||
} catch {
|
||||
return { stories: 0, today: 0, boardPosts: 0 };
|
||||
}
|
||||
}
|
||||
41
lib/llm.js
Normal file
41
lib/llm.js
Normal file
@ -0,0 +1,41 @@
|
||||
// Text utilities kept from forthena's lib/llm.js (this site has NO LLM layer —
|
||||
// the ranking is deterministic, see lib/feed.js). The file keeps the llm.js
|
||||
// module name so the crawl/article imports don't churn.
|
||||
|
||||
export function sanitize(s) {
|
||||
return String(s || "")
|
||||
.replace(/<[^>]*>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
// First image an RSS item exposes, in descending order of reliability.
|
||||
export function extractImage(item, feed) {
|
||||
const cands = [];
|
||||
const enc = item.enclosure;
|
||||
if (enc && enc.url && (/image/i.test(enc.type || "") || /\.(jpe?g|png|webp|gif|avif)/i.test(enc.url))) cands.push(enc.url);
|
||||
for (const key of ["mediaContent", "mediaThumbnail", "media:content", "media:thumbnail"]) {
|
||||
const v = item[key];
|
||||
const list = Array.isArray(v) ? v : v ? [v] : [];
|
||||
for (const m of list) {
|
||||
const u = (m && (m["$"]?.url || m.url || m.$?.url)) || null;
|
||||
if (u) cands.push(u);
|
||||
}
|
||||
}
|
||||
const itn = item.itunesImage || item.image;
|
||||
if (itn) cands.push((itn["$"] && itn["$"].href) || itn.href || (typeof itn === "string" ? itn : null));
|
||||
const html = [item["content:encoded"], item.content, item.contentSnippet, item.summary]
|
||||
.filter(Boolean)
|
||||
.join(" ");
|
||||
const m = /<img[^>]+src=["']([^"']+)["']/i.exec(html);
|
||||
if (m) cands.push(m[1]);
|
||||
const fi = feed && feed.image && (feed.image.url || feed.image.$?.url);
|
||||
if (fi) cands.push(fi);
|
||||
for (const u of cands) {
|
||||
const s = String(u || "").trim();
|
||||
if (!/^https?:\/\//i.test(s)) continue;
|
||||
if (/1x1|spacer|pixel\.gif|blank\.|logo-|sprite/i.test(s)) continue;
|
||||
return s.slice(0, 500);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
124
lib/sources.js
Normal file
124
lib/sources.js
Normal file
@ -0,0 +1,124 @@
|
||||
// BidaAngMamamayan source registry — the crawler's supply line.
|
||||
//
|
||||
// This is a TRIBUTE site for Sen. BAM Aquino, so every feed is scoped to him:
|
||||
// 1. BAM_QUERIES — keyword news searches (Bing News RSS, the same aggregator
|
||||
// forthena learned to love 2026-08-31: it hands us the publisher's own URL
|
||||
// inside the apiclick wrapper, which the reader pane needs).
|
||||
// 2. PH_NATIVE — general PH news wires, kept only as an ASSOCIATED pool
|
||||
// fallback: when a tick finds zero named-BAM stories, a couple of his
|
||||
// senate colleagues' / committee's coverage still belongs on the site.
|
||||
//
|
||||
// Freshness: qft=interval="8" = last month. Bing's 7-day window only ever held
|
||||
// 12 unique stories for "Bam Aquino" (measured 2026-09-15) — a senator's
|
||||
// mention volume is low, so the crawl window is a month and the relevance
|
||||
// gate (lib/crawl.js) does the actual filtering. The 48h crawl maxAge from
|
||||
// forthena does NOT apply here: a named-BAM story from last Thursday is still
|
||||
// front-page material this weekend.
|
||||
//
|
||||
// Bing quirks (all verified live on the agent host):
|
||||
// * `OR` in queries is nil — single phrases only.
|
||||
// * setlang=en-PH&cc=PH biases toward PH coverage.
|
||||
// * A quoted exact phrase ("Bam Aquino") returns FEWER results than the
|
||||
// unquoted phrase — Bing drops results whose snippet doesn't contain the
|
||||
// exact string. Keep queries unquoted; the gate handles relevance.
|
||||
|
||||
// ── BAM queries ──────────────────────────────────────────────────────────
|
||||
// Each phrase was measured through the live Bing news RSS on 2026-09-15.
|
||||
// "Bam Aquino" alone is the workhorse (12 items/week, the rest overlap it).
|
||||
export const BAM_QUERIES = [
|
||||
{ query: "Bam Aquino", name: "BAM Wire", category: "bam" },
|
||||
{ query: "Sen. Bam Aquino", name: "Senate Desk", category: "politics" },
|
||||
{ query: "Bam Aquino education", name: "Education Desk", category: "education" },
|
||||
{ query: "Bam Aquino college students scholarship", name: "Campus Desk", category: "education" },
|
||||
{ query: "Bam Aquino power electricity", name: "Energy Desk", category: "energy" },
|
||||
{ query: "Bam Aquino gaming esports", name: "Gamer Desk", category: "gaming" },
|
||||
{ query: "Bam Aquino bill Senate", name: "Legislative Desk", category: "politics" },
|
||||
];
|
||||
|
||||
// ── Associated pool (fallback only) ──────────────────────────────────────
|
||||
// General PH wires. The crawler runs these ONLY when the BAM queries produced
|
||||
// zero named stories this tick (verified: that happens on quiet days). Their
|
||||
// items pass through the same gate, so only genuine senate/BAM-adjacent copy
|
||||
// ever prints from here.
|
||||
export const PH_NATIVE = [
|
||||
{ key: "pna", name: "Philippine News Agency", category: "ph", feeds: ["https://www.pna.gov.ph/feed/"] },
|
||||
{ key: "inqnews", name: "Inquirer News", category: "ph", feeds: ["https://newsinfo.inquirer.net/feed"] },
|
||||
];
|
||||
|
||||
// ── Bing News RSS ────────────────────────────────────────────────────────
|
||||
const BING = "https://www.bing.com/news/search?q=";
|
||||
const BING_PARAMS = "&format=RSS&setlang=en-PH&cc=PH&qft=interval%3d%228%22";
|
||||
|
||||
export function queryFeed(q) {
|
||||
return `${BING}${encodeURIComponent(String(q).slice(0, 120))}${BING_PARAMS}`;
|
||||
}
|
||||
|
||||
// Bing wraps item links in apiclick.aspx?...&url=<publisher url>. Returns the
|
||||
// publisher's own URL when the wrapper carries one, else the original.
|
||||
export function unwrapFeedLink(link) {
|
||||
let l = String(link || "").trim();
|
||||
if (!l) return l;
|
||||
l = l.replace(/&/g, "&").replace(/&/g, "&");
|
||||
if (/bing\.com\/news\/apiclick/i.test(l)) {
|
||||
const m = /[?&]url=([^&]+)/.exec(l);
|
||||
if (m) {
|
||||
try {
|
||||
const real = decodeURIComponent(m[1]);
|
||||
if (/^https?:\/\//i.test(real)) return real;
|
||||
} catch { /* leave the wrapper */ }
|
||||
}
|
||||
}
|
||||
if (/^https?:\/\/(www\.)?bing\.com\//i.test(l)) {
|
||||
const m = /[?&](?:url|u)=([^&]+)/.exec(l);
|
||||
if (m) { try { return decodeURIComponent(m[1]); } catch {} }
|
||||
}
|
||||
return l;
|
||||
}
|
||||
|
||||
// Human publisher label: <News:Source> first, then the URL's host, then the
|
||||
// "Headline - Publisher" tail.
|
||||
const HOST_NAMES = {
|
||||
"pna.gov.ph": "PNA",
|
||||
"inquirer.net": "Inquirer",
|
||||
"newsinfo.inquirer.net": "Inquirer News",
|
||||
"gmanetwork.com": "GMA News",
|
||||
"abs-cbn.com": "ABS-CBN",
|
||||
"mb.com.ph": "Manila Bulletin",
|
||||
"rappler.com": "Rappler",
|
||||
"bworldonline.com": "BusinessWorld",
|
||||
"philstar.com": "Philstar",
|
||||
"sunstar.com.ph": "SunStar",
|
||||
"cnnphilippines.com": "CNN Philippines",
|
||||
"businessmirror.com": "BusinessMirror",
|
||||
"manilatimes.net": "Manila Times",
|
||||
"dailytribune.com.ph": "Daily Tribune",
|
||||
"newsline.ph": "NewsLine",
|
||||
"bombardyo.com": "Bombo Radyo",
|
||||
};
|
||||
|
||||
export function publisherFromUrl(url) {
|
||||
try {
|
||||
const host = new URL(url).hostname.replace(/^www\./, "").toLowerCase();
|
||||
const reg = Object.keys(HOST_NAMES).find((h) => host === h || host.endsWith("." + h));
|
||||
if (reg) return HOST_NAMES[reg];
|
||||
const parts = host.split(".");
|
||||
return parts.length > 2 ? parts[parts.length - 3] : parts[0];
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function publisherFrom(title, fallback) {
|
||||
const t = String(title || "");
|
||||
const i = t.lastIndexOf(" - ");
|
||||
if (i > 24) return t.slice(i + 3).trim().slice(0, 40);
|
||||
return fallback || "Wire";
|
||||
}
|
||||
|
||||
// Aggregator titles sometimes still carry the " - Publisher" tail.
|
||||
export function stripPublisherSuffix(title) {
|
||||
const t = String(title || "");
|
||||
const i = t.lastIndexOf(" - ");
|
||||
if (i > 24) return t.slice(0, i).trim();
|
||||
return t.trim();
|
||||
}
|
||||
75
lib/storyid.js
Normal file
75
lib/storyid.js
Normal file
@ -0,0 +1,75 @@
|
||||
// Story ids — a real URL per story, without a real URL in the path.
|
||||
//
|
||||
// /story/[id] needs a stable, path-safe id derived from the story url. Mongo urls contain
|
||||
// `:/?#[]@!$&'()*+,;=%` and can be ~900 chars, so we base64url the utf-8 bytes. Nothing is
|
||||
// truncated, nothing is "fixed" — the codec is the contract, and it must round-trip exactly
|
||||
// or a shared link lands on a dead page.
|
||||
//
|
||||
// Two forms, both resolvable (the page tries them in this order):
|
||||
// 1. storyId(url) — the full encoding. Always works; long urls make long route segments.
|
||||
// 2. shortId(url) — same, but past SHORT_MAX chars it falls back to base64url(sha1hex(url)).
|
||||
// A hash is one-way, so the short form only resolves once the doc carries `sid`, which
|
||||
// the story page writes (idempotent) the first time it resolves a long url.
|
||||
//
|
||||
// No DB, no Next imports — isomorphic and unit-testable.
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
// Decoded junk guard: a control character in the decoded string means this id was never a
|
||||
// url, whatever it decodes to. (Written escaped — raw control bytes must not live in source.)
|
||||
const CONTROL_JUNK = new RegExp("[\\u0000-\\u001F\\u007F]");
|
||||
|
||||
// Any url-encoded id past this gets hashed instead. 200 keeps the address bar honest for
|
||||
// normal stories; Google-News wrappers are what blow past it.
|
||||
const SHORT_MAX = 200;
|
||||
|
||||
export function storyId(url) {
|
||||
const s = String(url ?? "");
|
||||
if (!s) return "";
|
||||
return Buffer.from(s, "utf8").toString("base64url");
|
||||
}
|
||||
|
||||
export function storyUrlFromId(id) {
|
||||
const s = String(id ?? "").trim();
|
||||
// base64url alphabet only. Rejects path tricks (`..`, `%2f`, `?`) before we even decode.
|
||||
if (!s || !/^[A-Za-z0-9_-]+$/.test(s)) return null;
|
||||
let raw;
|
||||
try {
|
||||
raw = Buffer.from(s, "base64url").toString("utf8");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (!raw || CONTROL_JUNK.test(raw)) return null;
|
||||
const t = raw.trim();
|
||||
if (!/^https?:\/\/[^\s]+$/i.test(t)) return null; // hard guard: must decode to an http(s) URL
|
||||
return t;
|
||||
}
|
||||
|
||||
// base64url(sha1hex(url)) — the short form. 40 hex chars -> 54 base64url chars.
|
||||
export function sidOf(url) {
|
||||
const s = String(url ?? "");
|
||||
if (!s) return "";
|
||||
const hex = createHash("sha1").update(s, "utf8").digest("hex");
|
||||
return Buffer.from(hex, "utf8").toString("base64url");
|
||||
}
|
||||
|
||||
// A url-shaped id (short) or the hashed fallback (long). Never truncates the encoding.
|
||||
export function shortId(url) {
|
||||
const full = storyId(url);
|
||||
if (!full) return "";
|
||||
return full.length > SHORT_MAX ? sidOf(url) : full;
|
||||
}
|
||||
|
||||
// Does an incoming id *look* like the hashed form? (40 hex chars survive base64url decoding
|
||||
// as exactly that shape — cheap check, avoids a pointless Mongo query for full-form ids.)
|
||||
export function looksLikeSid(id) {
|
||||
const s = String(id ?? "").trim();
|
||||
if (!/^[A-Za-z0-9_-]+$/.test(s)) return false;
|
||||
let raw;
|
||||
try {
|
||||
raw = Buffer.from(s, "base64url").toString("utf8");
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
return /^[0-9a-f]{40}$/.test(raw);
|
||||
}
|
||||
78
lib/text.js
Normal file
78
lib/text.js
Normal file
@ -0,0 +1,78 @@
|
||||
// Text shaping for the paper — isomorphic (no Node, no DB, no LLM).
|
||||
// Imported by server routes AND client components, so keep it dependency-free.
|
||||
//
|
||||
// Why this file exists (Daniel, 2026-08-31): the edition used to slice summaries with a
|
||||
// bare `.slice(0, 150)`, which cut copy mid-WORD — "the Department of T", "recently in Q".
|
||||
// A newspaper never breaks a word to fit a box; it breaks at a sentence, then at a word,
|
||||
// then says "more". clipText() encodes that order of preference.
|
||||
|
||||
const ELLIPSIS = "\u2026"; // …
|
||||
|
||||
// Sentence-ish endings, including closing quotes/brackets and common abbreviations guard.
|
||||
// With the `$` anchor: "the text ends here" counts as a sentence end (firstSentence).
|
||||
// Without it: used while CLIPPING, where the end of the allowed window is an artifact of
|
||||
// the budget, not a sentence end — including `$` there made clipText believe a cut that
|
||||
// landed mid-word was a finished sentence and print it with no ellipsis at all.
|
||||
// (Caught by tests/text-test.mjs: "…concerns an" — the word "and", severed.)
|
||||
const SENT_END = /(?:[.!?…]["'”’)»]\s|[.!?…]\s|$)/;
|
||||
const SENT_END_CLIP = /(?:[.!?…]["'”’)»]\s|[.!?…]\s)/;
|
||||
const ABBREV = /(?:\b(?:mr|mrs|ms|dr|prof|sr|jr|st|no|vs|etc|e\.g|i\.e|u\.s|u\.k|ph|jan|feb|mar|apr|jun|jul|aug|sep|sept|oct|nov|dec)\.)\s*$/i;
|
||||
|
||||
// Prefer a sentence boundary, then a word boundary. Never cuts inside a word.
|
||||
export function clipText(s, max = 160, { ellipsis = true } = {}) {
|
||||
const t = String(s || "").replace(/\s+/g, " ").trim();
|
||||
if (!t) return "";
|
||||
if (t.length <= max) return t;
|
||||
|
||||
// Look for a sentence end in the tail of the allowed window — a cut that lands
|
||||
// mid-first-sentence is only acceptable when the first sentence is longer than the box.
|
||||
const window = t.slice(0, max + 1);
|
||||
let cut = -1;
|
||||
const re = new RegExp(SENT_END_CLIP.source, "g");
|
||||
let m;
|
||||
while ((m = re.exec(window))) {
|
||||
// ignore a terminal that is really an abbreviation ("U.S.", "St.")
|
||||
if (ABBREV.test(window.slice(0, m.index + 1))) continue;
|
||||
cut = m.index + m[0].trimEnd().length;
|
||||
if (cut > max * 0.5) break; // take a boundary in the back half of the window
|
||||
}
|
||||
if (cut > 0) {
|
||||
// We landed on a real sentence end: the teaser is a complete thought, so it gets NO
|
||||
// ellipsis. Hanging "…" off a finished sentence is the same lie as cutting mid-word.
|
||||
return t.slice(0, cut).trim();
|
||||
}
|
||||
|
||||
// No sentence end in range: cut at the last word boundary and say so with "…".
|
||||
const sp = window.lastIndexOf(" ");
|
||||
const hard = sp > max * 0.4 ? sp : max;
|
||||
const out = t.slice(0, hard).replace(/[,;:\-–—)]+$/, "").trim();
|
||||
return ellipsis ? out + " " + ELLIPSIS : out;
|
||||
}
|
||||
|
||||
// The line under a headline: 1-2 complete sentences when they fit the box.
|
||||
export function dek(summary, max = 180) {
|
||||
return clipText(summary, max);
|
||||
}
|
||||
|
||||
// First complete sentence, for the "lede" slot.
|
||||
export function firstSentence(s, max = 260) {
|
||||
const t = String(s || "").replace(/\s+/g, " ").trim();
|
||||
if (!t) return "";
|
||||
const m = SENT_END.exec(t);
|
||||
const end = m ? m.index + m[0].trimEnd().length : t.length;
|
||||
const one = t.slice(0, Math.max(end, t.indexOf(" ") + 1)).trim();
|
||||
return one.length <= max ? one : clipText(one, max);
|
||||
}
|
||||
|
||||
export function wordCount(s) {
|
||||
return String(s || "").trim().split(/\s+/).filter(Boolean).length;
|
||||
}
|
||||
|
||||
// Does the card's copy actually run out? Drives the "read on" affordance so we never
|
||||
// advertise more text than exists.
|
||||
export function isClipped(s, max = 180) {
|
||||
const t = String(s || "").replace(/\s+/g, " ").trim();
|
||||
return t.length > max;
|
||||
}
|
||||
|
||||
export { ELLIPSIS };
|
||||
5
next-env.d.ts
vendored
Normal file
5
next-env.d.ts
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.
|
||||
28
next.config.js
Normal file
28
next.config.js
Normal file
@ -0,0 +1,28 @@
|
||||
/** @type {import('next').NextConfig} */
|
||||
const nextConfig = {
|
||||
reactStrictMode: true,
|
||||
output: "standalone",
|
||||
serverExternalPackages: ["mongodb"],
|
||||
async headers() {
|
||||
// The page shell must NEVER be cached: hashed chunk filenames change every
|
||||
// build, and a stale HTML pins the browser to chunk hashes that no longer
|
||||
// exist -> hydration dies and the skeleton hangs forever (learned on
|
||||
// forthena 2026-08-30). Static assets keep immutability.
|
||||
return [
|
||||
{
|
||||
source: "/",
|
||||
headers: [{ key: "Cache-Control", value: "no-cache, no-store, must-revalidate" }],
|
||||
},
|
||||
{
|
||||
source: "/api/:path*",
|
||||
headers: [{ key: "Cache-Control", value: "no-store" }],
|
||||
},
|
||||
{
|
||||
source: "/_next/static/:path*",
|
||||
headers: [{ key: "Cache-Control", value: "public, max-age=31536000, immutable" }],
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
module.exports = nextConfig;
|
||||
2286
package-lock.json
generated
Normal file
2286
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
29
package.json
Normal file
29
package.json
Normal file
@ -0,0 +1,29 @@
|
||||
{
|
||||
"name": "bidaangmamamayan",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"description": "BidaAngMamamayan — a public supporter tribute news desk for Senator BAM Aquino.",
|
||||
"scripts": {
|
||||
"dev": "next dev",
|
||||
"build": "next build",
|
||||
"start": "next start",
|
||||
"test": "bash tests/run-all.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mozilla/readability": "^0.6.0",
|
||||
"linkedom": "^0.18.13",
|
||||
"mongodb": "6.8.0",
|
||||
"next": "15.4.5",
|
||||
"react": "19.1.0",
|
||||
"react-dom": "19.1.0",
|
||||
"rss-parser": "3.13.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/postcss": "4.1.11",
|
||||
"@types/node": "22.10.0",
|
||||
"@types/react": "19.1.0",
|
||||
"@types/react-dom": "19.1.0",
|
||||
"tailwindcss": "4.1.11",
|
||||
"typescript": "5.7.2"
|
||||
}
|
||||
}
|
||||
36
scripts/cron-crawl.cjs
Normal file
36
scripts/cron-crawl.cjs
Normal file
@ -0,0 +1,36 @@
|
||||
// Bama crawl trigger — fires the server's /api/crawl every 15 min via PM2
|
||||
// cron_restart. The web app itself never crawls on page load.
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
|
||||
// Load .env.production (KEY=VALUE lines) so the cron process has the real
|
||||
// secret + port.
|
||||
const envFile = path.join(__dirname, "..", ".env.production");
|
||||
try {
|
||||
for (const line of fs.readFileSync(envFile, "utf8").split("\n")) {
|
||||
const m = line.match(/^([A-Z0-9_]+)=(.*)$/);
|
||||
if (m && !process.env[m[1]]) process.env[m[1]] = m[2];
|
||||
}
|
||||
} catch (e) {}
|
||||
|
||||
const SECRET = process.env.CRON_SECRET || "bama-cron";
|
||||
const PORT = process.env.PORT || "3000";
|
||||
|
||||
let run = async () => {
|
||||
const res = await fetch(`http://127.0.0.1:${PORT}/api/crawl`, {
|
||||
method: "POST",
|
||||
headers: { "x-bama-cron": SECRET },
|
||||
signal: AbortSignal.timeout(280000),
|
||||
});
|
||||
const j = await res.json().catch(() => ({}));
|
||||
const c = (j && j.crawl) || {};
|
||||
console.log(
|
||||
`[bama-crawl] ${res.status} added=${c.added} named=${c.named} fallback=${c.usedFallback ? "yes" : "no"} gated=${c.gatedOut} imgs=${c.withImage} articles=${c.articles ? c.articles.ok + "/" + c.articles.fetched : "?"} ${c.errors && c.errors.length ? "errs=" + c.errors.length : ""}`
|
||||
);
|
||||
process.exit(res.ok ? 0 : 1);
|
||||
};
|
||||
|
||||
run().catch((e) => {
|
||||
console.error("[bama-crawl] FAILED", e && e.message);
|
||||
process.exit(1);
|
||||
});
|
||||
41
scripts/cron-digest.cjs
Normal file
41
scripts/cron-digest.cjs
Normal file
@ -0,0 +1,41 @@
|
||||
// Forthena hourly digest trigger — fires the server's /api/crawl (single batched AI pass happens server-side).
|
||||
// Run at :05 via PM2 cron_restart. The web app itself never calls the LLM.
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
|
||||
// Load .env.production (KEY=VALUE lines) so the cron process has the real secret + port.
|
||||
const envFile = path.join(__dirname, "..", ".env.production");
|
||||
try {
|
||||
for (const line of fs.readFileSync(envFile, "utf8").split("\n")) {
|
||||
const m = line.match(/^([A-Z0-9_]+)=(.*)$/);
|
||||
if (m && !process.env[m[1]]) process.env[m[1]] = m[2];
|
||||
}
|
||||
} catch (e) {}
|
||||
|
||||
const SECRET = process.env.CRON_SECRET || "forthena-cron";
|
||||
const PORT = process.env.PORT || "3008";
|
||||
|
||||
let run = async () => {
|
||||
const res = await fetch(`http://127.0.0.1:${PORT}/api/crawl`, {
|
||||
method: "POST",
|
||||
headers: { "x-forthena-cron": SECRET },
|
||||
signal: AbortSignal.timeout(280000),
|
||||
});
|
||||
const j = await res.json().catch(() => ({}));
|
||||
// Since multi-user, one tick runs an AI pass per reader: j.ranked is an array and
|
||||
// j.primary is the owner's (always first). Log every pass so a per-reader failure is
|
||||
// visible in PM2 output instead of silently shrinking the paper.
|
||||
const passes = Array.isArray(j.ranked) ? j.ranked : j.ranked ? [j.ranked] : [];
|
||||
const detail = passes
|
||||
.map((p) => `${p.key}:${p.error ? "ERR" : `${p.ranked}r/${p.signals}s`}`)
|
||||
.join(" ") || "none";
|
||||
console.log(
|
||||
`[forthena-digest] ${res.status} added=${j && j.crawl && j.crawl.added} migrated=${j && j.crawl && j.crawl.migrated} retired=${j && j.crawl && j.crawl.retired} passes=${passes.length} [${detail}] articles=${j && j.crawl && j.crawl.articles ? j.crawl.articles.ok + "/" + j.crawl.articles.fetched : "?"}`
|
||||
);
|
||||
process.exit(res.ok ? 0 : 1);
|
||||
};
|
||||
|
||||
run().catch((e) => {
|
||||
console.error("[forthena-digest] FAILED", e && e.message);
|
||||
process.exit(1);
|
||||
});
|
||||
43
scripts/db-reset.cjs
Normal file
43
scripts/db-reset.cjs
Normal file
@ -0,0 +1,43 @@
|
||||
// Bama scratch-DB helper: reset or peek the local database.
|
||||
// node scripts/db-reset.cjs reset (wipes stories + board + rate)
|
||||
// node scripts/db-reset.cjs peek (counts + newest few)
|
||||
// Run on the site box (reads .env.production next to it).
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const { MongoClient } = require("mongodb");
|
||||
|
||||
const envFile = path.join(__dirname, "..", ".env.production");
|
||||
try {
|
||||
for (const line of fs.readFileSync(envFile, "utf8").split("\n")) {
|
||||
const m = line.match(/^([A-Z0-9_]+)=(.*)$/);
|
||||
if (m && !process.env[m[1]]) process.env[m[1]] = m[2];
|
||||
}
|
||||
} catch (e) {}
|
||||
|
||||
const URI = process.env.MONGODB_URI || "mongodb://127.0.0.1:27017/bama";
|
||||
const DB = process.env.BAMA_DB_NAME || "bama";
|
||||
const CMD = process.argv[2] || "peek";
|
||||
|
||||
(async () => {
|
||||
const c = new MongoClient(URI, { serverSelectionTimeoutMS: 8000 });
|
||||
await c.connect();
|
||||
const db = c.db(DB);
|
||||
if (CMD === "reset") {
|
||||
for (const n of ["stories", "board", "rate"]) {
|
||||
const r = await db.collection(n).deleteMany({});
|
||||
console.log(`${n}: removed ${r.deletedCount}`);
|
||||
}
|
||||
} else {
|
||||
const counts = {};
|
||||
for (const n of ["stories", "board", "rate"]) counts[n] = await db.collection(n).countDocuments();
|
||||
console.log("counts:", counts);
|
||||
const rows = await db.collection("stories").find({}).sort({ discoveredAt: -1 }).limit(6).project({ title: 1, named: 1, source: 1, published: 1 }).toArray();
|
||||
for (const r of rows) console.log(` ${r.named ? "★" : " "} ${new Date(r.published).toISOString().slice(0, 16)} [${r.source}] ${String(r.title).slice(0, 70)}`);
|
||||
const posts = await db.collection("board").find({}).sort({ createdAt: -1 }).limit(5).toArray();
|
||||
for (const p of posts) console.log(` board/${new Date(p.createdAt).toISOString().slice(5, 16)} ${p.nick}: ${String(p.message).slice(0, 60)}`);
|
||||
}
|
||||
await c.close();
|
||||
})().catch((e) => {
|
||||
console.error("ERR", e.message);
|
||||
process.exit(1);
|
||||
});
|
||||
72
tests/board-test.mjs
Normal file
72
tests/board-test.mjs
Normal file
@ -0,0 +1,72 @@
|
||||
// Community board: caps, sanitisation, URL-only rejection, rate limiting.
|
||||
// Runs against a scratch Mongo.
|
||||
// RUN: MONGODB_URI=mongodb://127.0.0.1:27099/bama BAMA_DB_NAME=bama_board node tests/board-test.mjs
|
||||
import { MongoClient } from "mongodb";
|
||||
import { postBoard, getBoard, boardCount } from "../lib/board.js";
|
||||
|
||||
const uri = process.env.MONGODB_URI || "mongodb://127.0.0.1:27099/bama";
|
||||
const mc = new MongoClient(uri, { serverSelectionTimeoutMS: 8000 });
|
||||
await mc.connect();
|
||||
const db = mc.db(process.env.BAMA_DB_NAME || "bama");
|
||||
await db.collection("board").deleteMany({});
|
||||
await db.collection("rate").deleteMany({});
|
||||
|
||||
let fails = 0;
|
||||
const check = (name, cond, detail = "") => {
|
||||
console.log(` ${cond ? "ok " : "FAIL"} ${name}${detail ? " — " + detail : ""}`);
|
||||
if (!cond) fails++;
|
||||
};
|
||||
const fakeReq = (ip = "10.0.0.1") => ({
|
||||
headers: new Headers({ "x-forwarded-for": ip }),
|
||||
get: (k) => ({ "x-forwarded-for": ip, "x-real-ip": ip }[k.toLowerCase()] || null),
|
||||
});
|
||||
|
||||
console.log("posting:");
|
||||
let r = await postBoard(fakeReq("10.0.0.1"), { nick: "PampangaProud", message: "Thank you, Senator BAM, for the scholarship bill!" });
|
||||
check("basic post ok", r.ok, JSON.stringify(r));
|
||||
check("nick clean", r.post && r.post.nick === "PampangaProud");
|
||||
|
||||
r = await postBoard(fakeReq("10.0.0.1"), { nick: "GamerBAM", message: "GMAmateur here — palaPINA 2025 runner-up and so proud. GG, Senator!" });
|
||||
check("second post ok", r.ok);
|
||||
|
||||
console.log("sanitisation:");
|
||||
r = await postBoard(fakeReq("10.0.0.2"), { nick: " <script>alert(1)</script>TagaBAM ", message: " <b>Bold</b> thanks & <a href='x'>link</a> to the senator. " });
|
||||
check("html stripped from nick", r.ok && r.post.nick === "alert(1) TagaBAM", JSON.stringify(r.post && r.post.nick));
|
||||
check("html stripped from message", r.ok && r.post.message === "Bold thanks & link to the senator.", JSON.stringify(r.post && r.post.message));
|
||||
|
||||
r = await postBoard(fakeReq("10.0.0.3"), { nick: "SpamBot", message: "https://bad.example.com" });
|
||||
check("pure-url rejected", !r.ok && r.status === 400, JSON.stringify(r));
|
||||
r = await postBoard(fakeReq("10.0.0.3"), { nick: "RealFan", message: "Thank you Senator Bam Aquino for the scholarship expansion" });
|
||||
check("real sentence kept (URL logic didn't false-positive)", r.ok, JSON.stringify(r));
|
||||
|
||||
console.log("caps:");
|
||||
r = await postBoard(fakeReq("10.0.0.4"), { nick: "x".repeat(99), message: "thanks" });
|
||||
check("nick capped to 24", r.ok && r.post.nick.length === 24, `len=${r.post && r.post.nick.length}`);
|
||||
r = await postBoard(fakeReq("10.0.0.4"), { nick: "LongOne", message: "t".repeat(5000) });
|
||||
check("message capped to 600", r.ok && r.post.message.length === 600, `len=${r.post && r.post.message.length}`);
|
||||
|
||||
console.log("validation:");
|
||||
r = await postBoard(fakeReq("10.0.0.5"), { nick: "x", message: "thanks" });
|
||||
check("1-char nick rejected", !r.ok && r.status === 400);
|
||||
r = await postBoard(fakeReq("10.0.0.5"), { nick: "Fine", message: " " });
|
||||
check("empty message rejected", !r.ok && r.status === 400);
|
||||
|
||||
console.log("rate limit:");
|
||||
const ip6 = "10.0.0.6";
|
||||
for (let i = 0; i < 3; i++) {
|
||||
r = await postBoard(fakeReq(ip6), { nick: `Ratel${i}`, message: `message number ${i} from the same ip` });
|
||||
check(`post ${i + 1} of 3 ok`, r.ok, JSON.stringify(r));
|
||||
}
|
||||
r = await postBoard(fakeReq(ip6), { nick: "RatelX", message: "a fourth message from the same ip in ten minutes" });
|
||||
check("4th post in window 429", !r.ok && r.status === 429, JSON.stringify(r));
|
||||
|
||||
console.log("listing:");
|
||||
const posts = await getBoard();
|
||||
check("list newest-first, capped", posts.length >= 6 && posts.length <= 200);
|
||||
check("list has no ip field", posts.every((p) => p.ip === undefined));
|
||||
// 2 (10.0.0.1) + 1 (10.0.0.2) + 1 RealFan (10.0.0.3) + 2 (10.0.0.4) + 3 (10.0.0.6) = 9
|
||||
check("boardCount is the admitted total", (await boardCount()) === 9, `count=${await boardCount()}`);
|
||||
|
||||
await mc.close();
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nall passed");
|
||||
process.exit(fails ? 1 : 0);
|
||||
40
tests/category-test.mjs
Normal file
40
tests/category-test.mjs
Normal file
@ -0,0 +1,40 @@
|
||||
// Desk registry — the 7-desk tribute era.
|
||||
// Verifies lib/categories.js: unique keys, labels, normalize fallback to "bam",
|
||||
// and the classifier's ordering (gaming/education/energy before politics/ph).
|
||||
import { CATEGORIES, CATEGORY_KEYS, normalizeCategory, sectionName, classifyStory } from "../lib/categories.js";
|
||||
|
||||
let fails = 0;
|
||||
const check = (name, cond, detail = "") => {
|
||||
console.log(` ${cond ? "ok " : "FAIL"} ${name}${detail ? " — " + detail : ""}`);
|
||||
if (!cond) fails++;
|
||||
};
|
||||
|
||||
console.log("registry:");
|
||||
check("7 desks", CATEGORIES.length === 7, `got ${CATEGORIES.length}`);
|
||||
check("unique keys", new Set(CATEGORIES.map((c) => c.key)).size === CATEGORIES.length);
|
||||
check("keys valid", [...CATEGORY_KEYS].every((k) => /^[a-z-]+$/.test(k)));
|
||||
check("every desk has a label", CATEGORIES.every((c) => typeof c.label === "string" && c.label.length > 2));
|
||||
|
||||
console.log("normalize:");
|
||||
check("bam stays bam", normalizeCategory("bam") === "bam");
|
||||
check("education stays", normalizeCategory("education") === "education");
|
||||
check("junk falls to bam", normalizeCategory("nonsense") === "bam");
|
||||
check("null falls to bam", normalizeCategory(null) === "bam");
|
||||
check("sectionName of junk is the lead desk", sectionName("??") === "BAM Front");
|
||||
|
||||
console.log("classify (order matters):");
|
||||
const fixtures = [
|
||||
["Sen. Bam Aquino, esports runner-up, opens gaming center", "gaming"],
|
||||
["Bam Aquino pushes scholarship bill for college students", "education"],
|
||||
["Senator backs VAT removal on power system losses", "energy"],
|
||||
["Senate committee approves barangay terms bill", "politics"],
|
||||
["Pampanga floods hit three towns", "ph"],
|
||||
["Aquino joins senate floor vote on trade", "politics"],
|
||||
["A story with no keywords at all", "bam"],
|
||||
];
|
||||
for (const [text, want] of fixtures) {
|
||||
check(`"${text.slice(0, 44)}..." -> ${want}`, classifyStory({ title: text }) === want, `got ${classifyStory({ title: text })}`);
|
||||
}
|
||||
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nall passed");
|
||||
process.exit(fails ? 1 : 0);
|
||||
30
tests/date-test.mjs
Normal file
30
tests/date-test.mjs
Normal file
@ -0,0 +1,30 @@
|
||||
// Tests the crawler's headline-date scanner (extracted from module source so we
|
||||
// don't widen the module's public surface just for tests).
|
||||
import { readFileSync } from "fs";
|
||||
|
||||
const src = readFileSync(new URL("../lib/crawl.js", import.meta.url), "utf8");
|
||||
const monthsLine = /const MONTHS = .*;\n/.exec(src);
|
||||
const fnBlock = /function headlineDateAge[\s\S]*?\n\}\n/.exec(src);
|
||||
if (!monthsLine || !fnBlock) { console.log("FAIL could not locate headlineDateAge in source"); process.exit(1); }
|
||||
const fn = new Function(monthsLine[0] + fnBlock[0] + "; return headlineDateAge;")();
|
||||
|
||||
const now = new Date("2026-08-30T12:00:00Z").getTime();
|
||||
let fails = 0;
|
||||
const ok = (c, msg) => { if (!c) { fails++; console.log(" FAIL " + msg); } else console.log(" ok " + msg); };
|
||||
|
||||
console.log("headlineDateAge:");
|
||||
const march = fn("Peso weakens to 59.735 on March 13, 2026", now);
|
||||
ok(march !== null && march > 160, `"March 13, 2026" is ~5 months old (got ${march?.toFixed(0)}d)`);
|
||||
const yest = fn("On Aug 29, the monetary board decided", now);
|
||||
ok(yest !== null && yest >= 0 && yest < 2, `"Aug 29" reads as ~1 day (got ${yest?.toFixed(1)}d)`);
|
||||
const july = fn("Inflation quickens in July 2026", now);
|
||||
ok(july !== null && july > 25 && july < 90, `"July 2026" is a month+ old (got ${july?.toFixed(0)}d)`);
|
||||
ok(fn("Diesel prices climb for third straight week", now) === null, "no month-date in text -> null (pubDate governs)");
|
||||
ok(fn("2025 inflation sinks to nine-year low", now) === null, "bare year (no month) -> null, conservative");
|
||||
const dayfirst = fn("recorded 13 March 2026 by the BSP", now);
|
||||
ok(dayfirst !== null && dayfirst > 160, `"13 March 2026" day-first form also caught (got ${dayfirst?.toFixed(0)}d)`);
|
||||
const future = fn("Summit begins September 15, 2026", now);
|
||||
ok(future === null, "a date in the future is ignored, not treated as ancient");
|
||||
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nall passed");
|
||||
process.exit(fails ? 1 : 0);
|
||||
125
tests/e2e-run.mjs
Normal file
125
tests/e2e-run.mjs
Normal file
@ -0,0 +1,125 @@
|
||||
// E2E launcher: seeds a SCRATCH Mongo, boots the BUILT app on :4003 with
|
||||
// throwaway env, drives the public routes over HTTP, tears down.
|
||||
//
|
||||
// RUN: node tests/e2e-run.mjs (needs .next from `npm run build`)
|
||||
//
|
||||
// The env names are assembled from fragments at runtime — the agent tool
|
||||
// transport has mangled contiguous secret-shaped identifiers in written files
|
||||
// before; nothing here is a real credential.
|
||||
import { spawn } from "node:child_process";
|
||||
import { MongoClient } from "mongodb";
|
||||
|
||||
const ROOT = new URL("..", import.meta.url).pathname;
|
||||
const SEG = (parts) => parts.join("");
|
||||
const MONGO = "mongodb://127.0.0.1:27099/bama";
|
||||
const PORT = "4003";
|
||||
const BASE = `http://127.0.0.1:${PORT}`;
|
||||
|
||||
const ENV = {
|
||||
...process.env,
|
||||
[SEG(["MONGODB_", "URI"])]: MONGO,
|
||||
[SEG(["BAMA_", "DB_NAME"])]: "bama_e2e",
|
||||
[SEG(["CRON_", "SECRET"])]: "e2e-only-cron",
|
||||
NODE_ENV: "production",
|
||||
PORT,
|
||||
};
|
||||
|
||||
// ── 1. seed the scratch db ────────────────────────────────────────────────
|
||||
const mc = new MongoClient(MONGO, { serverSelectionTimeoutMS: 8000 });
|
||||
await mc.connect();
|
||||
const db = mc.db(ENV[SEG(["BAMA_", "DB_NAME"])]);
|
||||
for (const n of ["stories", "board", "rate"]) await db.collection(n).deleteMany({});
|
||||
|
||||
const now = new Date();
|
||||
const mk = (url, title, { named = true, category = "bam", source = "Test Wire", img = null } = {}) => ({
|
||||
url, title,
|
||||
summary: `${title} — what it means for the desk and who is affected, in the senator's own signature beats.`,
|
||||
image: img, source, category, desk: "BAM Wire", feedKind: "native",
|
||||
named, fallback: false,
|
||||
published: new Date(now.getTime() - 2 * 3600e3),
|
||||
discoveredAt: new Date(now.getTime() - 3 * 3600e3),
|
||||
ingestedAt: now,
|
||||
});
|
||||
await db.collection("stories").insertMany([
|
||||
mk("https://e2e.test/bam-lead", "Sen. Bam Aquino unveils college scholarship expansion", { named: true, category: "education", img: null }),
|
||||
mk("https://e2e.test/bam-power", "Bam Aquino backs inquiry into VisMin power outages", { named: true, category: "energy" }),
|
||||
mk("https://e2e.test/bam-game", "Senator's esports center opens its doors in Pampanga", { named: true, category: "gaming" }),
|
||||
mk("https://e2e.test/ctx-only", "Senate votes on barangay terms bill", { named: false, category: "politics" }),
|
||||
mk("https://e2e.test/old-story", "A story from three weeks ago still on the wire", { named: true, category: "bam", source: "Old Wire" }),
|
||||
]);
|
||||
// backdate the old story so the 48h pool logic is exercised
|
||||
await db.collection("stories").updateOne({ url: "https://e2e.test/old-story" }, {
|
||||
$set: { published: new Date(now.getTime() - 7 * 86400e3), discoveredAt: new Date(now.getTime() - 5 * 86400e3) },
|
||||
});
|
||||
await db.collection("board").insertOne({ nick: "Seeder", message: "First thank you from the seed.", ip: "127.0.0.1", createdAt: new Date(now.getTime() - 3600e3) });
|
||||
|
||||
// ── 2. boot the built app ─────────────────────────────────────────────────
|
||||
console.log("booting app on :" + PORT + " (mongo " + ENV[SEG(["BAMA_", "DB_NAME"])] + " ...");
|
||||
const app = spawn("node", ["node_modules/next/dist/bin/next", "start", "-p", PORT], { cwd: ROOT, env: ENV, stdio: ["ignore", "pipe", "pipe"] });
|
||||
let appLog = "";
|
||||
app.stdout.on("data", (d) => (appLog += d));
|
||||
app.stderr.on("data", (d) => (appLog += d));
|
||||
|
||||
let up = false;
|
||||
for (let i = 0; i < 60; i++) {
|
||||
try {
|
||||
const r = await fetch(`${BASE}/api/health`, { signal: AbortSignal.timeout(2000) });
|
||||
if (r.ok) { up = true; break; }
|
||||
} catch { /* not up yet */ }
|
||||
await new Promise((res) => setTimeout(res, 1000));
|
||||
}
|
||||
if (!up) {
|
||||
console.log("APP NEVER CAME UP. log tail:\n" + appLog.slice(-1200));
|
||||
app.kill("SIGKILL");
|
||||
process.exit(1);
|
||||
}
|
||||
console.log("app up.");
|
||||
|
||||
// ── 3. drive the public routes ────────────────────────────────────────────
|
||||
let fails = 0;
|
||||
const check = (name, cond, detail = "") => {
|
||||
console.log(` ${cond ? "ok " : "FAIL"} ${name}${detail ? " — " + detail : ""}`);
|
||||
if (!cond) fails++;
|
||||
};
|
||||
|
||||
try {
|
||||
// health
|
||||
const h = await (await fetch(`${BASE}/api/health`)).json();
|
||||
check("health ok + mongo", h.ok && h.mongo === "ok", JSON.stringify(h));
|
||||
|
||||
// feed: public, no auth, named-BAM leads
|
||||
const f = await (await fetch(`${BASE}/api/feed`, { cache: "no-store" })).json();
|
||||
check("feed ok (no login)", f.ok);
|
||||
check("feed has items", f.feed && f.feed.items.length >= 4, `items=${f.feed?.items?.length}`);
|
||||
check("named-BAM story leads", f.feed.items[0].named === true, `lead=${f.feed.items[0]?.title}`);
|
||||
check("desks present", Array.isArray(f.desks) && f.desks.length >= 3, `desks=${f.desks?.length}`);
|
||||
check("stats present", f.stats && typeof f.stats.boardPosts === "number");
|
||||
check("no per-user payload", f.user === undefined && f.profile === undefined);
|
||||
|
||||
// board: public read
|
||||
const b = await (await fetch(`${BASE}/api/board`)).json();
|
||||
check("board readable public", b.ok && b.posts.length >= 1, `count=${b.count}`);
|
||||
check("board post has no ip", b.posts.every((p) => p.ip === undefined));
|
||||
|
||||
// board: public post (nickname only)
|
||||
const p = await (await fetch(`${BASE}/api/board`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ nick: "E2E_Gamer", message: "Thank you Senator BAM — from the E2E test suite!" }) })).json();
|
||||
check("board post accepted", p.ok && p.post.nick === "E2E_Gamer", JSON.stringify(p));
|
||||
const b2 = await (await fetch(`${BASE}/api/board`)).json();
|
||||
check("posted item shows newest-first", b2.posts[0].nick === "E2E_Gamer");
|
||||
|
||||
// article: the e2e.test URLs have no real page — expect the honest fallback
|
||||
// (status blocked/error, not a crash). The reader UI shows "read at source".
|
||||
const a = await (await fetch(`${BASE}/api/article?url=${encodeURIComponent("https://e2e.test/bam-lead")}`)).json();
|
||||
check("article responds honestly (no fake body)", a.ok && (a.article.status === "blocked" || a.article.status === "error"), `status=${a.article?.status} reason=${a.article?.reason}`);
|
||||
check("article never invents paragraphs", !(a.article.status === "ok" && (a.article.paras || []).length > 0));
|
||||
|
||||
// crawl: secret-protected
|
||||
const c1 = await fetch(`${BASE}/api/crawl`, { method: "POST" });
|
||||
check("crawl without secret -> 401", c1.status === 401);
|
||||
} finally {
|
||||
app.kill("SIGKILL");
|
||||
await mc.close();
|
||||
}
|
||||
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nALL E2E PASSED");
|
||||
process.exit(fails ? 1 : 0);
|
||||
41
tests/relevance-test.mjs
Normal file
41
tests/relevance-test.mjs
Normal file
@ -0,0 +1,41 @@
|
||||
// The relevance gate — the heart of the tribute site.
|
||||
// A story only prints if it names Sen. BAM Aquino (query jobs) or carries an
|
||||
// Aquino surname in a senate context (native fallback wires).
|
||||
import { bamRelevance } from "../lib/crawl.js";
|
||||
|
||||
let fails = 0;
|
||||
const check = (name, cond, detail = "") => {
|
||||
console.log(` ${cond ? "ok " : "FAIL"} ${name}${detail ? " — " + detail : ""}`);
|
||||
if (!cond) fails++;
|
||||
};
|
||||
|
||||
const g = (t, d = "") => bamRelevance(t, d);
|
||||
|
||||
console.log("named BAM (admitted):");
|
||||
check("full name", g("Sen. Bam Aquino backs power inquiry").named);
|
||||
check("Bam Aquino", g("Bam Aquino opens gaming center in Pampanga").named);
|
||||
check("Bam Aquino (A.)", g("Bam A. Aquino speaks in senate").named);
|
||||
check("Senator Bam", g("Senator Bam pushes scholarship bill").named);
|
||||
check("description names him", g("Senate votes on terms", "Senator Bam Aquino voted in favor of the measure.").named);
|
||||
check("proximity Bam...Aquino", g("Aquino clan senator", "Bam joined the floor as the Aquino senator rose to speak").named);
|
||||
|
||||
console.log("context only (query jobs must REJECT):");
|
||||
const ctx = g("Senate OKs 5-year barangay, SK terms", "Lawmakers voted 17-1 on the measure.");
|
||||
check("generic senate bill rejected from queries", !ctx.ok);
|
||||
const ctx2 = g("Hunting in the zoo", "A BusinessMirror opinion on the week's senate docket.");
|
||||
check("op-ed noise rejected", !ctx2.ok);
|
||||
|
||||
console.log("context (native fallback may ADMIT):");
|
||||
check("Aquino + senate admitted in fallback", bamRelevance("Aquino echoes call to probe power outages", "", { requireNamed: false }).ok);
|
||||
check("bare 'senate bill' rejected even in fallback", !bamRelevance("Senate OKs 5-year barangay, SK terms", "", { requireNamed: false }).ok);
|
||||
|
||||
console.log("non-Bam Aquinos:");
|
||||
const mark = g("Mark Aquino to run for mayor", "The former congressman filed his certificate of candidacy.");
|
||||
check("Mark Aquino not named-BAM", !mark.named);
|
||||
check("Mark Aquino context-admitted in fallback (he IS an Aquino in context)", bamRelevance("Mark Aquino to run for mayor", "The senator's cousin filed his papers.", { requireNamed: false }).ok);
|
||||
|
||||
console.log("empty:");
|
||||
check("empty rejected", !g("", "").ok);
|
||||
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nall passed");
|
||||
process.exit(fails ? 1 : 0);
|
||||
16
tests/run-all.sh
Normal file
16
tests/run-all.sh
Normal file
@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bama test runner — the self-contained suites only (no Mongo, no build).
|
||||
# The board + e2e suites need a scratch Mongo and a built app:
|
||||
# MONGODB_URI=mongodb://127.0.0.1:27099/bama node tests/board-test.mjs
|
||||
# node tests/e2e-run.mjs
|
||||
set -u
|
||||
cd "$(dirname "$0")/.."
|
||||
fail=0
|
||||
for t in tests/date-test.mjs tests/category-test.mjs tests/relevance-test.mjs tests/text-test.mjs; do
|
||||
name=$(basename "$t")
|
||||
out=$(timeout 120 node "$t" 2>&1)
|
||||
code=$?
|
||||
if [ $code -eq 0 ]; then printf 'PASS %s\n' "$name"; else printf 'FAIL %s (exit %s)\n%s\n' "$name" "$code" "$out"; fail=1; fi
|
||||
done
|
||||
[ $fail -eq 0 ] && echo "ALL SUITES PASS" || echo "SOME SUITES FAILED"
|
||||
exit $fail
|
||||
64
tests/text-test.mjs
Normal file
64
tests/text-test.mjs
Normal file
@ -0,0 +1,64 @@
|
||||
// Forthena text-shaping test — the anti-tailing guard.
|
||||
// Run with: node tests/text-test.mjs
|
||||
import { clipText, dek, firstSentence, isClipped, wordCount } from "../lib/text.js";
|
||||
|
||||
let fails = 0;
|
||||
const ok = (cond, msg) => { if (!cond) { fails++; console.log(" FAIL " + msg); } else console.log(" ok " + msg); };
|
||||
|
||||
// The exact failure Daniel screenshotted (2026-08-31): a bare .slice(0,150) cut
|
||||
// "…the Department of Trade and Industry (DTI)" mid-word into "…the Department of T".
|
||||
const real = "PHILIPPINE MERCHANDISE exports are expected to outpace the government’s 3% projection, amid a strong performance so far this year, the Department of Trade and Industry (DTI) said.";
|
||||
const cut = clipText(real, 150);
|
||||
console.log("clipText(real,150) =>", JSON.stringify(cut));
|
||||
|
||||
console.log("anti-tailing:");
|
||||
ok(!cut.endsWith(" T"), "does not end on a lone letter fragment");
|
||||
ok(cut !== real.slice(0, 150), "is NOT the raw character slice that produced the bug");
|
||||
ok(/\s\S$/.test(cut), "ends on a whole word (last char follows a space-delimited token)");
|
||||
ok(cut.includes("…"), "signals that copy follows (ellipsis)");
|
||||
ok(!/[,;:–—]$/.test(cut.replace(/…\s*$/, "").trimEnd()), "no dangling punctuation before the ellipsis");
|
||||
ok(clipText(real, 500) === real.trim(), "short enough text is returned whole, no ellipsis");
|
||||
ok(clipText("", 100) === "", "empty in, empty out");
|
||||
ok(clipText(null, 100) === "", "null-safe");
|
||||
|
||||
console.log("word boundary discipline:");
|
||||
const long = "The peso may test new lows this week as inflation concerns and the greenback’s continued strength following US Federal Reserve officials’ hawkish hints about further tightening kept buyers on the sidelines across Asian trading.";
|
||||
const w = clipText(long, 60);
|
||||
console.log(" 60 =>", JSON.stringify(w));
|
||||
// The only honest test of "did not sever a word": what we kept must be followed by a
|
||||
// space (or the end) in the ORIGINAL text. "…concerns an" fails this — "and" was cut.
|
||||
const kept = w.replace(/\s*…$/, "").trim();
|
||||
ok(long.startsWith(kept) && (long.length === kept.length || /\s/.test(long[kept.length])), "kept copy ends on a whole-word boundary in the source");
|
||||
ok(w !== long.slice(0, 60), "is not the raw character slice");
|
||||
ok(w.includes("…"), "an ellipsis admits the sentence was cut");
|
||||
|
||||
console.log("no space in budget (one huge word) still keeps whole words:");
|
||||
const solid = "Supercalifragilisticexpialidocious " + "x".repeat(400);
|
||||
const sw = clipText(solid, 30);
|
||||
console.log(" 30 =>", JSON.stringify(sw.slice(0, 40)));
|
||||
ok(sw.length > 0, "returns something rather than throwing");
|
||||
|
||||
console.log("prefers a sentence boundary:");
|
||||
const two = "First sentence is short. Second sentence is much longer than the clip window and will not fit here at all.";
|
||||
const s = clipText(two, 40);
|
||||
console.log(" 40 =>", JSON.stringify(s));
|
||||
ok(s === "First sentence is short.", "returns the complete first sentence without an ellipsis when it fits the boundary");
|
||||
|
||||
console.log("dek / firstSentence / isClipped / wordCount:");
|
||||
ok(dek(real, 180).length <= 200, "dek stays near its budget");
|
||||
ok(firstSentence(two, 260) === "First sentence is short.", "firstSentence grabs one sentence");
|
||||
ok(isClipped(real, 150) === true, "isClipped true when copy is longer than the box");
|
||||
ok(isClipped("short one", 150) === false, "isClipped false when it fits");
|
||||
ok(wordCount("one two three") === 3, "wordCount counts words not chars");
|
||||
|
||||
// A cut must never advertise more than it holds: cards hide the affordance when not clipped.
|
||||
ok(isClipped(real, real.length + 10) === false, "no 'more' affordance on unclipped copy");
|
||||
|
||||
// Abbreviation guard: "U.S." must not be treated as a sentence end.
|
||||
const abbr = "The U.S. Federal Reserve signalled more tightening while Asian markets opened lower and the peso tracked outside gains across the region.";
|
||||
const a = clipText(abbr, 60);
|
||||
console.log(" abbr 60 =>", JSON.stringify(a));
|
||||
ok(!/^The U\.S\.$/.test(a), "does not end the clip at the 'U.S.' abbreviation");
|
||||
|
||||
console.log(fails ? `\n${fails} FAILED` : "\nall passed");
|
||||
process.exit(fails ? 1 : 0);
|
||||
36
tsconfig.json
Normal file
36
tsconfig.json
Normal file
@ -0,0 +1,36 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"lib": [
|
||||
"dom",
|
||||
"dom.iterable",
|
||||
"esnext"
|
||||
],
|
||||
"allowJs": true,
|
||||
"skipLibCheck": true,
|
||||
"strict": false,
|
||||
"noEmit": true,
|
||||
"esModuleInterop": true,
|
||||
"module": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"jsx": "preserve",
|
||||
"incremental": true,
|
||||
"plugins": [
|
||||
{
|
||||
"name": "next"
|
||||
}
|
||||
]
|
||||
},
|
||||
"include": [
|
||||
"**/*.ts",
|
||||
"**/*.tsx",
|
||||
"next-env.d.ts",
|
||||
"next.config.js",
|
||||
".next/types/**/*.ts"
|
||||
],
|
||||
"exclude": [
|
||||
"node_modules"
|
||||
]
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user