ScraperControl/docs/superpowers/plans/2026-03-17-buscarmisas-network-importer.md

# BuscarMisas Network Importer — Implementation Plan

> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add a single config-driven importer that scrapes ~15,294 Catholic churches and mass schedules from 5 Latin American WordPress-based directories (Brazil, Mexico, Argentina, Colombia, Chile).

**Architecture:** A `NETWORK_SITES` config map drives a single `import-buscarmisas-network.ts` script. Church HTML parsing extracts name, address, phone, coordinates, and weekly schedule. The external ID `{domain-slug}/{church-slug}` stored in a new `buscarmisasNetworkId` column prevents duplicate inserts on re-runs.

**Tech Stack:** TypeScript, tsx, Prisma 7 + pg adapter, existing `church-matcher.ts` + `day-names.ts` utilities.

---

## Chunk 1: Schema prerequisite + church-matcher update

### Task 1: Add `buscarmisasNetworkId` to BethelGuide schema

> ⚠️ BethelGuide is the schema source of truth. Never run `prisma migrate` in ScraperControl.

**Files:**
- Modify (in BethelGuide repo): `prisma/schema.prisma`
- Modify (in BethelGuide repo): migration SQL file

- [ ] **Step 1: In BethelGuide, open `prisma/schema.prisma` and add the column to the `Church` model**

Add after the existing `discovermassId` line:
```prisma
buscarmisasNetworkId  String?  @unique  @map("buscarmisas_network_id")
```

And add to the `@@index` block at the bottom of the `Church` model:
```prisma
@@index([buscarmisasNetworkId])
```

- [ ] **Step 2: In BethelGuide, create and run the migration**

```bash
npx prisma migrate dev --name add_buscarmisas_network_id
```

Expected: migration file created, column added to the shared PostgreSQL database.

- [ ] **Step 3: Sync the updated schema to ScraperControl**

```bash
cp prisma/schema.prisma ~/Documents/ScraperControl/prisma/schema.prisma
```

- [ ] **Step 4: Regenerate Prisma client in ScraperControl**

```bash
cd ~/Documents/ScraperControl
npx prisma generate
```

Expected: no errors, `@prisma/client` regenerated with `buscarmisasNetworkId` field.

- [ ] **Step 5: Verify the field is available**

```bash
npx tsx -e "
import { Pool } from 'pg';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '@prisma/client';
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const prisma = new PrismaClient({ adapter: new PrismaPg(pool) });
prisma.church.findFirst({ select: { buscarmisasNetworkId: true } }).then(r => {
  console.log('buscarmisasNetworkId field present:', JSON.stringify(r));
  return prisma.\$disconnect().then(() => pool.end());
});
"
```

Expected: prints `buscarmisasNetworkId field present: null` or `{}` (not a type error).

- [ ] **Step 6: Commit in ScraperControl**

```bash
git add prisma/schema.prisma
git commit -m "chore: sync schema — add buscarmisasNetworkId column"
```

---

### Task 2: Update `church-matcher.ts` with new field + ID-match pass

**Files:**
- Modify: `src/lib/church-matcher.ts`

- [ ] **Step 1: Add `buscarmisasNetworkId` to `ExistingChurch` interface**

In `src/lib/church-matcher.ts`, find the `ExistingChurch` interface (line ~11). The interface currently ends with `gottesdienstzeitenId: string | null;` followed by `source: string;`. Insert the two new fields immediately before the `source:` line:

```ts
  discovermassId: string | null;
  buscarmisasNetworkId: string | null;
  source: string;   // ← already exists, shown for placement only
```

Note: `discovermassId` was missing from the interface (pre-existing gap) — adding it here ensures the `loadExistingChurches` select in Task 5 compiles correctly.

- [ ] **Step 2: Add `buscarmisasNetworkId` to `ChurchCandidate` type**

Find the `ChurchCandidate` type (line ~122). After the existing `horariosMisasId?: string;` and all other existing optional ID fields, add:

```ts
  discovermassId?: string;
  buscarmisasNetworkId?: string;
```

- [ ] **Step 3: Add ID-match passes in `findDuplicateChurch`**

The existing passes run 1–13 (osmId through gottesdienstzeitenId), with pass 14 being proximity+name at line ~259. Find the **Thirteenth pass** block (gottesdienstzeitenId, line ~251):

```ts
// Thirteenth pass: exact gottesdienstzeitenId match
if (candidate.gottesdienstzeitenId) {
  ...
}
```

Insert two new passes **after** it and **before** the proximity pass comment (`// Fourteenth pass: proximity + name match`):

```ts
// Fourteenth pass: exact discovermassId match
if (candidate.discovermassId) {
  const match = existingChurches.find(
    (church) => church.discovermassId === candidate.discovermassId
  );
  if (match) return match;
}

// Fifteenth pass: exact buscarmisasNetworkId match
if (candidate.buscarmisasNetworkId) {
  const match = existingChurches.find(
    (church) => church.buscarmisasNetworkId === candidate.buscarmisasNetworkId
  );
  if (match) return match;
}
```

Then update the existing proximity pass comment from `// Fourteenth pass:` to `// Sixteenth pass:`.

- [ ] **Step 4: Verify TypeScript compiles**

```bash
npx tsc --noEmit
```

Expected: 0 errors.

- [ ] **Step 5: Commit**

```bash
git add src/lib/church-matcher.ts
git commit -m "feat: add buscarmisasNetworkId (and discovermassId) to church-matcher interfaces and ID-match passes"
```

---

## Chunk 2: Parsing functions

### Task 3: Write pure parsing functions with unit tests

**Files:**
- Create: `scripts/import-buscarmisas-network.ts` (scaffold with parsing functions only)

We write the parsing functions first as pure functions, then test them with real HTML snippets before wiring them to the HTTP layer.

- [ ] **Step 1: Create `scripts/import-buscarmisas-network.ts` with the file header and types**

```ts
#!/usr/bin/env tsx
/**
 * Import Catholic churches and mass schedules from the BuscarMisas network.
 *
 * A group of 5 identical WordPress-based directories covering Latin America:
 *   - horariosmissa.com.br  (Brazil,    ~4,732 churches)
 *   - buscarmisas.com.mx   (Mexico,    ~3,950 churches)
 *   - horariosmisa.com.ar  (Argentina, ~3,012 churches)
 *   - buscarmisas.co       (Colombia,  ~2,665 churches)
 *   - horariomisa.cl       (Chile,       ~935 churches)
 *
 * Usage:
 *   npx tsx scripts/import-buscarmisas-network.ts --domain horariosmissa.com.br
 *   npx tsx scripts/import-buscarmisas-network.ts --domain horariosmissa.com.br --resume-from 500
 *   npx tsx scripts/import-buscarmisas-network.ts --all
 *   npx tsx scripts/import-buscarmisas-network.ts --domain horariosmissa.com.br --dry-run
 */

import dotenv from 'dotenv';
import path from 'path';

dotenv.config({ path: path.resolve(process.cwd(), '.env.local') });
dotenv.config({ path: path.resolve(process.cwd(), '.env') });

import { Pool } from 'pg';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '@prisma/client';

import { findDuplicateChurch } from '../src/lib/church-matcher';
import type { ExistingChurch } from '../src/lib/church-matcher';
import { getDayNamesForCountry, buildDayPatterns } from '../src/scrapers/i18n/day-names';

// ─── Site Config ─────────────────────────────────────────────────────────────

interface SiteConfig {
  country: string;          // ISO 3166-1 alpha-2
  language: 'pt' | 'es';
  sitemapType: 'page' | 'post';
}

const NETWORK_SITES: Record<string, SiteConfig> = {
  'horariosmissa.com.br': { country: 'BR', language: 'pt', sitemapType: 'page' },
  'buscarmisas.com.mx':   { country: 'MX', language: 'es', sitemapType: 'page' },
  'horariosmisa.com.ar':  { country: 'AR', language: 'es', sitemapType: 'page' },
  'buscarmisas.co':       { country: 'CO', language: 'es', sitemapType: 'page' },
  'horariomisa.cl':       { country: 'CL', language: 'es', sitemapType: 'post' },
};

// ─── Types ────────────────────────────────────────────────────────────────────

interface ParsedChurch {
  name: string;
  address: string | null;
  city: string | null;
  state: string | null;
  phone: string | null;
  lat: number;
  lng: number;
  externalId: string;
  country: string;
}

interface ParsedMass {
  dayOfWeek: number;  // 0 = Sunday, 6 = Saturday
  time: string;       // HH:MM 24-hour
}

interface CLIArgs {
  domain: string | null;
  all: boolean;
  dryRun: boolean;
  resumeFrom: number;
  limit: number | null;
  jobId: string | null;
}

interface ImportStats {
  total: number;
  created: number;
  updated: number;
  skipped: number;
  errors: number;
  massSchedulesCreated: number;
}
```

- [ ] **Step 2: Add `buildExternalId` helper**

```ts
// ─── Helpers ─────────────────────────────────────────────────────────────────

/**
 * Build external ID for a church URL.
 * Format: "{domain-slug}/{church-slug}"
 * e.g. "horariosmissa-com-br/paroquia-nossa-senhora-dos-remedios"
 */
export function buildExternalId(domain: string, churchUrl: string): string {
  const domainSlug = domain.replace(/\./g, '-');
  // URL path: /{region}/{city}/{church-slug}/
  const segments = churchUrl.replace(/\/$/, '').split('/').filter(Boolean);
  const churchSlug = segments[segments.length - 1] || '';
  return `${domainSlug}/${churchSlug}`;
}
```

- [ ] **Step 3: Verify `buildExternalId` manually**

```bash
npx tsx -e "
import { buildExternalId } from './scripts/import-buscarmisas-network';
console.log(buildExternalId('horariosmissa.com.br', 'https://horariosmissa.com.br/sao-paulo/sao-paulo/paroquia-nossa-senhora-dos-remedios/'));
// Expected: horariosmissa-com-br/paroquia-nossa-senhora-dos-remedios
console.log(buildExternalId('buscarmisas.co', 'https://buscarmisas.co/bogota/bogota/parroquia-san-pedro/'));
// Expected: buscarmisas-co/parroquia-san-pedro
"
```

- [ ] **Step 4: Add `parseChurchPage` function**

```ts
/**
 * Parse church data from a church page HTML string.
 * Returns null if name or coordinates cannot be extracted.
 */
export function parseChurchPage(
  html: string,
  domain: string,
  churchUrl: string,
  config: SiteConfig,
): ParsedChurch | null {
  // Name: cell after <strong>Nome</strong> (PT) or <strong>Nombre</strong> (ES)
  const nameLabel = config.language === 'pt' ? 'Nome' : 'Nombre';
  const nameMatch = html.match(
    new RegExp(`<strong>${nameLabel}<\\/strong><\\/td>\\s*<td>([^<]+)<\\/td>`, 'i')
  );
  const name = nameMatch?.[1]?.trim() ?? '';
  if (!name) return null;

  // Coordinates: Google Maps iframe center= parameter
  const coordMatch = html.match(/center=([-\d.]+)%2C([-\d.]+)/i);
  if (!coordMatch) return null;
  const lat = parseFloat(coordMatch[1]);
  const lng = parseFloat(coordMatch[2]);
  if (!isFinite(lat) || !isFinite(lng) || Math.abs(lat) > 90 || Math.abs(lng) > 180) return null;

  // Address: cell after <strong>Endereço</strong> (PT) or <strong>Dirección</strong> (ES)
  const addrLabel = config.language === 'pt' ? 'Endere[çc]o' : 'Direcci[oó]n';
  const addrMatch = html.match(
    new RegExp(`<strong>${addrLabel}<\\/strong><\\/td>\\s*<td>([^<]+)<\\/td>`, 'i')
  );
  const address = addrMatch?.[1]?.trim() ?? null;

  // Phone: tel: href
  const phoneMatch = html.match(/href="tel:([^"]+)"/i);
  const phone = phoneMatch?.[1]?.trim() ?? null;

  // City and state from URL path segments
  const segments = churchUrl.replace(/\/$/, '').split('/').filter(Boolean);
  // segments[2] = region/state, segments[3] = city (after domain), but URL is full URL
  // URL form: https://{domain}/{state}/{city}/{slug}/
  const urlPath = new URL(churchUrl).pathname.split('/').filter(Boolean);
  const state = urlPath[0] ? decodeURIComponent(urlPath[0].replace(/-/g, ' ')) : null;
  const city  = urlPath[1] ? decodeURIComponent(urlPath[1].replace(/-/g, ' ')) : null;

  return {
    name,
    address,
    city,
    state,
    phone,
    lat,
    lng,
    externalId: buildExternalId(domain, churchUrl),
    country: config.country,
  };
}
```

- [ ] **Step 5: Add `parseMassSchedule` function**

```ts
/**
 * Parse the weekly mass schedule table from church page HTML.
 * Table format: day-name cell | time cell (comma-separated times, "-" = no mass)
 */
export function parseMassSchedule(html: string, countryCode: string): ParsedMass[] {
  const dayPatterns = buildDayPatterns(getDayNamesForCountry(countryCode));
  const results: ParsedMass[] = [];

  // Extract all <td> cells as pairs [day, time]
  const cells = [...html.matchAll(/<td[^>]*>(.*?)<\/td>/gis)].map(m =>
    m[1].replace(/<[^>]+>/g, '').trim()
  );

  for (let i = 0; i + 1 < cells.length; i += 2) {
    const dayCell = cells[i].toLowerCase();
    const timeCell = cells[i + 1];

    const dayOfWeek = dayPatterns[dayCell];
    if (dayOfWeek === undefined) continue;
    if (timeCell === '-' || !timeCell) continue;

    // Split comma-separated times: "10:00, 18:00" → ["10:00", "18:00"]
    for (const rawTime of timeCell.split(',')) {
      const time = rawTime.trim();
      if (/^\d{1,2}:\d{2}$/.test(time)) {
        results.push({ dayOfWeek, time });
      }
    }
  }
  return results;
}
```

- [ ] **Step 6: Test `parseChurchPage` and `parseMassSchedule` with real HTML**

```bash
npx tsx -e "
import { parseChurchPage, parseMassSchedule } from './scripts/import-buscarmisas-network';

const NETWORK_SITES = {
  'horariosmissa.com.br': { country: 'BR', language: 'pt', sitemapType: 'page' },
};

async function test() {
  const res = await fetch('https://horariosmissa.com.br/sao-paulo/sao-paulo/paroquia-nossa-senhora-dos-remedios/');
  const html = await res.text();
  const config = NETWORK_SITES['horariosmissa.com.br'];
  const parsed = parseChurchPage(html, 'horariosmissa.com.br', res.url, config);
  console.log('Church:', JSON.stringify(parsed, null, 2));
  const masses = parseMassSchedule(html, config.country);
  console.log('Masses:', JSON.stringify(masses, null, 2));
}
test().catch(console.error);
"
```

Expected output (exact values are illustrative — website content may change):
```
Church: {
  "name": "Paróquia Nossa Senhora dos Remédios",   // or current name
  "address": "R. Ten. Azevedo, 182 ...",
  "city": "sao paulo",
  "state": "sao paulo",
  "phone": "+55 11 ...",
  "lat": -23.56...,
  "lng": -46.62...,
  "externalId": "horariosmissa-com-br/paroquia-nossa-senhora-dos-remedios",
  "country": "BR"
}
Masses: [ { "dayOfWeek": 2, "time": "17:00" }, ... ]
```

Verify: `church` is non-null, `lat`/`lng` are non-zero finite numbers, `externalId` matches `horariosmissa-com-br/{slug}` pattern, `masses` array is non-empty with dayOfWeek 0–6 and HH:MM times.

- [ ] **Step 7: Test with a Spanish-language site (Mexico)**

```bash
npx tsx -e "
import { parseChurchPage, parseMassSchedule } from './scripts/import-buscarmisas-network';
const config = { country: 'MX', language: 'es', sitemapType: 'page' };
const domain = 'buscarmisas.com.mx';
const url = 'https://buscarmisas.com.mx/nuevo-leon/monterrey/parroquia-anunciacion-a-maria/';
fetch(url).then(r => r.text()).then(html => {
  console.log('Church:', JSON.stringify(parseChurchPage(html, domain, url, config), null, 2));
  console.log('Masses:', JSON.stringify(parseMassSchedule(html, config.country), null, 2));
}).catch(console.error);
"
```

Expected: name, coordinates, and Spanish-language schedule rows parsed correctly.

- [ ] **Step 8: Commit parsing scaffold**

```bash
git add scripts/import-buscarmisas-network.ts
git commit -m "feat: add buscarmisas-network importer — parsing functions"
```

---

### Task 4: Sitemap discovery function

**Files:**
- Modify: `scripts/import-buscarmisas-network.ts`

- [ ] **Step 1: Add HTTP helpers**

```ts
// ─── HTTP Helpers ─────────────────────────────────────────────────────────────

const USER_AGENT = 'NearestMass-Importer/1.0 (parish data aggregator; contact: privacy@nearestmass.com)';
const REQUEST_DELAY_MS = 2_000;
const DOMAIN_DELAY_MS  = 5_000;

async function fetchText(url: string): Promise<string> {
  const res = await fetch(url, { headers: { 'User-Agent': USER_AGENT } });
  if (!res.ok) throw new Error(`HTTP ${res.status} for ${url}`);
  return res.text();
}

async function fetchWithRetry(url: string, retries = 3): Promise<string> {
  for (let attempt = 1; attempt <= retries; attempt++) {
    try {
      return await fetchText(url);
    } catch (err) {
      const msg = err instanceof Error ? err.message : String(err);
      if (attempt === retries) throw err;
      const isRetryable = msg.includes('429') || msg.includes('503');
      if (!isRetryable) throw err;
      const backoff = attempt * 30_000; // 30s, 60s, 90s
      console.warn(`  [retry ${attempt}/${retries}] ${msg} — waiting ${backoff / 1000}s`);
      await sleep(backoff);
    }
  }
  throw new Error('unreachable');
}

function sleep(ms: number): Promise<void> {
  return new Promise(resolve => setTimeout(resolve, ms));
}
```

- [ ] **Step 2: Add `getChurchUrls` function**

```ts
// ─── Sitemap Discovery ────────────────────────────────────────────────────────

/**
 * Fetch all church page URLs for a domain from its sitemap.
 * Church URLs have exactly 3 path segments: /{region}/{city}/{slug}/
 */
export async function getChurchUrls(domain: string, config: SiteConfig): Promise<string[]> {
  const indexUrl = `https://${domain}/sitemap_index.xml`;
  console.log(`Fetching sitemap index: ${indexUrl}`);
  const indexXml = await fetchWithRetry(indexUrl);

  // Extract child sitemap URLs matching the sitemapType
  const childPattern = config.sitemapType === 'page'
    ? /https:\/\/[^<]*\/page-sitemap\d*\.xml/g
    : /https:\/\/[^<]*\/post-sitemap\.xml/g;

  const childUrls = [...indexXml.matchAll(childPattern)].map(m => m[0]);
  console.log(`  Found ${childUrls.length} child sitemaps`);

  const churchUrls: string[] = [];
  for (const sitemapUrl of childUrls) {
    const xml = await fetchWithRetry(sitemapUrl);
    const locs = [...xml.matchAll(/<loc>([^<]+)<\/loc>/g)].map(m => m[1].trim());
    for (const loc of locs) {
      // Church URLs: exactly 3 non-empty path segments after the domain
      try {
        const segments = new URL(loc).pathname.split('/').filter(Boolean);
        if (segments.length === 3) {
          churchUrls.push(loc);
        }
      } catch { /* skip malformed URLs */ }
    }
  }

  // Deduplicate
  const unique = [...new Set(churchUrls)];
  console.log(`  Total church URLs: ${unique.length}`);
  return unique;
}
```

- [ ] **Step 3: Verify sitemap discovery against known counts**

```bash
npx tsx -e "
import { getChurchUrls } from './scripts/import-buscarmisas-network';
const NETWORK_SITES = {
  'horariosmissa.com.br': { country: 'BR', language: 'pt', sitemapType: 'page' },
  'horariomisa.cl':       { country: 'CL', language: 'es', sitemapType: 'post' },
};
for (const [domain, config] of Object.entries(NETWORK_SITES)) {
  const urls = await getChurchUrls(domain, config);
  console.log(domain, '->', urls.length, 'churches');
  console.log('  Sample:', urls[0]);
}
"
```

Expected: Brazil ~4,700+ URLs, Chile ~930+ URLs.

- [ ] **Step 4: Commit**

```bash
git add scripts/import-buscarmisas-network.ts
git commit -m "feat: add buscarmisas-network importer — sitemap discovery"
```

---

## Chunk 3: Main importer

### Task 5: DB helpers and church processing loop

**Files:**
- Modify: `scripts/import-buscarmisas-network.ts`

- [ ] **Step 1: Add DB connection and `loadExistingChurches`**

At the top of the file (after dotenv), add the DB setup:

```ts
const dbUrl = process.env.DATABASE_URL || 'postgresql://postgres:postgres@localhost:5432/nearestmass';
console.log(`Connecting to: ${dbUrl.replace(/:[^:@]+@/, ':***@')}`);
const pool = new Pool({ connectionString: dbUrl, ssl: dbUrl.includes('neon') ? { rejectUnauthorized: false } : undefined });
const adapter = new PrismaPg(pool);
const prisma = new PrismaClient({ adapter });
```

Then add `loadExistingChurches`:

```ts
// ─── DB Helpers ───────────────────────────────────────────────────────────────

async function loadExistingChurches(country: string): Promise<ExistingChurch[]> {
  console.log(`Loading existing ${country} churches from DB...`);
  const churches = await prisma.church.findMany({
    where: { country },
    select: {
      id: true, name: true, latitude: true, longitude: true,
      osmId: true, baiduId: true, masstimesId: true,
      orarimesseId: true, massSchedulesPhId: true, philmassId: true,
      horariosMisasId: true, mszeInfoId: true, weekdayMassesId: true,
      messesInfoId: true, bohosluzbyId: true, miserendId: true,
      kerknetId: true, gottesdienstzeitenId: true, discovermassId: true,
      buscarmisasNetworkId: true,
      source: true, website: true, phone: true, address: true, country: true,
    },
  });
  console.log(`  Loaded ${churches.length} existing ${country} churches`);
  return churches as ExistingChurch[];
}
```

- [ ] **Step 2: Add `processChurch` function**

```ts
// ─── Church Processing ────────────────────────────────────────────────────────

async function processChurch(
  url: string,
  domain: string,
  config: SiteConfig,
  existingChurches: ExistingChurch[],
  args: CLIArgs,
  stats: ImportStats,
): Promise<void> {
  stats.total++;
  try {
    const html = await fetchWithRetry(url);
    const parsed = parseChurchPage(html, domain, url, config);
    if (!parsed) {
      console.log(`  [skip] No name/coords: ${url}`);
      stats.skipped++;
      return;
    }

    const masses = parseMassSchedule(html, config.country);

    if (args.dryRun) {
      console.log(`  [dry-run] ${parsed.name} — ${masses.length} masses`);
      return;
    }

    const candidate = {
      name: parsed.name,
      lat: parsed.lat,
      lng: parsed.lng,
      buscarmisasNetworkId: parsed.externalId,
    };
    const duplicate = findDuplicateChurch(candidate, existingChurches);

    if (duplicate) {
      const updateData: Record<string, unknown> = { buscarmisasNetworkId: parsed.externalId };
      if (!duplicate.phone && parsed.phone) updateData.phone = parsed.phone;
      if (parsed.lat !== 0 && duplicate.latitude === 0) {
        updateData.latitude = parsed.lat;
        updateData.longitude = parsed.lng;
      }

      await prisma.$transaction(async (tx) => {
        await tx.church.update({ where: { id: duplicate.id }, data: updateData });
        if (masses.length > 0) {
          await tx.massSchedule.deleteMany({ where: { churchId: duplicate.id } });
          await tx.massSchedule.createMany({
            data: masses.map(m => ({ churchId: duplicate.id, dayOfWeek: m.dayOfWeek, time: m.time, language: config.language === 'pt' ? 'Portuguese' : 'Spanish', notes: null })),
          });
        }
        await tx.church.update({ where: { id: duplicate.id }, data: { lastScrapedAt: new Date() } });
      });
      duplicate.buscarmisasNetworkId = parsed.externalId;
      stats.updated++;
    } else {
      const church = await prisma.church.create({
        data: {
          name: parsed.name,
          address: parsed.address,
          city: parsed.city,
          state: parsed.state,
          country: parsed.country,
          phone: parsed.phone,
          latitude: parsed.lat,
          longitude: parsed.lng,
          buscarmisasNetworkId: parsed.externalId,
          source: 'buscarmisas-network',
          hasWebsite: false,
        },
      });

      existingChurches.push({
        id: church.id, name: parsed.name, latitude: parsed.lat, longitude: parsed.lng,
        osmId: null, baiduId: null, masstimesId: null, orarimesseId: null,
        massSchedulesPhId: null, philmassId: null, horariosMisasId: null,
        mszeInfoId: null, weekdayMassesId: null, messesInfoId: null,
        bohosluzbyId: null, miserendId: null, kerknetId: null,
        gottesdienstzeitenId: null, discovermassId: null,
        buscarmisasNetworkId: parsed.externalId,
        source: 'buscarmisas-network', website: null, phone: parsed.phone,
        address: parsed.address, country: parsed.country,
      });

      if (masses.length > 0) {
        await prisma.massSchedule.createMany({
          data: masses.map(m => ({
            churchId: church.id,
            dayOfWeek: m.dayOfWeek,
            time: m.time,
            language: config.language === 'pt' ? 'Portuguese' : 'Spanish',
            notes: null,
          })),
        });
        await prisma.church.update({ where: { id: church.id }, data: { lastScrapedAt: new Date() } });
      }
      stats.created++;
    }

    stats.massSchedulesCreated += masses.length;
    console.log(
      `  [${duplicate ? 'update' : 'create'}] ${parsed.name} — ${masses.length} masses — ` +
      `${stats.total} total (${stats.created}↑ ${stats.updated}↻ ${stats.errors}✗)`
    );
  } catch (err) {
    stats.errors++;
    console.error(`  [error] ${url}: ${err instanceof Error ? err.message : err}`);
  }
}
```

- [ ] **Step 3: Compile-check the file so far**

```bash
npx tsc --noEmit
```

Expected: 0 errors.

- [ ] **Step 4: Commit**

```bash
git add scripts/import-buscarmisas-network.ts
git commit -m "feat: add buscarmisas-network importer — DB helpers and church processing"
```

---

### Task 6: CLI parsing and `main()` function

**Files:**
- Modify: `scripts/import-buscarmisas-network.ts`

- [ ] **Step 1: Add `parseCLIArgs`**

```ts
// ─── CLI ──────────────────────────────────────────────────────────────────────

function parseCLIArgs(): CLIArgs {
  const argv = process.argv.slice(2);
  const result: CLIArgs = { domain: null, all: false, dryRun: false, resumeFrom: 0, limit: null, jobId: null };
  for (let i = 0; i < argv.length; i++) {
    switch (argv[i]) {
      case '--domain':      result.domain     = argv[++i]; break;
      case '--all':         result.all        = true;      break;
      case '--dry-run':     result.dryRun     = true;      break;
      case '--resume-from': result.resumeFrom = parseInt(argv[++i], 10); break;
      case '--limit':       result.limit      = parseInt(argv[++i], 10); break;
      case '--job-id':      result.jobId      = argv[++i]; break;
    }
  }
  return result;
}

function validateArgs(args: CLIArgs): void {
  if (!args.domain && !args.all) {
    console.error('Usage:');
    console.error('  npx tsx scripts/import-buscarmisas-network.ts --domain <domain>');
    console.error('  npx tsx scripts/import-buscarmisas-network.ts --all');
    console.error('\nValid domains:', Object.keys(NETWORK_SITES).join(', '));
    process.exit(1);
  }
  if (args.domain && !NETWORK_SITES[args.domain]) {
    console.error(`Unknown domain: ${args.domain}`);
    console.error('Valid domains:', Object.keys(NETWORK_SITES).join(', '));
    process.exit(1);
  }
  if (args.all && args.resumeFrom > 0) {
    console.error('--resume-from cannot be used with --all. Use --domain to resume a specific site.');
    process.exit(1);
  }
}
```

- [ ] **Step 2: Add `runDomain` function**

```ts
async function runDomain(domain: string, config: SiteConfig, args: CLIArgs): Promise<ImportStats> {
  const stats: ImportStats = { total: 0, created: 0, updated: 0, skipped: 0, errors: 0, massSchedulesCreated: 0 };

  const allUrls = await getChurchUrls(domain, config);
  const existingChurches = await loadExistingChurches(config.country);

  // Build set of already-imported IDs for fast skip
  const importedIds = new Set(
    existingChurches.filter(c => c.buscarmisasNetworkId).map(c => c.buscarmisasNetworkId!)
  );

  let candidateUrls = allUrls.slice(args.resumeFrom).filter(url => {
    const externalId = buildExternalId(domain, url);
    return !importedIds.has(externalId);
  });
  if (args.limit !== null) candidateUrls = candidateUrls.slice(0, args.limit);

  console.log(`\n${domain}: ${allUrls.length} total | ${importedIds.size} already imported | ${candidateUrls.length} to process\n`);

  for (let i = 0; i < candidateUrls.length; i++) {
    const url = candidateUrls[i];
    console.log(`[${i + 1}/${candidateUrls.length}] ${url}`);
    await processChurch(url, domain, config, existingChurches, args, stats);
    if (i < candidateUrls.length - 1) await sleep(REQUEST_DELAY_MS);
  }

  return stats;
}
```

- [ ] **Step 3: Add `main()` function**

```ts
// ─── Main ─────────────────────────────────────────────────────────────────────

async function main() {
  const args = parseCLIArgs();
  validateArgs(args);

  if (args.jobId) {
    try {
      await prisma.backgroundJob.update({
        where: { id: args.jobId },
        data: { status: 'running', startedAt: new Date() },
      });
    } catch { /* job may not exist yet */ }
  }

  const domainsToRun: [string, SiteConfig][] = args.all
    ? Object.entries(NETWORK_SITES)
    : [[args.domain!, NETWORK_SITES[args.domain!]]];

  const totalStats: ImportStats = { total: 0, created: 0, updated: 0, skipped: 0, errors: 0, massSchedulesCreated: 0 };

  try {
    for (let d = 0; d < domainsToRun.length; d++) {
      const [domain, config] = domainsToRun[d];
      console.log(`\n${'─'.repeat(60)}`);
      console.log(`Domain ${d + 1}/${domainsToRun.length}: ${domain} (${config.country})`);
      console.log('─'.repeat(60));
      const stats = await runDomain(domain, config, args);
      totalStats.total              += stats.total;
      totalStats.created            += stats.created;
      totalStats.updated            += stats.updated;
      totalStats.skipped            += stats.skipped;
      totalStats.errors             += stats.errors;
      totalStats.massSchedulesCreated += stats.massSchedulesCreated;
      if (d < domainsToRun.length - 1) await sleep(DOMAIN_DELAY_MS);
    }
  } finally {
    console.log('\n─── Import Complete ───────────────────────────────────────');
    console.log(`Total processed:   ${totalStats.total}`);
    console.log(`Created:           ${totalStats.created}`);
    console.log(`Updated:           ${totalStats.updated}`);
    console.log(`Skipped:           ${totalStats.skipped}`);
    console.log(`Errors:            ${totalStats.errors}`);
    console.log(`Mass schedules:    ${totalStats.massSchedulesCreated}`);

    if (args.jobId) {
      const status = totalStats.errors > totalStats.total * 0.1 ? 'failed' : 'completed';
      try {
        await prisma.backgroundJob.update({
          where: { id: args.jobId },
          data: {
            status,
            completedAt: new Date(),
            processed: totalStats.total,
            succeeded: totalStats.created + totalStats.updated,
            failed: totalStats.errors,
            itemsFound: totalStats.massSchedulesCreated,
          },
        });
      } catch { /* ignore */ }
    }

    await prisma.$disconnect();
    await pool.end();
  }
}

main().catch(err => {
  console.error('Fatal error:', err);
  process.exit(1);
});
```

- [ ] **Step 4: Final compile check**

```bash
npx tsc --noEmit
```

Expected: 0 errors.

- [ ] **Step 5: Commit**

```bash
git add scripts/import-buscarmisas-network.ts
git commit -m "feat: add buscarmisas-network importer — CLI + main loop"
```

---

## Chunk 4: Integration + smoke test

### Task 7: package.json and scheduler integration

**Files:**
- Modify: `package.json`
- Modify: `scripts/scheduler.ts`

- [ ] **Step 1: Add npm script to `package.json`**

In the `"scripts"` section, add after `"import:gcatholic"`:
```json
"import:buscarmisas-network": "tsx scripts/import-buscarmisas-network.ts",
```

- [ ] **Step 2: Add 5 case blocks to `getJobCommand` in `scheduler.ts`**

In `scripts/scheduler.ts`, find the `case 'discovermass-import':` block (around line 240). After it, before the `default:` case, add:

```ts
case 'buscarmisas-network-BR': {
  const args = ['tsx', 'scripts/import-buscarmisas-network.ts', '--domain', 'horariosmissa.com.br'];
  if (config?.resumeFrom) args.push('--resume-from', String(config.resumeFrom));
  return { command: 'npx', args };
}
case 'buscarmisas-network-MX': {
  const args = ['tsx', 'scripts/import-buscarmisas-network.ts', '--domain', 'buscarmisas.com.mx'];
  if (config?.resumeFrom) args.push('--resume-from', String(config.resumeFrom));
  return { command: 'npx', args };
}
case 'buscarmisas-network-AR': {
  const args = ['tsx', 'scripts/import-buscarmisas-network.ts', '--domain', 'horariosmisa.com.ar'];
  if (config?.resumeFrom) args.push('--resume-from', String(config.resumeFrom));
  return { command: 'npx', args };
}
case 'buscarmisas-network-CO': {
  const args = ['tsx', 'scripts/import-buscarmisas-network.ts', '--domain', 'buscarmisas.co'];
  if (config?.resumeFrom) args.push('--resume-from', String(config.resumeFrom));
  return { command: 'npx', args };
}
case 'buscarmisas-network-CL': {
  const args = ['tsx', 'scripts/import-buscarmisas-network.ts', '--domain', 'horariomisa.cl'];
  if (config?.resumeFrom) args.push('--resume-from', String(config.resumeFrom));
  return { command: 'npx', args };
}
```

- [ ] **Step 3: Add 5 pipeline phases to `PIPELINE_GROUPS[0].phases` in `scheduler.ts`**

In `scripts/scheduler.ts`, find the `PIPELINE_GROUPS` array. Inside the first group (`name: 'imports'`), add after the `discovermass-import` phase:

```ts
{ name: 'buscarmisas-network-BR', type: 'buscarmisas-network-BR', config: {} },
{ name: 'buscarmisas-network-MX', type: 'buscarmisas-network-MX', config: {} },
{ name: 'buscarmisas-network-AR', type: 'buscarmisas-network-AR', config: {} },
{ name: 'buscarmisas-network-CO', type: 'buscarmisas-network-CO', config: {} },
{ name: 'buscarmisas-network-CL', type: 'buscarmisas-network-CL', config: {} },
```

- [ ] **Step 4: TypeScript compile check**

```bash
npx tsc --noEmit
```

Expected: 0 errors.

- [ ] **Step 5: Commit**

```bash
git add package.json scripts/scheduler.ts
git commit -m "feat: add buscarmisas-network to package.json and scheduler pipeline"
```

---

### Task 8: Smoke test against live sites

- [ ] **Step 1: Dry-run Brazil (verifies parsing + sitemap, no DB writes)**

```bash
npx tsx scripts/import-buscarmisas-network.ts --domain horariosmissa.com.br --dry-run
```

Expected: prints church names with mass counts, no DB errors, >4,000 URLs discovered.

- [ ] **Step 2: Live run — 3 churches from Brazil**

```bash
npx tsx scripts/import-buscarmisas-network.ts --domain horariosmissa.com.br --limit 3
```

Expected: 3 churches created in DB, mass schedules created, no errors.

- [ ] **Step 3: Verify in DB**

```bash
npx tsx -e "
import { Pool } from 'pg';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '@prisma/client';
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const prisma = new PrismaClient({ adapter: new PrismaPg(pool) });
const churches = await prisma.church.findMany({
  where: { source: 'buscarmisas-network' },
  select: { name: true, country: true, buscarmisasNetworkId: true, latitude: true, longitude: true },
  take: 5,
});
console.table(churches);
const massCount = await prisma.massSchedule.count({
  where: { church: { source: 'buscarmisas-network' } },
});
console.log('Mass schedules created:', massCount);
await prisma.\$disconnect(); await pool.end();
"
```

Expected: 3 rows with source `buscarmisas-network`, valid lat/lng, `buscarmisasNetworkId` populated.

- [ ] **Step 4: Test idempotency (re-run should skip already-imported)**

Re-run the same limited test. Expected: `0 to process` (all skipped via the `importedIds` Set).

- [ ] **Step 5: Dry-run Chile (verifies post-sitemap path)**

```bash
npx tsx scripts/import-buscarmisas-network.ts --domain horariomisa.cl --dry-run
```

Expected: ~935 URLs discovered, Spanish day names parsed correctly.

- [ ] **Step 6: Final commit**

```bash
git add scripts/import-buscarmisas-network.ts package.json scripts/scheduler.ts
git commit -m "feat: complete buscarmisas-network importer — Brazil, Mexico, Argentina, Colombia, Chile"
```