How We Translated an Entire Website Overnight with a Swarm of Claude Sub-Agents

On the evening of October 16, 2026, SkillverseHub existed in one language: French. Eight hundred and sixteen source files, roughly nineteen million characters of courses, tutorials, blog posts, page templates and UI components. By the next morning, the whole site existed in English, Spanish, Portuguese and Arabic, with 754 generated pages per language, a working language switcher, and a 3D control room where you could replay the entire night minute by minute.

This article explains how it was done. Not the marketing version, the engineering version: the pipeline, the numbers, the dashboard, the agent-to-agent coordination, and the five things that broke.

The constraint that shaped everything

We had prepaid Anthropic API credits that were about to expire within ten days. The question was not “should we translate the site?” but “how do we convert expiring credits into the largest possible amount of finished, deployable work, unattended, in a single night?”

That constraint ruled out the obvious approach of calling the API file by file from a loop. A sequential loop over 6,000 requests would have taken days and would have died at the first network hiccup. We needed something that behaved like a factory: submit everything at once, let the provider parallelise it, collect the results whenever they arrive, and survive crashes, credit exhaustion and operator mistakes without losing paid work.

The answer was the Anthropic Message Batches API, wrapped in a small, resumable orchestrator we called the translation engine, and watched through a real-time dashboard we called the Atelier.

The pipeline, stage by stage

1. Inventory

The engine walks src/ and classifies every French source file into one of seven families: blog articles, course chapters (“categories”), tutorials, Astro pages, layouts (“gabarits”), components, and data files. Each family gets its own translation rules, because translating a Markdown course and translating an .astro template are not the same job.

Family	Files	Characters
Tutorials	322	8.0 M
Blog	215	7.8 M
Categories (courses)	111	1.0 M
Pages	103	1.4 M
Layouts	32	0.6 M
Components	29	0.2 M
Data	2	< 0.1 M

That is 18.9 million characters per target language, four languages, so about 76 million characters of output to produce and verify.

2. Chunking

Some course chapters are 60,000 characters long. Sending them whole risks truncated output and makes a single failure expensive. The engine splits Markdown at heading boundaries into chunks of about 16,000 characters, never cutting inside a fenced code block or a front-matter block. Code blocks longer than a threshold are replaced by a placeholder before sending and restored verbatim afterwards: the model never gets the chance to “improve” a Dockerfile.

Each (chunk, language) pair becomes one request. Its custom_id is the SHA-1 of language + file type + chunk text. This is the single most important design decision in the whole system: the identifier is derived from the content, so the same chunk resubmitted after a crash maps to the same result, and results can be matched back to their chunk even if every local state file has been deleted. We will come back to why that mattered.

3. One system prompt per (language, family)

The prompt for a blog article says: translate the prose, keep every Markdown structure, do not touch code, keep front-matter keys but translate their values, keep internal links but rewrite /fr/ to /en/. The prompt for an Astro component says something very different: translate only user-visible strings, never touch imports, props, expressions or class names, and keep the file compilable. Arabic prompts add right-to-left considerations for punctuation and mixed Latin terms.

4. Submission to the Batch API

Requests are grouped into batches of up to 200 and submitted to /v1/messages/batches. Anthropic then runs them in parallel on its side, at half the price of synchronous calls, and lets you poll for completion. Each batch is what the dashboard calls a sub-bot: an autonomous worker that takes thousands of requests and comes back with results.

Over the night, 45 batches were submitted, totalling 6,380 requests, of which 6,227 came back successfully. The engine rotated between two API keys with automatic failover and attached the organisation workspace header required by workspace-scoped keys.

5. Validation, not trust

Every result is checked before it is accepted:

• same number of headings, same heading levels, in the same order as the source;
• same number of fenced code blocks, with identical content;
• front-matter still parses and contains the same keys;
• for .astro and .ts files: same imports, same exported symbols, no French left in the template strings;
• no leftover model chatter (“Here is the translation…”).

A chunk that fails validation is resubmitted, up to three attempts. After three failures, the French original is used as a fallback for that file, flagged in the state, so the site still builds and the gap is visible rather than silent. By morning, 3,136 files were translated and 128 had fallen back; a second agent later brought the fallbacks down to 64.

6. Assembly and code rewriting

Chunks are reassembled in order. Then a rewriting pass adapts code to the target language’s place in the project:

• getCollection('blog') becomes getCollection('blog_en'), because each language has its own Astro content collection;
• import paths pointing into src/components/ are redirected to src/l10n/en/components/;
• /fr/ link prefixes become /en/, /es/, /pt/, /ar/;
• Mermaid diagrams, relative image paths and placeholder code blocks are restored.

7. Build with self-healing

The engine then runs a full astro build. If the build fails, it parses the error, identifies the culprit file, replaces it with the French fallback, and rebuilds, up to thirty times. Four languages times 754 pages built cleanly after this loop.

8. Publication

Languages that are at least 90 % translated are added to PUBLISHED_LOCALES, which drives both the hreflang tags and the language switcher. The result is committed and pushed to a dedicated branch, where Vercel produces a preview to be validated before anything reaches production.

The state machine that saved the night

Everything the engine knows lives in a .traduction/ folder: etat.json (every unit, its status, its batch id), one result file per custom_id, a journal, and historique.jsonl, a snapshot of the whole state taken every 30 seconds.

Around midnight an operator error deleted that folder. Forty-two batches had already been paid for and were sitting on Anthropic’s servers with no local record of them.

Because results are kept for 29 days and can be downloaded for free, and because every custom_id is a content hash, the engine gained a rattacher (“re-attach”) command: list every batch in the account from the last 29 days, download their results, recompute the hash of every local chunk, and match them. 6,478 results were recovered without spending a cent. The night continued.

Later, the credits ran out mid-run. The engine detected the credit balance is too low error on both keys, marked the state as credits exhausted, stopped submitting, let in-flight batches finish, and proceeded straight to assembly and build with fallbacks for whatever was missing. Nobody was awake to make that decision; the engine made it.

The 3D control room

A log file scrolling at 3 a.m. tells you nothing. We wanted anyone, including non-engineers, to open a page and understand what was happening at a glance. So the dashboard, built with three.js, is a city:

• Every source file is a building. Its height is proportional to its size in characters. Districts group the families: Blog, Tutorials, Categories, Pages, Layouts, Components.
• Every building has four floors, one per language, English at the bottom, Arabic at the top. A floor lights up in its language’s colour the moment that translation is validated. A dark floor is pending; a red floor is a fallback.
• At the centre glows the Anthropic core. When a batch is submitted, a drone (the sub-bot) leaves the core carrying a label with its batch id, request count and family, orbits while the batch runs, and streams particles back to the buildings as results arrive.
• Four reservoirs fill up with the percentage of each language completed.
• A side panel explains the six stages in plain language, and a log shows the latest engine decisions.
• Buttons let you launch a run (10 files, 60 files, or the whole site), pause, resume, and, most importantly, replay the night at ×150, ×300 or ×900 speed from the 1,210 recorded snapshots, with a timeline slider.

The replay was the deliverable for the morning after: a four-minute film of four and a half hours of work, showing 45 drones leaving the core, buildings lighting floor by floor, the credit counter climbing to $309.95, and the moment at 01:37 when the final build passed and the engine pushed the branch.

The dashboard is a static page served by the engine itself (and mirrored on the site under /atelier-traduction/); it reads etat.json, plan.json and historique.json over HTTP and accepts lancer, pause and reprendre commands with an origin allow-list.

Two agents, one Markdown file

The translation engine was written and operated by one AI coding agent (Claude, inside Cursor). A second agent, Grok, was available in parallel. Rather than let them collide in the same repository, we gave them a protocol:

• a single file, grok-cursor-bridge.md, used as a message board: each agent appends dated entries, states what it is doing, and asks questions;
• a grok-inbox/ folder where Claude drops task lists (for example replis-a-traduire.tsv, the files that fell back to French);
• a grok-outbox/<lang>/<path> folder where Grok drops finished translations, mirroring the project’s src/ layout;
• an importer command in the engine that validates everything in the outbox with the same checks as batch results and integrates it into the state.

Grok translated the hardest files, the ones Claude’s batches had rejected three times, and delivered them in waves of 20 to 50; the engine imported about seventy of them. Grok also worked on a separate branch for UI dictionary keys, which was merged normally. The two agents never edited the same file, and every question Grok asked got a written answer in the bridge within minutes.

The numbers

Metric	Value
Source files per language	816
Characters per language	18.9 M
Target languages	4 (EN, ES, PT, AR)
Translation requests	6,380
Batches (sub-bots)	45
Input tokens	35.3 M
Output tokens	34.3 M
Cost	≈ $310
Generated pages	754 per language
Files translated / fallbacks (morning)	3,136 / 128
Results recovered after state loss	6,478
Recorded snapshots for the replay	1,210
Wall-clock time	21:07 → 01:37, unattended

Equivalent human effort, at a professional rate of about 2,500 words per day per translator: roughly 12 million words of output, or in the order of twenty translator-years.

Five things that broke

1. The temperature parameter. The model used in batches no longer accepts it; every request in the first batch errored. Removed it, resubmitted, lost ten minutes.
2. A port collision. The dashboard’s HTTP server was started twice (EADDRINUSE). The engine now runs detached, as a proper background process.
3. The state deletion described above, which produced the rattacher command. The best features come from the worst nights.
4. Arabic was missing 215 pages. Generated pages still contained isLocaleEntry('ar'), a filter that is meaningless inside a per-language collection and therefore filtered everything. The rewriter now replaces it with () => true. Fifty-two files fixed, rebuilt, 754 pages.
5. The host. A 5-language site bundles the whole content data store into the Vercel serverless function: 391 MB against a hard limit of 250 MB, and the 8 GB build container ran out of memory. Production currently ships French and English (180 MB); Spanish, Portuguese and Arabic are complete on the branch and appear greyed out as “Soon” in the language menu until we move them to a static-only output. Translating the site turned out to be the easy part. Deploying it was the hard part.

What we would keep

• Content-addressed identifiers. Hashing the chunk into the custom_id is what made the system crash-proof and the state recoverable.
• Batches over loops. 6,380 requests in 45 submissions, parallelised by the provider, at half price. Any one-shot bulk job over a few hundred items should be a batch.
• Validate everything, trust nothing. Structural checks caught truncated outputs, swallowed code blocks and chatty preambles that would otherwise have shipped.
• Fallback, never block. A file that cannot be translated after three attempts falls back to the source and is flagged; the build never waits for perfection.
• Make the state visible. The 3D city was not decoration. It is how a non-engineer verified the next morning, in four minutes, that the work was really done, and it is how we found the Arabic bug: a whole district with its top floor dark.

Start the control room yourself

Everything below runs on a laptop. The engine is a single Node.js file with no dependency beyond Node 20; the dashboard is three static files.

1. What you need

• Node.js 20 or newer (fetch and node:crypto are built in).
• The project, with the French sources under src/.
• An Anthropic API key with credits. Put it in a .env file at the project root, never in the code:

# .env (git-ignored)
ANTHROPIC_API_KEY=sk-ant-...
# Only for keys scoped to an organisation workspace:
# ANTHROPIC_WORKSPACE_ID=wrkspc_...

The engine also accepts a second key (key_anthropic_2) and switches to it automatically when the first one runs out of credits.

2. The files

scripts/traduction-batch/traduire.mjs        the engine (Appendix A, 752 lines)
public/atelier-traduction/index.html          the dashboard page
public/atelier-traduction/app.js              the three.js scene and the controls
public/atelier-traduction/style.css           the look
.traduction/                                  created at first run: state, results, journal, history

The .traduction/ folder is listed in .gitignore: it is the engine’s memory, not source code.

3. Look before you spend

node scripts/traduction-batch/traduire.mjs plan

No API call is made. The engine inventories the sources, cuts them into chunks and prints the number of files, requests and characters. Multiply the characters by your number of languages and by the batch price and you know your budget before launching anything.

4. Start the dashboard

node scripts/traduction-batch/traduire.mjs serve

Then open http://localhost:4599/. On start the engine:

1. builds the work plan (every file, every language, every chunk, each with its content hash);
2. opens the HTTP server on port 4599 (--port 4600 to change it);
3. calls Anthropic once to re-attach any batch from the last 29 days whose results match your chunks, so a previous night’s work is recognised immediately and for free;
4. takes a snapshot of the state every 10 seconds, and appends one to historique.jsonl every 30 seconds.

The page connects to http://localhost:4599/etat.json. If it shows “the engine is not responding”, the engine is simply not running.

5. Drive it from the browser

• Tick the languages, pick a size (10 files for a first test, 60, or the whole site) and press Launch. A drone leaves the Anthropic core for every batch submitted.
• Pause stops submitting new batches; the ones already at Anthropic keep running and their results are still collected.
• Resume lifts the pause and resets the “credits exhausted” flag, so you can top up your account and continue without restarting.
• Replay the night reads historique.jsonl and plays every snapshot at ×150, ×300 or ×900, with a timeline slider. Back to live returns to the real state.

The same actions exist on the command line:

node scripts/traduction-batch/traduire.mjs run --langs en --limit 10      # trial: 10 files, English only
node scripts/traduction-batch/traduire.mjs run                            # everything, then build, then push
node scripts/traduction-batch/traduire.mjs run --no-build --no-push       # translation only
node scripts/traduction-batch/traduire.mjs importer ./grok-outbox         # integrate files translated elsewhere
node scripts/traduction-batch/traduire.mjs replis                         # list files still in French

6. Leave it running overnight

Launch it detached from your terminal so that closing the window does not kill it:

# Windows PowerShell
Start-Process node -ArgumentList 'scripts/traduction-batch/traduire.mjs','run' `
  -RedirectStandardOutput .traduction-console.log -RedirectStandardError .traduction-console.err

# macOS / Linux
nohup node scripts/traduction-batch/traduire.mjs run > .traduction-console.log 2>&1 &

If anything crashes, the laptop reboots, or you stop it on purpose, just run the same command again: the state file knows which chunks are done, which are in flight and which have failed how many times. Nothing paid for is ever redone.

7. What the engine exposes

Endpoint	Method	Content
/etat.json	GET	Current snapshot: phase, per-language progress, batches, tokens, cost, build, last log lines
/plan.json	GET	The inventory: every source file with its family and size (the city’s buildings)
/historique.json	GET	All recorded snapshots, for the replay
/api/commande	POST	{ "action": "lancer" | "pause" | "reprendre", "langs": ["en"], "limite": 10 }
/	GET	The dashboard itself

Commands are accepted only from an allow-list of origins (localhost, the production domain and its previews), so a random web page cannot start spending your credits.

Build your own sub-agent system, from zero

Forget the 3D city for a minute. Underneath, a “sub-agent system” is a very small number of ideas, and you can have a working one in about a hundred lines. Here is the mental model, then the code.

What a sub-agent is, and is not

In this system a sub-agent is not a chatbot with a personality that you talk to. It is a worker: a bundle of requests, each with the same contract (a system prompt that says exactly what to return), sent off to run in parallel and come back with results. The worker holds the intelligence. The orchestrator, your script, holds the discipline: what remains to be done, what came back, whether it is acceptable, what to do if not. The orchestrator never “thinks”; it is boring, deterministic code, and that is precisely why the whole thing can run unattended.

The seven principles

1. Define the unit of work. Something small enough that one failure is cheap, large enough that the model has context: a file, or a 16,000-character chunk of one. Everything else is counted in units.
2. Name every unit by its content. id = hash(language + text). Two runs over the same text produce the same id; a changed text produces a new one. This is what makes retries, crashes and recoveries trivial: a result is valid as long as its id still exists in the plan.
3. Keep workers stateless and the orchestrator stateful. A worker knows nothing but its requests. The orchestrator writes a small JSON file after every decision. Memory on disk, not in RAM.
4. Write the contract down and check it mechanically. The prompt says “output only the file, keep every code block”. The validator counts the code blocks. A model’s promise is not a verification.
5. Give failures a budget, then a fallback. Three attempts. Then the original, clearly flagged. The pipeline never waits for a unit that will not come.
6. Make the loop idempotent. One “tick” = collect finished workers → compute what remains → launch a worker for it → save → sleep. Kill the process at any line and restart: the next tick recomputes everything from the state file and the results that exist.
7. Expose the state. Serve the JSON. Any front end, from a curl to a three.js city, can then show what is happening, and a human can intervene with a button instead of a terminal.

The loop, in words

forever:
    for each worker not yet collected:
        if finished: download results, validate each, keep the good ones, count a strike against the bad ones
    remaining = units with no result, not in flight, with fewer than 3 strikes
    if remaining is not empty: launch a worker with them
    if nothing remaining and nothing in flight: stop
    save state, sleep one minute
assemble: for each unit, the result if any, otherwise the original marked as fallback

A complete, minimal engine (one file, about 100 lines)

This script translates every .md file of a folder into one language with Anthropic batch workers. It is resumable, it validates, it retries, it falls back. Save it as mini-swarm.mjs, set ANTHROPIC_API_KEY, and run node mini-swarm.mjs ./docs ./docs-en English.

#!/usr/bin/env node
// mini-swarm.mjs - translate every .md file of a folder with Anthropic batch workers.
// Resumable: run it again after any crash, it continues where it stopped.
import { createHash } from 'node:crypto';
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';

const [SOURCE = './docs', TARGET = './docs-en', LANGUAGE = 'English'] = process.argv.slice(2);
const KEY = process.env.ANTHROPIC_API_KEY;
const MODEL = 'claude-sonnet-5-5';
const STATE_FILE = join(TARGET, '.state.json');
const MAX_ATTEMPTS = 3;

mkdirSync(TARGET, { recursive: true });
const state = existsSync(STATE_FILE)
  ? JSON.parse(readFileSync(STATE_FILE, 'utf8'))
  : { workers: [], attempts: {}, results: {} };
const save = () => writeFileSync(STATE_FILE, JSON.stringify(state, null, 1));
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

// 1. Units of work: one per file, named by the hash of its content (principle 2).
const units = new Map();
for (const name of readdirSync(SOURCE).filter((n) => n.endsWith('.md'))) {
  const text = readFileSync(join(SOURCE, name), 'utf8');
  const id = createHash('sha1').update(LANGUAGE + text).digest('hex').slice(0, 40);
  units.set(id, { id, name, text });
}

// 2. One call to the API, with the headers Anthropic expects.
async function api(path, init = {}) {
  const r = await fetch(`https://api.anthropic.com${path}`, {
    ...init,
    headers: { 'x-api-key': KEY, 'anthropic-version': '2023-06-01', 'content-type': 'application/json' },
  });
  if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
  return init.raw ? r.text() : r.json();
}

// 3. The contract every result must honour before being accepted (principle 4).
function validate(unit, output, stopReason) {
  if (stopReason === 'max_tokens') return 'truncated';
  const ratio = output.length / unit.text.length;
  if (ratio < 0.5 || ratio > 2.5) return `suspicious length (${ratio.toFixed(2)})`;
  const fences = (t) => (t.match(/^```/gm) ?? []).length;
  if (fences(output) !== fences(unit.text)) return 'code blocks differ';
  return null;
}

// 4. Launch a worker (a batch) carrying every unit that still needs doing.
async function launch(ids) {
  const requests = ids.map((id) => ({
    custom_id: id,
    params: {
      model: MODEL,
      max_tokens: 16000,
      system: `Translate the Markdown file from French into ${LANGUAGE}. Keep headings, lists and code blocks exactly as they are. Output only the translated file, nothing else.`,
      messages: [{ role: 'user', content: units.get(id).text }],
    },
  }));
  const batch = await api('/v1/messages/batches', { method: 'POST', body: JSON.stringify({ requests }) });
  state.workers.push({ id: batch.id, ids, collected: false });
  save();
  console.log(`worker ${batch.id} launched with ${ids.length} files`);
}

// 5. Collect finished workers: validate, keep, or count a strike (principle 5).
async function collect() {
  for (const worker of state.workers.filter((w) => !w.collected)) {
    const info = await api(`/v1/messages/batches/${worker.id}`);
    if (info.processing_status !== 'ended') continue;
    const jsonl = await api(`/v1/messages/batches/${worker.id}/results`, { raw: true });
    for (const line of jsonl.split('\n').filter(Boolean)) {
      const r = JSON.parse(line);
      const unit = units.get(r.custom_id);
      if (!unit) continue; // result of a file that changed since: ignore
      const message = r.result?.message;
      const output = message ? message.content.filter((c) => c.type === 'text').map((c) => c.text).join('') : '';
      const error = r.result?.type === 'succeeded'
        ? validate(unit, output, message.stop_reason)
        : (r.result?.error?.error?.message ?? r.result?.type ?? 'unknown error');
      if (!error) { state.results[unit.id] = output; continue; }
      state.attempts[unit.id] = (state.attempts[unit.id] ?? 0) + 1;
      console.log(`rejected ${unit.name}: ${error} (attempt ${state.attempts[unit.id]})`);
    }
    worker.collected = true;
    save();
  }
}

// 6. The loop. Every turn is idempotent: crash anywhere, run again, nothing is lost (principle 6).
for (;;) {
  await collect();
  const inFlight = new Set(state.workers.filter((w) => !w.collected).flatMap((w) => w.ids));
  const remaining = [...units.keys()].filter(
    (id) => !state.results[id] && !inFlight.has(id) && (state.attempts[id] ?? 0) < MAX_ATTEMPTS,
  );
  if (remaining.length) await launch(remaining);
  if (!remaining.length && !inFlight.size) break;
  console.log(`${Object.keys(state.results).length}/${units.size} done, ${inFlight.size + remaining.length} in flight`);
  await sleep(60_000);
}

// 7. Assemble: the translation, or the original clearly flagged as a fallback.
let fallbacks = 0;
for (const unit of units.values()) {
  const translated = state.results[unit.id];
  if (!translated) fallbacks++;
  writeFileSync(join(TARGET, unit.name), translated ?? `<!-- FALLBACK: not translated after ${MAX_ATTEMPTS} attempts -->\n${unit.text}`);
}
console.log(`finished: ${units.size - fallbacks} translated, ${fallbacks} fallbacks`);

Read it top to bottom once and you have read the whole production engine in miniature: the same seven steps, the same state file, the same hash ids. Everything the 752-line version adds is either scale (chunking, seven file families with their own prompts, 200-request batches, two keys), safety (import path rewriting, build-and-repair, re-attaching lost batches) or visibility (the HTTP server and the snapshots).

Growing it into the real thing

• Chunking. When a file can exceed the model’s comfortable output, split it at heading boundaries, never inside a code block, and make each chunk a unit. Reassemble in order. See decouperMarkdown in Appendix A.
• One prompt per kind of file. Markdown prose, Astro templates and TypeScript data files need different contracts. Classify at inventory time; pick the prompt from the type. See systeme.
• Post-processing you can prove. Anything that can be done with a regular expression after the fact (link prefixes, import paths, collection names) should not be asked of the model. Deterministic rewrites are free and never wrong twice. See reecrireCode and reecrireMarkdown.
• Build as the final validator. If the output is code, compile it. If the compiler names a file, replace that file by its fallback and compile again. See construire.
• A snapshot function instead of a logger. photographier computes one JSON object describing the whole state. Serve it. Append it to a history file. The dashboard, the replay and the final report all read that one object.
• Several agents. If a second AI, or a human, can help, give them an inbox (a list of units), an outbox (finished files in a mirror folder structure) and run the same validator on what they return. Never let two writers touch the same file; coordinate through a dated, append-only Markdown file.

Mistakes we made so you do not have to

• Sending a parameter the model had deprecated (temperature): the entire first batch errored. Submit a batch of 5 requests before a batch of 5,000.
• Deleting the state folder by hand. Content-hashed ids and the re-attach command saved us; without them 42 paid batches would have been lost.
• Trusting that a per-language filter (isLocaleEntry('ar')) would be harmless in a per-language collection. It filtered everything. Count your output pages per language and compare.
• Not checking the host’s limits before translating. Translation is cheap; a 250 MB function limit is not negotiable.

Appendix A: the complete engine (traduire.mjs)

The exact file that ran the night, 752 lines, unedited. It is also available as a raw download: traduire.mjs. French identifiers are kept as they are in the repository; the section comments mark the eight stages described above.

#!/usr/bin/env node
/**
 * Traduction complete du site par l'API Message Batches d'Anthropic.
 *
 *   node scripts/traduction-batch/traduire.mjs plan            estimation, rien n'est envoye
 *   node scripts/traduction-batch/traduire.mjs run             boucle complete (reprend la ou elle s'est arretee)
 *   node scripts/traduction-batch/traduire.mjs run --langs en --limit 20     essai reduit
 *
 * Etapes de `run`, toutes reprenables (etat dans .traduction/) :
 *   1. inventaire des sources francaises (contenus Markdown, pages, composants, layouts, donnees) ;
 *   2. decoupage des gros Markdown en morceaux, une requete par morceau et par langue ;
 *   3. envoi en lots (Message Batches), suivi toutes les minutes, telechargement des resultats ;
 *   4. validation de chaque morceau (structure, blocs de code, imports) ; renvoi en cas d'echec ;
 *   5. assemblage des fichiers cibles et reecritures deterministes (imports, collections, liens) ;
 *   6. build Astro, remplacement par la version francaise des fichiers qui le cassent ;
 *   7. commit et push de la branche, rapport final.
 *
 * Le tableau de bord (public/atelier-traduction/) lit http://localhost:4599/etat.json.
 */

import { createHash } from 'node:crypto';
import { createServer } from 'node:http';
import { spawn } from 'node:child_process';
import {
  existsSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync, appendFileSync, rmSync, copyFileSync
} from 'node:fs';
import { dirname, join, relative, resolve, extname, sep } from 'node:path';
import { fileURLToPath } from 'node:url';

const RACINE = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
const ETAT_DIR = join(RACINE, '.traduction');
const RES_DIR = join(ETAT_DIR, 'resultats');
const FICHIER_ETAT = join(ETAT_DIR, 'etat.json');
const JOURNAL = join(ETAT_DIR, 'journal.log');
const PUBLIC_TABLEAU = join(RACINE, 'public', 'atelier-traduction');

const LANGUES = {
  en: { nom: 'English (United States)', bcp47: 'en-US' },
  es: { nom: 'Spanish (Spain)', bcp47: 'es-ES' },
  pt: { nom: 'Portuguese (Brazil)', bcp47: 'pt-BR' },
  ar: { nom: 'Modern Standard Arabic', bcp47: 'ar' }
};
const COLLECTIONS = ['blog', 'tutoriels', 'category'];
const PRIX = { in: 1.5, out: 7.5 }; // $ par million de jetons, tarif Batch (moitie du tarif standard)
const TAILLE_MORCEAU = 16000;
const MAX_CODE = 120000;
const MAX_TENTATIVES = 3;
const REQUETES_PAR_LOT = 200; // petits lots : progression fine et un sous-bot par lot dans la vue 3D

/* ------------------------------------------------------------------ options */

function options(argv) {
  const o = { cmd: argv[0] && !argv[0].startsWith('--') ? argv[0] : 'run', langs: Object.keys(LANGUES), model: 'claude-sonnet-5-5',
    port: 4599, limit: 0, build: true, push: true, intervalle: 60 };
  for (let i = 0; i < argv.length; i++) {
    const a = argv[i];
    if (a === '--langs') o.langs = argv[++i].split(',');
    else if (a === '--model') o.model = argv[++i];
    else if (a === '--port') o.port = Number(argv[++i]);
    else if (a === '--limit') o.limit = Number(argv[++i]);
    else if (a === '--no-build') o.build = false;
    else if (a === '--no-push') o.push = false;
    else if (a === '--intervalle') o.intervalle = Number(argv[++i]);
  }
  return o;
}
const O = options(process.argv.slice(2));

function lireEnv() {
  const candidats = [join(RACINE, '.env'), resolve(RACINE, '..', '07-contactFormApp', '.env')];
  const vals = {};
  for (const f of candidats) {
    if (!existsSync(f)) continue;
    for (const l of readFileSync(f, 'utf8').split(/\r?\n/)) {
      const m = l.match(/^\s*([A-Za-z0-9_]+)\s*=\s*(.+?)\s*$/);
      if (m && !(m[1].toLowerCase() in vals)) vals[m[1].toLowerCase()] = m[2];
    }
  }
  return vals;
}
const ENV = lireEnv();
const CLEFS = [...new Set([process.env.ANTHROPIC_API_KEY, ENV.key_anthropic, ENV.key_anthropic_to_use, ENV.key_anthropic_2, ENV.other_key_anthropic].filter(Boolean))];
const WORKSPACE = process.env.ANTHROPIC_WORKSPACE_ID ?? ENV.key_id_workspace ?? ENV.id_1_claude_key;
let clefActive = 0;

/* ------------------------------------------------------------------ etat */

mkdirSync(RES_DIR, { recursive: true });
const etat = existsSync(FICHIER_ETAT)
  ? JSON.parse(readFileSync(FICHIER_ETAT, 'utf8'))
  : { debut: new Date().toISOString(), lots: [], tentatives: {}, usage: { in: 0, out: 0 }, fichiers: {}, build: null, phase: 'demarrage', git: null };
const journalRecent = [];

function log(msg) {
  const ligne = `[${new Date().toISOString().slice(11, 19)}] ${msg}`;
  console.log(ligne);
  appendFileSync(JOURNAL, ligne + '\n');
  journalRecent.push(ligne);
  if (journalRecent.length > 60) journalRecent.shift();
}
function sauver() { writeFileSync(FICHIER_ETAT, JSON.stringify(etat, null, 1)); }

/* ------------------------------------------------------------------ inventaire */

function parcourir(dossier, filtre, acc = []) {
  if (!existsSync(dossier)) return acc;
  for (const n of readdirSync(dossier)) {
    const f = join(dossier, n);
    if (statSync(f).isDirectory()) parcourir(f, filtre, acc);
    else if (filtre(f)) acc.push(f);
  }
  return acc;
}
const posix = (p) => p.split(sep).join('/');

/** Fichiers de code mirroirs par langue : composants, layouts, donnees, et les lib qui lisent une collection. */
function ensembleMiroir() {
  const src = join(RACINE, 'src');
  const res = [
    ...parcourir(join(src, 'components'), (f) => f.endsWith('.astro')),
    ...parcourir(join(src, 'layouts'), (f) => f.endsWith('.astro')),
    ...parcourir(join(src, 'data'), (f) => /\.(ts|js)$/.test(f)),
    ...parcourir(join(src, 'lib'), (f) => /\.(ts|js)$/.test(f) && /getCollection\(|getEntry\(/.test(readFileSync(f, 'utf8')))
  ];
  return new Set(res);
}
const MIROIR = ensembleMiroir();

function destination(source, lang) {
  const rel = posix(relative(join(RACINE, 'src'), source));
  const [tete, ...reste] = rel.split('/');
  if (tete === 'content') return join(RACINE, 'src', 'content', reste[0], lang, ...reste.slice(1));
  if (tete === 'pages') return join(RACINE, 'src', 'pages', lang, ...reste.slice(1));
  return join(RACINE, 'src', 'l10n', lang, tete, ...reste);
}

function inventaire() {
  const src = join(RACINE, 'src');
  const fichiers = [];
  for (const c of COLLECTIONS) {
    const base = join(src, 'content', c);
    for (const f of parcourir(base, (f) => f.endsWith('.md'))) {
      const premier = posix(relative(base, f)).split('/')[0];
      if (premier in LANGUES || premier === 'fr') continue;
      fichiers.push({ source: f, type: 'md', groupe: c });
    }
  }
  for (const f of parcourir(join(src, 'pages', 'fr'), (f) => /\.(astro|ts|js)$/.test(f))) {
    fichiers.push({ source: f, type: 'code', groupe: 'pages' });
  }
  for (const f of MIROIR) fichiers.push({ source: f, type: 'code', groupe: posix(relative(src, f)).split('/')[0] });
  return fichiers;
}

/* ------------------------------------------------------------------ decoupage */

function decouperMarkdown(texte) {
  if (texte.length <= TAILLE_MORCEAU) return [texte];
  const lignes = texte.split(/(?<=\n)/);
  const morceaux = [];
  let courant = '', dansBloc = false, frontmatter = texte.startsWith('---'), finFm = false, nFm = 0;
  let coupure = -1; // longueur de `courant` au dernier point de coupure sur
  for (const l of lignes) {
    if (frontmatter && !finFm) { if (/^---\s*$/.test(l)) { nFm++; if (nFm === 2) finFm = true; } courant += l; continue; }
    if (/^\s*(```|~~~)/.test(l)) dansBloc = !dansBloc;
    if (!dansBloc && (/^#{1,3}\s/.test(l) || /^\s*$/.test(l))) coupure = courant.length;
    courant += l;
    if (courant.length > TAILLE_MORCEAU && coupure > 0 && !dansBloc) {
      morceaux.push(courant.slice(0, coupure));
      courant = courant.slice(coupure);
      coupure = -1;
    }
  }
  if (courant) morceaux.push(courant);
  return morceaux;
}

/* ------------------------------------------------------------------ consignes */

function systeme(lang, type) {
  const L = LANGUES[lang];
  const commun = [
    `You are a senior technical translator for SkillverseHub, a French e-learning platform (software development, DevOps, cloud, AI, cybersecurity).`,
    `Translate from French into ${L.nom}. The result is published as-is on the ${lang} version of the site.`,
    `Output ONLY the translated file content. No preamble, no explanation, no surrounding code fence.`,
    `Natural, fluent, professional ${L.nom}; keep technical terms, product names, commands and acronyms in their usual form.`,
    `Internal links: replace the /fr/ language prefix with /${lang}/ (for example /fr/blog/x/ becomes /${lang}/blog/x/). Never translate URL paths or slugs themselves.`
  ];
  if (lang === 'ar') commun.push(`Arabic text uses Arabic punctuation; Latin technical identifiers stay in Latin script, unaltered.`);
  if (type === 'md') {
    commun.push(
      `The input is a Markdown file (or a consecutive part of one). Preserve its exact structure: headings, lists, tables, HTML tags and attributes, blank lines, admonition markers such as ":::note".`,
      `YAML frontmatter, if present: keep every key and every value unchanged EXCEPT the values of title, description, sidebarTitle, subtitle, prevTitle, nextTitle, which you translate. Keep quoting valid YAML.`,
      `Code blocks: keep code, commands, outputs, file names and identifiers byte-for-byte; translate only human comments inside them. Mermaid blocks stay strictly unchanged.`,
      `Keep image paths, link targets, anchors and inline code unchanged.`
    );
  } else {
    commun.push(
      `The input is a source file of an Astro 5 site (.astro, .ts or .js). Translate ONLY human-visible French text: text between tags, title/description strings, alt, aria-label, placeholder, meta content, button labels, user-facing strings in scripts and data objects.`,
      `Do not change imports, identifiers, variable or function names, props, object keys, CSS, class names, slugs, API URLs, collection names, logic or formatting. Leave code comments unchanged.`,
      `Where the code states the page language as French (lang="fr", inLanguage "fr", og:locale fr_FR, toLocaleDateString('fr-FR'), Intl formats 'fr-FR'), use ${L.bcp47} instead.`,
      `Return the complete file, every line, in the same order.`
    );
  }
  return commun.join('\n');
}

/* ------------------------------------------------------------------ unites */

const hash = (s) => createHash('sha1').update(s).digest('hex');

/** Toutes les langues sont planifiees d'emblee : un lot lance plus tot reste exploitable quel que soit le lancement en cours. */
function construireUnites() {
  const fichiers = inventaire();
  const unites = new Map(); // custom_id -> requete
  const plans = [];         // un par fichier et langue
  const sources = [];       // un par fichier, pour la vue 3D
  for (const f of fichiers) {
    const texte = readFileSync(f.source, 'utf8');
    f.index = sources.length;
    sources.push({ g: f.groupe, n: posix(relative(join(RACINE, 'src'), f.source)), c: texte.length, t: f.type });
    if (f.type === 'code' && texte.length > MAX_CODE) { for (const lang of Object.keys(LANGUES)) plans.push({ ...f, lang, ids: [], tropGros: true }); continue; }
    const morceaux = f.type === 'md' ? decouperMarkdown(texte) : [texte];
    for (const lang of Object.keys(LANGUES)) {
      const ids = morceaux.map((m, i) => {
        const id = `${lang}_${hash(lang + f.type + m).slice(0, 40)}`;
        if (!unites.has(id)) {
          const partie = morceaux.length > 1 ? `\n\n(Part ${i + 1} of ${morceaux.length} of the same document${i > 0 ? '; this part has no frontmatter' : ''}.)` : '';
          unites.set(id, {
            id, lang, type: f.type, source: m, groupe: f.groupe,
            max: Math.min(64000, Math.max(4000, Math.ceil(m.length / 2) + 2000)),
            system: systeme(lang, f.type), user: `File: ${posix(relative(RACINE, f.source))}${partie}\n\n${m}`
          });
        }
        return id;
      });
      plans.push({ ...f, lang, ids, morceaux });
    }
  }
  return { unites, plans, sources };
}

/* ------------------------------------------------------------------ API */

async function api(chemin, init = {}) {
  for (let essai = 1; essai <= 6; essai++) {
    const headers = { 'x-api-key': CLEFS[clefActive], 'anthropic-version': '2023-06-01', 'content-type': 'application/json' };
    if (WORKSPACE) headers['anthropic-workspace-id'] = WORKSPACE;
    let r;
    try { r = await fetch(`https://api.anthropic.com${chemin}`, { ...init, headers }); }
    catch (e) { log(`reseau : ${e.message}, nouvel essai`); await pause(10000 * essai); continue; }
    if (r.status === 429 || r.status >= 500) { await pause(10000 * essai); continue; }
    if (!r.ok) {
      const t = await r.text();
      if (/credit|billing|balance/i.test(t)) {
        if (clefActive < CLEFS.length - 1) { clefActive++; log(`credits epuises sur la clef ${clefActive}, bascule sur la suivante`); continue; }
        etat.credits = 'epuises';
        const e = new Error('credits Anthropic epuises sur toutes les clefs'); e.credits = true; throw e;
      }
      throw new Error(`${r.status} ${t.slice(0, 300)}`);
    }
    return init.brut ? r.text() : r.json();
  }
  throw new Error(`API injoignable : ${chemin}`);
}
const pause = (ms) => new Promise((ok) => setTimeout(ok, ms));

async function soumettre(ids, unites) {
  const parLangue = {};
  for (const id of ids) (parLangue[unites.get(id).lang] ??= []).push(id);
  for (const [lang, liste] of Object.entries(parLangue)) {
    for (let i = 0; i < liste.length; i += REQUETES_PAR_LOT) {
      const tranche = liste.slice(i, i + REQUETES_PAR_LOT);
      const requests = tranche.map((id) => {
        const u = unites.get(id);
        const tent = etat.tentatives[id] ?? 0;
        return { custom_id: id, params: { model: O.model, max_tokens: tent > 0 ? Math.min(u.max, 32000) : u.max,
          system: u.system, messages: [{ role: 'user', content: u.user }] } };
      });
      const lot = await api('/v1/messages/batches', { method: 'POST', body: JSON.stringify({ requests }) });
      etat.lots.push({ id: lot.id, lang, ids: tranche, statut: lot.processing_status, cree: new Date().toISOString(),
        compte: lot.request_counts, recupere: false, types: compterTypes(tranche, unites) });
      log(`lot ${lot.id} envoye : ${tranche.length} requetes (${lang})`);
      sauver();
    }
  }
}
function compterTypes(ids, unites) {
  const c = {};
  for (const id of ids) { const g = unites.get(id)?.groupe ?? '?'; c[g] = (c[g] ?? 0) + 1; }
  return c;
}

/* ------------------------------------------------------------------ validation */

const fences = (t) => (t.match(/^\s*(```|~~~)/gm) ?? []).length;
const imports = (t) => (t.match(/^\s*import\s/gm) ?? []).length;

function nettoyer(sortie, source) {
  let t = sortie;
  if (!/^\s*```/.test(source) && /^\s*```[a-z]*\n/.test(t) && /\n```\s*$/.test(t)) t = t.replace(/^\s*```[a-z]*\n/, '').replace(/\n```\s*$/, '\n');
  if (source.endsWith('\n') && !t.endsWith('\n')) t += '\n';
  return t;
}

function valider(u, t, stop) {
  if (stop === 'max_tokens') return 'tronque';
  const ratio = t.length / Math.max(1, u.source.length);
  if (ratio < 0.45 || ratio > 2.6) return `longueur ${ratio.toFixed(2)}`;
  if (u.type === 'md') {
    if (fences(t) !== fences(u.source)) return 'blocs de code';
    if (u.source.startsWith('---') && !t.startsWith('---')) return 'frontmatter';
  } else {
    if (imports(t) !== imports(u.source)) return 'imports';
    if (u.source.startsWith('---') && !t.startsWith('---')) return 'frontmatter astro';
  }
  return null;
}

/**
 * Rattache les lots deja soumis chez Anthropic (etat local perdu, ou relance depuis une autre
 * machine). Les custom_id sont des empreintes du texte source : un resultat reste exploitable
 * tant que la source n'a pas change, et le telechargement des resultats ne coute aucun credit.
 */
async function rattacher(unites) {
  const connus = new Set(etat.lots.map((l) => l.id));
  const depuis = new Date(Date.now() - 29 * 86400000).toISOString();
  let apres = '', ajoutes = 0;
  for (;;) {
    const page = await api(`/v1/messages/batches?limit=100${apres ? `&after_id=${apres}` : ''}`);
    for (const b of page.data ?? []) {
      if (connus.has(b.id) || b.created_at < depuis) continue;
      etat.lots.push({ id: b.id, lang: '?', ids: [], statut: b.processing_status, cree: b.created_at, compte: b.request_counts, recupere: false, types: {}, rattache: true });
      ajoutes++;
    }
    if (!page.has_more || !page.last_id) break;
    apres = page.last_id;
  }
  if (ajoutes) { log(`${ajoutes} lots existants rattaches depuis le compte Anthropic`); sauver(); }
  await suivreLots(unites);
  // Les lots rattaches sans aucune requete connue (essais d'un autre projet) sont ecartes.
  etat.lots = etat.lots.filter((l) => !(l.rattache && l.recupere && !l.ids.length));
  sauver();
}

async function suivreLots(unites) {
  for (const lot of etat.lots.filter((l) => !l.recupere)) {
    const info = await api(`/v1/messages/batches/${lot.id}`);
    lot.statut = info.processing_status;
    lot.compte = info.request_counts;
    if (info.processing_status !== 'ended') continue;
    const jsonl = await api(`/v1/messages/batches/${lot.id}/results`, { brut: true });
    let ok = 0, ko = 0;
    const rattache = lot.rattache && !lot.ids.length;
    for (const ligne of jsonl.split('\n')) {
      if (!ligne.trim()) continue;
      const r = JSON.parse(ligne);
      const u = unites.get(r.custom_id);
      if (rattache) {
        if (!u) continue; // resultat d'un autre projet ou d'une source modifiee depuis
        lot.ids.push(r.custom_id);
        lot.lang = u.lang;
        lot.types[u.groupe] = (lot.types[u.groupe] ?? 0) + 1;
      }
      if (r.result?.type === 'succeeded') {
        const m = r.result.message;
        etat.usage.in += m.usage?.input_tokens ?? 0;
        etat.usage.out += m.usage?.output_tokens ?? 0;
        if (!u) continue;
        const texte = nettoyer((m.content ?? []).filter((c) => c.type === 'text').map((c) => c.text).join(''), u.source);
        const erreur = valider(u, texte, m.stop_reason);
        if (!erreur) { writeFileSync(join(RES_DIR, `${r.custom_id}.txt`), texte); ok++; continue; }
        log(`rejet ${r.custom_id} (${u.groupe}) : ${erreur}`);
      } else {
        const msg = r.result?.error?.error?.message ?? r.result?.type ?? 'inconnu';
        if (!ko) log(`erreur API sur ${r.custom_id} : ${msg}`);
        etat.erreursApi = (etat.erreursApi ?? 0) + 1;
      }
      etat.tentatives[r.custom_id] = (etat.tentatives[r.custom_id] ?? 0) + 1;
      ko++;
    }
    lot.recupere = true;
    lot.ok = ok; lot.ko = ko;
    log(`lot ${lot.id} termine : ${ok} acceptes, ${ko} a refaire`);
    sauver();
  }
}

/* ------------------------------------------------------------------ assemblage */

function versSpec(depuis, cible, specOrigine) {
  let r = posix(relative(dirname(depuis), cible));
  if (!r.startsWith('.')) r = './' + r;
  if (!extname(specOrigine) && extname(r)) r = r.slice(0, -extname(r).length);
  return r;
}

function resoudre(base, spec) {
  const abs = resolve(dirname(base), spec);
  for (const c of [abs, abs + '.ts', abs + '.js', abs + '.astro', join(abs, 'index.ts'), join(abs, 'index.js')]) {
    if (existsSync(c) && statSync(c).isFile()) return c;
  }
  return abs;
}

function reecrireCode(texte, source, dest, lang) {
  let t = texte.replace(/(from\s+|import\s+|import\(\s*)(['"])(\.{1,2}\/[^'"]+)\2/g, (tout, avant, q, spec) => {
    const cible = resoudre(source, spec);
    const finale = MIROIR.has(cible) ? destination(cible, lang) : cible;
    return `${avant}${q}${versSpec(dest, finale, spec)}${q}`;
  });
  t = t.replace(/(getCollection|getEntry|getEntries)\(\s*(['"])(blog|tutoriels|category)\2/g, `$1($2$3_${lang}$2`);
  t = t.replace(/CollectionEntry<\s*(['"])(blog|tutoriels|category)\1\s*>/g, `CollectionEntry<$1$2_${lang}$1>`);
  // Dans une collection par langue, les identifiants n'ont plus de prefixe de langue : le filtre
  // de langue de la collection francaise ne doit plus rien exclure.
  t = t.replace(/isLocaleEntry\(\s*['"][a-z]{2}['"]\s*\)/g, '() => true');
  t = t.replace(/entriesForLocale\(([^,()]+),\s*['"][a-z]{2}['"]\s*\)/g, '$1');
  t = t.replace(/(["'`(=]\s*)\/fr(?=[/"'`])/g, `$1/${lang}`);
  t = t.replace(/skillversehub\.com\/fr\//g, `skillversehub.com/${lang}/`);
  return t;
}

function reecrireMarkdown(texte, original, lang) {
  let t = texte;
  const mOrig = original.match(/```mermaid[\s\S]*?```/g) ?? [];
  const mTrad = t.match(/```mermaid[\s\S]*?```/g) ?? [];
  if (mOrig.length && mOrig.length === mTrad.length) { let i = 0; t = t.replace(/```mermaid[\s\S]*?```/g, () => mOrig[i++]); }
  t = t.replace(/(\]\(|href=["']|src=["'])\/fr\//g, `$1/${lang}/`);
  t = t.replace(/skillversehub\.com\/fr\//g, `skillversehub.com/${lang}/`);
  t = t.replace(/(\]\(|src=["'])(\.{1,2}\/)/g, '$1../$2');
  return t;
}

function assembler(plans, forcer = false) {
  let ecrits = 0;
  for (const p of plans) {
    const dest = destination(p.source, p.lang);
    const cle = posix(relative(RACINE, dest));
    const deja = etat.fichiers[cle];
    if (deja && (deja.statut === 'traduit' || deja.statut === 'conserve')) continue;
    const original = readFileSync(p.source, 'utf8');
    // Fichier deja present sans etat connu (ecrit par un lancement precedent, ou a la main) : il
    // est conserve seulement si on ne dispose d'aucun resultat et qu'il differe du repli francais.
    if (!deja && existsSync(dest) && !p.tropGros && !p.ids.every((id) => existsSync(join(RES_DIR, `${id}.txt`)))) {
      const actuel = readFileSync(dest, 'utf8');
      const repliFr = p.type === 'md' ? reecrireMarkdown(original, original, p.lang) : reecrireCode(original, p.source, dest, p.lang);
      if (actuel !== repliFr && actuel !== original) { etat.fichiers[cle] = { statut: 'conserve', groupe: p.groupe, lang: p.lang, source: posix(relative(RACINE, p.source)), type: p.type }; continue; }
    }
    let texte = null, statut = null;
    if (p.tropGros) { texte = original; statut = 'repli'; }
    else {
      const morceaux = p.ids.map((id) => { const f = join(RES_DIR, `${id}.txt`); return existsSync(f) ? readFileSync(f, 'utf8') : null; });
      if (morceaux.every((m) => m !== null)) { texte = morceaux.join(''); statut = 'traduit'; }
      else if (!forcer && p.ids.some((id, i) => morceaux[i] === null && (etat.tentatives[id] ?? 0) < MAX_TENTATIVES)) continue;
      else { texte = original; statut = 'repli'; }
    }
    if (deja?.statut === 'repli' && statut === 'repli') continue;
    texte = p.type === 'md' ? reecrireMarkdown(texte, original, p.lang) : reecrireCode(texte, p.source, dest, p.lang);
    mkdirSync(dirname(dest), { recursive: true });
    writeFileSync(dest, texte);
    etat.fichiers[cle] = { statut, groupe: p.groupe, lang: p.lang, source: posix(relative(RACINE, p.source)), type: p.type };
    ecrits++;
  }
  if (ecrits) { log(`${ecrits} fichiers assembles`); sauver(); }
}

/* ------------------------------------------------------------------ build */

function executer(cmd, args, cwd = RACINE) {
  return new Promise((ok) => {
    // Cinq langues de contenu : la synchronisation Astro depasse le tas Node par defaut (4 Go).
    const p = spawn(cmd, args, { cwd, shell: true, env: { ...process.env, NODE_OPTIONS: '--max-old-space-size=16384' } });
    let sortie = '';
    p.stdout.on('data', (d) => { sortie += d; });
    p.stderr.on('data', (d) => { sortie += d; });
    p.on('close', (code) => ok({ code, sortie }));
  });
}

async function construire(plans) {
  etat.phase = 'build';
  etat.build = { statut: 'en cours', essais: 0, replis: [] };
  sauver();
  for (let essai = 1; essai <= 30; essai++) {
    etat.build.essais = essai;
    log(`build Astro, essai ${essai}`);
    const r = await executer('npm', ['run', 'build']);
    writeFileSync(join(ETAT_DIR, `build-${essai}.log`), r.sortie);
    if (r.code === 0) { etat.build.statut = 'reussi'; log('build reussi'); sauver(); return true; }
    const fautif = trouverFautif(r.sortie);
    if (!fautif) { etat.build.statut = 'echec'; etat.build.erreur = r.sortie.slice(-1500); log('build en echec sans fichier traduit identifiable'); sauver(); return false; }
    const info = etat.fichiers[fautif];
    const dest = join(RACINE, fautif);
    if (info?.statut === 'traduit') {
      const p = plans.find((x) => posix(relative(RACINE, destination(x.source, x.lang))) === fautif);
      const original = readFileSync(join(RACINE, info.source), 'utf8');
      writeFileSync(dest, p.type === 'md' ? reecrireMarkdown(original, original, p.lang) : reecrireCode(original, p.source, dest, p.lang));
      info.statut = 'repli';
      log(`build : ${fautif} remplace par sa version francaise`);
    } else if (info && info.type === 'md') {
      rmSync(dest, { force: true }); info.statut = 'retire'; log(`build : ${fautif} retire`);
    } else if (info && info.groupe === 'pages') {
      rmSync(dest, { force: true }); info.statut = 'retire'; log(`build : page ${fautif} retiree`);
    } else { etat.build.statut = 'echec'; etat.build.erreur = r.sortie.slice(-1500); log(`build : ${fautif} casse meme en repli`); sauver(); return false; }
    etat.build.replis.push(fautif);
    sauver();
  }
  etat.build.statut = 'echec';
  sauver();
  return false;
}

function trouverFautif(sortie) {
  const connus = Object.keys(etat.fichiers);
  const norm = sortie.replace(/\\/g, '/');
  return connus.find((cle) => norm.includes(cle)) ?? null;
}

async function publier(langs) {
  etat.phase = 'git';
  sauver();
  const config = join(RACINE, 'src', 'i18n', 'config.ts');
  const prets = Object.keys(LANGUES).filter((l) => {
    const f = Object.values(etat.fichiers).filter((x) => x.lang === l);
    return f.length && f.filter((x) => x.statut === 'traduit' || x.statut === 'conserve').length / f.length >= 0.9;
  });
  const c = readFileSync(config, 'utf8').replace(/PUBLISHED_LOCALES: readonly Locale\[\] = \[[^\]]*\]/, `PUBLISHED_LOCALES: readonly Locale[] = ['fr'${prets.map((l) => `, '${l}'`).join('')}]`);
  writeFileSync(config, c);
  const etapes = [['git', ['add', '-A', 'src', 'public', 'scripts', '.gitignore']],
    ['git', ['commit', '-q', '-m', `"i18n - traduction complete (${langs.join(', ')}) par l'API Batch Anthropic"`]]];
  if (O.push) etapes.push(['git', ['push', '-q', '-u', 'origin', 'i18n-batch']]);
  for (const [cmd, args] of etapes) {
    const r = await executer(cmd, args);
    log(`${cmd} ${args[0]} : ${r.code === 0 ? 'ok' : r.sortie.slice(0, 300)}`);
  }
  etat.git = { branche: 'i18n-batch', publiees: prets, at: new Date().toISOString() };
}

/* ------------------------------------------------------------------ tableau de bord */

const HISTORIQUE = join(ETAT_DIR, 'historique.jsonl');
const ORIGINES = [/^http:\/\/(localhost|127\.0\.0\.1)(:\d+)?$/, /^https:\/\/(www\.)?skillversehub\.com$/, /^https:\/\/[a-z0-9-]+\.vercel\.app$/];
let U = null;           // { unites, plans, sources }
let travailEnCours = null;
const controle = { pause: false, langs: [], limite: 0, demarre: null };
let instantane = {};
let dernierInstantane = 0;
let derniereEcritureHisto = 0;

const cleDest = (p) => posix(relative(RACINE, destination(p.source, p.lang)));

function photographier() {
  if (!U) return instantane;
  const { unites, plans, sources } = U;
  const vol = new Set(etat.lots.filter((l) => !l.recupere).flatMap((l) => l.ids));
  const fait = (id) => existsSync(join(RES_DIR, `${id}.txt`));
  const langues = {};
  for (const lang of Object.keys(LANGUES)) {
    const p = plans.filter((x) => x.lang === lang);
    const statuts = new Array(sources.length).fill('.');
    let caracteres = 0, caracteresFaits = 0, requetes = 0, faites = 0, traduits = 0, replis = 0, enVol = 0;
    for (const x of p) {
      const s = etat.fichiers[cleDest(x)]?.statut;
      const taille = sources[x.index].c;
      caracteres += taille;
      const okIds = x.ids.filter(fait).length;
      requetes += x.ids.length; faites += okIds;
      if (s === 'traduit' || s === 'conserve') { statuts[x.index] = 't'; traduits++; caracteresFaits += taille; }
      else if (s === 'repli' || s === 'retire') { statuts[x.index] = 'r'; replis++; caracteresFaits += taille; }
      else if (x.ids.some((id) => vol.has(id))) { statuts[x.index] = 'v'; enVol++; caracteresFaits += x.ids.length ? taille * okIds / x.ids.length : 0; }
    }
    langues[lang] = { active: controle.langs.includes(lang), fichiers: p.length, traduits, replis, enVol, requetes, faites, caracteres, caracteresFaits, statuts: statuts.join('') };
  }
  const anthropicFaites = etat.lots.reduce((n, l) => n + (l.compte?.succeeded ?? 0) + (l.compte?.errored ?? 0), 0);
  instantane = {
    maj: new Date().toISOString(), debut: etat.debut, phase: etat.phase, modele: O.model, pause: controle.pause,
    actif: !!travailEnCours, langsActives: controle.langs, langues, anthropicFaites, credits: controle.creditsEpuises ? 'epuises' : 'ok',
    lots: etat.lots.map(({ id, lang, statut, compte, cree, ok, ko, types }) => ({ id, lang, statut, compte, cree, ok, ko, types, taille: Object.values(types ?? {}).reduce((a, b) => a + b, 0) })),
    usage: etat.usage, cout: +((etat.usage.in * PRIX.in + etat.usage.out * PRIX.out) / 1e6).toFixed(2), erreursApi: etat.erreursApi ?? 0,
    build: etat.build ? { statut: etat.build.statut, essais: etat.build.essais, replis: etat.build.replis?.length ?? 0 } : null,
    git: etat.git, journal: journalRecent.slice(-25)
  };
  dernierInstantane = Date.now();
  if (Date.now() - derniereEcritureHisto > 30000) {
    derniereEcritureHisto = Date.now();
    const { journal, ...reste } = instantane;
    appendFileSync(HISTORIQUE, JSON.stringify({ ...reste, journal: journal.slice(-6) }) + '\n');
  }
  return instantane;
}

function lireCorps(req) {
  return new Promise((ok) => { let d = ''; req.on('data', (c) => { d += c; if (d.length > 1e5) req.destroy(); }); req.on('end', () => { try { ok(JSON.parse(d || '{}')); } catch { ok({}); } }); });
}

async function commande(c) {
  const langs = (c.langs ?? []).filter((l) => l in LANGUES);
  switch (c.action) {
    case 'lancer':
      if (travailEnCours) return { ok: false, message: 'Un travail tourne deja.' };
      if (!langs.length) return { ok: false, message: 'Choisis au moins une langue.' };
      travailEnCours = travail(langs, Number(c.limite) || 0, c.build !== false).finally(() => { travailEnCours = null; photographier(); });
      return { ok: true, message: `Lancement : ${langs.join(', ')}${c.limite ? `, ${c.limite} fichiers` : ', site complet'}` };
    case 'pause': controle.pause = true; log('pause demandee : plus aucun nouveau lot, les lots en cours continuent chez Anthropic'); return { ok: true };
    case 'reprendre': controle.pause = false; controle.creditsEpuises = false; etat.credits = 'ok'; log('reprise (les clefs seront reessayees)'); return { ok: true };
    default: return { ok: false, message: 'Action inconnue.' };
  }
}

function serveur() {
  const types = { '.html': 'text/html; charset=utf-8', '.js': 'text/javascript; charset=utf-8', '.css': 'text/css', '.json': 'application/json' };
  createServer(async (req, res) => {
    const origine = req.headers.origin;
    const origineOk = !origine || ORIGINES.some((re) => re.test(origine));
    if (origine && origineOk) res.setHeader('Access-Control-Allow-Origin', origine);
    res.setHeader('Access-Control-Allow-Private-Network', 'true');
    res.setHeader('Access-Control-Allow-Headers', 'content-type');
    res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
    if (req.method === 'OPTIONS') { res.writeHead(204); return res.end(); }
    const url = new URL(req.url, 'http://x');
    const json = (code, o) => { res.writeHead(code, { 'content-type': types['.json'], 'cache-control': 'no-store' }); res.end(JSON.stringify(o)); };
    if (url.pathname === '/etat.json') return json(200, Date.now() - dernierInstantane > 2000 ? photographier() : instantane);
    if (url.pathname === '/plan.json') return json(200, { sources: U?.sources ?? [], langues: Object.fromEntries(Object.entries(LANGUES).map(([k, v]) => [k, v.nom])) });
    if (url.pathname === '/historique.json') {
      const lignes = existsSync(HISTORIQUE) ? readFileSync(HISTORIQUE, 'utf8').trim().split('\n').filter(Boolean) : [];
      return json(200, lignes.map((l) => JSON.parse(l)));
    }
    if (url.pathname === '/api/commande' && req.method === 'POST') {
      if (!origineOk) return json(403, { ok: false, message: 'Origine refusee.' });
      return json(200, await commande(await lireCorps(req)));
    }
    const f = join(PUBLIC_TABLEAU, url.pathname === '/' ? 'index.html' : url.pathname.replace(/^\/+/, ''));
    if (!f.startsWith(PUBLIC_TABLEAU) || !existsSync(f)) { res.writeHead(404); return res.end(); }
    res.writeHead(200, { 'content-type': types[extname(f)] ?? 'application/octet-stream', 'cache-control': 'no-store' });
    res.end(readFileSync(f));
  }).listen(O.port, () => log(`tableau de bord : http://localhost:${O.port}/`));
}

/* ------------------------------------------------------------------ principal */

function selection(langs, limite) {
  let plans = U.plans.filter((p) => langs.includes(p.lang));
  if (limite) {
    const indices = (type, n) => U.sources.map((s, i) => [s.t, i]).filter(([t]) => t === type).slice(0, n).map(([, i]) => i);
    const garde = new Set([...indices('md', limite), ...indices('code', Math.ceil(limite / 4))]);
    plans = plans.filter((p) => garde.has(p.index));
  }
  return plans;
}

async function travail(langs, limite, build) {
  const { unites } = U;
  const plans = selection(langs, limite);
  const ids = new Set(plans.flatMap((p) => p.ids));
  controle.langs = langs; controle.limite = limite; controle.demarre = new Date().toISOString();
  const caracteres = [...ids].reduce((n, id) => n + unites.get(id).source.length, 0);
  log(`lancement : ${plans.length} fichiers, ${ids.size} requetes (${langs.join(', ')}), ${(caracteres / 1e6).toFixed(1)} M caracteres, environ ${((caracteres / 3.3 * PRIX.in + caracteres / 3 * PRIX.out) / 1e6).toFixed(0)} $`);
  etat.phase = 'traduction';
  const enVol = () => new Set(etat.lots.filter((l) => !l.recupere).flatMap((l) => l.ids));
  for (;;) {
    let aEnvoyer = [];
    try {
      await suivreLots(unites);
      const vol = enVol();
      aEnvoyer = [...ids].filter((id) => !existsSync(join(RES_DIR, `${id}.txt`)) && !vol.has(id) && (etat.tentatives[id] ?? 0) < MAX_TENTATIVES);
      if (aEnvoyer.length && !controle.pause && !controle.creditsEpuises) await soumettre(aEnvoyer, unites);
      assembler(plans);
      sauver();
      const resteEnVol = [...enVol()].some((id) => ids.has(id));
      if (!aEnvoyer.length && !resteEnVol) break;
      // Plus de credits et plus rien en vol : on termine avec ce qu'on a, le reste reste en francais.
      if (controle.creditsEpuises && !resteEnVol) { log(`${aEnvoyer.length} requetes restent sans credit : repli francais pour ces fichiers, on passe au build`); break; }
    } catch (e) {
      if (e.credits && !controle.creditsEpuises) { controle.creditsEpuises = true; log('credits Anthropic epuises sur toutes les clefs : plus aucun envoi'); }
      else log(`erreur : ${e.message}`);
    }
    photographier();
    await pause(controle.creditsEpuises ? 5000 : O.intervalle * 1000);
  }
  assembler(plans, controle.creditsEpuises);
  log('traduction terminee');
  if (build) {
    photographier();
    const ok = await construire(plans);
    if (ok) await publier(langs);
  }
  etat.phase = 'termine';
  photographier();
  sauver();
  ecrireRapport();
  log('travail termine');
}

/**
 * Importe des traductions faites a la main ou par un autre agent (Grok Bot) : le dossier
 * contient `<lang>/<chemin relatif a src/>` de la source francaise, par exemple
 * `en/content/blog/linux/mon-article.md`. Le fichier est valide puis ecrit a sa destination,
 * et le fichier source est marque traduit.
 *   node scripts/traduction-batch/traduire.mjs importer C:\...\grok-outbox
 */
function importer(dossier) {
  const plansParCle = new Map(U.plans.map((p) => [`${p.lang}/${posix(relative(join(RACINE, 'src'), p.source))}`, p]));
  let ok = 0, ko = 0;
  for (const f of parcourir(dossier, () => true)) {
    const cle = posix(relative(dossier, f));
    const p = plansParCle.get(cle);
    if (!p) { log(`import : ${cle} ne correspond a aucune source, ignore`); ko++; continue; }
    const texte = readFileSync(f, 'utf8');
    const original = readFileSync(p.source, 'utf8');
    const erreur = valider({ type: p.type, source: original }, texte, null);
    if (erreur) { log(`import : ${cle} refuse (${erreur})`); ko++; continue; }
    const dest = destination(p.source, p.lang);
    mkdirSync(dirname(dest), { recursive: true });
    writeFileSync(dest, p.type === 'md' ? reecrireMarkdown(texte, original, p.lang) : reecrireCode(texte, p.source, dest, p.lang));
    etat.fichiers[posix(relative(RACINE, dest))] = { statut: 'traduit', groupe: p.groupe, lang: p.lang, source: posix(relative(RACINE, p.source)), type: p.type, origine: 'import' };
    ok++;
  }
  sauver();
  log(`import : ${ok} fichiers integres, ${ko} ignores`);
}

/** Liste des fichiers restes en francais (repli), pour les confier a un autre agent. */
function listerReplis() {
  return Object.entries(etat.fichiers)
    .filter(([, v]) => v.statut === 'repli' || v.statut === 'retire')
    .map(([dest, v]) => ({ lang: v.lang, source: v.source, dest, groupe: v.groupe }));
}

async function main() {
  if (!CLEFS.length) throw new Error('Aucune clef Anthropic (key_anthropic dans .env).');
  U = construireUnites();
  if (O.cmd === 'importer') return importer(resolve(process.argv[3]));
  if (O.cmd === 'replis') { const l = listerReplis(); console.log(l.map((r) => `${r.lang}\t${r.source}`).join('\n')); console.log(`${l.length} fichiers`); return; }
  const caracteres = [...U.unites.values()].reduce((n, u) => n + u.source.length, 0);
  log(`${U.sources.length} fichiers sources, ${U.unites.size} requetes pour toutes les langues, ${(caracteres / 1e6).toFixed(1)} M caracteres`);
  if (O.cmd === 'plan') return;
  serveur();
  photographier();
  try { await rattacher(U.unites); } catch (e) { log(`rattachement : ${e.message}`); }
  if (etat.credits === 'epuises') controle.creditsEpuises = true;
  // Le suivi des lots tourne meme sans travail actif, pour que la vue reste vivante.
  setInterval(() => { if (!travailEnCours) photographier(); }, 10000);
  if (O.cmd === 'run') await commande({ action: 'lancer', langs: O.langs, limite: O.limit, build: O.build });
  else log('en attente d\'un lancement depuis le tableau de bord');
}

function ecrireRapport() {
  const l = Object.entries(instantane.langues ?? {}).map(([k, v]) => `| ${k} | ${v.fichiers} | ${v.traduits} | ${v.replis} |`).join('\n');
  writeFileSync(join(ETAT_DIR, 'RAPPORT.md'), `# Rapport de traduction\n\nDebut ${etat.debut}, fin ${new Date().toISOString()}.\n\n| Langue | Fichiers | Traduits | Repli FR / retires |\n|---|---|---|---|\n${l}\n\nJetons : ${etat.usage.in} en entree, ${etat.usage.out} en sortie, environ ${instantane.cout} $.\n\nBuild : ${etat.build?.statut ?? 'non lance'} (${etat.build?.essais ?? 0} essais, ${etat.build?.replis?.length ?? 0} fichiers remplaces).\n\nGit : ${etat.git ? `branche ${etat.git.branche}, langues publiees ${etat.git.publiees.join(', ')}` : 'aucun commit'}.\n`);
}

main().catch((e) => { log(`arret : ${e.stack ?? e.message}`); process.exitCode = 1; });

Appendix B: the dashboard

Three static files, served by the engine itself and mirrored on this site:

• index.html: the layout: header counters, “how it works” panel, the 3D canvas, per-language progress, the log and the control panel.
• app.js: the three.js scene (530 lines). Buildings are a single InstancedMesh holding four floor boxes per file, heights taken from plan.json; the instance colour of each floor follows the language status in etat.json: t translated, v in flight, r fallback, . pending. Drones are small meshes with CSS2D labels, created from the lots array and animated in orbit while statut !== 'ended'. The replay simply feeds recorded snapshots to the same render function at a chosen speed.
• style.css: dark theme, panels, buttons.

The live page is at /atelier-traduction/. It reads the engine on localhost:4599, so it will only show data on a machine where the engine is running; the screenshots in this article come from the night replay.