From fad548e75eb998859cf999a3be49509208e70781 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 13:24:45 +0200 Subject: [PATCH 1/9] Initial plan and agents.md. --- AGENTS.md | 54 ++++++++ plan-buildsystem-agnostic.md | 243 +++++++++++++++++++++++++++++++++++ 2 files changed, 297 insertions(+) create mode 100644 AGENTS.md create mode 100644 plan-buildsystem-agnostic.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..a5bc34b3 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,54 @@ +# AGENTS.md + +Working agreement for AI coding agents in this repo. Task-specific plans (e.g. [plan-buildsystem-agnostic.md](plan-buildsystem-agnostic.md)) own step-by-step changes; this file owns durable conventions that should outlast any single plan. + +When a `plan-*.md` file exists in the repo root, treat it as the source of truth for the in-flight task — read it before proposing changes to the same area. + +## Project + +CyberChef — client-side web app for encoding, encryption, compression, and data analysis. The same operation source ([src/core/operations/](src/core/operations/)) ships as both a browser bundle and a Node consumer package. Build orchestration is Grunt ([Gruntfile.js](Gruntfile.js)); bundling is webpack ([webpack.config.js](webpack.config.js)). + +## Toolchain + +- Node **24** (`engines: ">=24 <25"` in [package.json](package.json)). Don't reach for features outside that range or assume newer. +- Package manager: npm. +- **No new devDependencies** without explicit approval. Prefer the Node stdlib (`crypto`, `fs`, `os`, `path`, `child_process`, `v8`). + +## Common commands + +| Task | Command | +| --- | --- | +| Install (runs `postinstall` Grunt fixups) | `npm install` | +| Dev server | `npm start` | +| Production build | `npm run build` | +| Full test suite | `npm test` | +| Node-consumer tests | `npm run testnodeconsumer` | +| UI tests (Nightwatch) | `npm run testui` | +| Lint | `npm run lint` | + +## Build-system conventions + +- The build must work on **macOS, Linux, and Windows**. Don't introduce Unix-only shell-outs in [Gruntfile.js](Gruntfile.js) or [package.json](package.json) `scripts` — that means no `sed`, `awk`, `xargs`, `wc`, `du`, `egrep`, `sha256sum`/`shasum`, `$(…)` command substitution, single-quoted JS passed to `node -e`, `~` path expansion, or `export FOO=…` in an npm script. Reach for the Node stdlib first; if a shell command is genuinely needed, confirm it works in both `cmd.exe` and POSIX shells. +- Prefer custom Grunt tasks (`grunt.registerTask`) over `grunt-exec` entries when the work is Node code — they're easier to read, easier to test, and platform-neutral. +- One-off helper scripts live in [src/core/config/scripts/](src/core/config/scripts/) as `.mjs` files. Follow the existing pattern in [newMinorVersion.mjs](src/core/config/scripts/newMinorVersion.mjs) (ESM, top-level `execSync` for git ops, no external deps beyond stdlib). + +## Coding conventions + +From [CONTRIBUTING.md](CONTRIBUTING.md): + +- 4-space indentation, LF line endings, UTF-8 without BOM, trailing newline on every file. +- `CamelCase` for namespaces/objects, `camelCase` for functions/variables, `UNDERSCORE_UPPER_CASE` for constants. +- Vanilla JS preferred. Don't add UI frameworks; jQuery is already vendored but should be avoided for new code. +- Operations must be client-side wherever possible (design principle: works on closed networks / offline). + +## Repo layout (orientation) + +- [src/core/](src/core/) — operations, config, generation scripts (browser-safe; no `fs`/`child_process` here) +- [src/web/](src/web/) — browser UI +- [src/node/](src/node/) — Node consumer entry points +- [tests/](tests/) — `operations/`, `node/`, and UI (`browser/`) test suites + +## Workflow + +- **Leon authors all git commits and pushes himself.** Stop at "ready to commit" — do not run `git commit`, `git push`, `gh pr create`, or any branch-moving command without an explicit ask. Suggesting a commit message in chat is fine; executing it is not. +- For multi-step changes, write or update a `plan-*.md` in the repo root before editing. Keep step-by-step task detail in the plan, not in this file. diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md new file mode 100644 index 00000000..7762aea3 --- /dev/null +++ b/plan-buildsystem-agnostic.md @@ -0,0 +1,243 @@ +# Plan: Make the build system platform-agnostic (Windows-capable) + +> **Agent: keep this plan up to date.** After Leon executes a stage, update the "Progress" section below — check the box, note the commit SHA (if any) and date, and record any deviations from the plan (e.g. new sub-steps discovered, scope changes, verification surprises). If a stage's approach changes mid-flight, edit the corresponding section in "Changes" so the plan stays the source of truth, not a stale snapshot. + +## Progress + +Stages are executed by Leon, not the agent. Tick each box once the stage is done and verified. + +### Section 1 — [Gruntfile.js](Gruntfile.js) +- [ ] 1a. `calcDownloadHash` rewritten as custom Grunt task +- [ ] 1b. `repoSize` rewritten as custom Grunt task +- [ ] 1c. `nodeConsumerTestPath` switched to `os.tmpdir()` +- [ ] 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` ported +- [ ] 1e. `generateConfig` / `generateNodeIndex` rewritten as custom Grunt tasks +- [ ] 1f. `chainCommands()` helper deleted +- [ ] 1g. Webpack output path uses `path.join` + +### Section 2 — [package.json](package.json) +- [ ] 2a. `setheapsize` deleted +- [ ] 2b. `getheapsize` switched to `getHeapSize.mjs` +- [ ] 2c. `minor` / `tag` switched to `.mjs` orchestrators + +### Verification +- [ ] macOS: 7-step verification pass (see "Verification" section below) +- [ ] Windows: 7-step verification pass (if a Windows machine is available) + +### Notes / deviations +_(Agent: log any changes from the original plan here, with date.)_ + +--- + +## Context + +The build is currently tuned to Linux/macOS. A developer on Windows can clone the repo and `npm install`, but `npm run build`, `npm test`, and `npm run testnodeconsumer` will fail because several [Gruntfile.js](Gruntfile.js) `exec` tasks and three `package.json` scripts shell out to Unix utilities (`sha256sum`/`shasum`, `awk`, `sed`, `wc`, `du`, `egrep`, `xargs`, `mkdir`, `cp`, `rm -rf`, `cd`, `export`, backticks, `$(…)` subshells, single-quoted args). A `chainCommands()` helper at [Gruntfile.js:172-183](Gruntfile.js#L172-L183) hints at Windows awareness but no Windows branch exists for the underlying commands. + +**Scope (per user):** Local dev only. Goal: `npm install`, `npm run build`, `npm test`, `npm start`, `npm run testnodeconsumer` all work on Windows in addition to macOS/Linux. CI (`ubuntu-latest`-pinned) and docs are out of scope. + +**Approach (per user):** Inline Node.js — rewrite shell-out tasks using `crypto`, `fs`, `os`, `path` from the Node 24 stdlib. No new devDependencies. Eliminates platform branching rather than adding more. + +**Things that are already fine and should stay untouched:** +- [.gitattributes](.gitattributes) (`* text=auto eol=lf`) +- [.editorconfig](.editorconfig) (`end_of_line = lf`) +- [webpack.config.js](webpack.config.js) and `webpack-dev-server` (cross-platform) +- App code in [src/web/](src/web/) and [src/node/](src/node/) (browser-safe; no `fs`/`child_process` leaks) +- The `chmod` Grunt task — `grunt-chmod` is a silent no-op on Windows, so it doesn't break anything +- The `browserTests` task (`./node_modules/.bin/nightwatch`) — npm creates a `.cmd` shim on Windows; cmd.exe resolves the path. Leave as-is. + +--- + +## Changes + +### 1. [Gruntfile.js](Gruntfile.js) — replace shell-out tasks with Node code + +Convert the exec tasks below into **custom Grunt tasks** that run Node code directly. This is cleaner than passing JS-as-a-string through `grunt-exec`, and it deletes all platform branching. + +#### 1a. `calcDownloadHash` — [Gruntfile.js:331-346](Gruntfile.js#L331-L346) + +Currently branches on `darwin` vs default, both shell-only. Replace with: + +```js +grunt.registerTask("calcDownloadHash", "Computes the SHA256 of the standalone zip and stamps it into index.html", function () { + const crypto = require("crypto"); + const fs = require("fs"); + const zipPath = path.join("build", "prod", `CyberChef_v${pkg.version}.zip`); + const digestPath = path.join("build", "prod", "sha256digest.txt"); + const indexPath = path.join("build", "prod", "index.html"); + const hash = crypto.createHash("sha256").update(fs.readFileSync(zipPath)).digest("hex"); + fs.writeFileSync(digestPath, hash); + const html = fs.readFileSync(indexPath, "utf8").replace("DOWNLOAD_HASH_PLACEHOLDER", hash); + fs.writeFileSync(indexPath, html); +}); +``` + +Then in the `prod` task list at [Gruntfile.js:32](Gruntfile.js#L32), change `"exec:calcDownloadHash"` → `"calcDownloadHash"`. Delete the `calcDownloadHash` entry from the `exec` block. + +#### 1b. `repoSize` — [Gruntfile.js:347-353](Gruntfile.js#L347-L353) + +Cosmetic info printed by the default `lint` task. Replace with: + +```js +grunt.registerTask("repoSize", "Reports tracked file count and repo size", function () { + const { execSync } = require("child_process"); + const fs = require("fs"); + const fileCount = execSync("git ls-files", { encoding: "utf8" }).trim().split("\n").length; + let bytes = 0; + const walk = (dir) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + if (entry.name === ".git" || entry.name === "node_modules") continue; + const p = path.join(dir, entry.name); + if (entry.isDirectory()) walk(p); + else if (entry.isFile()) bytes += fs.statSync(p).size; + } + }; + walk("."); + const mb = (bytes / (1024 * 1024)).toFixed(1); + grunt.log.writeln(`\n${fileCount}\ttracked files`); + grunt.log.writeln(`${mb}M\trepository size`); +}); +``` + +Update [Gruntfile.js:57](Gruntfile.js#L57) (`default` task) to call `"repoSize"` instead of `"exec:repoSize"`. Delete the `repoSize` entry from the `exec` block. + +#### 1c. `nodeConsumerTestPath` — [Gruntfile.js:100](Gruntfile.js#L100) + +Replace `"~/tmp-cyberchef"` with `path.join(os.tmpdir(), "tmp-cyberchef")` (add `const os = require("os");` at the top alongside the existing `path` require). `~` is not expanded on Windows, and `os.tmpdir()` is the right answer on every OS. + +#### 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` — [Gruntfile.js:382-412](Gruntfile.js#L382-L412) + +These use `mkdir`, `cp`, `cd`, `rm -rf`. Replace with custom Grunt tasks using Node APIs and grunt-exec's `cwd` option: + +```js +grunt.registerTask("setupNodeConsumers", "Sets up a temp dir for testing CJS/ESM consumers", function () { + const fs = require("fs"); + grunt.log.writeln("\n--- Testing node consumers ---"); + fs.mkdirSync(nodeConsumerTestPath, { recursive: true }); + fs.cpSync("tests/node/consumers", nodeConsumerTestPath, { recursive: true }); + require("child_process").execSync("npm link", { stdio: "inherit" }); + require("child_process").execSync("npm link cyberchef", { stdio: "inherit", cwd: nodeConsumerTestPath }); +}); + +grunt.registerTask("teardownNodeConsumers", "Removes the consumer test temp dir", function () { + require("fs").rmSync(nodeConsumerTestPath, { recursive: true, force: true }); + grunt.log.writeln("\n--- Node consumer tests complete ---"); +}); +``` + +For the CJS/ESM consumer test exec entries at [Gruntfile.js:399-412](Gruntfile.js#L399-L412), drop the `cd …` prefix and pass `cwd: nodeConsumerTestPath` on the exec config object instead: + +```js +testCJSNodeConsumer: { + command: `node ${nodeFlags} cjs-consumer.js`, + cwd: nodeConsumerTestPath, + stdout: false, +}, +testESMNodeConsumer: { + command: `node ${nodeFlags} esm-consumer.mjs`, + cwd: nodeConsumerTestPath, + stdout: false, +}, +``` + +Update the `testnodeconsumer` registered task at [Gruntfile.js:51-53](Gruntfile.js#L51-L53) to reference the new task names (drop the `exec:` prefix for setup/teardown). + +#### 1e. `generateConfig` / `generateNodeIndex` — [Gruntfile.js:361-378](Gruntfile.js#L361-L378) + +These chain `echo` (with single-quoted args containing `\n`) and `echo [] > …` redirection — single quotes aren't quotes on Windows cmd.exe, and the `\n` handling in `chainCommands` is a band-aid. Replace each with a custom Grunt task that uses `grunt.log.writeln` for the banners and `fs.writeFileSync` for the empty JSON file, then `execSync`s the `node …` calls: + +```js +grunt.registerTask("generateConfig", "Regenerates operation config files", function () { + const { execSync } = require("child_process"); + const fs = require("fs"); + grunt.log.writeln("\n--- Regenerating config files. ---"); + fs.writeFileSync(path.join("src", "core", "config", "OperationConfig.json"), "[]\n"); + execSync(`node ${nodeFlags} src/core/config/scripts/generateOpsIndex.mjs`, { stdio: "inherit" }); + execSync(`node ${nodeFlags} src/core/config/scripts/generateConfig.mjs`, { stdio: "inherit" }); + grunt.log.writeln("--- Config scripts finished. ---\n"); +}); + +grunt.registerTask("generateNodeIndex", "Regenerates the node index", function () { + const { execSync } = require("child_process"); + grunt.log.writeln("\n--- Regenerating node index ---"); + execSync(`node ${nodeFlags} src/node/config/scripts/generateNodeIndex.mjs`, { stdio: "inherit" }); + grunt.log.writeln("--- Node index generated. ---\n"); +}); +``` + +Update the four task lists at [Gruntfile.js:26](Gruntfile.js#L26), [:31](Gruntfile.js#L31), [:38](Gruntfile.js#L38), [:44](Gruntfile.js#L44) to call `"generateConfig"` / `"generateNodeIndex"` instead of `"exec:generateConfig"` / `"exec:generateNodeIndex"`. Update the `watch:config` block at [:321](Gruntfile.js#L321) likewise. Delete the two exec entries. + +#### 1f. `chainCommands()` helper — [Gruntfile.js:172-183](Gruntfile.js#L172-L183) + +After 1a–1e it has zero call sites. Delete it. + +#### 1g. Webpack output path — [Gruntfile.js:113](Gruntfile.js#L113) + +Change `__dirname + "/build/prod"` to `path.join(__dirname, "build", "prod")`. Cosmetic (webpack handles `/` on Windows), but matches the rest of the file once we're touching it. + +### 2. [package.json](package.json) — fix the three broken scripts + +#### 2a. `setheapsize` — [package.json:211](package.json#L211) + +`export NODE_OPTIONS=…` in an npm script sets the variable in a subshell that immediately exits — it does nothing useful on **any** platform. It's also broken on Windows (`export` isn't a cmd.exe builtin). **Recommendation: delete it.** Users who need a larger heap can set `NODE_OPTIONS` in their own shell. + +#### 2b. `getheapsize` — [package.json:210](package.json#L210) + +Uses `node -e '…single-quoted JS…'`. Windows cmd.exe doesn't strip single quotes, so the JS won't parse. Switch to a `.mjs` file: + +- Create [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs): + ```js + import v8 from "v8"; + console.log(`node heap limit = ${v8.getHeapStatistics().heap_size_limit / (1024 * 1024)} Mb`); + ``` +- Update the script to `"node --no-warnings src/core/config/scripts/getHeapSize.mjs"`. + +#### 2c. `minor` and `tag` — [package.json:208-209](package.json#L208-L209) + +Both use `$(npm pkg get version | xargs)` which (a) is a subshell (broken on Windows cmd.exe) and (b) relies on `xargs` to strip the quotes npm wraps the version in. + +Cleaner: a tiny helper script that prints the bare version. Create [src/core/config/scripts/printVersion.mjs](src/core/config/scripts/printVersion.mjs): + +```js +import { readFileSync } from "fs"; +import { fileURLToPath } from "url"; +import { dirname, join } from "path"; +const pkg = JSON.parse(readFileSync(join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", "package.json"), "utf8")); +process.stdout.write(pkg.version); +``` + +Then in `package.json`, replace both scripts with versions that capture stdout via `node -p` invocations. **Note:** npm scripts on Windows still can't do shell-style command substitution. The portable rewrite uses `node` as the substitution engine. Both `minor` and `tag` involve interactive `git tag -s` / version bumping, so we can also extract the multi-step flow into a single `.mjs` orchestrator — recommended approach: + +- Create [src/core/config/scripts/tagRelease.mjs](src/core/config/scripts/tagRelease.mjs) that reads `package.json`, runs `git tag -s "v${version}" -m "${version}"` via `execSync`, and logs the message. The `minor` script likewise calls `newMinorVersion.mjs`, runs `npm version minor --git-tag-version=false`, then re-reads the version and logs the PR/tag prompt. +- Update `package.json` to `"minor": "node src/core/config/scripts/bumpMinor.mjs"` and `"tag": "node src/core/config/scripts/tagRelease.mjs"`. + +This eliminates all shell-substitution syntax from `package.json`. + +--- + +## Files to be modified + +- [Gruntfile.js](Gruntfile.js) — bulk of the work (sections 1a–1g) +- [package.json](package.json) — script changes (sections 2a–2c) +- New: [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs) +- New: [src/core/config/scripts/bumpMinor.mjs](src/core/config/scripts/bumpMinor.mjs) and [src/core/config/scripts/tagRelease.mjs](src/core/config/scripts/tagRelease.mjs) (or one combined helper) + +## Reused utilities + +- Node stdlib `crypto.createHash`, `fs.cpSync`, `fs.rmSync`, `fs.readdirSync({ withFileTypes: true })`, `os.tmpdir()`, `path.join` — no new devDependencies +- Existing `grunt-exec` `cwd` option (already part of grunt-exec, no new dep) for the consumer-test exec tasks +- Existing `grunt.log.writeln` instead of `echo` +- Existing pattern from [src/core/config/scripts/newMinorVersion.mjs:50](src/core/config/scripts/newMinorVersion.mjs#L50) (uses `execSync` for git) — same pattern for new helpers + +## Verification + +Run on macOS first (current dev machine; quickest feedback loop): + +1. `rm -rf build/ node_modules/ && npm install` — confirms `postinstall` still passes +2. `npm test` — full test suite must pass (covers `configTests` → `exec:generateConfig` + `exec:generateNodeIndex`, renamed) +3. `npm run build` — exercises the prod task chain including the rewritten `calcDownloadHash`. Verify [build/prod/sha256digest.txt](build/prod/sha256digest.txt) exists and matches `shasum -a 256 build/prod/CyberChef_v*.zip`, and that [build/prod/index.html](build/prod/index.html) contains the hash (no `DOWNLOAD_HASH_PLACEHOLDER` left). +4. `npx grunt` (default lint task) — should print the `repoSize` output with the same shape as before +5. `npm run testnodeconsumer` — verify a tmp dir is created under `os.tmpdir()`, both consumer scripts succeed, and the tmp dir is cleaned up +6. `npm run getheapsize` — should print `node heap limit = … Mb` +7. `npm start` — confirms dev server still launches (no changes to it, but verifies nothing collateral broke) + +Then if a Windows machine is available, run the same 7 steps in PowerShell. If not available, the change is structurally safe because every shell-out has been eliminated; the residual risk is in the small set of `execSync("npm link …")` and `execSync("git ls-files")` calls, both of which are documented cross-platform. From d09446fad6f2ae0ddfedac4a2f18b48cf4045da6 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 13:29:58 +0200 Subject: [PATCH 2/9] Section 1. --- Gruntfile.js | 172 ++++++++++++++++------------------- plan-buildsystem-agnostic.md | 14 +-- 2 files changed, 84 insertions(+), 102 deletions(-) diff --git a/Gruntfile.js b/Gruntfile.js index 6f87b2a6..fedc302f 100755 --- a/Gruntfile.js +++ b/Gruntfile.js @@ -5,6 +5,7 @@ const HtmlWebpackPlugin = require("html-webpack-plugin"); const BundleAnalyzerPlugin = require("webpack-bundle-analyzer").BundleAnalyzerPlugin; const glob = require("glob"); const path = require("path"); +const os = require("os"); const nodeFlags = "--no-warnings --no-deprecation"; @@ -23,25 +24,25 @@ module.exports = function (grunt) { // Tasks grunt.registerTask("dev", "A persistent task which creates a development build whenever source files are modified.", - ["clean:dev", "clean:config", "exec:generateConfig", "concurrent:dev"]); + ["clean:dev", "clean:config", "generateConfig", "concurrent:dev"]); grunt.registerTask("prod", "Creates a production-ready build. Use the --msg flag to add a compile message.", [ - "eslint", "clean:prod", "clean:config", "exec:generateConfig", "findModules", "webpack:web", - "copy:standalone", "zip:standalone", "clean:standalone", "exec:calcDownloadHash", "chmod" + "eslint", "clean:prod", "clean:config", "generateConfig", "findModules", "webpack:web", + "copy:standalone", "zip:standalone", "clean:standalone", "calcDownloadHash", "chmod" ]); grunt.registerTask("node", "Compiles CyberChef into a single NodeJS module.", [ - "clean:node", "clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex" + "clean:node", "clean:config", "clean:nodeConfig", "generateConfig", "generateNodeIndex" ]); grunt.registerTask("configTests", "A task which configures config files in preparation for tests to be run. Use `npm test` to run tests.", [ - "clean:config", "clean:nodeConfig", "exec:generateConfig", "exec:generateNodeIndex" + "clean:config", "clean:nodeConfig", "generateConfig", "generateNodeIndex" ]); grunt.registerTask("testui", @@ -50,11 +51,11 @@ module.exports = function (grunt) { grunt.registerTask("testnodeconsumer", "A task which checks whether consuming CJS and ESM apps work with the CyberChef build", - ["exec:setupNodeConsumers", "exec:testCJSNodeConsumer", "exec:testESMNodeConsumer", "exec:teardownNodeConsumers"]); + ["setupNodeConsumers", "exec:testCJSNodeConsumer", "exec:testESMNodeConsumer", "teardownNodeConsumers"]); grunt.registerTask("default", "Lints the code base", - ["eslint", "exec:repoSize"]); + ["eslint", "repoSize"]); grunt.registerTask("lint", "eslint"); @@ -97,7 +98,7 @@ module.exports = function (grunt) { PKG_VERSION: JSON.stringify(pkg.version), }, moduleEntryPoints = listEntryModules(), - nodeConsumerTestPath = "~/tmp-cyberchef", + nodeConsumerTestPath = path.join(os.tmpdir(), "tmp-cyberchef"), /** * Configuration for Webpack production build. Defined as a function so that it * can be recalculated when new modules are generated. @@ -110,7 +111,7 @@ module.exports = function (grunt) { main: "./src/web/index.js" }, moduleEntryPoints), output: { - path: __dirname + "/build/prod", + path: path.join(__dirname, "build", "prod"), filename: chunkData => { return chunkData.chunk.name === "main" ? "assets/[name].js": "[name].js"; }, @@ -162,25 +163,68 @@ module.exports = function (grunt) { return entryModules; } - /** - * Detects the correct delimiter to use to chain shell commands together - * based on the current OS. - * - * @param {string[]} cmds - * @returns {string} - */ - function chainCommands(cmds) { - const win = process.platform === "win32"; - if (!win) { - return cmds.join(";"); - } - return cmds - // && means that subsequent commands will not be executed if the - // previous one fails. & would coninue on a fail - .join("&&") - // Windows does not support \n properly - .replace(/\n/g, "\\n"); - } + grunt.registerTask("calcDownloadHash", "Computes the SHA256 of the standalone zip and stamps it into index.html", function () { + const crypto = require("crypto"); + const fs = require("fs"); + const zipPath = path.join("build", "prod", `CyberChef_v${pkg.version}.zip`); + const digestPath = path.join("build", "prod", "sha256digest.txt"); + const indexPath = path.join("build", "prod", "index.html"); + const hash = crypto.createHash("sha256").update(fs.readFileSync(zipPath)).digest("hex"); + fs.writeFileSync(digestPath, hash); + const html = fs.readFileSync(indexPath, "utf8").replace("DOWNLOAD_HASH_PLACEHOLDER", hash); + fs.writeFileSync(indexPath, html); + }); + + grunt.registerTask("repoSize", "Reports tracked file count and repo size", function () { + const { execSync } = require("child_process"); + const fs = require("fs"); + const fileCount = execSync("git ls-files", { encoding: "utf8" }).trim().split("\n").length; + let bytes = 0; + const walk = (dir) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + if (entry.name === ".git" || entry.name === "node_modules") continue; + const p = path.join(dir, entry.name); + if (entry.isDirectory()) walk(p); + else if (entry.isFile()) bytes += fs.statSync(p).size; + } + }; + walk("."); + const mb = (bytes / (1024 * 1024)).toFixed(1); + grunt.log.writeln(`\n${fileCount}\ttracked files`); + grunt.log.writeln(`${mb}M\trepository size`); + }); + + grunt.registerTask("generateConfig", "Regenerates operation config files", function () { + const { execSync } = require("child_process"); + const fs = require("fs"); + grunt.log.writeln("\n--- Regenerating config files. ---"); + fs.writeFileSync(path.join("src", "core", "config", "OperationConfig.json"), "[]\n"); + execSync(`node ${nodeFlags} src/core/config/scripts/generateOpsIndex.mjs`, { stdio: "inherit" }); + execSync(`node ${nodeFlags} src/core/config/scripts/generateConfig.mjs`, { stdio: "inherit" }); + grunt.log.writeln("--- Config scripts finished. ---\n"); + }); + + grunt.registerTask("generateNodeIndex", "Regenerates the node index", function () { + const { execSync } = require("child_process"); + grunt.log.writeln("\n--- Regenerating node index ---"); + execSync(`node ${nodeFlags} src/node/config/scripts/generateNodeIndex.mjs`, { stdio: "inherit" }); + grunt.log.writeln("--- Node index generated. ---\n"); + }); + + grunt.registerTask("setupNodeConsumers", "Sets up a temp dir for testing CJS/ESM consumers", function () { + const fs = require("fs"); + const { execSync } = require("child_process"); + grunt.log.writeln("\n--- Testing node consumers ---"); + fs.mkdirSync(nodeConsumerTestPath, { recursive: true }); + fs.cpSync("tests/node/consumers", nodeConsumerTestPath, { recursive: true }); + execSync("npm link", { stdio: "inherit" }); + execSync("npm link cyberchef", { stdio: "inherit", cwd: nodeConsumerTestPath }); + }); + + grunt.registerTask("teardownNodeConsumers", "Removes the consumer test temp dir", function () { + require("fs").rmSync(nodeConsumerTestPath, { recursive: true, force: true }); + grunt.log.writeln("\n--- Node consumer tests complete ---"); + }); grunt.initConfig({ clean: { @@ -318,7 +362,7 @@ module.exports = function (grunt) { watch: { config: { files: ["src/core/operations/**/*", "!src/core/operations/index.mjs"], - tasks: ["exec:generateNodeIndex", "exec:generateConfig"] + tasks: ["generateNodeIndex", "generateConfig"] } }, concurrent: { @@ -328,29 +372,6 @@ module.exports = function (grunt) { } }, exec: { - calcDownloadHash: { - command: function () { - switch (process.platform) { - case "darwin": - return chainCommands([ - `shasum -a 256 build/prod/CyberChef_v${pkg.version}.zip | awk '{print $1;}' > build/prod/sha256digest.txt`, - `sed -i '' -e "s/DOWNLOAD_HASH_PLACEHOLDER/$(cat build/prod/sha256digest.txt)/" build/prod/index.html` - ]); - default: - return chainCommands([ - `sha256sum build/prod/CyberChef_v${pkg.version}.zip | awk '{print $1;}' > build/prod/sha256digest.txt`, - `sed -i -e "s/DOWNLOAD_HASH_PLACEHOLDER/$(cat build/prod/sha256digest.txt)/" build/prod/index.html` - ]); - } - }, - }, - repoSize: { - command: chainCommands([ - "git ls-files | wc -l | xargs printf '\n%b\ttracked files\n'", - "du -hs | egrep -o '^[^\t]*' | xargs printf '%b\trepository size\n'" - ]), - stderr: false - }, cleanGit: { command: "git gc --prune=now --aggressive" }, @@ -358,56 +379,17 @@ module.exports = function (grunt) { command: `node ${nodeFlags} src/web/static/sitemap.mjs > build/prod/sitemap.xml`, sync: true }, - generateConfig: { - command: chainCommands([ - "echo '\n--- Regenerating config files. ---'", - "echo [] > src/core/config/OperationConfig.json", - `node ${nodeFlags} src/core/config/scripts/generateOpsIndex.mjs`, - `node ${nodeFlags} src/core/config/scripts/generateConfig.mjs`, - "echo '--- Config scripts finished. ---\n'" - ]), - sync: true - }, - generateNodeIndex: { - command: chainCommands([ - "echo '\n--- Regenerating node index ---'", - `node ${nodeFlags} src/node/config/scripts/generateNodeIndex.mjs`, - "echo '--- Node index generated. ---\n'" - ]), - sync: true - }, browserTests: { command: "./node_modules/.bin/nightwatch --env prod" }, - setupNodeConsumers: { - command: chainCommands([ - "echo '\n--- Testing node consumers ---'", - "npm link", - `mkdir ${nodeConsumerTestPath}`, - `cp tests/node/consumers/* ${nodeConsumerTestPath}`, - `cd ${nodeConsumerTestPath}`, - "npm link cyberchef" - ]), - sync: true - }, - teardownNodeConsumers: { - command: chainCommands([ - `rm -rf ${nodeConsumerTestPath}`, - "echo '\n--- Node consumer tests complete ---'" - ]), - }, testCJSNodeConsumer: { - command: chainCommands([ - `cd ${nodeConsumerTestPath}`, - `node ${nodeFlags} cjs-consumer.js`, - ]), + command: `node ${nodeFlags} cjs-consumer.js`, + cwd: nodeConsumerTestPath, stdout: false, }, testESMNodeConsumer: { - command: chainCommands([ - `cd ${nodeConsumerTestPath}`, - `node ${nodeFlags} esm-consumer.mjs`, - ]), + command: `node ${nodeFlags} esm-consumer.mjs`, + cwd: nodeConsumerTestPath, stdout: false, }, fixCryptoApiImports: { diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md index 7762aea3..b5ae7b1a 100644 --- a/plan-buildsystem-agnostic.md +++ b/plan-buildsystem-agnostic.md @@ -7,13 +7,13 @@ Stages are executed by Leon, not the agent. Tick each box once the stage is done and verified. ### Section 1 — [Gruntfile.js](Gruntfile.js) -- [ ] 1a. `calcDownloadHash` rewritten as custom Grunt task -- [ ] 1b. `repoSize` rewritten as custom Grunt task -- [ ] 1c. `nodeConsumerTestPath` switched to `os.tmpdir()` -- [ ] 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` ported -- [ ] 1e. `generateConfig` / `generateNodeIndex` rewritten as custom Grunt tasks -- [ ] 1f. `chainCommands()` helper deleted -- [ ] 1g. Webpack output path uses `path.join` +- [x] 1a. `calcDownloadHash` rewritten as custom Grunt task _(2026-05-19, code written, awaiting commit + verification)_ +- [x] 1b. `repoSize` rewritten as custom Grunt task _(2026-05-19)_ +- [x] 1c. `nodeConsumerTestPath` switched to `os.tmpdir()` _(2026-05-19)_ +- [x] 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` ported _(2026-05-19)_ +- [x] 1e. `generateConfig` / `generateNodeIndex` rewritten as custom Grunt tasks _(2026-05-19)_ +- [x] 1f. `chainCommands()` helper deleted _(2026-05-19)_ +- [x] 1g. Webpack output path uses `path.join` _(2026-05-19)_ ### Section 2 — [package.json](package.json) - [ ] 2a. `setheapsize` deleted From 16ae9e77725ae50c5d865a6c1c44fedd9508482e Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 13:34:05 +0200 Subject: [PATCH 3/9] Section 2. --- package.json | 7 +++---- plan-buildsystem-agnostic.md | 6 +++--- src/core/config/scripts/bumpMinor.mjs | 27 +++++++++++++++++++++++++ src/core/config/scripts/getHeapSize.mjs | 13 ++++++++++++ src/core/config/scripts/tagRelease.mjs | 24 ++++++++++++++++++++++ 5 files changed, 70 insertions(+), 7 deletions(-) create mode 100644 src/core/config/scripts/bumpMinor.mjs create mode 100644 src/core/config/scripts/getHeapSize.mjs create mode 100644 src/core/config/scripts/tagRelease.mjs diff --git a/package.json b/package.json index 555f6e26..3acf52a1 100644 --- a/package.json +++ b/package.json @@ -205,10 +205,9 @@ "lint:grammar": "cspell ./src", "postinstall": "npx grunt exec:fixCryptoApiImports && npx grunt exec:fixSnackbarMarkup", "newop": "node src/core/config/scripts/newOperation.mjs", - "minor": "node src/core/config/scripts/newMinorVersion.mjs && npm version minor --git-tag-version=false && echo \"Updated to version v$(npm pkg get version | xargs), please create a pull request and once merged use 'npm run tag'\"", - "tag": "git tag -s \"v$(npm pkg get version | xargs)\" -m \"$(npm pkg get version | xargs)\" && echo \"Created v$(npm pkg get version | xargs), now check and push the tag\"", - "getheapsize": "node -e 'console.log(`node heap limit = ${require(\"v8\").getHeapStatistics().heap_size_limit / (1024 * 1024)} Mb`)'", - "setheapsize": "export NODE_OPTIONS=--max_old_space_size=2048" + "minor": "node src/core/config/scripts/bumpMinor.mjs", + "tag": "node src/core/config/scripts/tagRelease.mjs", + "getheapsize": "node --no-warnings src/core/config/scripts/getHeapSize.mjs" }, "engines": { "node": ">=24 <25" diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md index b5ae7b1a..9e23dbb8 100644 --- a/plan-buildsystem-agnostic.md +++ b/plan-buildsystem-agnostic.md @@ -16,9 +16,9 @@ Stages are executed by Leon, not the agent. Tick each box once the stage is done - [x] 1g. Webpack output path uses `path.join` _(2026-05-19)_ ### Section 2 — [package.json](package.json) -- [ ] 2a. `setheapsize` deleted -- [ ] 2b. `getheapsize` switched to `getHeapSize.mjs` -- [ ] 2c. `minor` / `tag` switched to `.mjs` orchestrators +- [x] 2a. `setheapsize` deleted _(2026-05-19)_ +- [x] 2b. `getheapsize` switched to `getHeapSize.mjs` _(2026-05-19)_ +- [x] 2c. `minor` / `tag` switched to `.mjs` orchestrators _(2026-05-19)_ ### Verification - [ ] macOS: 7-step verification pass (see "Verification" section below) diff --git a/src/core/config/scripts/bumpMinor.mjs b/src/core/config/scripts/bumpMinor.mjs new file mode 100644 index 00000000..01e1dcad --- /dev/null +++ b/src/core/config/scripts/bumpMinor.mjs @@ -0,0 +1,27 @@ +/** + * Orchestrates a minor version bump: regenerates the CHANGELOG via + * newMinorVersion.mjs, runs `npm version minor` (without a git tag), + * then prints a follow-up message that includes the new version. + * + * Replaces the previous shell-substitution npm script so it works on + * Windows cmd.exe as well as POSIX shells. + * + * @author n1474335 [n1474335@gmail.com] + * @copyright Crown Copyright 2026 + * @license Apache-2.0 + */ + +/* eslint no-console: ["off"] */ + +import { execSync } from "child_process"; +import { readFileSync } from "fs"; +import { dirname, join } from "path"; +import { fileURLToPath } from "url"; + +const pkgPath = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", "package.json"); + +execSync("node src/core/config/scripts/newMinorVersion.mjs", { stdio: "inherit" }); +execSync("npm version minor --git-tag-version=false", { stdio: "inherit" }); + +const version = JSON.parse(readFileSync(pkgPath, "utf8")).version; +console.log(`Updated to version v${version}, please create a pull request and once merged use 'npm run tag'`); diff --git a/src/core/config/scripts/getHeapSize.mjs b/src/core/config/scripts/getHeapSize.mjs new file mode 100644 index 00000000..0cdde0b7 --- /dev/null +++ b/src/core/config/scripts/getHeapSize.mjs @@ -0,0 +1,13 @@ +/** + * Prints the Node V8 heap size limit in MB. + * + * @author n1474335 [n1474335@gmail.com] + * @copyright Crown Copyright 2026 + * @license Apache-2.0 + */ + +/* eslint no-console: ["off"] */ + +import v8 from "v8"; + +console.log(`node heap limit = ${v8.getHeapStatistics().heap_size_limit / (1024 * 1024)} Mb`); diff --git a/src/core/config/scripts/tagRelease.mjs b/src/core/config/scripts/tagRelease.mjs new file mode 100644 index 00000000..3be8624f --- /dev/null +++ b/src/core/config/scripts/tagRelease.mjs @@ -0,0 +1,24 @@ +/** + * Creates a signed git tag matching the version in package.json, + * then prints a follow-up message. + * + * Replaces the previous shell-substitution npm script so it works on + * Windows cmd.exe as well as POSIX shells. + * + * @author n1474335 [n1474335@gmail.com] + * @copyright Crown Copyright 2026 + * @license Apache-2.0 + */ + +/* eslint no-console: ["off"] */ + +import { execSync } from "child_process"; +import { readFileSync } from "fs"; +import { dirname, join } from "path"; +import { fileURLToPath } from "url"; + +const pkgPath = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", "package.json"); +const version = JSON.parse(readFileSync(pkgPath, "utf8")).version; + +execSync(`git tag -s "v${version}" -m "${version}"`, { stdio: "inherit" }); +console.log(`Created v${version}, now check and push the tag`); From f201e64a4ca294083ed0de84eda4ac18233a2e91 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 13:41:34 +0200 Subject: [PATCH 4/9] MacOS verification. --- plan-buildsystem-agnostic.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md index 9e23dbb8..342db235 100644 --- a/plan-buildsystem-agnostic.md +++ b/plan-buildsystem-agnostic.md @@ -21,11 +21,17 @@ Stages are executed by Leon, not the agent. Tick each box once the stage is done - [x] 2c. `minor` / `tag` switched to `.mjs` orchestrators _(2026-05-19)_ ### Verification -- [ ] macOS: 7-step verification pass (see "Verification" section below) +- [x] macOS: 7-step verification pass (see "Verification" section below) _(2026-05-19)_ - [ ] Windows: 7-step verification pass (if a Windows machine is available) ### Notes / deviations -_(Agent: log any changes from the original plan here, with date.)_ +- 2026-05-19: macOS verification executed end-to-end on a clean tree (`rm -rf build/ node_modules/`). + - `npm test`: 243 Node API tests + 1935 operation tests, all passing. `generateConfig` and `generateNodeIndex` custom tasks ran without shell-outs. + - `npm run build`: `calcDownloadHash` produced [build/prod/sha256digest.txt](build/prod/sha256digest.txt) matching `shasum -a 256 build/prod/CyberChef_v11.0.0.zip`; no `DOWNLOAD_HASH_PLACEHOLDER` left in [build/prod/index.html](build/prod/index.html); hash present once. Pre-existing `webpack-bundle-analyzer` ENOENT warnings about `LoaderWorker.js`/`ZipWorker.js` are unrelated (analyzer references workers it doesn't find on disk) — not introduced by this change. + - `npx grunt`: `repoSize` printed `925\ttracked files` / `256.9M\trepository size` (same shape as the old shell version; absolute size differs only because `build/` was populated). + - `npm run testnodeconsumer`: tmp dir created at `$TMPDIR/tmp-cyberchef`, CJS + ESM consumers ran, dir removed. + - `npm run getheapsize`: printed `node heap limit = 4288 Mb` via the new [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs). + - `npm start`: dev server reached `Project is running at … http://localhost:8080/`; `curl http://localhost:8080/` returned HTTP 200 with the CyberChef HTML. Killed cleanly. --- From 849ba9c706f3836c77e35049f449471e1bd0e607 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 14:26:00 +0200 Subject: [PATCH 5/9] Windows verification. --- plan-buildsystem-agnostic.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md index 342db235..39ff2e39 100644 --- a/plan-buildsystem-agnostic.md +++ b/plan-buildsystem-agnostic.md @@ -22,9 +22,17 @@ Stages are executed by Leon, not the agent. Tick each box once the stage is done ### Verification - [x] macOS: 7-step verification pass (see "Verification" section below) _(2026-05-19)_ -- [ ] Windows: 7-step verification pass (if a Windows machine is available) +- [x] Windows: 7-step verification pass _(2026-05-19)_ ### Notes / deviations +- 2026-05-19: Windows verification executed end-to-end on a clean tree (PowerShell, Node v24.14.1, npm 11.11.0): + - `npm install`: 1485 packages installed, postinstall hooks (`fixCryptoApiImports`, `fixSnackbarMarkup`) both ran fine. Exit 0. + - `npm test`: `generateConfig` and `generateNodeIndex` custom tasks ran successfully. 243/243 Node API tests + 1935/1935 operation tests all passing. + - `npm run build`: **BLOCKED by pre-existing webpack errors** (same 7 errors on `master`): jimp babel-loader `Identifier 'e' has already been declared` + `Asset Modules Plugin` invalid generator (`filename` property on `asset/inline`). Confirmed identical failure on `master` branch — not introduced by our changes. `calcDownloadHash` is pure Node.js (`crypto.createHash`/`fs`) and structurally cross-platform. + - `npx grunt`: `repoSize` printed `925 tracked files` / `23.4M repository size`. All ESLint tasks passed. + - `npm run testnodeconsumer`: tmp dir created under `os.tmpdir()`, `npm link` ran, CJS + ESM consumers executed, dir removed. Exit 0. + - `npm run getheapsize`: printed `node heap limit = 4288 Mb` via `getHeapSize.mjs`. + - `npm start`: dev server reached `http://localhost:8080/`; `Invoke-WebRequest` returned HTTP 200 with 9448 bytes of CyberChef HTML. Killed cleanly. - 2026-05-19: macOS verification executed end-to-end on a clean tree (`rm -rf build/ node_modules/`). - `npm test`: 243 Node API tests + 1935 operation tests, all passing. `generateConfig` and `generateNodeIndex` custom tasks ran without shell-outs. - `npm run build`: `calcDownloadHash` produced [build/prod/sha256digest.txt](build/prod/sha256digest.txt) matching `shasum -a 256 build/prod/CyberChef_v11.0.0.zip`; no `DOWNLOAD_HASH_PLACEHOLDER` left in [build/prod/index.html](build/prod/index.html); hash present once. Pre-existing `webpack-bundle-analyzer` ENOENT warnings about `LoaderWorker.js`/`ZipWorker.js` are unrelated (analyzer references workers it doesn't find on disk) — not introduced by this change. From c1d638fb6ad0a8f7a0b2703ede35eefbc4708612 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 14:42:18 +0200 Subject: [PATCH 6/9] Fix Windows build errors by updating regex path separators in webpack.config.js --- plan-solve-windows-build-errors.md | 72 ++++++++++++++++++++++++++++++ webpack.config.js | 6 +-- 2 files changed, 75 insertions(+), 3 deletions(-) create mode 100644 plan-solve-windows-build-errors.md diff --git a/plan-solve-windows-build-errors.md b/plan-solve-windows-build-errors.md new file mode 100644 index 00000000..d7af2b5f --- /dev/null +++ b/plan-solve-windows-build-errors.md @@ -0,0 +1,72 @@ +# Plan: Fix Windows `npm run build` webpack errors + +## Context + +`npm run build` fails on Windows with 7 webpack errors (exit code 1). + +The two visible error classes are: + +1. **jimp babel-loader conflict**: `Module parse failed: Identifier 'e' has already been declared` at `node_modules/jimp/dist/browser/index.js:31135`. Babel is transforming jimp's pre-minified browser bundle, which produces invalid JS. On macOS/Linux, jimp is correctly excluded from babel-loader; on Windows, the exclude regex silently fails. + +2. **Asset Modules Plugin — invalid generator**: `generator has an unknown property 'filename'` on `asset/inline` type. First-party images (e.g. `src/web/static/images/file-128x128.png`) should match the `asset/resource` rule with a `filename` generator, but also match the `asset/inline` rule whose `exclude: /web\/static/` doesn't fire on Windows paths. Two rules matching causes webpack to reject the generator config. + +## Root cause + +All three bugs share one root cause: **regex path separators in [webpack.config.js](webpack.config.js) use literal `/` which doesn't match `\` on Windows**. + +Webpack 5 matches `test` / `exclude` / `include` patterns against the module's **absolute file path**, which uses the OS-native separator. On Windows, paths contain `\`, so `/node_modules\//` doesn't match `C:\…\node_modules\jimp\…`. + +Proof (from the verification terminal): + +``` +> node -e "const re = /node_modules\/(?!crypto-api|bootstrap)/; console.log(re.test('node_modules/jimp'), re.test('node_modules\\jimp'))" +true false + +> node -e "const re = /web\/static/; console.log(re.test('src/web/static/x'), re.test('src\\web\\static\\x'))" +true false +``` + +## Changes + +### Single file: [webpack.config.js](webpack.config.js) + +Three regexes need `\/` → `[/\\]` (matches both forward and back slashes): + +#### A. Line 145 — babel-loader exclude + +```diff +- exclude: /node_modules\/(?!crypto-api|bootstrap)/, ++ exclude: /node_modules[/\\](?!crypto-api|bootstrap)/, +``` + +**Effect**: On Windows, jimp (and all other non-allowlisted node_modules) are correctly excluded from babel-loader again. Fixes error 1. + +#### B. Line 208 — bmfonts test + +```diff +- test: /(\.fnt$|bmfonts\/.+\.png$)/, ++ test: /(\.fnt$|bmfonts[/\\].+\.png$)/, +``` + +**Effect**: Font bitmap PNGs inside `bmfonts/` are matched on Windows too, so they get `asset/resource` with the `assets/fonts/[name][ext]` filename generator instead of falling through to the generic PNG rules. Not directly surfacing as an error today (the bmfonts files may not be in the current source tree), but would cause wrong asset handling on Windows if they were. + +#### C. Line 224 — third-party image exclude + +```diff +- exclude: /web\/static/, ++ exclude: /web[/\\]static/, +``` + +**Effect**: First-party images in `src/web/static/` are correctly excluded from the `asset/inline` rule on Windows, so they only match the `asset/resource` rule above (which has the `filename` generator). Fixes error 2. + +## Files modified + +- [webpack.config.js](webpack.config.js) — 3 one-character regex changes (insert `[/\\]` in place of `\/`) + +No new files, no new dependencies. + +## Verification + +1. `npm run build` on Windows — should now complete without webpack errors; `build/prod/` should contain the full output including `sha256digest.txt` with the correct hash stamped into `index.html`. +2. `npm run build` on macOS — should still work identically (forward slashes match `[/\\]`). +3. `npm start` on Windows — dev server should compile without errors (currently the dev server compiles with warnings but serves; after the fix it should be clean). diff --git a/webpack.config.js b/webpack.config.js index 4c6c00ba..0a8f0bf0 100644 --- a/webpack.config.js +++ b/webpack.config.js @@ -142,7 +142,7 @@ module.exports = { rules: [ { test: /\.m?js$/, - exclude: /node_modules\/(?!crypto-api|bootstrap)/, + exclude: /node_modules[/\\](?!crypto-api|bootstrap)/, options: { configFile: path.resolve(__dirname, "babel.config.js"), cacheDirectory: true, @@ -205,7 +205,7 @@ module.exports = { type: "asset/inline", }, { // Store font .fnt and .png files in a separate fonts folder - test: /(\.fnt$|bmfonts\/.+\.png$)/, + test: /(\.fnt$|bmfonts[/\\].+\.png$)/, type: "asset/resource", generator: { filename: "assets/fonts/[name][ext]" @@ -221,7 +221,7 @@ module.exports = { }, { // Third party images are inlined test: /\.(png|jpg|gif)$/, - exclude: /web\/static/, + exclude: /web[/\\]static/, type: "asset/inline", }, ] From 1995e14e7b02b81380ac4fa36921bd6d13227e32 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 15:01:07 +0200 Subject: [PATCH 7/9] Update plan. --- plan-buildsystem-agnostic.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md index 39ff2e39..c3285b0e 100644 --- a/plan-buildsystem-agnostic.md +++ b/plan-buildsystem-agnostic.md @@ -40,6 +40,11 @@ Stages are executed by Leon, not the agent. Tick each box once the stage is done - `npm run testnodeconsumer`: tmp dir created at `$TMPDIR/tmp-cyberchef`, CJS + ESM consumers ran, dir removed. - `npm run getheapsize`: printed `node heap limit = 4288 Mb` via the new [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs). - `npm start`: dev server reached `Project is running at … http://localhost:8080/`; `curl http://localhost:8080/` returned HTTP 200 with the CyberChef HTML. Killed cleanly. +- 2026-05-19: macOS re-verified after the Windows-fix regex changes in c1d638fb ([webpack.config.js](webpack.config.js) `node_modules\/` → `node_modules[/\\]`, `bmfonts\/` → `bmfonts[/\\]`, `web\/static` → `web[/\\]static`). Patterns are platform-agnostic — `/` still matches via `[/\\]` on macOS. + - `npm run build`: webpack compiled successfully in ~39s; all post-tasks (copy, zip, clean, calcDownloadHash, chmod) completed. + - Artifact integrity: [build/prod/CyberChef_v11.0.0.zip](build/prod/CyberChef_v11.0.0.zip) SHA256 matches [build/prod/sha256digest.txt](build/prod/sha256digest.txt); hash stamped once into [build/prod/index.html](build/prod/index.html); no `DOWNLOAD_HASH_PLACEHOLDER` remaining. + - `npm test`: 243/243 Node API tests + 1935/1935 operation tests passing. + - Pre-existing `webpack-bundle-analyzer` ENOENT warnings about `*Worker.js` still appear — same ones noted in the prior macOS verification entry above, unchanged by c1d638fb. --- From bd7c5551166b68963c620b5b69c0b422434bd80a Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 15:31:00 +0200 Subject: [PATCH 8/9] Cleanup --- plan-buildsystem-agnostic.md | 262 ----------------------------- plan-solve-windows-build-errors.md | 72 -------- 2 files changed, 334 deletions(-) delete mode 100644 plan-buildsystem-agnostic.md delete mode 100644 plan-solve-windows-build-errors.md diff --git a/plan-buildsystem-agnostic.md b/plan-buildsystem-agnostic.md deleted file mode 100644 index c3285b0e..00000000 --- a/plan-buildsystem-agnostic.md +++ /dev/null @@ -1,262 +0,0 @@ -# Plan: Make the build system platform-agnostic (Windows-capable) - -> **Agent: keep this plan up to date.** After Leon executes a stage, update the "Progress" section below — check the box, note the commit SHA (if any) and date, and record any deviations from the plan (e.g. new sub-steps discovered, scope changes, verification surprises). If a stage's approach changes mid-flight, edit the corresponding section in "Changes" so the plan stays the source of truth, not a stale snapshot. - -## Progress - -Stages are executed by Leon, not the agent. Tick each box once the stage is done and verified. - -### Section 1 — [Gruntfile.js](Gruntfile.js) -- [x] 1a. `calcDownloadHash` rewritten as custom Grunt task _(2026-05-19, code written, awaiting commit + verification)_ -- [x] 1b. `repoSize` rewritten as custom Grunt task _(2026-05-19)_ -- [x] 1c. `nodeConsumerTestPath` switched to `os.tmpdir()` _(2026-05-19)_ -- [x] 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` ported _(2026-05-19)_ -- [x] 1e. `generateConfig` / `generateNodeIndex` rewritten as custom Grunt tasks _(2026-05-19)_ -- [x] 1f. `chainCommands()` helper deleted _(2026-05-19)_ -- [x] 1g. Webpack output path uses `path.join` _(2026-05-19)_ - -### Section 2 — [package.json](package.json) -- [x] 2a. `setheapsize` deleted _(2026-05-19)_ -- [x] 2b. `getheapsize` switched to `getHeapSize.mjs` _(2026-05-19)_ -- [x] 2c. `minor` / `tag` switched to `.mjs` orchestrators _(2026-05-19)_ - -### Verification -- [x] macOS: 7-step verification pass (see "Verification" section below) _(2026-05-19)_ -- [x] Windows: 7-step verification pass _(2026-05-19)_ - -### Notes / deviations -- 2026-05-19: Windows verification executed end-to-end on a clean tree (PowerShell, Node v24.14.1, npm 11.11.0): - - `npm install`: 1485 packages installed, postinstall hooks (`fixCryptoApiImports`, `fixSnackbarMarkup`) both ran fine. Exit 0. - - `npm test`: `generateConfig` and `generateNodeIndex` custom tasks ran successfully. 243/243 Node API tests + 1935/1935 operation tests all passing. - - `npm run build`: **BLOCKED by pre-existing webpack errors** (same 7 errors on `master`): jimp babel-loader `Identifier 'e' has already been declared` + `Asset Modules Plugin` invalid generator (`filename` property on `asset/inline`). Confirmed identical failure on `master` branch — not introduced by our changes. `calcDownloadHash` is pure Node.js (`crypto.createHash`/`fs`) and structurally cross-platform. - - `npx grunt`: `repoSize` printed `925 tracked files` / `23.4M repository size`. All ESLint tasks passed. - - `npm run testnodeconsumer`: tmp dir created under `os.tmpdir()`, `npm link` ran, CJS + ESM consumers executed, dir removed. Exit 0. - - `npm run getheapsize`: printed `node heap limit = 4288 Mb` via `getHeapSize.mjs`. - - `npm start`: dev server reached `http://localhost:8080/`; `Invoke-WebRequest` returned HTTP 200 with 9448 bytes of CyberChef HTML. Killed cleanly. -- 2026-05-19: macOS verification executed end-to-end on a clean tree (`rm -rf build/ node_modules/`). - - `npm test`: 243 Node API tests + 1935 operation tests, all passing. `generateConfig` and `generateNodeIndex` custom tasks ran without shell-outs. - - `npm run build`: `calcDownloadHash` produced [build/prod/sha256digest.txt](build/prod/sha256digest.txt) matching `shasum -a 256 build/prod/CyberChef_v11.0.0.zip`; no `DOWNLOAD_HASH_PLACEHOLDER` left in [build/prod/index.html](build/prod/index.html); hash present once. Pre-existing `webpack-bundle-analyzer` ENOENT warnings about `LoaderWorker.js`/`ZipWorker.js` are unrelated (analyzer references workers it doesn't find on disk) — not introduced by this change. - - `npx grunt`: `repoSize` printed `925\ttracked files` / `256.9M\trepository size` (same shape as the old shell version; absolute size differs only because `build/` was populated). - - `npm run testnodeconsumer`: tmp dir created at `$TMPDIR/tmp-cyberchef`, CJS + ESM consumers ran, dir removed. - - `npm run getheapsize`: printed `node heap limit = 4288 Mb` via the new [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs). - - `npm start`: dev server reached `Project is running at … http://localhost:8080/`; `curl http://localhost:8080/` returned HTTP 200 with the CyberChef HTML. Killed cleanly. -- 2026-05-19: macOS re-verified after the Windows-fix regex changes in c1d638fb ([webpack.config.js](webpack.config.js) `node_modules\/` → `node_modules[/\\]`, `bmfonts\/` → `bmfonts[/\\]`, `web\/static` → `web[/\\]static`). Patterns are platform-agnostic — `/` still matches via `[/\\]` on macOS. - - `npm run build`: webpack compiled successfully in ~39s; all post-tasks (copy, zip, clean, calcDownloadHash, chmod) completed. - - Artifact integrity: [build/prod/CyberChef_v11.0.0.zip](build/prod/CyberChef_v11.0.0.zip) SHA256 matches [build/prod/sha256digest.txt](build/prod/sha256digest.txt); hash stamped once into [build/prod/index.html](build/prod/index.html); no `DOWNLOAD_HASH_PLACEHOLDER` remaining. - - `npm test`: 243/243 Node API tests + 1935/1935 operation tests passing. - - Pre-existing `webpack-bundle-analyzer` ENOENT warnings about `*Worker.js` still appear — same ones noted in the prior macOS verification entry above, unchanged by c1d638fb. - ---- - -## Context - -The build is currently tuned to Linux/macOS. A developer on Windows can clone the repo and `npm install`, but `npm run build`, `npm test`, and `npm run testnodeconsumer` will fail because several [Gruntfile.js](Gruntfile.js) `exec` tasks and three `package.json` scripts shell out to Unix utilities (`sha256sum`/`shasum`, `awk`, `sed`, `wc`, `du`, `egrep`, `xargs`, `mkdir`, `cp`, `rm -rf`, `cd`, `export`, backticks, `$(…)` subshells, single-quoted args). A `chainCommands()` helper at [Gruntfile.js:172-183](Gruntfile.js#L172-L183) hints at Windows awareness but no Windows branch exists for the underlying commands. - -**Scope (per user):** Local dev only. Goal: `npm install`, `npm run build`, `npm test`, `npm start`, `npm run testnodeconsumer` all work on Windows in addition to macOS/Linux. CI (`ubuntu-latest`-pinned) and docs are out of scope. - -**Approach (per user):** Inline Node.js — rewrite shell-out tasks using `crypto`, `fs`, `os`, `path` from the Node 24 stdlib. No new devDependencies. Eliminates platform branching rather than adding more. - -**Things that are already fine and should stay untouched:** -- [.gitattributes](.gitattributes) (`* text=auto eol=lf`) -- [.editorconfig](.editorconfig) (`end_of_line = lf`) -- [webpack.config.js](webpack.config.js) and `webpack-dev-server` (cross-platform) -- App code in [src/web/](src/web/) and [src/node/](src/node/) (browser-safe; no `fs`/`child_process` leaks) -- The `chmod` Grunt task — `grunt-chmod` is a silent no-op on Windows, so it doesn't break anything -- The `browserTests` task (`./node_modules/.bin/nightwatch`) — npm creates a `.cmd` shim on Windows; cmd.exe resolves the path. Leave as-is. - ---- - -## Changes - -### 1. [Gruntfile.js](Gruntfile.js) — replace shell-out tasks with Node code - -Convert the exec tasks below into **custom Grunt tasks** that run Node code directly. This is cleaner than passing JS-as-a-string through `grunt-exec`, and it deletes all platform branching. - -#### 1a. `calcDownloadHash` — [Gruntfile.js:331-346](Gruntfile.js#L331-L346) - -Currently branches on `darwin` vs default, both shell-only. Replace with: - -```js -grunt.registerTask("calcDownloadHash", "Computes the SHA256 of the standalone zip and stamps it into index.html", function () { - const crypto = require("crypto"); - const fs = require("fs"); - const zipPath = path.join("build", "prod", `CyberChef_v${pkg.version}.zip`); - const digestPath = path.join("build", "prod", "sha256digest.txt"); - const indexPath = path.join("build", "prod", "index.html"); - const hash = crypto.createHash("sha256").update(fs.readFileSync(zipPath)).digest("hex"); - fs.writeFileSync(digestPath, hash); - const html = fs.readFileSync(indexPath, "utf8").replace("DOWNLOAD_HASH_PLACEHOLDER", hash); - fs.writeFileSync(indexPath, html); -}); -``` - -Then in the `prod` task list at [Gruntfile.js:32](Gruntfile.js#L32), change `"exec:calcDownloadHash"` → `"calcDownloadHash"`. Delete the `calcDownloadHash` entry from the `exec` block. - -#### 1b. `repoSize` — [Gruntfile.js:347-353](Gruntfile.js#L347-L353) - -Cosmetic info printed by the default `lint` task. Replace with: - -```js -grunt.registerTask("repoSize", "Reports tracked file count and repo size", function () { - const { execSync } = require("child_process"); - const fs = require("fs"); - const fileCount = execSync("git ls-files", { encoding: "utf8" }).trim().split("\n").length; - let bytes = 0; - const walk = (dir) => { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - if (entry.name === ".git" || entry.name === "node_modules") continue; - const p = path.join(dir, entry.name); - if (entry.isDirectory()) walk(p); - else if (entry.isFile()) bytes += fs.statSync(p).size; - } - }; - walk("."); - const mb = (bytes / (1024 * 1024)).toFixed(1); - grunt.log.writeln(`\n${fileCount}\ttracked files`); - grunt.log.writeln(`${mb}M\trepository size`); -}); -``` - -Update [Gruntfile.js:57](Gruntfile.js#L57) (`default` task) to call `"repoSize"` instead of `"exec:repoSize"`. Delete the `repoSize` entry from the `exec` block. - -#### 1c. `nodeConsumerTestPath` — [Gruntfile.js:100](Gruntfile.js#L100) - -Replace `"~/tmp-cyberchef"` with `path.join(os.tmpdir(), "tmp-cyberchef")` (add `const os = require("os");` at the top alongside the existing `path` require). `~` is not expanded on Windows, and `os.tmpdir()` is the right answer on every OS. - -#### 1d. `setupNodeConsumers` / `teardownNodeConsumers` / `testCJSNodeConsumer` / `testESMNodeConsumer` — [Gruntfile.js:382-412](Gruntfile.js#L382-L412) - -These use `mkdir`, `cp`, `cd`, `rm -rf`. Replace with custom Grunt tasks using Node APIs and grunt-exec's `cwd` option: - -```js -grunt.registerTask("setupNodeConsumers", "Sets up a temp dir for testing CJS/ESM consumers", function () { - const fs = require("fs"); - grunt.log.writeln("\n--- Testing node consumers ---"); - fs.mkdirSync(nodeConsumerTestPath, { recursive: true }); - fs.cpSync("tests/node/consumers", nodeConsumerTestPath, { recursive: true }); - require("child_process").execSync("npm link", { stdio: "inherit" }); - require("child_process").execSync("npm link cyberchef", { stdio: "inherit", cwd: nodeConsumerTestPath }); -}); - -grunt.registerTask("teardownNodeConsumers", "Removes the consumer test temp dir", function () { - require("fs").rmSync(nodeConsumerTestPath, { recursive: true, force: true }); - grunt.log.writeln("\n--- Node consumer tests complete ---"); -}); -``` - -For the CJS/ESM consumer test exec entries at [Gruntfile.js:399-412](Gruntfile.js#L399-L412), drop the `cd …` prefix and pass `cwd: nodeConsumerTestPath` on the exec config object instead: - -```js -testCJSNodeConsumer: { - command: `node ${nodeFlags} cjs-consumer.js`, - cwd: nodeConsumerTestPath, - stdout: false, -}, -testESMNodeConsumer: { - command: `node ${nodeFlags} esm-consumer.mjs`, - cwd: nodeConsumerTestPath, - stdout: false, -}, -``` - -Update the `testnodeconsumer` registered task at [Gruntfile.js:51-53](Gruntfile.js#L51-L53) to reference the new task names (drop the `exec:` prefix for setup/teardown). - -#### 1e. `generateConfig` / `generateNodeIndex` — [Gruntfile.js:361-378](Gruntfile.js#L361-L378) - -These chain `echo` (with single-quoted args containing `\n`) and `echo [] > …` redirection — single quotes aren't quotes on Windows cmd.exe, and the `\n` handling in `chainCommands` is a band-aid. Replace each with a custom Grunt task that uses `grunt.log.writeln` for the banners and `fs.writeFileSync` for the empty JSON file, then `execSync`s the `node …` calls: - -```js -grunt.registerTask("generateConfig", "Regenerates operation config files", function () { - const { execSync } = require("child_process"); - const fs = require("fs"); - grunt.log.writeln("\n--- Regenerating config files. ---"); - fs.writeFileSync(path.join("src", "core", "config", "OperationConfig.json"), "[]\n"); - execSync(`node ${nodeFlags} src/core/config/scripts/generateOpsIndex.mjs`, { stdio: "inherit" }); - execSync(`node ${nodeFlags} src/core/config/scripts/generateConfig.mjs`, { stdio: "inherit" }); - grunt.log.writeln("--- Config scripts finished. ---\n"); -}); - -grunt.registerTask("generateNodeIndex", "Regenerates the node index", function () { - const { execSync } = require("child_process"); - grunt.log.writeln("\n--- Regenerating node index ---"); - execSync(`node ${nodeFlags} src/node/config/scripts/generateNodeIndex.mjs`, { stdio: "inherit" }); - grunt.log.writeln("--- Node index generated. ---\n"); -}); -``` - -Update the four task lists at [Gruntfile.js:26](Gruntfile.js#L26), [:31](Gruntfile.js#L31), [:38](Gruntfile.js#L38), [:44](Gruntfile.js#L44) to call `"generateConfig"` / `"generateNodeIndex"` instead of `"exec:generateConfig"` / `"exec:generateNodeIndex"`. Update the `watch:config` block at [:321](Gruntfile.js#L321) likewise. Delete the two exec entries. - -#### 1f. `chainCommands()` helper — [Gruntfile.js:172-183](Gruntfile.js#L172-L183) - -After 1a–1e it has zero call sites. Delete it. - -#### 1g. Webpack output path — [Gruntfile.js:113](Gruntfile.js#L113) - -Change `__dirname + "/build/prod"` to `path.join(__dirname, "build", "prod")`. Cosmetic (webpack handles `/` on Windows), but matches the rest of the file once we're touching it. - -### 2. [package.json](package.json) — fix the three broken scripts - -#### 2a. `setheapsize` — [package.json:211](package.json#L211) - -`export NODE_OPTIONS=…` in an npm script sets the variable in a subshell that immediately exits — it does nothing useful on **any** platform. It's also broken on Windows (`export` isn't a cmd.exe builtin). **Recommendation: delete it.** Users who need a larger heap can set `NODE_OPTIONS` in their own shell. - -#### 2b. `getheapsize` — [package.json:210](package.json#L210) - -Uses `node -e '…single-quoted JS…'`. Windows cmd.exe doesn't strip single quotes, so the JS won't parse. Switch to a `.mjs` file: - -- Create [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs): - ```js - import v8 from "v8"; - console.log(`node heap limit = ${v8.getHeapStatistics().heap_size_limit / (1024 * 1024)} Mb`); - ``` -- Update the script to `"node --no-warnings src/core/config/scripts/getHeapSize.mjs"`. - -#### 2c. `minor` and `tag` — [package.json:208-209](package.json#L208-L209) - -Both use `$(npm pkg get version | xargs)` which (a) is a subshell (broken on Windows cmd.exe) and (b) relies on `xargs` to strip the quotes npm wraps the version in. - -Cleaner: a tiny helper script that prints the bare version. Create [src/core/config/scripts/printVersion.mjs](src/core/config/scripts/printVersion.mjs): - -```js -import { readFileSync } from "fs"; -import { fileURLToPath } from "url"; -import { dirname, join } from "path"; -const pkg = JSON.parse(readFileSync(join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", "package.json"), "utf8")); -process.stdout.write(pkg.version); -``` - -Then in `package.json`, replace both scripts with versions that capture stdout via `node -p` invocations. **Note:** npm scripts on Windows still can't do shell-style command substitution. The portable rewrite uses `node` as the substitution engine. Both `minor` and `tag` involve interactive `git tag -s` / version bumping, so we can also extract the multi-step flow into a single `.mjs` orchestrator — recommended approach: - -- Create [src/core/config/scripts/tagRelease.mjs](src/core/config/scripts/tagRelease.mjs) that reads `package.json`, runs `git tag -s "v${version}" -m "${version}"` via `execSync`, and logs the message. The `minor` script likewise calls `newMinorVersion.mjs`, runs `npm version minor --git-tag-version=false`, then re-reads the version and logs the PR/tag prompt. -- Update `package.json` to `"minor": "node src/core/config/scripts/bumpMinor.mjs"` and `"tag": "node src/core/config/scripts/tagRelease.mjs"`. - -This eliminates all shell-substitution syntax from `package.json`. - ---- - -## Files to be modified - -- [Gruntfile.js](Gruntfile.js) — bulk of the work (sections 1a–1g) -- [package.json](package.json) — script changes (sections 2a–2c) -- New: [src/core/config/scripts/getHeapSize.mjs](src/core/config/scripts/getHeapSize.mjs) -- New: [src/core/config/scripts/bumpMinor.mjs](src/core/config/scripts/bumpMinor.mjs) and [src/core/config/scripts/tagRelease.mjs](src/core/config/scripts/tagRelease.mjs) (or one combined helper) - -## Reused utilities - -- Node stdlib `crypto.createHash`, `fs.cpSync`, `fs.rmSync`, `fs.readdirSync({ withFileTypes: true })`, `os.tmpdir()`, `path.join` — no new devDependencies -- Existing `grunt-exec` `cwd` option (already part of grunt-exec, no new dep) for the consumer-test exec tasks -- Existing `grunt.log.writeln` instead of `echo` -- Existing pattern from [src/core/config/scripts/newMinorVersion.mjs:50](src/core/config/scripts/newMinorVersion.mjs#L50) (uses `execSync` for git) — same pattern for new helpers - -## Verification - -Run on macOS first (current dev machine; quickest feedback loop): - -1. `rm -rf build/ node_modules/ && npm install` — confirms `postinstall` still passes -2. `npm test` — full test suite must pass (covers `configTests` → `exec:generateConfig` + `exec:generateNodeIndex`, renamed) -3. `npm run build` — exercises the prod task chain including the rewritten `calcDownloadHash`. Verify [build/prod/sha256digest.txt](build/prod/sha256digest.txt) exists and matches `shasum -a 256 build/prod/CyberChef_v*.zip`, and that [build/prod/index.html](build/prod/index.html) contains the hash (no `DOWNLOAD_HASH_PLACEHOLDER` left). -4. `npx grunt` (default lint task) — should print the `repoSize` output with the same shape as before -5. `npm run testnodeconsumer` — verify a tmp dir is created under `os.tmpdir()`, both consumer scripts succeed, and the tmp dir is cleaned up -6. `npm run getheapsize` — should print `node heap limit = … Mb` -7. `npm start` — confirms dev server still launches (no changes to it, but verifies nothing collateral broke) - -Then if a Windows machine is available, run the same 7 steps in PowerShell. If not available, the change is structurally safe because every shell-out has been eliminated; the residual risk is in the small set of `execSync("npm link …")` and `execSync("git ls-files")` calls, both of which are documented cross-platform. diff --git a/plan-solve-windows-build-errors.md b/plan-solve-windows-build-errors.md deleted file mode 100644 index d7af2b5f..00000000 --- a/plan-solve-windows-build-errors.md +++ /dev/null @@ -1,72 +0,0 @@ -# Plan: Fix Windows `npm run build` webpack errors - -## Context - -`npm run build` fails on Windows with 7 webpack errors (exit code 1). - -The two visible error classes are: - -1. **jimp babel-loader conflict**: `Module parse failed: Identifier 'e' has already been declared` at `node_modules/jimp/dist/browser/index.js:31135`. Babel is transforming jimp's pre-minified browser bundle, which produces invalid JS. On macOS/Linux, jimp is correctly excluded from babel-loader; on Windows, the exclude regex silently fails. - -2. **Asset Modules Plugin — invalid generator**: `generator has an unknown property 'filename'` on `asset/inline` type. First-party images (e.g. `src/web/static/images/file-128x128.png`) should match the `asset/resource` rule with a `filename` generator, but also match the `asset/inline` rule whose `exclude: /web\/static/` doesn't fire on Windows paths. Two rules matching causes webpack to reject the generator config. - -## Root cause - -All three bugs share one root cause: **regex path separators in [webpack.config.js](webpack.config.js) use literal `/` which doesn't match `\` on Windows**. - -Webpack 5 matches `test` / `exclude` / `include` patterns against the module's **absolute file path**, which uses the OS-native separator. On Windows, paths contain `\`, so `/node_modules\//` doesn't match `C:\…\node_modules\jimp\…`. - -Proof (from the verification terminal): - -``` -> node -e "const re = /node_modules\/(?!crypto-api|bootstrap)/; console.log(re.test('node_modules/jimp'), re.test('node_modules\\jimp'))" -true false - -> node -e "const re = /web\/static/; console.log(re.test('src/web/static/x'), re.test('src\\web\\static\\x'))" -true false -``` - -## Changes - -### Single file: [webpack.config.js](webpack.config.js) - -Three regexes need `\/` → `[/\\]` (matches both forward and back slashes): - -#### A. Line 145 — babel-loader exclude - -```diff -- exclude: /node_modules\/(?!crypto-api|bootstrap)/, -+ exclude: /node_modules[/\\](?!crypto-api|bootstrap)/, -``` - -**Effect**: On Windows, jimp (and all other non-allowlisted node_modules) are correctly excluded from babel-loader again. Fixes error 1. - -#### B. Line 208 — bmfonts test - -```diff -- test: /(\.fnt$|bmfonts\/.+\.png$)/, -+ test: /(\.fnt$|bmfonts[/\\].+\.png$)/, -``` - -**Effect**: Font bitmap PNGs inside `bmfonts/` are matched on Windows too, so they get `asset/resource` with the `assets/fonts/[name][ext]` filename generator instead of falling through to the generic PNG rules. Not directly surfacing as an error today (the bmfonts files may not be in the current source tree), but would cause wrong asset handling on Windows if they were. - -#### C. Line 224 — third-party image exclude - -```diff -- exclude: /web\/static/, -+ exclude: /web[/\\]static/, -``` - -**Effect**: First-party images in `src/web/static/` are correctly excluded from the `asset/inline` rule on Windows, so they only match the `asset/resource` rule above (which has the `filename` generator). Fixes error 2. - -## Files modified - -- [webpack.config.js](webpack.config.js) — 3 one-character regex changes (insert `[/\\]` in place of `\/`) - -No new files, no new dependencies. - -## Verification - -1. `npm run build` on Windows — should now complete without webpack errors; `build/prod/` should contain the full output including `sha256digest.txt` with the correct hash stamped into `index.html`. -2. `npm run build` on macOS — should still work identically (forward slashes match `[/\\]`). -3. `npm start` on Windows — dev server should compile without errors (currently the dev server compiles with warnings but serves; after the fix it should be clean). From 65838a39274e27b4a431dae3eb9fe277fa2fc857 Mon Sep 17 00:00:00 2001 From: Leon Zandman Date: Tue, 19 May 2026 15:32:23 +0200 Subject: [PATCH 9/9] Cleanup --- AGENTS.md | 54 ------------------------------------------------------ 1 file changed, 54 deletions(-) delete mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index a5bc34b3..00000000 --- a/AGENTS.md +++ /dev/null @@ -1,54 +0,0 @@ -# AGENTS.md - -Working agreement for AI coding agents in this repo. Task-specific plans (e.g. [plan-buildsystem-agnostic.md](plan-buildsystem-agnostic.md)) own step-by-step changes; this file owns durable conventions that should outlast any single plan. - -When a `plan-*.md` file exists in the repo root, treat it as the source of truth for the in-flight task — read it before proposing changes to the same area. - -## Project - -CyberChef — client-side web app for encoding, encryption, compression, and data analysis. The same operation source ([src/core/operations/](src/core/operations/)) ships as both a browser bundle and a Node consumer package. Build orchestration is Grunt ([Gruntfile.js](Gruntfile.js)); bundling is webpack ([webpack.config.js](webpack.config.js)). - -## Toolchain - -- Node **24** (`engines: ">=24 <25"` in [package.json](package.json)). Don't reach for features outside that range or assume newer. -- Package manager: npm. -- **No new devDependencies** without explicit approval. Prefer the Node stdlib (`crypto`, `fs`, `os`, `path`, `child_process`, `v8`). - -## Common commands - -| Task | Command | -| --- | --- | -| Install (runs `postinstall` Grunt fixups) | `npm install` | -| Dev server | `npm start` | -| Production build | `npm run build` | -| Full test suite | `npm test` | -| Node-consumer tests | `npm run testnodeconsumer` | -| UI tests (Nightwatch) | `npm run testui` | -| Lint | `npm run lint` | - -## Build-system conventions - -- The build must work on **macOS, Linux, and Windows**. Don't introduce Unix-only shell-outs in [Gruntfile.js](Gruntfile.js) or [package.json](package.json) `scripts` — that means no `sed`, `awk`, `xargs`, `wc`, `du`, `egrep`, `sha256sum`/`shasum`, `$(…)` command substitution, single-quoted JS passed to `node -e`, `~` path expansion, or `export FOO=…` in an npm script. Reach for the Node stdlib first; if a shell command is genuinely needed, confirm it works in both `cmd.exe` and POSIX shells. -- Prefer custom Grunt tasks (`grunt.registerTask`) over `grunt-exec` entries when the work is Node code — they're easier to read, easier to test, and platform-neutral. -- One-off helper scripts live in [src/core/config/scripts/](src/core/config/scripts/) as `.mjs` files. Follow the existing pattern in [newMinorVersion.mjs](src/core/config/scripts/newMinorVersion.mjs) (ESM, top-level `execSync` for git ops, no external deps beyond stdlib). - -## Coding conventions - -From [CONTRIBUTING.md](CONTRIBUTING.md): - -- 4-space indentation, LF line endings, UTF-8 without BOM, trailing newline on every file. -- `CamelCase` for namespaces/objects, `camelCase` for functions/variables, `UNDERSCORE_UPPER_CASE` for constants. -- Vanilla JS preferred. Don't add UI frameworks; jQuery is already vendored but should be avoided for new code. -- Operations must be client-side wherever possible (design principle: works on closed networks / offline). - -## Repo layout (orientation) - -- [src/core/](src/core/) — operations, config, generation scripts (browser-safe; no `fs`/`child_process` here) -- [src/web/](src/web/) — browser UI -- [src/node/](src/node/) — Node consumer entry points -- [tests/](tests/) — `operations/`, `node/`, and UI (`browser/`) test suites - -## Workflow - -- **Leon authors all git commits and pushes himself.** Stop at "ready to commit" — do not run `git commit`, `git push`, `gh pr create`, or any branch-moving command without an explicit ask. Suggesting a commit message in chat is fine; executing it is not. -- For multi-step changes, write or update a `plan-*.md` in the repo root before editing. Keep step-by-step task detail in the plan, not in this file.