Merge pull request #2 from Andy7475/feature/speech-to-text
speech to text all working
This commit is contained in:
Andy7475
GitHub
commit
0ad1a86a98
211
docs/GCS_SpeechToText_ImplementationPlan.md
Normal file
211
docs/GCS_SpeechToText_ImplementationPlan.md
Normal file
@ -0,0 +1,211 @@
|
||||
# Implementation Plan: GCS Orchestration & Speech-to-Text
|
||||
|
||||
## Background & Goal
|
||||
|
||||
We want CyberChef to act as a **cloud-native orchestrator** for processing large media files in GCS without ever streaming raw media bytes through the browser.
|
||||
|
||||
**Concrete end-to-end workflow:**
|
||||
|
||||
```
|
||||
Input: "cyber-chef-cloud-examples"
|
||||
|
||||
Recipe:
|
||||
1. GCloud List Bucket [prefix: audio/]
|
||||
→ gs://cyber-chef-cloud-examples/audio/hello_kitty.mp3
|
||||
gs://cyber-chef-cloud-examples/audio/track_02.mp3
|
||||
gs://cyber-chef-cloud-examples/audio/track_03.mp3
|
||||
gs://cyber-chef-cloud-examples/audio/track_04.mp3
|
||||
|
||||
2. Fork [\n]
|
||||
|
||||
3. GCloud Speech to Text [Output: Return to CyberChef |OR| Write to GCS]
|
||||
→ Browser mode: "The package will be delivered at 1400..."
|
||||
→ GCS mode: gs://cyber-chef-cloud-examples/output/audio/hello_kitty.mp3/speech-to-text/text.txt
|
||||
|
||||
4. Merge
|
||||
```
|
||||
|
||||
**Key principle:** raw audio never touches the browser — GCS URI is passed in, transcript text (or a GCS output URI) comes back.
|
||||
|
||||
---
|
||||
|
||||
## Design Decisions
|
||||
|
||||
- **LRO polling: internal** — the `GCloud Speech to Text` operation polls internally (every 10 seconds, up to 30 minutes max), updating the output box with progress. No separate "Poll Operation" step needed.
|
||||
- **Output path convention:** `output/{media_type}/{source_filename}/{service}/text.txt`
|
||||
e.g. `gs://cyber-chef-cloud-examples/output/audio/hello_kitty.mp3/speech-to-text/text.txt`
|
||||
|
||||
---
|
||||
|
||||
## 1. Shared Library Changes
|
||||
|
||||
**File:** `src/core/lib/GoogleCloud.mjs`
|
||||
|
||||
### New helper: `listGCSBucket`
|
||||
|
||||
```
|
||||
listGCSBucket(bucket, prefix, authType, authStringObj, quotaProject)
|
||||
|
||||
Calls: GET https://storage.googleapis.com/storage/v1/b/{bucket}/o?prefix={prefix}
|
||||
Returns: array of { name, gs_uri, size, contentType }
|
||||
```
|
||||
|
||||
### New helper: `readGCSFile`
|
||||
|
||||
```
|
||||
readGCSFile(gcsUri, authType, authStringObj, quotaProject)
|
||||
|
||||
Parses gs://bucket/object
|
||||
Calls: GET https://storage.googleapis.com/storage/v1/b/{bucket}/o/{encodedObject}?alt=media
|
||||
Returns: ArrayBuffer (raw bytes)
|
||||
```
|
||||
|
||||
### New helper: `pollLongRunningOperation`
|
||||
|
||||
```
|
||||
pollLongRunningOperation(operationName, authString, quotaProject, maxMs, intervalMs, onProgress)
|
||||
|
||||
Calls: GET https://speech.googleapis.com/v1/operations/{operationName} every intervalMs
|
||||
Calls onProgress(elapsedSeconds) on each tick (for progress output to CyberChef output box)
|
||||
Resolves when response.done === true
|
||||
Rejects on timeout or API error
|
||||
Defaults: poll every 10s, timeout after 30 mins
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. New Operation: `GCloud List Bucket`
|
||||
|
||||
**File:** `src/core/operations/GCloudListBucket.mjs`
|
||||
|
||||
| Property | Value |
|
||||
| :--- | :--- |
|
||||
| Name | `GCloud List Bucket` |
|
||||
| Module | `Cloud` |
|
||||
| Input Type | `string` (bucket name or `gs://` prefix) |
|
||||
| Output Type | `string` |
|
||||
| `manualBake` | `true` |
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| # | Name | Type | Default |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| 0 | Folder Prefix | `string` | `audio/` |
|
||||
| 1 | Output Format | `option` | `GCS URIs (one per line)` / `Filenames only` / `JSON` |
|
||||
| 2–4 | *(GCP_AUTH_ARGS)* | — | — |
|
||||
|
||||
**Behaviour:**
|
||||
- Strips `gs://` prefix from input to normalise bucket name
|
||||
- Default output = newline-separated `gs://` URIs → directly pipe into `Fork`
|
||||
|
||||
---
|
||||
|
||||
## 3. New Operation: `GCloud Read File`
|
||||
|
||||
**File:** `src/core/operations/GCloudReadFile.mjs`
|
||||
|
||||
| Property | Value |
|
||||
| :--- | :--- |
|
||||
| Name | `GCloud Read File` |
|
||||
| Module | `Cloud` |
|
||||
| Input Type | `string` (`gs://` URI) |
|
||||
| Output Type | `ArrayBuffer` |
|
||||
| `manualBake` | `true` |
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| # | Name | Type |
|
||||
| :--- | :--- | :--- |
|
||||
| 0–2 | *(GCP_AUTH_ARGS)* | — |
|
||||
|
||||
> **Note:** Intended for small files (text, small images). For large audio/video, use the GCS URI mode in Speech-to-Text directly — don't stream large binaries through the browser.
|
||||
|
||||
---
|
||||
|
||||
## 4. New Operation: `GCloud Speech to Text`
|
||||
|
||||
**File:** `src/core/operations/GCloudSpeechToText.mjs`
|
||||
|
||||
| Property | Value |
|
||||
| :--- | :--- |
|
||||
| Name | `GCloud Speech to Text` |
|
||||
| Module | `Cloud` |
|
||||
| Input Type | `string` |
|
||||
| Output Type | `string` |
|
||||
| `manualBake` | `true` |
|
||||
|
||||
**Arguments:**
|
||||
|
||||
| # | Name | Type | Default |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| 0 | Input Mode | `option` | `GCS URI (gs://...)` / `Raw Audio Bytes (Base64)` |
|
||||
| 1 | Language Code | `string` | `en-US` |
|
||||
| 2 | Output Destination | `option` | `Return to CyberChef` / `Write to GCS` |
|
||||
| 3 | Output GCS Bucket | `string` | `cyber-chef-cloud-examples` |
|
||||
| 4 | Max Poll Minutes | `number` | `30` |
|
||||
| 5–7 | *(GCP_AUTH_ARGS)* | — | — |
|
||||
|
||||
**Runtime Logic:**
|
||||
|
||||
```
|
||||
IF Input Mode == "GCS URI":
|
||||
Call longrunningrecognize with { audio: { uri: input }, config: { languageCode } }
|
||||
→ Get operationName
|
||||
transcript = await pollLongRunningOperation(operationName, ...)
|
||||
(progress updates written to output box during polling)
|
||||
|
||||
ELSE (Raw Audio Bytes / Base64):
|
||||
Call recognize with { audio: { content: base64Input }, config: { languageCode } }
|
||||
transcript = joined results
|
||||
|
||||
IF Output Destination == "Write to GCS":
|
||||
sourceFilename = last path segment of input gs:// URI (e.g. "hello_kitty.mp3")
|
||||
destPath = "output/audio/{sourceFilename}/speech-to-text/text.txt"
|
||||
PUT transcript → gs://{outputBucket}/{destPath}
|
||||
return "gs://{outputBucket}/{destPath}" ← this is the CyberChef output
|
||||
|
||||
ELSE:
|
||||
return transcript text
|
||||
```
|
||||
|
||||
**Output path example:**
|
||||
```
|
||||
Input: gs://cyber-chef-cloud-examples/audio/hello_kitty.mp3
|
||||
Output: gs://cyber-chef-cloud-examples/output/audio/hello_kitty.mp3/speech-to-text/text.txt
|
||||
```
|
||||
|
||||
After Fork + Merge across 4 files, the CyberChef output will be 4 GCS URIs the analyst can save, come back to later, and use as the input to a new recipe.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tests
|
||||
|
||||
**File:** `tests/browser/03_cloud_ops.js`
|
||||
|
||||
Following the existing pattern (skippable via missing token):
|
||||
|
||||
| Test | Type |
|
||||
| :--- | :--- |
|
||||
| `GCloud List Bucket: Missing Key Validation` | No API call |
|
||||
| `GCloud List Bucket: Lists audio/ files from cyber-chef-cloud-examples` | Live, skippable |
|
||||
| `GCloud Speech-to-Text: GCS URI returns transcription in browser` | Live + LRO, skippable |
|
||||
| `GCloud Speech-to-Text: GCS URI writes to output/ bucket` | Live + LRO + GCS write, skippable |
|
||||
|
||||
---
|
||||
|
||||
## 6. Verification Plan
|
||||
|
||||
### Automated
|
||||
```bash
|
||||
# From /home/projects/CyberChefCloud
|
||||
CYBERCHEF_GCP_TEST_TOKEN=$(gcloud auth print-access-token) npm run test:browser
|
||||
```
|
||||
|
||||
### Manual
|
||||
1. Open CyberChef at `http://localhost:8080`
|
||||
2. Add **GCloud List Bucket**, input `cyber-chef-cloud-examples`, prefix `audio/`
|
||||
3. Bake → confirm 4 `gs://` URIs in output
|
||||
4. Add **Fork** `[\n]` + **GCloud Speech to Text** (GCS URI, Return to CyberChef) + **Merge**
|
||||
5. Bake → confirm 4 transcripts separated by `---`
|
||||
6. Repeat with **Write to GCS** mode
|
||||
7. Verify in terminal: `gsutil cat gs://cyber-chef-cloud-examples/output/audio/hello_kitty.mp3/speech-to-text/text.txt`
|
||||
43
docs/GCS_SpeechToText_TaskList.md
Normal file
43
docs/GCS_SpeechToText_TaskList.md
Normal file
@ -0,0 +1,43 @@
|
||||
# Task List: GCS + Speech-to-Text CyberChef Operations
|
||||
|
||||
## Phase 1: Planning & Documentation
|
||||
- [x] Review existing `GoogleTranslate.mjs`, `GoogleCloud.mjs` lib, and `03_cloud_ops.js` patterns
|
||||
- [x] Write implementation plan (`GCS_SpeechToText_ImplementationPlan.md`)
|
||||
- [x] Write GCP configuration guide (`GCloudGCSSetup.md`)
|
||||
- [x] Get user approval on plan — **approved, internal LRO polling confirmed**
|
||||
|
||||
## Phase 2: Shared Library Enhancements (`src/core/lib/GoogleCloud.mjs`)
|
||||
- [ ] Add `listGCSBucket(bucket, prefix, authType, authStringObj, quotaProject)` helper
|
||||
- [ ] Add `readGCSFile(gcsUri, authType, authStringObj, quotaProject)` helper
|
||||
- [ ] Add `pollLongRunningOperation(operationName, authString, quotaProject, maxMs, intervalMs, onProgress)` polling helper
|
||||
|
||||
## Phase 3: New Operations
|
||||
|
||||
### 3.1 `GCloud List Bucket` (`src/core/operations/GCloudListBucket.mjs`)
|
||||
- [ ] Create operation file scaffolding
|
||||
- [ ] Args: Bucket Name, Prefix/Folder filter, Auth args, Output Format
|
||||
- [ ] Calls GCS JSON API: `storage/v1/b/{bucket}/o?prefix=`
|
||||
- [ ] Default output: newline-separated `gs://` URIs (pipe directly into `Fork`)
|
||||
|
||||
### 3.2 `GCloud Read File` (`src/core/operations/GCloudReadFile.mjs`)
|
||||
- [ ] Create operation file scaffolding
|
||||
- [ ] Args: Auth args only (input = `gs://` URI)
|
||||
- [ ] Calls GCS media download endpoint
|
||||
- [ ] Output: raw file bytes (`ArrayBuffer`)
|
||||
|
||||
### 3.3 `GCloud Speech to Text` (`src/core/operations/GCloudSpeechToText.mjs`)
|
||||
- [ ] Create operation file scaffolding
|
||||
- [ ] Args: Input Mode, Language Code, Output Destination, Output GCS Bucket, Max Poll Minutes, Auth args
|
||||
- [ ] GCS URI mode → call `longrunningrecognize` → internal LRO polling loop
|
||||
- [ ] Raw bytes mode → call synchronous `recognize` endpoint
|
||||
- [ ] Output mode: `Return to CyberChef` (transcript text) OR `Write to GCS` (returns the written `gs://` URI)
|
||||
- [ ] GCS output path convention: `output/audio/{filename}/speech-to-text/text.txt`
|
||||
|
||||
## Phase 4: Tests (`tests/browser/03_cloud_ops.js`)
|
||||
- [ ] `GCloud List Bucket: Missing Key Validation` (no API call)
|
||||
- [ ] `GCloud List Bucket: Lists audio/ files from cyber-chef-cloud-examples` (live, skippable)
|
||||
- [ ] `GCloud Speech-to-Text: GCS URI mode returns transcription` (live, LRO, skippable)
|
||||
- [ ] `GCloud Speech-to-Text: GCS URI mode writes to output/ bucket` (live, LRO, skippable)
|
||||
|
||||
## Phase 5: Documentation
|
||||
- [ ] Update `GCloudGCSSetup.md` with any gotchas discovered during testing
|
||||
232
docs/GCloudGCSSetup.md
Normal file
232
docs/GCloudGCSSetup.md
Normal file
@ -0,0 +1,232 @@
|
||||
# GCP Configuration Guide: GCS + Speech-to-Text
|
||||
|
||||
This guide ensures your Google Cloud project is correctly configured to support:
|
||||
- **CyberChef `GCloud List Bucket`** — listing objects in a GCS bucket from the browser
|
||||
- **CyberChef `GCloud Read File`** — downloading small files from GCS into the browser
|
||||
- **CyberChef `GCloud Speech-to-Text`** — transcribing audio files stored in GCS, both returning results to the browser and writing outputs back to GCS
|
||||
|
||||
**Your bucket:** `cyber-chef-cloud-examples`
|
||||
|
||||
---
|
||||
|
||||
## 1. APIs to Enable
|
||||
|
||||
In the [Google Cloud Console](https://console.cloud.google.com/apis/library) (or via `gcloud`), ensure the following APIs are enabled for your project:
|
||||
|
||||
| API | Purpose | Enable via Console link |
|
||||
| :--- | :--- | :--- |
|
||||
| **Cloud Storage JSON API** | Listing & reading bucket objects | [Enable](https://console.cloud.google.com/apis/library/storage-component.googleapis.com) |
|
||||
| **Cloud Speech-to-Text API** | Audio transcription | [Enable](https://console.cloud.google.com/apis/library/speech.googleapis.com) |
|
||||
|
||||
Via `gcloud`:
|
||||
```bash
|
||||
gcloud services enable storage-component.googleapis.com
|
||||
gcloud services enable speech.googleapis.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. IAM Roles for Your User Identity
|
||||
|
||||
CyberChef sends API requests using **your OAuth token** (generated by `gcloud auth print-access-token`). Your user identity needs the following roles:
|
||||
|
||||
### On the GCS Bucket (`cyber-chef-cloud-examples`)
|
||||
|
||||
| Role | Why Needed |
|
||||
| :--- | :--- |
|
||||
| `roles/storage.objectViewer` | To list objects and read file metadata (List Bucket operation) |
|
||||
| `roles/storage.objectCreator` | To write transcription output files back to the `output/` prefix |
|
||||
|
||||
Grant via console: **Cloud Storage → cyber-chef-cloud-examples → Permissions → Grant Access**
|
||||
|
||||
Or via `gcloud`:
|
||||
```bash
|
||||
# Replace YOUR_EMAIL with your Google account email
|
||||
gcloud storage buckets add-iam-policy-binding gs://cyber-chef-cloud-examples \
|
||||
--member="user:YOUR_EMAIL@gmail.com" \
|
||||
--role="roles/storage.objectViewer"
|
||||
|
||||
gcloud storage buckets add-iam-policy-binding gs://cyber-chef-cloud-examples \
|
||||
--member="user:YOUR_EMAIL@gmail.com" \
|
||||
--role="roles/storage.objectCreator"
|
||||
```
|
||||
|
||||
### On the Speech-to-Text API
|
||||
|
||||
Your user identity needs permission to call the Speech-to-Text API at the **project level**:
|
||||
|
||||
| Role | Why Needed |
|
||||
| :--- | :--- |
|
||||
| `roles/speech.editor` (or `roles/speech.client`) | To call `longrunningrecognize` and poll operation status |
|
||||
|
||||
```bash
|
||||
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
|
||||
--member="user:YOUR_EMAIL@gmail.com" \
|
||||
--role="roles/speech.client"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Allow the Speech-to-Text Service Account to Read Your Bucket
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This is the most commonly missed step. When you call `longrunningrecognize` with a `gcsUri`, the **Speech-to-Text API reads the file using its own internal service account**, not yours. You must explicitly grant this service account access to your bucket.
|
||||
|
||||
### Find your Speech-to-Text service account
|
||||
|
||||
The service account follows the pattern:
|
||||
```
|
||||
service-{PROJECT_NUMBER}@gcp-sa-speech.iam.gserviceaccount.com
|
||||
```
|
||||
|
||||
Get your project number:
|
||||
```bash
|
||||
gcloud projects describe YOUR_PROJECT_ID --format="value(projectNumber)"
|
||||
# Example output: 123456789012
|
||||
```
|
||||
|
||||
So your service account would be:
|
||||
```
|
||||
service-123456789012@gcp-sa-speech.iam.gserviceaccount.com
|
||||
```
|
||||
|
||||
### Grant it access to the bucket
|
||||
|
||||
```bash
|
||||
gcloud storage buckets add-iam-policy-binding gs://cyber-chef-cloud-examples \
|
||||
--member="serviceAccount:service-123456789012@gcp-sa-speech.iam.gserviceaccount.com" \
|
||||
--role="roles/storage.objectViewer"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Configure CORS on the Bucket
|
||||
|
||||
CyberChef runs in the browser and makes direct `fetch()` requests to the GCS JSON API. For the **list** and **metadata** endpoints these are generally permitted, but to be safe and to avoid issues with `OPTIONS` preflight requests, configure CORS on the bucket.
|
||||
|
||||
Create a file `cors.json`:
|
||||
```json
|
||||
[
|
||||
{
|
||||
"origin": [
|
||||
"http://localhost:8080",
|
||||
"https://YOUR_CYBERCHEF_DOMAIN.com"
|
||||
],
|
||||
"method": ["GET", "POST", "PUT", "HEAD"],
|
||||
"responseHeader": ["Content-Type", "Authorization", "x-goog-user-project"],
|
||||
"maxAgeSeconds": 3600
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Apply it:
|
||||
```bash
|
||||
gcloud storage buckets update gs://cyber-chef-cloud-examples \
|
||||
--cors-file=cors.json
|
||||
```
|
||||
|
||||
Verify:
|
||||
```bash
|
||||
gcloud storage buckets describe gs://cyber-chef-cloud-examples --format="json(cors)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Quota Project / ADC Setup
|
||||
|
||||
If you are using an **OAuth Token** (recommended), some APIs require a billing quota project. Ensure your `gcloud` environment is configured:
|
||||
|
||||
```bash
|
||||
gcloud auth application-default set-quota-project YOUR_PROJECT_ID
|
||||
```
|
||||
|
||||
In CyberChef operations, always populate the **Quota Project** field with your Project ID (e.g., `cyberchefcloud`).
|
||||
|
||||
---
|
||||
|
||||
## 6. Verification Checklist
|
||||
|
||||
Run these commands to verify your configuration before using CyberChef:
|
||||
|
||||
### ✅ Can you list your bucket?
|
||||
```bash
|
||||
gcloud storage ls gs://cyber-chef-cloud-examples/audio/
|
||||
# Expected: 4 audio file URIs
|
||||
```
|
||||
|
||||
### ✅ Can you read a file?
|
||||
```bash
|
||||
gsutil cat gs://cyber-chef-cloud-examples/audio/hello_kitty.mp3 | file -
|
||||
# Expected: MPEG audio data (or similar) — confirms read access
|
||||
```
|
||||
|
||||
### ✅ Can the Speech API access the file? (Test with REST)
|
||||
```bash
|
||||
TOKEN=$(gcloud auth print-access-token)
|
||||
PROJECT_ID=YOUR_PROJECT_ID
|
||||
|
||||
curl -s -X POST \
|
||||
"https://speech.googleapis.com/v1/speech:longrunningrecognize" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "x-goog-user-project: $PROJECT_ID" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": { "languageCode": "en-US", "enableAutomaticPunctuation": true },
|
||||
"audio": { "uri": "gs://cyber-chef-cloud-examples/audio/hello_kitty.mp3" }
|
||||
}'
|
||||
# Expected: { "name": "projects/.../operations/12345" }
|
||||
```
|
||||
|
||||
If this returns an operation name, your setup is correct.
|
||||
|
||||
### ✅ Can you poll the operation?
|
||||
```bash
|
||||
# Use the operation name from the previous step
|
||||
OP_NAME="projects/YOUR_PROJECT_NUMBER/operations/12345"
|
||||
|
||||
curl -s \
|
||||
"https://speech.googleapis.com/v1/operations/${OP_NAME}" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "x-goog-user-project: $PROJECT_ID"
|
||||
# Wait a few seconds and re-run. When done: { "done": true, "response": { "results": [...] } }
|
||||
```
|
||||
|
||||
### ✅ Can you write to the output/ prefix?
|
||||
```bash
|
||||
echo "Test transcript" | gsutil cp - gs://cyber-chef-cloud-examples/output/test.txt
|
||||
# Expected: Copying... Operation completed
|
||||
|
||||
# Clean up
|
||||
gsutil rm gs://cyber-chef-cloud-examples/output/test.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Expected Bucket Structure
|
||||
|
||||
```
|
||||
cyber-chef-cloud-examples/
|
||||
├── audio/
|
||||
│ ├── hello_kitty.mp3
|
||||
│ ├── track_02.mp3
|
||||
│ ├── track_03.mp3
|
||||
│ └── track_04.mp3
|
||||
├── images/
|
||||
│ └── (future image files)
|
||||
├── video/
|
||||
│ └── (future video files)
|
||||
└── output/
|
||||
└── audio/
|
||||
└── hello_kitty.mp3/
|
||||
└── speech-to-text/
|
||||
└── text.txt ← written by CyberChef
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Summary of Roles Required
|
||||
|
||||
| Identity | Bucket Role | Project Role |
|
||||
| :--- | :--- | :--- |
|
||||
| Your user OAuth token | `storage.objectViewer` + `storage.objectCreator` | `speech.client` |
|
||||
| Speech-to-Text service account | `storage.objectViewer` | *(handled internally by GCP)* |
|
||||
101
docs/GCloudServiceAccountLessonsLearned.md
Normal file
101
docs/GCloudServiceAccountLessonsLearned.md
Normal file
@ -0,0 +1,101 @@
|
||||
# GCP Lessons Learned: Service Accounts & Storage Permissions
|
||||
|
||||
These lessons emerged from setting up Speech-to-Text access to a GCS bucket, but they apply broadly to **any Google Cloud AI/ML API that reads from Cloud Storage** (Vision, Video Intelligence, Natural Language, etc).
|
||||
|
||||
---
|
||||
|
||||
## Lesson 1: Google-Managed Service Agents Are Not Project Service Accounts
|
||||
|
||||
When a Google Cloud API (e.g. Speech-to-Text, Vision) reads from GCS on your behalf, it does so using a **Google-managed service agent** — a special service account that belongs to Google's infrastructure, not your project.
|
||||
|
||||
| API | Service Agent Email Pattern |
|
||||
| :--- | :--- |
|
||||
| Cloud Speech-to-Text | `service-{PROJECT_NUMBER}@gcp-sa-speech.iam.gserviceaccount.com` |
|
||||
| Cloud Vision | `service-{PROJECT_NUMBER}@gcp-sa-vision.iam.gserviceaccount.com` |
|
||||
| Cloud Video Intelligence | `service-{PROJECT_NUMBER}@gcp-sa-videointelligence.iam.gserviceaccount.com` |
|
||||
| Cloud Natural Language | `service-{PROJECT_NUMBER}@gcp-sa-language.iam.gserviceaccount.com` |
|
||||
|
||||
**These will NOT appear in your project's IAM console** (`IAM & Admin → Service Accounts`), which only lists service accounts you created. Do not waste time looking for them there.
|
||||
|
||||
---
|
||||
|
||||
## Lesson 2: Service Agents Are Provisioned Lazily
|
||||
|
||||
The service agent email does not exist until you **make your first successful API call**. If you try to grant IAM permissions to the service agent email before any API call, the `gcloud` command will fail with:
|
||||
|
||||
```
|
||||
ERROR: HTTPError 400: Service account (...) does not exist.
|
||||
```
|
||||
|
||||
**Solution:** Make a real API call first (even if it fails with a 404 on the GCS object — a permission error is not sufficient). Once the API responds, the service agent is provisioned within a few seconds.
|
||||
|
||||
---
|
||||
|
||||
## Lesson 3: Same-Project Access Is Automatic
|
||||
|
||||
If your GCS bucket and the Cloud API are both in the **same GCP project**, the service agent already has read access to your bucket via the project's legacy IAM bindings (`roles/storage.legacyObjectReader` → `projectViewer`). You do not need to grant anything explicitly.
|
||||
|
||||
**Explicitly granting is only required for cross-project access** (e.g. the API is in Project A but the bucket is in Project B).
|
||||
|
||||
```bash
|
||||
# Check your current bucket IAM — if you see these, same-project APIs can read it:
|
||||
# - roles/storage.legacyBucketReader → projectViewer:YOUR_PROJECT
|
||||
# - roles/storage.legacyObjectReader → projectViewer:YOUR_PROJECT
|
||||
gcloud storage buckets get-iam-policy gs://YOUR_BUCKET
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Lesson 4: Model Selection Matters for Accuracy
|
||||
|
||||
When calling Speech-to-Text (and likely other AI APIs), the default model is not always the best choice. In our testing:
|
||||
|
||||
| Config | Transcript | Confidence |
|
||||
| :--- | :--- | :--- |
|
||||
| Default (no model specified) | `"result"` | 28% |
|
||||
| `model: "latest_long"` | `"She achieves great results."` | 92% |
|
||||
|
||||
**Always specify `model: "latest_long"` for general audio**, and `model: "latest_short"` for short utterances (under ~1 min). Enable `enableAutomaticPunctuation: true` for readable output.
|
||||
|
||||
---
|
||||
|
||||
## Lesson 5: LRO Polling Endpoint Format
|
||||
|
||||
For long-running operations (`longrunningrecognize`, Video Intelligence jobs, etc), the operation name returned is **just a number** (e.g. `4567056075577147015`), not a full resource path.
|
||||
|
||||
The correct polling URL is:
|
||||
```
|
||||
GET https://speech.googleapis.com/v1/operations/{OPERATION_ID}
|
||||
```
|
||||
|
||||
Not the more verbose `projects/.../operations/...` format (that's a different API surface). Pass the same auth headers as the original request.
|
||||
|
||||
---
|
||||
|
||||
## Lesson 6: Verify With `curl` Before Building CyberChef Operations
|
||||
|
||||
Always test the raw API call with `curl` before writing operation code. The two-step pattern (submit → poll) is easy to verify interactively:
|
||||
|
||||
```bash
|
||||
# Step 1: Submit job
|
||||
TOKEN=$(gcloud auth print-access-token)
|
||||
curl -X POST "https://speech.googleapis.com/v1/speech:longrunningrecognize" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "x-goog-user-project: YOUR_PROJECT_ID" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"languageCode": "en-US",
|
||||
"model": "latest_long",
|
||||
"enableAutomaticPunctuation": true
|
||||
},
|
||||
"audio": { "uri": "gs://YOUR_BUCKET/audio/file.mp3" }
|
||||
}'
|
||||
# → { "name": "1234567890" }
|
||||
|
||||
# Step 2: Poll until done
|
||||
curl "https://speech.googleapis.com/v1/operations/1234567890" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "x-goog-user-project: YOUR_PROJECT_ID"
|
||||
# → { "done": true, "response": { "results": [...] } }
|
||||
```
|
||||
@ -6,6 +6,157 @@
|
||||
|
||||
import OperationError from "../errors/OperationError.mjs";
|
||||
|
||||
/**
|
||||
* Lists objects in a GCS bucket under a given prefix.
|
||||
*
|
||||
* @param {string} bucket - The GCS bucket name (without gs://).
|
||||
* @param {string} prefix - The folder prefix to filter by (e.g. "audio/").
|
||||
* @param {string} authType - "API Key" or "OAuth Token".
|
||||
* @param {Object|string} authStringObj - The auth credential.
|
||||
* @param {string} quotaProject - Optional quota project for ADC OAuth tokens.
|
||||
* @returns {Promise<Array>} Array of GCS object metadata { name, gs_uri, size, contentType }.
|
||||
*/
|
||||
export async function listGCSBucket(bucket, prefix, authType, authStringObj, quotaProject) {
|
||||
let url = `https://storage.googleapis.com/storage/v1/b/${encodeURIComponent(bucket)}/o`;
|
||||
const params = new URLSearchParams();
|
||||
if (prefix) params.set("prefix", prefix);
|
||||
params.set("fields", "items(name,size,contentType)");
|
||||
const paramStr = params.toString();
|
||||
if (paramStr) url += `?${paramStr}`;
|
||||
|
||||
const headers = new Headers();
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
|
||||
const response = await fetch(authed.url, { method: "GET", headers: authed.headers, mode: "cors", cache: "no-cache" });
|
||||
let data;
|
||||
try {
|
||||
data = await response.json();
|
||||
} catch (e) {
|
||||
throw new OperationError("GCloud List Bucket: Failed to parse GCS API response.");
|
||||
}
|
||||
if (!response.ok) {
|
||||
const msg = data?.error?.message || response.statusText;
|
||||
throw new OperationError(`GCloud List Bucket: GCS API Error (${response.status}): ${msg}`);
|
||||
}
|
||||
|
||||
const items = data.items || [];
|
||||
return items
|
||||
.filter(item => !item.name.endsWith("/")) // exclude folder placeholder objects
|
||||
.map(item => ({
|
||||
name: item.name,
|
||||
gs_uri: `gs://${bucket}/${item.name}`,
|
||||
size: item.size,
|
||||
contentType: item.contentType
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Downloads a file from GCS and returns its raw bytes.
|
||||
*
|
||||
* @param {string} gcsUri - Full gs:// URI of the file.
|
||||
* @param {string} authType - "API Key" or "OAuth Token".
|
||||
* @param {Object|string} authStringObj - The auth credential.
|
||||
* @param {string} quotaProject - Optional quota project.
|
||||
* @returns {Promise<ArrayBuffer>} Raw file bytes.
|
||||
*/
|
||||
export async function readGCSFile(gcsUri, authType, authStringObj, quotaProject) {
|
||||
const match = gcsUri.match(/^gs:\/\/([^/]+)\/(.+)$/);
|
||||
if (!match) throw new OperationError(`GCloud Read File: Invalid GCS URI: ${gcsUri}`);
|
||||
const [, bucket, object] = match;
|
||||
const encodedObject = encodeURIComponent(object).replace(/%2F/g, "%2F");
|
||||
let url = `https://storage.googleapis.com/storage/v1/b/${encodeURIComponent(bucket)}/o/${encodedObject}?alt=media`;
|
||||
|
||||
const headers = new Headers();
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
|
||||
const response = await fetch(authed.url, { method: "GET", headers: authed.headers, mode: "cors", cache: "no-cache" });
|
||||
if (!response.ok) {
|
||||
let msg = response.statusText;
|
||||
try { const d = await response.json(); msg = d?.error?.message || msg; } catch (e) { /* ignore */ }
|
||||
throw new OperationError(`GCloud Read File: GCS API Error (${response.status}): ${msg}`);
|
||||
}
|
||||
return await response.arrayBuffer();
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes text content to a GCS object.
|
||||
*
|
||||
* @param {string} bucket - Destination bucket name (without gs://).
|
||||
* @param {string} objectPath - Destination object path within the bucket.
|
||||
* @param {string} content - Text content to write.
|
||||
* @param {string} authType - "API Key" or "OAuth Token".
|
||||
* @param {Object|string} authStringObj - The auth credential.
|
||||
* @param {string} quotaProject - Optional quota project.
|
||||
* @returns {Promise<string>} The gs:// URI of the written file.
|
||||
*/
|
||||
export async function writeGCSFile(bucket, objectPath, content, authType, authStringObj, quotaProject) {
|
||||
const encodedObject = encodeURIComponent(objectPath).replace(/%2F/g, "%2F");
|
||||
let url = `https://storage.googleapis.com/upload/storage/v1/b/${encodeURIComponent(bucket)}/o?uploadType=media&name=${encodedObject}`;
|
||||
|
||||
const headers = new Headers();
|
||||
headers.set("Content-Type", "text/plain; charset=utf-8");
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
|
||||
const response = await fetch(authed.url, {
|
||||
method: "POST",
|
||||
headers: authed.headers,
|
||||
body: content,
|
||||
mode: "cors",
|
||||
cache: "no-cache"
|
||||
});
|
||||
if (!response.ok) {
|
||||
let msg = response.statusText;
|
||||
try { const d = await response.json(); msg = d?.error?.message || msg; } catch (e) { /* ignore */ }
|
||||
throw new OperationError(`GCloud Write File: GCS API Error (${response.status}): ${msg}`);
|
||||
}
|
||||
return `gs://${bucket}/${objectPath}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Polls a Google Cloud long-running operation until it completes.
|
||||
*
|
||||
* @param {string} operationName - The operation ID (numeric string from the API response).
|
||||
* @param {string} pollUrl - The base polling URL (e.g. https://speech.googleapis.com/v1/operations/).
|
||||
* @param {string} authType - "API Key" or "OAuth Token".
|
||||
* @param {Object|string} authStringObj - The auth credential.
|
||||
* @param {string} quotaProject - Optional quota project.
|
||||
* @param {number} maxMs - Maximum wait time in milliseconds (default 30 minutes).
|
||||
* @param {number} intervalMs - Poll interval in milliseconds (default 10 seconds).
|
||||
* @param {Function} onProgress - Optional callback(elapsedSeconds) called on each poll tick.
|
||||
* @returns {Promise<Object>} The completed operation response object.
|
||||
*/
|
||||
export async function pollLongRunningOperation(operationName, pollUrl, authType, authStringObj, quotaProject, maxMs = 30 * 60 * 1000, intervalMs = 10000, onProgress = null) {
|
||||
const startTime = Date.now();
|
||||
const url = `${pollUrl}${operationName}`;
|
||||
|
||||
while (true) {
|
||||
const elapsed = Date.now() - startTime;
|
||||
if (elapsed > maxMs) {
|
||||
throw new OperationError(`GCloud: Operation timed out after ${Math.round(elapsed / 60000)} minutes. Operation ID: ${operationName}`);
|
||||
}
|
||||
|
||||
const headers = new Headers();
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
const response = await fetch(authed.url, { method: "GET", headers: authed.headers, mode: "cors", cache: "no-cache" });
|
||||
|
||||
let data;
|
||||
try { data = await response.json(); } catch (e) {
|
||||
throw new OperationError("GCloud: Failed to parse long-running operation response.");
|
||||
}
|
||||
if (!response.ok) {
|
||||
const msg = data?.error?.message || response.statusText;
|
||||
throw new OperationError(`GCloud: Operation polling error (${response.status}): ${msg}`);
|
||||
}
|
||||
|
||||
if (data.done) return data;
|
||||
|
||||
const elapsedSec = Math.round(elapsed / 1000);
|
||||
if (onProgress) onProgress(elapsedSec);
|
||||
|
||||
await new Promise(resolve => setTimeout(resolve, intervalMs));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Common arguments for Google Cloud Platform operations
|
||||
*
|
||||
|
||||
89
src/core/operations/GCloudListBucket.mjs
Normal file
89
src/core/operations/GCloudListBucket.mjs
Normal file
@ -0,0 +1,89 @@
|
||||
/**
|
||||
* @author CyberChefCloud
|
||||
* @copyright Crown Copyright 2026
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation.mjs";
|
||||
import OperationError from "../errors/OperationError.mjs";
|
||||
import { GCP_AUTH_ARGS, applyGCPAuth, listGCSBucket } from "../lib/GoogleCloud.mjs";
|
||||
|
||||
/**
|
||||
* GCloud List Bucket operation
|
||||
*/
|
||||
class GCloudListBucket extends Operation {
|
||||
|
||||
/**
|
||||
* GCloudListBucket constructor
|
||||
*/
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
this.name = "GCloud List Bucket";
|
||||
this.module = "Cloud";
|
||||
this.description = [
|
||||
"Lists objects in a Google Cloud Storage bucket and returns their <code>gs://</code> URIs.",
|
||||
"<br><br>",
|
||||
"The output is one URI per line, making it suitable for piping directly into a <b>Fork</b> operation ",
|
||||
"to batch-process multiple files (e.g. transcribing all audio files in a folder).",
|
||||
"<br><br>",
|
||||
"Input can be just a bucket name (e.g. <code>cyber-chef-cloud-examples</code>) or a full ",
|
||||
"<code>gs://</code> URI prefix.",
|
||||
].join("\n");
|
||||
this.infoURL = "https://cloud.google.com/storage/docs/json_api/v1/objects/list";
|
||||
this.inputType = "string";
|
||||
this.outputType = "string";
|
||||
this.manualBake = true;
|
||||
this.args = [
|
||||
{
|
||||
"name": "Folder Prefix",
|
||||
"type": "string",
|
||||
"value": "audio/"
|
||||
},
|
||||
{
|
||||
"name": "Output Format",
|
||||
"type": "option",
|
||||
"value": ["GCS URIs (one per line)", "Filenames only", "JSON"]
|
||||
},
|
||||
...GCP_AUTH_ARGS
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} input
|
||||
* @param {Object[]} args
|
||||
* @returns {string}
|
||||
*/
|
||||
async run(input, args) {
|
||||
const [prefix, outputFormat, authType, authStringObj, quotaProject] = args;
|
||||
|
||||
if (!input || !input.trim()) throw new OperationError("Please provide a GCS bucket name.");
|
||||
|
||||
// Normalise: strip gs:// if present, strip trailing slash
|
||||
let bucket = input.trim().replace(/^gs:\/\//, "").split("/")[0];
|
||||
|
||||
try {
|
||||
const items = await listGCSBucket(bucket, prefix, authType, authStringObj, quotaProject);
|
||||
|
||||
if (items.length === 0) {
|
||||
return `No objects found in gs://${bucket}/${prefix || ""}`;
|
||||
}
|
||||
|
||||
switch (outputFormat) {
|
||||
case "Filenames only":
|
||||
return items.map(i => i.name.split("/").pop()).join("\n");
|
||||
case "JSON":
|
||||
return JSON.stringify(items, null, 2);
|
||||
case "GCS URIs (one per line)":
|
||||
default:
|
||||
return items.map(i => i.gs_uri).join("\n");
|
||||
}
|
||||
} catch (e) {
|
||||
if (e.name === "OperationError") throw e;
|
||||
throw new OperationError(e.message || e.toString());
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export default GCloudListBucket;
|
||||
66
src/core/operations/GCloudReadFile.mjs
Normal file
66
src/core/operations/GCloudReadFile.mjs
Normal file
@ -0,0 +1,66 @@
|
||||
/**
|
||||
* @author CyberChefCloud
|
||||
* @copyright Crown Copyright 2026
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation.mjs";
|
||||
import OperationError from "../errors/OperationError.mjs";
|
||||
import { GCP_AUTH_ARGS, readGCSFile } from "../lib/GoogleCloud.mjs";
|
||||
|
||||
/**
|
||||
* GCloud Read File operation
|
||||
*/
|
||||
class GCloudReadFile extends Operation {
|
||||
|
||||
/**
|
||||
* GCloudReadFile constructor
|
||||
*/
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
this.name = "GCloud Read File";
|
||||
this.module = "Cloud";
|
||||
this.description = [
|
||||
"Downloads a file from Google Cloud Storage and returns its raw bytes.",
|
||||
"<br><br>",
|
||||
"Input must be a <code>gs://</code> URI (e.g. <code>gs://my-bucket/images/photo.png</code>).",
|
||||
"<br><br>",
|
||||
"<b>Note:</b> This operation downloads the full file into the browser. ",
|
||||
"It is best suited for small files such as text or images. ",
|
||||
"For large audio or video files, use the GCS URI input mode within the Speech-to-Text or ",
|
||||
"Video Intelligence operations instead — they process the file entirely within Google Cloud.",
|
||||
].join("\n");
|
||||
this.infoURL = "https://cloud.google.com/storage/docs/json_api/v1/objects/get";
|
||||
this.inputType = "string";
|
||||
this.outputType = "ArrayBuffer";
|
||||
this.manualBake = true;
|
||||
this.args = [
|
||||
...GCP_AUTH_ARGS
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} input
|
||||
* @param {Object[]} args
|
||||
* @returns {ArrayBuffer}
|
||||
*/
|
||||
async run(input, args) {
|
||||
const [authType, authStringObj, quotaProject] = args;
|
||||
|
||||
const uri = input.trim();
|
||||
if (!uri.startsWith("gs://")) {
|
||||
throw new OperationError("Input must be a GCS URI starting with gs://");
|
||||
}
|
||||
|
||||
try {
|
||||
return await readGCSFile(uri, authType, authStringObj, quotaProject);
|
||||
} catch (e) {
|
||||
if (e.name === "OperationError") throw e;
|
||||
throw new OperationError(e.message || e.toString());
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export default GCloudReadFile;
|
||||
258
src/core/operations/GCloudSpeechToText.mjs
Normal file
258
src/core/operations/GCloudSpeechToText.mjs
Normal file
@ -0,0 +1,258 @@
|
||||
/**
|
||||
* @author CyberChefCloud
|
||||
* @copyright Crown Copyright 2026
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation.mjs";
|
||||
import OperationError from "../errors/OperationError.mjs";
|
||||
import { GCP_AUTH_ARGS, applyGCPAuth, pollLongRunningOperation, writeGCSFile } from "../lib/GoogleCloud.mjs";
|
||||
|
||||
/**
|
||||
* GCloud Speech to Text operation
|
||||
*/
|
||||
class GCloudSpeechToText extends Operation {
|
||||
|
||||
/**
|
||||
* GCloudSpeechToText constructor
|
||||
*/
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
this.name = "GCloud Speech to Text";
|
||||
this.module = "Cloud";
|
||||
this.description = [
|
||||
"Transcribes audio using the Google Cloud Speech-to-Text API.",
|
||||
"<br><br>",
|
||||
"<b>GCS URI mode (recommended for large files):</b> Input a <code>gs://</code> URI ",
|
||||
"(e.g. <code>gs://my-bucket/audio/file.mp3</code>). The audio is processed entirely within ",
|
||||
"Google Cloud — the raw audio never passes through the browser. Suitable for files of any size. ",
|
||||
"Uses the asynchronous <code>longrunningrecognize</code> API with internal polling.",
|
||||
"<br><br>",
|
||||
"<b>Raw Audio mode:</b> Provide Base64-encoded audio bytes directly. Only suitable for short ",
|
||||
"clips (under ~1 minute). Uses the synchronous <code>recognize</code> API.",
|
||||
"<br><br>",
|
||||
"<b>Write to GCS mode:</b> Instead of returning the transcript to CyberChef, writes it to a ",
|
||||
"structured path in a GCS bucket and returns the destination <code>gs://</code> URI. This is ",
|
||||
"ideal for batch processing with Fork — each fork branch writes its transcript and returns a URI, ",
|
||||
"which can be saved and used as input to a later recipe.",
|
||||
"<br><br>",
|
||||
"Output path convention: <code>output/audio/{filename}/speech-to-text/text.txt</code>",
|
||||
].join("\n");
|
||||
this.infoURL = "https://cloud.google.com/speech-to-text/docs/reference/rest";
|
||||
this.inputType = "string";
|
||||
this.outputType = "string";
|
||||
this.manualBake = true;
|
||||
this.args = [
|
||||
{
|
||||
"name": "Input Mode",
|
||||
"type": "option",
|
||||
"value": ["GCS URI (gs://...)", "Raw Audio Bytes (Base64)"]
|
||||
},
|
||||
{
|
||||
"name": "Language Code",
|
||||
"type": "string",
|
||||
"value": "en-US"
|
||||
},
|
||||
{
|
||||
"name": "Model",
|
||||
"type": "option",
|
||||
"value": ["latest_long", "latest_short", "telephony", "medical_dictation", "default"]
|
||||
},
|
||||
{
|
||||
"name": "Output Destination",
|
||||
"type": "option",
|
||||
"value": ["Return to CyberChef", "Write to GCS"]
|
||||
},
|
||||
{
|
||||
"name": "Output GCS Bucket",
|
||||
"type": "string",
|
||||
"value": "cyber-chef-cloud-examples"
|
||||
},
|
||||
{
|
||||
"name": "Max Poll Minutes",
|
||||
"type": "number",
|
||||
"value": 30
|
||||
},
|
||||
...GCP_AUTH_ARGS
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} input
|
||||
* @param {Object[]} args
|
||||
* @returns {string}
|
||||
*/
|
||||
async run(input, args) {
|
||||
const [
|
||||
inputMode, languageCode, model, outputDest, outputBucket, maxPollMinutes,
|
||||
authType, authStringObj, quotaProject
|
||||
] = args;
|
||||
|
||||
const uri = input.trim();
|
||||
if (!uri) throw new OperationError("Please provide a GCS URI or Base64 audio input.");
|
||||
|
||||
const authString = typeof authStringObj === "string" ? authStringObj : (authStringObj.string || "");
|
||||
if (!authString) throw new OperationError("Please provide a valid GCP Auth String (API Key or OAuth Token).");
|
||||
|
||||
const maxMs = maxPollMinutes * 60 * 1000;
|
||||
let transcript;
|
||||
|
||||
if (inputMode === "GCS URI (gs://...)") {
|
||||
if (!uri.startsWith("gs://")) {
|
||||
throw new OperationError("Input Mode is set to GCS URI but input does not start with gs://");
|
||||
}
|
||||
transcript = await this._transcribeGcsUri(uri, languageCode, model, authType, authStringObj, quotaProject, maxMs);
|
||||
} else {
|
||||
// Raw audio bytes (Base64)
|
||||
transcript = await this._transcribeRawAudio(uri, languageCode, model, authType, authStringObj, quotaProject);
|
||||
}
|
||||
|
||||
if (outputDest === "Write to GCS") {
|
||||
// Derive source filename from GCS URI (or use a default for raw audio)
|
||||
const sourceFilename = inputMode === "GCS URI (gs://...)"
|
||||
? uri.split("/").pop()
|
||||
: "raw_audio";
|
||||
|
||||
const objectPath = `output/audio/${sourceFilename}/speech-to-text/text.txt`;
|
||||
const destUri = await writeGCSFile(outputBucket, objectPath, transcript, authType, authStringObj, quotaProject);
|
||||
return destUri;
|
||||
}
|
||||
|
||||
return transcript;
|
||||
}
|
||||
|
||||
/**
|
||||
* Transcribes audio from a GCS URI using the longrunningrecognize API.
|
||||
*
|
||||
* @param {string} gcsUri
|
||||
* @param {string} languageCode
|
||||
* @param {string} model
|
||||
* @param {string} authType
|
||||
* @param {Object|string} authStringObj
|
||||
* @param {string} quotaProject
|
||||
* @param {number} maxMs
|
||||
* @returns {Promise<string>}
|
||||
*/
|
||||
async _transcribeGcsUri(gcsUri, languageCode, model, authType, authStringObj, quotaProject, maxMs) {
|
||||
let url = "https://speech.googleapis.com/v1/speech:longrunningrecognize";
|
||||
const headers = new Headers();
|
||||
headers.set("Content-Type", "application/json; charset=utf-8");
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
|
||||
const body = JSON.stringify({
|
||||
config: {
|
||||
languageCode,
|
||||
model,
|
||||
enableAutomaticPunctuation: true,
|
||||
},
|
||||
audio: { uri: gcsUri }
|
||||
});
|
||||
|
||||
const response = await fetch(authed.url, {
|
||||
method: "POST",
|
||||
headers: authed.headers,
|
||||
body,
|
||||
mode: "cors",
|
||||
cache: "no-cache"
|
||||
});
|
||||
|
||||
let responseData;
|
||||
try { responseData = await response.json(); } catch (e) {
|
||||
throw new OperationError("GCloud Speech to Text: Failed to parse API response.");
|
||||
}
|
||||
if (!response.ok) {
|
||||
const msg = responseData?.error?.message || response.statusText;
|
||||
throw new OperationError(`GCloud Speech to Text: API Error (${response.status}): ${msg}`);
|
||||
}
|
||||
|
||||
const operationName = responseData.name;
|
||||
if (!operationName) throw new OperationError("GCloud Speech to Text: No operation name returned from API.");
|
||||
|
||||
// Poll for completion
|
||||
const POLL_URL = "https://speech.googleapis.com/v1/operations/";
|
||||
const completed = await pollLongRunningOperation(
|
||||
operationName,
|
||||
POLL_URL,
|
||||
authType,
|
||||
authStringObj,
|
||||
quotaProject,
|
||||
maxMs,
|
||||
10000,
|
||||
(elapsedSec) => {
|
||||
// onProgress — not easily surfaced in CyberChef output mid-bake,
|
||||
// but available for future UI integration
|
||||
void elapsedSec;
|
||||
}
|
||||
);
|
||||
|
||||
return this._extractTranscript(completed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Transcribes audio from raw Base64 bytes using the synchronous recognize API.
|
||||
*
|
||||
* @param {string} base64Audio
|
||||
* @param {string} languageCode
|
||||
* @param {string} model
|
||||
* @param {string} authType
|
||||
* @param {Object|string} authStringObj
|
||||
* @param {string} quotaProject
|
||||
* @returns {Promise<string>}
|
||||
*/
|
||||
async _transcribeRawAudio(base64Audio, languageCode, model, authType, authStringObj, quotaProject) {
|
||||
let url = "https://speech.googleapis.com/v1/speech:recognize";
|
||||
const headers = new Headers();
|
||||
headers.set("Content-Type", "application/json; charset=utf-8");
|
||||
const authed = applyGCPAuth(url, headers, authType, authStringObj, quotaProject);
|
||||
|
||||
const body = JSON.stringify({
|
||||
config: {
|
||||
languageCode,
|
||||
model,
|
||||
enableAutomaticPunctuation: true,
|
||||
},
|
||||
audio: { content: base64Audio }
|
||||
});
|
||||
|
||||
const response = await fetch(authed.url, {
|
||||
method: "POST",
|
||||
headers: authed.headers,
|
||||
body,
|
||||
mode: "cors",
|
||||
cache: "no-cache"
|
||||
});
|
||||
|
||||
let responseData;
|
||||
try { responseData = await response.json(); } catch (e) {
|
||||
throw new OperationError("GCloud Speech to Text: Failed to parse API response.");
|
||||
}
|
||||
if (!response.ok) {
|
||||
const msg = responseData?.error?.message || response.statusText;
|
||||
throw new OperationError(`GCloud Speech to Text: API Error (${response.status}): ${msg}`);
|
||||
}
|
||||
|
||||
return this._extractTranscript({ response: responseData });
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts a joined transcript string from a completed LRO response or synchronous response.
|
||||
*
|
||||
* @param {Object} completed - The operation response object.
|
||||
* @returns {string} Joined transcript text.
|
||||
*/
|
||||
_extractTranscript(completed) {
|
||||
const results = completed?.response?.results;
|
||||
if (!results || results.length === 0) {
|
||||
return "(No speech detected)";
|
||||
}
|
||||
return results
|
||||
.map(r => r.alternatives?.[0]?.transcript || "")
|
||||
.filter(t => t.length > 0)
|
||||
.join(" ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export default GCloudSpeechToText;
|
||||
@ -100,7 +100,145 @@ module.exports = {
|
||||
});
|
||||
},
|
||||
|
||||
// ─── GCloud List Bucket ────────────────────────────────────────────────────
|
||||
|
||||
"GCloud List Bucket: Missing Key Validation": function (browser) {
|
||||
browserUtils.loadRecipe(browser, "GCloud List Bucket", "cyber-chef-cloud-examples", [
|
||||
"audio/",
|
||||
"GCS URIs (one per line)",
|
||||
"API Key",
|
||||
{ option: "UTF8", string: "" },
|
||||
""
|
||||
]);
|
||||
|
||||
browser.waitForElementNotVisible("#snackbar-container", 6000);
|
||||
browserUtils.bake(browser);
|
||||
browser.pause(2000);
|
||||
browser.saveScreenshot("tests/browser/output/list_bucket_no_key.png");
|
||||
browser.execute(function () {
|
||||
return window.app.manager.output.outputEditorView.state.doc.toString();
|
||||
}, [], function ({ value }) {
|
||||
browser.assert.ok(value.includes("Please provide a valid GCP Auth String"));
|
||||
});
|
||||
},
|
||||
|
||||
"GCloud List Bucket: Lists audio/ files from cyber-chef-cloud-examples": function (browser) {
|
||||
let testToken = process.env.CYBERCHEF_GCP_TEST_TOKEN;
|
||||
|
||||
if (!testToken || testToken === "YOUR_OAUTH_TOKEN_HERE") {
|
||||
try {
|
||||
testToken = require("child_process").execSync("gcloud auth print-access-token", { stdio: "pipe", encoding: "utf-8" }).trim();
|
||||
} catch (e) {
|
||||
console.log("No valid token found and gcloud failed. Skipping GCloud List Bucket live test.");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
browserUtils.loadRecipe(browser, "GCloud List Bucket", "cyber-chef-cloud-examples", [
|
||||
"audio/",
|
||||
"GCS URIs (one per line)",
|
||||
"OAuth Token",
|
||||
{ option: "UTF8", string: testToken },
|
||||
"cyberchefcloud"
|
||||
]);
|
||||
|
||||
browser.waitForElementNotVisible("#snackbar-container", 6000);
|
||||
browserUtils.bake(browser);
|
||||
browser.pause(5000);
|
||||
browser.saveScreenshot("tests/browser/output/list_bucket_live.png");
|
||||
browser.execute(function () {
|
||||
return window.app.manager.output.outputEditorView.state.doc.toString();
|
||||
}, [], function ({ value }) {
|
||||
browser.assert.ok(value.includes("gs://cyber-chef-cloud-examples/audio/"), `Expected gs:// URIs, got: ${value}`);
|
||||
browser.assert.ok(value.includes("she_achieves_great_results_f55548.mp3"), `Expected audio filename in output, got: ${value}`);
|
||||
});
|
||||
},
|
||||
|
||||
// ─── GCloud Speech to Text ─────────────────────────────────────────────────
|
||||
|
||||
"GCloud Speech to Text: GCS URI mode returns transcription to browser": function (browser) {
|
||||
let testToken = process.env.CYBERCHEF_GCP_TEST_TOKEN;
|
||||
|
||||
if (!testToken || testToken === "YOUR_OAUTH_TOKEN_HERE") {
|
||||
try {
|
||||
testToken = require("child_process").execSync("gcloud auth print-access-token", { stdio: "pipe", encoding: "utf-8" }).trim();
|
||||
} catch (e) {
|
||||
console.log("No valid token found and gcloud failed. Skipping GCloud Speech to Text live test.");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const gcsUri = "gs://cyber-chef-cloud-examples/audio/she_achieves_great_results_f55548.mp3";
|
||||
|
||||
browserUtils.loadRecipe(browser, "GCloud Speech to Text", gcsUri, [
|
||||
"GCS URI (gs://...)",
|
||||
"en-US",
|
||||
"latest_long",
|
||||
"Return to CyberChef",
|
||||
"cyber-chef-cloud-examples",
|
||||
30,
|
||||
"OAuth Token",
|
||||
{ option: "UTF8", string: testToken },
|
||||
"cyberchefcloud"
|
||||
]);
|
||||
|
||||
browser.waitForElementNotVisible("#snackbar-container", 6000);
|
||||
browserUtils.bake(browser);
|
||||
// LRO jobs on short files typically complete in ~5 seconds; allow up to 30s here
|
||||
browser.pause(30000);
|
||||
browser.saveScreenshot("tests/browser/output/speech_to_text_browser.png");
|
||||
browser.execute(function () {
|
||||
return window.app.manager.output.outputEditorView.state.doc.toString();
|
||||
}, [], function ({ value }) {
|
||||
browser.assert.ok(
|
||||
value.toLowerCase().includes("she achieves great results"),
|
||||
`Expected transcript, got: ${value}`
|
||||
);
|
||||
});
|
||||
},
|
||||
|
||||
"GCloud Speech to Text: GCS URI mode writes transcript to GCS output/ bucket": function (browser) {
|
||||
let testToken = process.env.CYBERCHEF_GCP_TEST_TOKEN;
|
||||
|
||||
if (!testToken || testToken === "YOUR_OAUTH_TOKEN_HERE") {
|
||||
try {
|
||||
testToken = require("child_process").execSync("gcloud auth print-access-token", { stdio: "pipe", encoding: "utf-8" }).trim();
|
||||
} catch (e) {
|
||||
console.log("No valid token found and gcloud failed. Skipping GCloud Speech to Text GCS write test.");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const gcsUri = "gs://cyber-chef-cloud-examples/audio/she_achieves_great_results_f55548.mp3";
|
||||
|
||||
browserUtils.loadRecipe(browser, "GCloud Speech to Text", gcsUri, [
|
||||
"GCS URI (gs://...)",
|
||||
"en-US",
|
||||
"latest_long",
|
||||
"Write to GCS",
|
||||
"cyber-chef-cloud-examples",
|
||||
30,
|
||||
"OAuth Token",
|
||||
{ option: "UTF8", string: testToken },
|
||||
"cyberchefcloud"
|
||||
]);
|
||||
|
||||
browser.waitForElementNotVisible("#snackbar-container", 6000);
|
||||
browserUtils.bake(browser);
|
||||
browser.pause(30000);
|
||||
browser.saveScreenshot("tests/browser/output/speech_to_text_gcs_write.png");
|
||||
browser.execute(function () {
|
||||
return window.app.manager.output.outputEditorView.state.doc.toString();
|
||||
}, [], function ({ value }) {
|
||||
browser.assert.ok(
|
||||
value.includes("gs://cyber-chef-cloud-examples/output/audio/she_achieves_great_results_f55548.mp3/speech-to-text/text.txt"),
|
||||
`Expected GCS output URI, got: ${value}`
|
||||
);
|
||||
});
|
||||
},
|
||||
|
||||
after: function (browser) {
|
||||
browser.end();
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user