feat: Release v0.10 - Modular Architecture & External Config

- Refactor codebase into src/ (preprocessing, translation, assembly) - Add pipeline/ scripts for individual stages - Externalize configuration to config/config.yaml - Fix Cover Image preservation - Update documentation and manuals
2026-01-31 22:49:44 +08:00
parent 9ef82393be
commit 7a93c52b42
306 changed files with 30313 additions and 1071 deletions
@@ -0,0 +1,90 @@
+# Operation Manual & Change Log
+
+## Core Principles
+
+1.  **Modularity**: The system is divided into three distinct phases (Preprocessing, Translation, Assembly) with clear boundaries.
+2.  **Immutability**: `book_structure.json` is generated once during preprocessing and should not be modified by subsequent steps.
+3.  **Source of Truth**: `manifest.json` is the single source of truth for translations.
+4.  **Idempotency**: Translation steps can be retried without side effects (existing translations are preserved).
+
+## Directory Structure
+
+*   `pipeline/`: Executable scripts for each stage.
+    *   `01_preprocess.py`: Clean EPUB, generate structure, extract text.
+    *   `02_translate.py`: Translate text in manifest.
+    *   `03_assemble.py`: Apply translations and build final EPUB.
+*   `src/`: Core logic modules.
+    *   `preprocessing/`: Cleaning, extraction, profiling.
+    *   `translation/`: LLM integration, manifest management.
+    *   `assembly/`: Backfilling, EPUB building.
+    *   `common/`: Shared data models and utils.
+*   `work/`: Working directory for intermediate files (ignored by git).
+
+## Pipeline Usage
+
+### Step 1: Preprocessing
+```bash
+python pipeline/01_preprocess.py inputs/my_book.epub
+```
+Generates `work/my_book/book_structure.json` and `manifest.json`.
+
+### Step 2: Translation
+```bash
+python pipeline/02_translate.py --input-epub inputs/my_book.epub
+```
+Translates entries in `manifest.json`. ensuring `.env` has `OPENAI_API_KEY`.
+
+### Step 3: Assembly
+```bash
+python pipeline/03_assemble.py inputs/my_book.epub --mode bilingual
+```
+Generates `output/my_book_bilingual.epub`.
+
+## Configuration
+
+System settings are managed via `config/config.yaml` and environment variables.
+
+### `config/config.yaml`
+Control LLM parameters and translation behavior:
+```yaml
+llm:
+  model: "gpt-3.5-turbo"           # LLM Model Name
+  base_url: "https://api.openai.com/v1" 
+  timeout: 60
+  requests_per_minute: 60          # Rate limiting
+  concurrent_requests: 5           # Parallel chunks
+
+translation:
+  chunk_size: 4000                 # Characters per chunk
+```
+
+### Environment Variables (`.env`)
+Security-sensitive credentials must be set here:
+```bash
+OPENAI_API_KEY=sk-...              # Required
+OPENAI_BASE_URL=...                # Optional override for config
+```
+
+## Known Issues & Troubleshooting
+
+### Missing Placeholders Warning
+During assembly, you may see logs like:
+`WARNING - Restoration warning: missing placeholders {'1'}`
+This indicates that the LLM translation missed a placeholder tag (e.g. `φ1φ`). The system attempts to recover, but this warning is logged for review. These are usually minor and do not prevent EPUB generation.
+
+## Change Log
+
+### [2026-01-28] Bug Fixes
+*   **Fix Cover Image**: Resolved issue where book cover execution was missing in the final EPUB. Added `cover_image_id` tracking in `BookStructure` and restored proper OPF metadata in `BilingualBuilder`.
+
+### [2026-01-27] Externalized Configuration
+*   **Config**: Added `config/config.yaml` for tuning parameters (LLM model, RPM, Chunk Size).
+*   **Logic**: `pipeline/02_translate.py` now loads settings from `config.yaml`.
+*   **Dependency**: Added `PyYAML` to `requirements.txt`.
+
+### [2026-01-27] Architecture Refactoring
+*   **Restructured**: Moved source files into `src/preprocessing`, `src/translation`, `src/assembly`, `src/common`.
+*   **Pipeline**: Created individual pipeline scripts in `pipeline/`.
+*   **Refactor**: Renamed `fine_grained_extractor` to `text_extractor`, `translator` to `translator_engine`, etc.
+*   **Logic**: Enforced 100% text coverage check in `format_extractor.py` (removed 95% threshold).
+*   **Docs**: Created this Operation Manual.