epub_bilingual_translator/archive/v0.11/doc/OPERATION_MANUAL.md

Operation Manual & Change Log

Core Principles

1. Modularity: The system is divided into three distinct phases (Preprocessing, Translation, Assembly) with clear boundaries.
2. Immutability: book_structure.json is generated once during preprocessing and should not be modified by subsequent steps.
3. Source of Truth: manifest.json is the single source of truth for translations.
4. Idempotency: Translation steps can be retried without side effects (existing translations are preserved).

Directory Structure

• pipeline/: Executable scripts for each stage.
  • 01_preprocess.py: Clean EPUB, generate structure, extract text.
  • 02_translate.py: Translate text in manifest.
  • 03_assemble.py: Apply translations and build final EPUB.
• src/: Core logic modules.
  • preprocessing/: Cleaning, extraction, profiling.
  • translation/: LLM integration, manifest management.
  • assembly/: Backfilling, EPUB building.
  • common/: Shared data models and utils.
• work/: Working directory for intermediate files (ignored by git).

Pipeline Usage

Step 1: Preprocessing

python pipeline/01_preprocess.py inputs/my_book.epub

Generates work/my_book/book_structure.json and manifest.json.

Step 2: Translation

python pipeline/02_translate.py --input-epub inputs/my_book.epub

Translates entries in manifest.json. ensuring .env has OPENAI_API_KEY.

Step 3: Assembly

Step 3: Assembly

# Bilingual Output (Default: output/bilingual_my_book.epub)
python pipeline/03_assemble.py inputs/my_book.epub --bilingual

# Target Language Output (Default: output/translated_my_book.epub)
python pipeline/03_assemble.py inputs/my_book.epub

Note: The Assembly step now includes an LLM-based Placeholder Repair mechanism. If format_restorer detects broken placeholders in the translation, it will query the LLM (using your configured credentials) to attempt an automatic fix. Ensure your OPENAI_API_KEY is set if you want this feature enabled.

Configuration

System settings are managed via config/config.yaml and environment variables.

config/config.yaml

Control LLM parameters and translation behavior:

llm:
  model: "gpt-3.5-turbo"           # LLM Model Name
  base_url: "https://api.openai.com/v1" 
  timeout: 60
  requests_per_minute: 60          # Rate limiting
  concurrent_requests: 5           # Parallel chunks

translation:
  chunk_size: 4000                 # Characters per chunk

Environment Variables (.env)

Security-sensitive credentials must be set here:

OPENAI_API_KEY=sk-...              # Required
OPENAI_BASE_URL=...                # Optional override for config

Known Issues & Troubleshooting

AuthenticationError (OpenRouter etc.)

If you see AuthenticationError despite having the correct base_url in config:

1. Check if you have a stale OPENAI_API_KEY in your shell environment.
2. Environment variables override .env files.
3. Fix: Run unset OPENAI_API_KEY (and OPENAI_BASE_URL) before running the script.

Missing/Unknown Placeholders

• Logs: WARNING - Restoration warning: missing placeholders...
• Cause: The LLM translation didn't preserve the exact φXφ tags.
• Fix:
  1. The system will now attempt to auto-repair using the LLM during Assembly.
  2. If that fails, check logs. In some cases (e.g., complex HTML entities like &), the extractor might have degraded to plain text.
  3. (Fixed in v0.11) Enhanced FormatExtractor now handles HTML entities correctly, preventing phantom placeholder hallucinations.

Change Log

[2026-01-31] Robustness & Repair

• Feature: Added LLM-based Placeholder Repair in Assembly stage. If placeholders mismatch, the system asks the LLM to fix the tags without changing text.
• Fix: Solved FormatExtractor "phantom placeholders" issue by correctly unescaping HTML entities during integrity checks.
• Fix: Resolved Duplicate ID issue in LLM response parsing. Now recursively strips repeated headers (e.g., #12: #12: ...) to prevent them from leaking into the translation.
• Tweak: Updated pipeline/03_assemble.py to be async and load LLM config.

[2026-01-30] Performance Improvements

• Concurrency Fix: Resolved issue where concurrent_requests in config.yaml was ignored by the Translator engine. Now main.py and pipeline/02_translate.py correctly propagate this setting, allowing faster translation with higher limits (e.g., for local LLMs or high-rate-limit providers).

[2026-01-28] Bug Fixes

• Fix Cover Image: Resolved issue where book cover execution was missing in the final EPUB. Added cover_image_id tracking in BookStructure and restored proper OPF metadata in BilingualBuilder.

[2026-01-27] Externalized Configuration

• Config: Added config/config.yaml for tuning parameters (LLM model, RPM, Chunk Size).
• Logic: pipeline/02_translate.py now loads settings from config.yaml.
• Dependency: Added PyYAML to requirements.txt.

[2026-01-27] Architecture Refactoring

• Restructured: Moved source files into src/preprocessing, src/translation, src/assembly, src/common.
• Pipeline: Created individual pipeline scripts in pipeline/.
• Refactor: Renamed fine_grained_extractor to text_extractor, translator to translator_engine, etc.
• Logic: Enforced 100% text coverage check in format_extractor.py (removed 95% threshold).
• Docs: Created this Operation Manual.