
from opendatalab
Run, validate, and parse OmniDocBench document parsing evaluations with Docker/conda workflows and result parsing.
python pdf_validation.py --config ... inside recommended Docker image ghcr.io/zeng-weijun/omnidocbench-eval:repro-ubuntu2204.\n- Parse and summarize *_metric_result.json, *_run_summary.json, *_stage_execution.json, and *_runtime_environment.json.\n- Troubleshooting: Docker access, CDM OOMs, ImageMagick policy issues, GT capitalization, and prediction folder nesting.\n\n## Recommended prompts / usage\n- "How do I run OmniDocBench end2end evaluation on my GT JSON and markdown predictions?"\n- "Help me parse an OmniDocBench result directory and extract overall/text/formula/table scores."\n- "I get CDM OOMs — how should I set cdm_workers on a 4CPU/8GB node?"\n\n## Installation / scripts\nThe skill bundles scripts: scripts/generate_end2end_config.py and scripts/parse_results.py for config generation and result parsing. Follow Docker workflow in the skill to avoid local dependency issues.\nA well-documented skill for running OmniDocBench document parsing evaluations via Docker or conda. The SKILL.md is thorough with input validation, Docker/SSH workflows, config guidance, and a detailed troubleshooting playbook. No bundled scripts to test. Security posture is strong: read-only mounts for inputs, no destructive commands, no credentials, non-interactive sudo only as fallback. The main limitation is narrow audience — only useful to researchers evaluating document parsing models on OmniDocBench.
No scripts to run. Security deductions: -3 for shell variable interpolation in heredoc/Docker commands (minor injection risk if user paths contain special characters, though paths are quoted in validation), -2 for sudo -n fallback pattern. Quality is high but no idempotent scripts to verify. Architecture is good — clear sections, progressive disclosure, result parsing guidance — but no scripts/ or references/ directories. Usefulness is niche: only researchers running OmniDocBench evaluations would use this.