Extract vs chunk
The tab extracts. pip can see layout.
A PDF in the playground becomes text via PDF.js, then a text strategy runs. That is useful and honest. It is not pdf_chunker. Same story for Word via mammoth. If you need tables, images, scene cuts, or silence, you are on pip.
| Input | Lab | pip |
|---|---|---|
| txt, md, html, json, csv, code | Native, same names | Same names, streaming |
PDF.js extract → text strategy. No pdf_chunker | chunking-strategy[pdf] then pdf_chunker (layout, images, tables) or extract + any text strategy | |
| docx | mammoth extract → text strategy | doc_chunker when you want structure; otherwise extract |
| OLE .doc, xlsx, pptx | Not in the tab | Library / Tika |
| images | Not in the tab | grid_based_image, patch_based_image via [media] |
| audio | Not in the tab | time_based_audio, silence_based_audio |
| video | Not in the tab | time_based_video, scene_based_video |
| anything else | — | [tika] / universal_document, POSIX |
Extract in Python without picking a chunker
from chunking_strategy import extract_content
extracted = extract_content("report.pdf")
print(extracted.text_content[:400])That is the library extract path (PDF, Office, Tika when installed). Then pass extracted.content to any text strategy. The lab does the same job with PDF.js / mammoth instead.
Offsets
Lab goldens compare Unicode scalar offsets, not UTF-16 code units. If you port a chunker, fixtures under fixtures/ are the contract. See fixtures/README.md.
Size
Lab cap is 150 MB because the bytes sit in the tab. pip StreamingChunker / mmap reads from disk. Dropping a 2 GB PDF into the playground is the wrong tool; chunk_file is the right one.