OpenFang adds in-memory PDF text extraction and chunked stream limits to web retrieval

OpenFang contributors submitted PR #1281, addressing severe context corruption vulnerabilities in the agent's web_fetch retrieval tool. Previously, web_fetch decoded all HTTP responses directly into string format. When an agent fetched PDF documents, the tool ingested raw FlateDecode-compressed binary byte streams into the conversational context. This flooded model prompts with hundreds of thousands of unparseable binary tokens, instantly blowing context limits and poisoning session history. Furthermore, chunked HTTP streams lacking Content-Length headers bypassed size restrictions, causing unbounded memory consumption. PR #1281 integrates the 'pdf-extract' and 'lopdf' crates to parse PDF text cleanly in memory, adds %PDF magic byte sniffing to detect binary payloads, and implements read_bounded_body to cap chunked streams dynamically. This technical patch represents rare engineering movement for OpenFang, which remains categorized in ClawCharts' inactive tier with zero commits in thirty days and no tagged release since v0.6.9 in May 2026. Simultaneously, community proposals in PR #1272 and issue #1282 request integration with Atlas Cloud and RunInfra model providers.

The facts

- PR #1281 fixes issue #1271 by integrating pdf-extract and lopdf crates to extract clean text from PDF documents in memory. - Stops raw FlateDecode binary streams and invalid token sequences from entering model prompts when fetching PDF links. - Replaces unmetered HTTP body decoding with read_bounded_body, enforcing strict byte limits on chunked responses without Content-Length. - Detects %PDF magic byte headers and samples null bytes to reject arbitrary binary media files before text conversion. - OpenFang remains inactive on ClawCharts with 18,204 stars and no formal version release since v0.6.9 in May 2026.

Why it matters

Web-browsing tools are high-risk ingestion points for autonomous agents. If retrieval pipelines do not sanitize MIME types, sniff magic bytes, or parse complex file formats into clean text, models end up ingesting raw binary garbage. In addition to wasting expensive input tokens, binary ingestion frequently triggers provider-level API errors and degrades reasoning performance. Enforcing strict binary detection and in-memory text parsing protects context hygiene and ensures consistent agent operation.

Current

Inspected on 2026-09-20. The OpenFang stable-channel baseline is v0.6.9 published 2026-05-12T18:42:42Z. The primary source was open (not merged) when captured. Mainline merges, open proposals and packaged releases are distinct availability states; the release baseline does not establish that a proposal has shipped.

Evidence

Primary evidence: RightNow-AI/openfang PR #1281 (https://github.com/RightNow-AI/openfang/pull/1281). Supporting context: RightNow-AI/openfang PR #1272 (https://github.com/RightNow-AI/openfang/pull/1272); RightNow-AI/openfang issue #1282 (https://github.com/RightNow-AI/openfang/issues/1282). The source bodies and linked context were inspected. Test results quoted from contributors remain attributed reports, not independently reproduced experiments.

Operator take

Unchecked web retrieval is a recipe for prompt poisoning and context overflow. Agent runtimes must enforce strict content boundaries on all external inputs before passing them to the model. While OpenFang's PDF parsing patch addresses a critical retrieval defect, operators must weigh individual PR fixes against the project's broader release dormancy. High-assurance deployments should ensure all web retrieval tools implement binary sniffing and bounded streaming guards regardless of framework choice.

Caveat

PR #1281 is an open pull request that has not been merged into OpenFang's main branch. OpenFang has not cut a formal software release in four months, and community provider requests in #1272 and #1282 remain unreviewed proposals.