Skip to content

Optional You.com web provider behind the WebProvider contract (no key, DuckDuckGo stays default) #1338

Open

Description

@brainsparker

I work at You.com, so treat the recommendation accordingly. Asking before writing any code.

The default keyless-http provider searches DuckDuckGo's HTML form and parses .result nodes. That keeps core running with no hosted vendor, which VISION.md asks for. When DuckDuckGo serves its anti-bot page instead of results, there's no other keyless option to set in WEB_PROVIDER.

The idea is one optional adapter, with nothing about the default changing:

  • packages/adapters/src/you-web.ts, selected with WEB_PROVIDER=you in web-provider-factory.ts.
  • search calls GET https://api.you.com/v1/agents/search?query=...&count=.... No key, no account, 100 searches a day. It returns JSON (results.web[] with url, title, description, snippets), so it maps onto WebSearchHit without HTML parsing. If YDC_API_KEY is set in the deployment env, it uses the keyed Search API at https://ydc-index.io/v1/search with X-API-Key instead.
  • fetch reuses what KeylessHttpWebProvider already does (fetchSafeWebText plus Readability), so page reads stay local and keep the SSRF rules.
  • describe() reports search, fetch, readability, contract version 1, and it gets its own case in web-conformance.test.ts with an injected fetch, offline like the others.

Builtin tool names don't change, and anyone who never sets WEB_PROVIDER=you sees no difference.

For transparency: requests to You.com would send User-Agent: Rakazo/0.1 (+https://github.com/elie222/rakazo; web tools) youdotcom-integration/elie222-rakazo (your web-tools UA plus our tag) so we can see traffic that comes from Rakazo. It's only on You.com requests.

If you'd take it, I'll open a small PR with the conformance case and pnpm test output. If you'd rather build it yourself, or keep the default the only built-in, say so and I'll close this without fuss.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions