gem-hunter
Language: TypeScript
Field Theory CLI for bookmarks, Library, commands, and agent workflows
Sync and store bookmarks locally, manage Field Theory Library and command workflows, and make local context available to Claude Code, Codex, or any agent with shell access.
Free and open source. Designed for Mac.
Field Theory CLI also acts as the command-line companion to the Field Theory Mac app: it knows the local ~/.fieldtheory paths, can open Library pages in the packaged app, installs agent skills, and can download packaged app releases from the release feed.
Field Theory is split across sibling repositories:
afar1/fieldtheory: Field Theory Mac app source.afar1/fieldtheory-cli: this CLI, licensed under MIT.afar1/fieldtheory-plugin: Field Theory Codex plugin and skills.afar1/field-releases: packaged Mac app release artifacts and updater metadata.Use the app source repo for Mac app issues, this repo for CLI issues, the plugin repo for agent-plugin issues, and the release feed only for installer, download, update-feed, or release-artifact problems.
npm install -g fieldtheory
Requires Node.js 20+. A Chrome-family browser or Firefox is recommended for session sync; OAuth is available for all platforms.
# 1. Sync your bookmarks (needs a supported browser logged into X) ft sync # 2. Search them ft search "distributed systems" # 3. Explore ft viz ft categories ft stats
On first run, ft sync extracts your X session from your browser and downloads your bookmarks into ~/.fieldtheory/bookmarks/.
| Command | Description |
|---|---|
ft sync |
Download and sync bookmarks, then fetch any missing media (photos, video posters, capped videos). No API required. |
ft sync --no-media |
Sync bookmarks only; skip the media download pass |
ft sync --skip-profile-images |
Sync bookmarks and post media but skip author profile images |
ft sync --rebuild |
Full re-crawl of all bookmarks |
ft sync --continue |
Resume a paused or interrupted sync from the saved cursor |
ft sync --gaps |
Backfill quoted tweets, expand truncated/X Article text, enrich linked articles, and fill any media gaps |
ft sync --folders |
Also sync X bookmark folder tags (read-only mirror of X state) |
ft sync --folder <name> |
Sync a single folder by name (exact or unambiguous prefix) |
ft sync --classify |
Sync then classify new bookmarks with LLM |
ft sync --api |
Sync via OAuth API (cross-platform) |
ft auth |
Set up OAuth for API-based sync (optional) |
| Command | Description |
|---|---|
ft search <query> |
Full-text search with BM25 ranking |
ft list |
Filter by author, date, category, domain, or folder |
ft list --folder <name> |
Show bookmarks in an X bookmark folder |
ft show <id> |
Show one bookmark in detail |
ft sample <category> |
Random sample from a category |
ft stats |
Top authors, languages, date range |
ft viz |
Terminal dashboard with sparklines, categories, and domains |
ft categories |
Show category distribution |
ft domains |
Subject domain distribution |
ft folders |
Show X bookmark folder distribution (requires ft sync --folders first) |
| Command | Description |
|---|---|
ft classify |
Classify by category and domain using LLM |
ft classify --regex |
Classify by category using simple regex |
ft classify-domains |
Classify by subject domain only (LLM) |
ft classify --engine <name> |
Override the LLM engine for one run (also works on ft sync --classify and ft classify-domains) |
ft model |
View or change the default LLM engine |
| Command | Description |
|---|---|
ft md |
Export bookmarks as individual markdown files, including enriched article text |
ft md --changed |
Re-export only markdown files whose source bookmark data changed |
ft wiki |
Compile a Karpathy-style interlinked knowledge base |
ft ask <question> |
Ask questions against the knowledge base |
ft ask <question> --save |
Ask and save the answer as a concept page |
ft lint |
Health-check the wiki for broken links and missing pages |
ft lint --fix |
Auto-fix fixable wiki issues |
| Command | Description |
|---|---|
ft seeds search "<query>" --create |
Save a bookmark-grounded seed |
ft repos add <path> |
Add a repo to the default repo set |
ft possible |
Interactive seed + repo + frame wizard |
ft possible run --defaults |
Re-run with the most-recently-used seed and saved repos |
ft possible run --background |
Start a run as a background job |
ft possible prompt <node-id> |
Print the goal prompt for one plotted node |
ft possible nightly install |
Install a nightly Possible run on macOS |
| Command | Description |
|---|---|
ft paths --json |
Show canonical bookmarks, Library, Commands, and compatibility paths |
ft status --json |
Show bookmark/classification status plus Field Theory paths |
ft library search <query> |
Search local Field Theory Library markdown |
ft library show <path> |
Print a Library page and its version metadata with --json |
ft library create <path> --stdin |
Create a new Library page under ~/.fieldtheory/library |
ft library update <path> --stdin --expected-sha256 <hash> |
Replace a Library page with conflict protection |
ft library delete <path> |
Move a Library page to Trash; the Mac app owns remote sync tombstones |
ft library open <path> |
Open a Library page in the Field Theory Mac app |
ft commands list |
List portable commands under ~/.fieldtheory/commands |
ft commands new <name> |
Create a reusable portable command |
ft commands validate [name] |
Check command shape and guardrails |
ft install app |
Download and install the latest Field Theory Mac app from afar1/field-releases |
ft library open targets the packaged Field Theory app by bundle id (com.fieldtheory.app) instead of trusting the system-wide fieldtheory:// handler. That avoids accidentally opening a generic Electron development app when another checkout registered the same URL scheme.
For local Field Theory app development, point the CLI at the dev checkout:
export FT_APP_DEV_DIR=/Users/you/dev/fieldtheory/mac-app
ft library open notes/example.mdPackaged variants can override the bundle id with FT_APP_BUNDLE_ID. Advanced development launchers can set FT_APP_OPEN_COMMAND to an executable that receives the deep-link URL as its first argument.
| Command | Description |
|---|---|
ft skill install |
Install /fieldtheory skill for Claude Code and Codex |
ft skill show |
Print skill content to stdout |
ft skill uninstall |
Remove installed skill files |
| Command | Description |
|---|---|
ft index |
Rebuild search index from JSONL cache (preserves classifications) |
ft fetch-media |
Backfill/download X media assets for existing bookmarks (default: all pending bookmarks) |
ft fetch-media --skip-profile-images |
Download post media without author profile images |
ft status |
Show sync/classification status and data location |
ft path |
Print data directory path |
Install the /fieldtheory skill so your agent automatically searches your bookmarks when relevant:
ft skill install # Auto-detects Claude Code and CodexThen ask your agent:
"What have I bookmarked about cancer research in the last three years and how has it progressed?"
"I bookmarked a number of new open source AI memory tools. Pick the best one and figure out how to incorporate it in this repo."
"Your goal is to look at AI agent bookmarks and come up with a roadmap plotted in the grid of what I should do next across the Field Theory CLI and Mac app projects."
"Every day please sync any new X bookmarks using the Field Theory CLI."
Works with Claude Code, Codex, or any agent with shell access.
Sync with cron:
# Sync every morning at 7am 0 7 * * * ft sync # Sync and classify every morning 0 7 * * * ft sync --classify
Run Possible every night on macOS with LaunchAgent:
ft seeds search "agents" --days 90 --limit 8 --frame leverage-specificity --create ft repos add ~/dev/fieldtheory ft repos add ~/dev/fieldtheory-cli ft possible nightly install --time 02:00 --defaults --model opus --effort medium --nodes 5 ft possible nightly show
Nightly schedules are stored under ~/.fieldtheory/ideas/nightly/. Each tick starts a normal background job under ~/.fieldtheory/ideas/jobs/, using your local logged-in CLI sessions and the current PATH captured in the LaunchAgent plist.
ft respects standard proxy environment variables for network requests: HTTPS_PROXY, HTTP_PROXY, ALL_PROXY, and NO_PROXY.
Data is stored locally under ~/.fieldtheory/:
~/.fieldtheory/bookmarks/
bookmarks.jsonl # raw bookmark cache (one per line)
bookmarks.db # SQLite FTS5 search index
bookmarks-meta.json # sync metadata
oauth-token.json # OAuth token (if using API mode, chmod 600)
~/.fieldtheory/library/
index.md # markdown knowledge base (ft wiki / ft md)
~/.fieldtheory/commands/
*.md # portable commands used by Field Theory and agents
~/.fieldtheory/ideas/
seeds/runs/nodes/ # Possible seeds, runs, and node prompt artifacts
batches/jobs/nightly/ # Multi-repo batches, background jobs, and schedules
Override locations with FT_DATA_DIR, FT_LIBRARY_DIR, and FT_COMMANDS_DIR:
export FT_DATA_DIR=/path/to/custom/dir export FT_LIBRARY_DIR=/path/to/custom/library export FT_COMMANDS_DIR=/path/to/custom/commands
To remove bookmark and Library data: rm -rf ~/.fieldtheory/bookmarks ~/.fieldtheory/library
The CLI is one piece of the Field Theory repo family.
afar1/fieldtheory: Field Theory Mac app source, licensed under AGPL-3.0-or-later.afar1/fieldtheory-cli: this CLI, licensed under MIT.afar1/fieldtheory-plugin: Field Theory Codex plugin and skills, licensed under MIT.afar1/field-releases: packaged Mac app release artifacts and updater metadata only.File app-source issues in the app source repo. Keep binary installer, updater, and release-feed issues out of this CLI repo unless the bug is caused by ft install app or CLI launch behavior.
| Category | What it catches |
|---|---|
| tool | GitHub repos, CLI tools, npm packages, open-source projects |
| security | CVEs, vulnerabilities, exploits, supply chain |
| technique | Tutorials, demos, code patterns, "how I built X" |
| launch | Product launches, announcements, "just shipped" |
| research | ArXiv papers, studies, academic findings |
| opinion | Takes, analysis, commentary, threads |
| commerce | Products, shopping, physical goods |
Use ft classify for LLM-powered classification that catches what regex misses.
In PowerShell, use fieldtheory or ft.cmd instead of ft because ft is already a built-in alias for Format-Table.
If browser session sync cannot find the right profile, pass the browser and profile explicitly:
fieldtheory sync --browser chrome --chrome-profile-directory "Default" fieldtheory sync --browser edge --chrome-profile-directory "Default"
For Firefox, if profile detection misses the profile, pass the profile directory explicitly with --firefox-profile-dir.
If cookie extraction still fails, close the browser completely and retry. As a last resort, pass cookies manually:
fieldtheory sync --cookies <ct0> <auth_token>
Treat ct0 and auth_token like passwords. Do not paste them into logs, issues, or chat.
| Feature | macOS | Linux | Windows |
|---|---|---|---|
Session sync (ft sync) |
Chrome, Chromium, Brave, Edge, Helium, Comet, Dia, Firefox | Chrome, Chromium, Brave, Edge, Firefox | Chrome, Chromium, Brave, Edge, Firefox |
OAuth API sync (ft sync --api) |
Yes | Yes | Yes |
| Search, list, classify, viz, wiki | Yes | Yes | Yes |
Session sync extracts cookies from your browser's local database. Use ft sync --browser <name> to pick a browser. On Windows, Firefox requires Node.js 22.5+ or sqlite3 on PATH. For unsupported browsers or platforms, use ft auth + ft sync --api.
Your data stays local. No telemetry, no analytics, nothing phoned home. The CLI only makes network requests to X's API during sync.
Chrome session sync reads cookies from Chrome's local database, uses them for the sync request, and discards them. Cookies are never stored separately.
OAuth tokens are stored with chmod 600 (owner-only). Treat ~/.fieldtheory/bookmarks/oauth-token.json like a password.
The default sync uses X's internal GraphQL API, the same API that x.com uses in your browser. For the official v2 API, use ft auth + ft sync --api.
Do not open a public issue for suspected vulnerabilities, exposed credentials, auth bypasses, token-handling bugs, or private data exposure. Use the private maintainer contact in SECURITY.md.
By submitting a PR, patch, or contribution, you certify that you have the right to contribute it and agree to license it under the MIT License.
No CLA is required. Maintainers may ask for explicit signoff on larger, sensitive, or ambiguous contributions.
MIT — fieldtheory.dev/cli