iansinnott/full-text-tabs-forever

README

Full Text Tabs Forever

Search everything you read online. FTTF lets you search the full text of every web page you visit.

IMPORTANT FOR v2.0 USERS: If you're upgrading from v1.x, see the Database Migration section for instructions on migrating your existing data.

Firefox requires additional permissions. See below.

Doesn't the browser do that already? How is this different?

Chrome does not let you search the text on pages you've visited, only the URLs and titles, and it deletes your history after a number of months. Firefox will keep your history for longer, but likewise doesn't let you search page content, only URLs and titles.

FTTF is different:

Full-Text Search Capabilities: The full content of every page you've visited becomes searchable.
Permanent History: Your digital footprints are yours to keep. Your data is yours, so it should not be removed without your approval. Nothing is deleted automatically.
Instant indexing: FTTF creates a search index as you browse, so pages are immediately available for searching right after you land on a page.
For your eyes only: Your browsing history is stored locally on your device, and not on any external servers. Beware that if you switch computers your FTTF history will not automatically come with you. It can be exported though.

Who is it for?

Data hoarders like myself that never want to delete anything, and want everything to be searchable. More generally, if you've ever felt limited by the standard history search you should try this out.

How it works:

Browser extensions have access to the pages you visit, which lets FTTF make an index of the content on any page. When a page loads, its content is extracted and indexed.

Extracted? Yes, or "distilled" if you prefer. Full web pages are huge and have a lot of information that's not related to the content itself. FTTF will ignore all of that. It acts like "reader mode" to find relevant content on a page and only index that.

Installation

Install in your browser via the Chrome Web Store or the Firefox Add-ons Store.

Testing

This project uses bun as a unit testing framework, but not (currently) as a bundler. You will need to install bun, then:

bun test

Or, bun run test if you prefer.

Note to self: Submitting a new version manually

How could this be automated?

Manually bump the version in the manifest file
Run the build
- bun run build:chrome
- bun run build:firefox
Submit
- Chrome
  - Go to: https://chrome.google.com/webstore/devconsole/bc898ad5-018e-4774-b9ab-c4bef7b7f92b/gfmbnlbnapjmffgcnbopfgmflmlfghel/edit/package
  - Upload the fttf-chrome.zip file
- Firefox
  - Go to: https://addons.mozilla.org/en-US/developers/addon/full-text-tabs-forever/edit
  - Upload the fttf-firefox.zip file
  - Zip the original source code and upload that too: zip -r src.zip src

Firefox

Install here: https://addons.mozilla.org/en-US/firefox/addon/full-text-tabs-forever/

Currently you have to manually enable additional permissions in Firefox like so:

See this comment for more details: #3 (comment)

Support was added in: #4.

Database Migration (v2.0)

With version 2.0, Full Text Tabs Forever has migrated from SQLite (VLCN) to PostgreSQL (PgLite) as its database backend. This change brings several improvements:

Better full-text search capabilities with PostgreSQL's advanced text search
Support for vector embeddings for semantic search (coming soon)
Improved performance for large databases
More efficient storage of document fragments

For Existing Users

If you're upgrading from a previous version (v1.x), your data will not be lost! The extension includes a migration system that will:

Detect your existing VLCN (SQLite) database
Provide a simple one-click migration option in the Settings page
Transfer all your saved pages to the new PostgreSQL database
Show real-time progress during migration
Preserve all your searchable content

To migrate your data:

After upgrading, open the extension
Go to the Settings page
Find the "Import VLCN Database (v1)" section
Click the "Import VLCN Database" button
Wait for the migration to complete - this may take several minutes depending on how many pages you've saved
Your data is now accessible in the new database system!

The migration happens entirely on your device, and no data is sent to external servers. Your privacy remains protected throughout the process.

TODO

Backfill history Currently only new pages you visit are indexed, but we could backfill by opening every page in the browser's history that hasn't yet been indexed. An optional feature, but a useful one.
Backup and sync Improved export/import capabilities for moving data between devices.
Semantic search Leverage vector embeddings in the new PostgreSQL backend for more intelligent searching.
Integrate with browser-gopher Browser gopher and BrowserParrot were the initial impetus to create a better way to ingest full text web pages, without triggering a Cloudflare captcha party on your home connection.
Migrate to PostgreSQL Replace SQLite with a more powerful database backend using PgLite.
Improve discoverability of functionality. There is now a button to open the command palette. Still not much GUI, but enough to be discovered.
Firefox ~~This should not be too difficult since this project was started with web extension polyfills. However, there is currently some chrome specific code.~~ It appears that the APIs do not have to be rewritten to work in Firefox. See this PR for details: #4

Contributing

PRs welcome!