gildas-lormeau/single-file-cli

README

SingleFile CLI (Command Line Interface)

Introduction

SingleFile can be launched from the command line by running it into a (headless) browser.

It runs through Deno as a standalone script injected into the web page via the Chrome DevTools Protocol, or via WebDriver BiDi when using Firefox, instead of being embedded into a WebExtension.

Installation

SingleFile can be run without installing it, just download the executable file and save it in the directory of your choice here: https://github.com/gildas-lormeau/single-file-cli/releases

Make sure Chrome or a Chromium-based browser is installed in the default folder. Installations for the current user are also found (%LOCALAPPDATA% on Windows, ~/Applications on macOS, snap and the PATH on Linux). Otherwise you might need to set the --browser-executable-path option to help SingleFile locating the path of the executable file. Firefox can be used instead by setting the --browser-engine firefox option, with some limitations described in the help. The Flatpak version of Firefox needs access to /tmp, where the temporary profile is created. Allow it with flatpak override --user --filesystem=/tmp org.mozilla.firefox and set --browser-executable-path to /var/lib/flatpak/exports/bin/org.mozilla.firefox, or to ~/.local/share/flatpak/exports/bin/org.mozilla.firefox for a per-user installation.

Installation with Docker

Installation from Docker Hub

docker pull capsulecode/singlefile

docker tag capsulecode/singlefile singlefile
Manual installation

git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git

cd single-file-cli

docker build --no-cache -t singlefile .
Run

docker run singlefile "https://www.wikipedia.org"
Run with Firefox

docker run singlefile "https://www.wikipedia.org" --browser-engine firefox
Run and redirect the result into a file

docker run singlefile "https://www.wikipedia.org" > wikipedia.html
Run and mount a volume to get the saved file in the current directory
- Save one page
  
  docker run -v %cd%:/usr/src/app/out singlefile "https://www.wikipedia.org" wikipedia.html (Windows)
  
  docker run -v $(pwd):/usr/src/app/out singlefile "https://www.wikipedia.org" wikipedia.html (Linux/UNIX)
- Save one or multiple pages by using the filename template (see --filename-template option)
  
  docker run -v %cd%:/usr/src/app/out singlefile "https://www.wikipedia.org" --dump-content=false (Windows)
  
  docker run -v $(pwd):/usr/src/app/out singlefile "https://www.wikipedia.org" --dump-content=false (Linux/UNIX)
An alternative docker file can be found here https://github.com/screenbreak/SingleFile-dockerized. It allows you to save pages from the command line interface or through an HTTP server.

Manual installation

Install Deno
There are 3 ways to download the code of SingleFile, choose the one you prefer:
- Install with npm and run single-file via npx (npm and npx are installed with Node.js)
```
npm install "single-file-cli"
npx single-file ...
```
  You can also install SingleFile globally with -g when running npm install.
- Download and unzip manually the main archive provided by Github
```
unzip main.zip .
cd single-file-cli-main
```
- Download with git
```
git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git
cd single-file-cli
```
Make single-file executable (Linux/Unix/BSD etc.).
```
chmod +x single-file
```

Run

Syntax

single-file <url> [output] [options ...]

Display help
```
single-file --help
```
Record what was observed while saving, for later verification: see MANIFEST.md
```
single-file https://www.wikipedia.org wikipedia.html --manifest
```

Examples

Dump the HTML content of https://www.wikipedia.org into the console

single-file https://www.wikipedia.org --dump-content

Save https://www.wikipedia.org into wikipedia.html in the current folder

single-file https://www.wikipedia.org wikipedia.html

Save a page needing a logged-in session. Create the profile first, log in to the website in the browser window, and quit the browser to save it. Then reuse the profile as many times as needed, it is copied and left unmodified.

single-file --create-browser-profile ./profiles/example https://www.example.com/login
single-file https://www.example.com/account account.html --browser-profile=./profiles/example

Save https://www.wikipedia.org with Firefox

single-file https://www.wikipedia.org wikipedia.html --browser-engine firefox

Save a list of URLs stored into list-urls.txt in the current folder

single-file --urls-file=list-urls.txt

Save https://www.wikipedia.org and crawl its internal links with the query parameters removed from the URL

single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"

Save https://www.wikipedia.org and external links only

single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=false --crawl-external-links-max-depth=1 --crawl-rewrite-rule="^.*wikipedia.*$"

Save http://info.cern.ch (i.e. the first website) and its internal links into a single self-extracting ZIP file, and mark the links to pages missing from the archive

single-file http://info.cern.ch first-website.html --crawl-links=true --crawl-max-depth=2 --crawl-save-archive=true --crawl-save-archive-mark-unarchived-links=true --compress-content=true

Exit code

The exit code is 0 when all the pages have been saved, 1 when at least one capture failed (the other pages of the batch or the crawl are still saved), and 255 when an error prevented the process from running.

Compile executables

Compile executables into /dist

./compile.sh

License

SingleFile and SingleFile CLI are licensed under AGPL. Code derived from third-party projects is licensed under MIT. Please contact me at gildas.lormeau <at> gmail.com if you are interested in licensing the SingleFile code for a commercial service or product.