gem-hunterThe Steampipe foreign data wrapper (FDW) is a zero-ETL product that provides Postgres foreign tables which translate queries into API calls to cloud services and APIs. It's bundled with Steampipe and also available as a set of standalone extensions for use in your own Postgres database.
The Steampipe Postgres Foreign Data Wrapper (FDW) is a Postgres extension that translates APIs to foreign tables. It does not directly interface with external systems, but instead relies on plugins to implement API- or provider-specific code that returns data in a standard format via gRPC. See the Writing Plugins guide to get started writing Steampipe plugins.
The FDW is part of the Steampipe project. Bundled with the Steampipe CLI, it works with one or more of the plugins you install in Steampipe. You can also install one or more plugin-specific extensions in your own instance of Postgres.
To use the FDW with Steampipe, download Steampipe and use it to install one or more plugins.
You can also use a standalone installer that enables you to choose a plugin and download the FDW for that plugin.
Make sure that you have the following installed in your system:
Postgresql v14gogcc for LinuxFor instructions on how to install PostgreSQL, please visit: https://www.postgresql.org/download/
For instruction on how to install
golang, please visit: https://go.dev/dl/
Steps:
$ make install
This will compile the FDW (steampipe_postgres_fdw.so) along with the control and sql file in the build-$PLATFORM directory. This will install the compiled FDW into the default Steampipe installation directory (~/.steampipe) - if it exists.
To build the FDW for one particular plugin, and run it as a standalone extension in any PostgreSQL database without relying on Steampipe:
Make sure that you have the following installed in your system:
Postgresql v14gogcc for LinuxSteps:
$ make standalone plugin="<plugin alias>"
Replace plugin alias with the alias or short name of your plugin.
This command will compile the FDW specifically for the chosen plugin, and the resulting binary, control file, and SQL files will be generated.
Suppose you want to build the FDW for a plugin with an alias aws from a GitHub repository located at https://github.com/turbot/steampipe-plugin-aws. You would run the following command:
$ make standalone plugin="aws"
Suppose you want to build the FDW for your own plugin(not maintained by Turbot) located at https://github.com/francois2metz/steampipe-plugin-scalingo. You would need to build the FDW by running the following command:
$ make standalone plugin="scalingo" plugin_github_url="github.com/francois2metz/steampipe-plugin-scalingo"
Once you have built the standalone FDW, the binaries will be available in a folder build-Darwin or build-Linux depending on your OS. Run the install.sh script available in that directory. This will detect the installed PostrgeSQL version and location and copy the binaries there.
➜ steampipe-postgres-fdw ✗ cd build-Darwin
➜ build-Darwin ✗ ./install.sh
Discovered:
- PostgreSQL version: 14
- PostgreSQL location: /opt/homebrew/Cellar/postgresql@14/14.13_1
Install Steampipe PostgreSQL FDW for version 14 in /opt/homebrew/Cellar/postgresql@14/14.13_1? (Y/n):
Installing...
Successfully installed steampipe_postgres_scalingo extension!
Files have been copied to:
- Library directory: /opt/homebrew/lib/postgresql@14
- Extension directory: /opt/homebrew/share/postgresql@14/extension/
This repository is published under the Apache 2.0 license. Please see our code of conduct. We look forward to collaborating with you!
Steampipe is a product produced exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.