gem-hunter
Language: Go
Expose shellscripts, postgres queries, redis commands and other as rest server endpoints
Goexpose is lightweight json api server that maps url path to various tasks. Goexpose can be used in various scenarios: either make call commands on your servers (or farm of servers), or you can use it as monitoring tool. Builtin tasks are currently:
I have a plan to implement other tasks with support for: memcache, mongodb, sqlite, file..
All these commands can accepts variables from route (gorilla mux is used). GOexpose has system for authorization, currently basic (username password) is implemented. In the future more types of authorization will be implemented.
Lets see example configuration file:
{
"host": "127.0.0.1",
"port": 9900,
"ssl": {
"cert": "./cert.pem",
"key": "./key.pem"
},
"reload_env": true,
"endpoints": [{
"path": "/info",
"authorizers": ["basic"],
"methods": {
"GET": {
"type": "info",
"description": "Info task"
}
}
}],
"authorizers": {
"basic": {
"type": "basic",
"config": {
"username": "hello",
"password": "world"
}
}
}
}This means that Goexpose will listen on https://127.0.0.1:9900 "endpoints" is a list of defined endpoints that goexpose responds to.
You can also write your configuration in yaml format(command line arg -format)
Run go install
go install github.com/phonkee/goexpose
Goexpose provides various variables from url, query, request. This data is available in commands to interpolate various strings. text/template is used and available data is in this structure:
{
"url": {},
"query": {},
"request": {
"method": "",
"body": ""
},
"env": {}
}Goexpose has support to use query parameters. But you have to configure all params that you want to use. Goexpose gives you possibility to validate params values with regular expressions and provide default value. You can provide "query_params" on two levels. You can specify them on endpoint level and also on task level.
Configuration:
{
"query_params": {
"return_params": true,
"params": [{
"name": "page",
"regexp": "^[0-9]+$",
"default": "0"
}, {
"name": "limit",
"regexp": "^[0-9]+$",
"default": "10"
}]
}
}http task and shell task have possibility to set format of response. Currently available formats are: "json", "jsonlines", "lines", "text". Format can be combination of multiple formats. e.g.
"format": "json|jsonlines"
First format that returns result without error will be used. If "text" is not found in format, it is automatically inserted to the end.
Tasks can be configured in config["methods"] which is a map[string]TaskConfig - http method to task. Every task config has common part and configuration for given task. Common configuration is:
{
"type": "http",
"authorizers": [],
"config": {},
"query_params": {
"params": [{
"name": "id",
"regexp": "^[0-9]+$",
"default": "0"
}],
"return_params": true
}
}Http task is task that can do external request. Task configuration is following:
{
"type": "http",
"config": {
"single_result": 0,
"urls": [{
"url": "http://127.0.0.1:8000/{{.url.id}}",
"post_body": false,
"format": "json",
"return_headers": false
}, {
"url": "http://127.0.0.1:8000/{{.url.id}}",
"method": "PUT",
"post_body": false,
"format": "json",
"return_headers": false,
"post_body": true,
}]
}
}Configuration:
ShellTask is task that is able to run shell commands on target server. Every command is interpolated (see Interpolation)
Warning!!! Use appropriate regular expressions for inputs so you don't expose your server to shell injection. Example:
{
"type": "shell",
"config": {
"env": {
"key": "value"
},
"shell": "/bin/bash",
"commands": [{
"command": "echo \"{{.url.id}}\"",
"chdir": "/tmp",
"format": "json",
"return_command": true
}]
}
}Configuration:
Info task returns information about goexpose. In result you can find version of goexpose and also all registered tasks with info. Task info has no configuration.
Run queries on postgres database. Configuration for postgres task:
{
"type": "postgres",
"config": {
"return_queries": true,
"queries": [{
"url": "postgres://username:password@localhost/database",
"query": "SELECT * FROM product WHERE id = $1",
"args": [
"{{.url.id}}"
]
}]
}
}Configuration:
Task that can run multiple commands on redis. Example:
{
"type": "redis",
"config": {
"address": "127.0.0.1:6379",
"network": "tcp",
"database": 1,
"return_queries": true,
"queries": [{
"command": "GET",
"args": [
"product:{{.url.id}}"
],
"type": "string"
}]
}
}Configuration:
Run cassandra queries task. Example:
{
"type": "cassandra",
"config": {
"return_queries": true,
"queries": [{
"query": "SELECT * from user WHERE id = ?",
"args": [
"{{.url.id}}"
],
"cluster": [
"192.168.1.1",
"192.168.1.2"
],
"keyspace": "keyspace"
}]
}
}Configuration:
Run mysql queries. Example:
{
"type": "mysql",
"config": {
"return_queries": true,
"queries": [{
"url": "user:password@localhost/dbname",
"query": "SELECT * FROM auth_user WHERE id = ?",
"args": [
"{{.url.id}}"
]
}]
}
}Configuration:
Multi task gives possibility to run multiple tasks in one task. These task can be any tasks (except of embedded multi task).
{
"type": "multi",
"config": {
"single_result": 0,
"tasks": [{
"type": "http",
"config": {
"single_result": 0,
"urls": [{
"url": "http://www.google.com"
}]
}
}]
}Configuration:
Filesystem task is simple yet powerful task. It can be configured to serve single file or serve all files in directory with optional index page for directories.
In following example we serve only one file on url /file/some. The output will be json with base encoded file content.
{
"path": "/file/some",
"methods": {
"GET": {
"type": "filesystem",
"config": {
"file": "/tmp/file"
}
}
}
}In next example we will serve files in directory and provide index page for directories and also give possibility to return raw file as response.
{
"path": "/static/{file:.+}",
"methods": {
"GET": {
"query_params": {
"params": [{
"name": "output",
"regexp": "^raw$",
"default": ""
}],
},
"config": {
"file": "{{.url.file}}",
"output": "{{.query.output}}",
"directory": "/tmp",
"index": true
}
}
}
}Configuration:
Types of authentication ( I know it's silly name..) First you have to define your authorizers in top level "authorizers" and then you can use them in your tasks defined by name. e.g.:
{
"endpoints": [{
"path": "/info",
"authorizers": ["username_pass"],
"methods": {
"GET": {
"type": "info",
}
}
}],
"authorizers": {
"username_pass": {
"type": "basic",
"config": {
"username": "hello",
"password": "world"
}
}
}
}You can set your authorizers in endpoint configuration, or you can set in every task for fine tuned configuration.
Support for basic authentication.
{
"type": "basic",
"config": {
"username": "hello",
"password": "world"
}
}Support for LDAP authentication.
{
"type": "ldap",
"config": {
"host": "localhost",
"port": 1234,
"network": "tls"
}
}Configuration:
localhost)389)tcp, tls (default tcp)@TODO - add tls support for certificates
Support to call external web services for authentication (such as rest).
Request to goexpose is allowed when the http call returns status code 200. Otherwise Unauthorized is raised.
{
"type": "http",
"config": {
"url": "http://localhost:8080/api/users/login",
"data": "{\"username\": \"{{.username}}\", \"password\": \"{{.password}}\"}",
"method": "post"
}
}Configuration:
in folder example/ there is complete example for couple of tasks. You can find example here! or yaml!
@TODO: Add tasks for: sqlite, memcached, mongodb
phonkee