Overview

Before we dive into the code, lets talk about the abstract concepts.

What Is SHADY-STACK?

As the name implies, this is a less than kosher web stack based of the SWAG stack. I developed it for my projects as I am in university and web hosting can be expensive. The shady stack consists of the following:

1. A S tatic site hosted (GitHub Pages, GitLab Pages, etc.)

2. Simplistic pages that communicate to the backend via a already existing web-app’s Web H ooks (Slack, Discord, Gsuit, GitHub Actions, etc.).

3. An A plication Bridge program that can read the calls to the webhooks and pass them to the next component.

4. A Backend D eamon that acutely presses the requests from the webhooks and updates a local copy of the site tree as needed.

5. An application that regularly s Y ncs the local tree with the remote tree served as the static site in step 1.

What Is This?

In this webpage you will find the documentation for using the SHADY-STACK.

Note

This sample documentation was generated on Mar 15, 2023, and is rebuilt with each release.

Is SHADY-STACK Right For You?

There are some ups and downs to using this stack.

Pros	Cons
Backend is hidden behind a proxy of sorts.	Failure or delay in the webhook provider means failure/delay in your site.
It’s free!	One must be carefull not to violate any TOS when using certian webhook providers.
It’s kinda fail-safe. If the backend breaks, the static parts of your site will still work.	The number of components between your frontend and backend means it’s slow to update.

Remember to consult your physician to see if SHADY-STACK is right for you.

Design Philosophy

With the above in mind. The code for this project will follow these goals:

1. Secure defaults - It may be shady, but let’s still not get pwned…

2. Dead simple to set up - When using shady, time from project design to working product so be as fast as possible.

3. Minimize the amount of JS needed in a project - I hate writing it.

(These are mostly for me to keep in mind while developing.)

What does SHADY-STACK Provide?

You may be asking yourself, what tooling exists for this SHADY-STACK? Well, while anyone is invited to build on these concepts, the SHADY-STACK repo provides the following:

• Bridge Applications For:

  • Discord

• A Default API Demon

• Default Hooks For Syncing Via Git

Each of these parts are explained in their respective sections.

Install

The resources provided by the SHADY-STACK repo are in the form of a python model of the name shadybackend.

Currently, the default demon and a specified bridge can be run as a nix flake with:

nix run github:user-1103/shady-stack - <args>
# To enter a development environment use:
nix develop github:user-1103/shady-stack

For those of you who have yet to see the glory of nix the package can be installed as a poetry project.

# Clone the repo.
git clone https://github.com/user-1103/shady-stack
# Enter the repo:
cd shady-stack
# Install with poetry_
poetry install
# Run the backend components
shadybackend <args>

If you have something against poetry, you can install the dependencies your self (found in pyproject.toml). And run:

python3 top_level.py <args>

Note

Need to set up a pip package…

Shady Backend

The shadybackend python module is it the collection of resources provided by the repo. The Install section outlines how to install and run the module from the command line, subsequently allowing for access to all the parts provided by the repo. To progamaticly use the module, see Pragmatic Use.

CLI Args

No matter how you call shady, the following is roughly how it will be used.

<command to call shady> '<name of bridge>' '<JSON describing the initial G value>'

See default bridges and `G`_ for more info.

Optional Args

Additionally, one may use the following:

-h, --help

show this help message and exit

-v, --verbose

Be chatty.

--tree TREE

Provide the location of the web root.

--api API

Provide the location of the api.py file.

The Default API Demon (DAD)

DAD is an extremely bare-bones Application Demon. When shadybackend is run, it will be started. When run, DAD first looks for user defined APIs to load in from api.py in the current directory.

Defining APIs

An api.py file is a normal python file where you can define how your API.

Take the sample below:

api.py

from shadybackend.api_tools import define_api

@define_api({"example_arg": "default_value", "required_arg_": 1})
def example_api(G, ARGS):
    ... # Do some api stuff

Let’s break down these lines one at a time:

1. Here we import the wrapper function for defining an API call.

2. A little space to breath.

3. We use the wrapper to define the `baseline arguments`_ for the API.

4. We declare a function that will do the actual backend processing of the API. The name of the API call is determined by the name of the function and is passed G_, and ARGS_.

API baselines are one of the default security features of SHADY-STACK. The baseline for an API exists as a dictionary keyed by strings representing the name of the argument. The values represent the default values for the argument if none is provided when the API is called. Before the API function is called, the types of each supplied argument will be compared to the type of the default value. If they do not match, the API will not be called. If the name of an argument ends with a _, it will be marked as required. If a required arg is missing when an API call is requested, the API will not be called. All arguments provided that are not found in the baseline will be silently dropped before being called. Internal the default JSON lib is used and as such, only types parsed by it can be used in the baseline.

When an API is called, it has access to a global dictionary named G a la flask. The data in this variable is shared across all API calls, hooks, and Application Bridges.

Note

A feature that is being considered is to save the state of G across runs of the backend. For now though, the state is purged on shutdown.

By default, DAD sets the following G_ variables:

Default bridges may define further values.

Warning

It is probably best to not touch Q or req unless you know what you are doing. The Requests stored in these variables are not sanitized.

After the above parsing is done to the arguments provided to the webhook, they are then provided to the API function via the ARGS_ variable.

HOOKS

DAD provides a way to ensure certain actions happen when certain events happen during the execution of DAD. Take the following addition to our api.py file:

api.py

from shadybackend.api_tools import define_api, define_hook, HookTypes
import time

@define_api({"example_arg": "default_value", "required_arg_": 1})
def example_api(G, ARGS):
    ... # Do some api stuff

@define_hook(HookTypes.ERR)
def log_error(G):
    with open("error.log", "a") as f:
        f.write(f"Failed to process request @{time.time()}\n")

This trivial example writes a log file every time DAD fails to process a request. The available events you can hook are:

Name	Runs On…
HookTypes.ERR	…a failure in processing a request.
HookTypes.INIT	…demon startup.
HookTypes.PRE	…the start of processing a request
HookTypes.OK	…the successful processing of a request
HookTypes.POST	…the end of the processing of a request
HookTypes.EXIT	…demon safe exit.
HookTypes.FATAL	…demon hitting an unliveable error.
HookTypes.WAIT	…demon idle.

Hint

You can use the OK hook to sync your local copy of the site to the remote static site.

Default Bridges

The SHADY-STACK repo provides a bunch of default bridges that can be activated with DAD.

Call Structure

When the user using the static front end of your site needs to do something dynamic, the JS in the page should call the webhook watched by the Application Bridge. In this call two things need to be passed the name of the API call to use, and the arguments to pass to the API call. The bridges defined in the repo expects the following data fields as a minimum:

Name	Description
api_call	The name of the api to use for the request.
data	A dictionary of args to pass to the request.

Discord Bridge

The discord bridge can be used by requesting the discord from DAD. First though, you need to set up a new discord server, a webhook for that server (save this for latter), and then register a new bot.

Warning

As of the most recent discord API update, you will need to give your bot the message content intent for shady to work.

When you register a new bot, you will get a token that will allow the discord bridge to act as your newly created bot. You will need to pass this token through the G variable like so:

python3 shadybackend "discord" '{"discord_token": "<bot token>"}'

This tells shady backend to start DAD with the discord bridge, and to connect to the bot token provided. You can test your webhook with the following commands:

test_hook.sh

export WEBHOOK_URL="<your web hook url>"
export SHADY_MSG='{"api_call":"example_api_call","data":{"example_arg":"foobar"}}'
export WEBHOOK_MSG="{\"username\": \"test\", \"content\": \"$SHADY_MSG\"}"
curl -H "Content-Type: application/json" -d $WEBHOOK_MSG $WEBHOOK_URL

Note

Note how the shady Call Structure is wrapped by the discord call structure.

Hacking

If you intend to modify the code provided in the repo, the following should be helpful.

Pragmatic Use

The code is divided into a few logical modules, the first of which to consider is top_level.py:

Serves as the entry point for runing the SHADY backend.

shadybackend.top_level.on_exit(signum, stack) → None

Calls the redigested EXIT hooks on kill signal

shadybackend.top_level.run() → None

Runs the CLI parser and then starts the top level.

shadybackend.top_level.run_top_level(start_g: Dict[str, Any], bridge: str, root: str = './tree', api: str = './api.py') → None

Starts the provided hook bridge and runs the default demon.

Args start_g:

JSON string that will be loaded as the default global

dictionary. :args bridge: The bridge app to use. See the bridge module for options. :args root: The path as a string to the root of the web tree. :args api: The path as a string to the api to be run.

The above code primaraly loads the requested bridge from bridges.py and starts DAD (located in demon.py. These two modules are documented below:

A central module to look up the available backends.

class shadybackend.bridges.DiscordBridge

This is just the example code from: https://discordpy.readthedocs.io/en/stable/quickstart.html

build_bridge(g: Dict[str, Any]) → None

name = 'discord'

This is the default API demon that is used for the shadybackend.

shadybackend.demon.run_api_demon(root: Path, api_paths: List[Path]) → None

Run the app, with self restarting on recoverable errors.

Args root:

The root of the web tree.

Args api_paths:

The paths where API definitions are found.

Finally, shady backend also provides two utility classes to deal with the two most common data types. The documentation for request_tools.py and api_tools.py follows:

A module to define the Request type.

exception shadybackend.request_tools.BadRequest

Used when a request is received that does not folow the API baseline.

class shadybackend.request_tools.BaselineValue(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

NOT_FOUND = 1

class shadybackend.request_tools.Request(api_call: str, data: Dict[str, Any], shady_request_version: int = 1, _type: str = 'ἐπιούσιον')

Represents a request to a SHADY backend. May be warped in other json as may be needed for the webhook provider to work.

_type: str = 'ἐπιούσιον'

api_call: str

data: Dict[str, Any]

sanitize(baseline: Dict[str, Any]) → Dict[str, Any]

Takes the data element and the provied basline and checks that for every element in the basline, there is a matching attribute at the same location, name, and type in the data element. If these conditions are matched it will be added to the output dictionary. If the name and location exists in the basline but not the data element, and the name does not end in _ (IE it is required), then the value in the basline will be used as a default. If it is required, then the BadRequest Error will be thrown. In all other cases, if there are extra names / locations in the data element, they will be silently dropped.

Args baseline:

The baseline of the API to use (will be coppied) for

the return value. :return: The sanitized baseline

shady_request_version: int = 1

shadybackend.request_tools._check_required(tmp: Dict[str, Any]) → List[str]

Checks to see if there are any values with a name ending in _ (IE it is required) in tmp and returns a list of attribute names if any are found.

Args tmp:

The data struck to search through

Returns:

A list of attributes with names that end in _

shadybackend.request_tools._update(data: Dict[str, Any], baseline: Dict[str, Any]) → List[Tuple[str, Type, Type]]

Takes baseline and updates any shared attributes with the contents of data. If the types do not match in the shared attributes, baseline will not be updated. In staid, a tuple of (attribute name, expected, got) will be added to the list that is returned.

Args data:

The arguments to pass to the API

Args baseline:

The default args of the API

Returns:

A list of missmatched types

Module for handeling the definition and execution of the API.

class shadybackend.api_tools.API(name: str, call_function: Callable, baseline: Dict[str, Any])

Represents an api call.

baseline: Dict[str, Any]

call_function: Callable

name: str

class shadybackend.api_tools.HookTypes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

The types of hooks that are understood by the system.

ERR = 5

EXIT = 2

FATAL = 7

INIT = 1

OK = 8

POST = 4

PRE = 3

WAIT = 6

exception shadybackend.api_tools.MalformedAPI

Used when the defined api can not be loaded.

shadybackend.api_tools.call_hooks(hook: HookTypes) → None

Calls all hooks registered with a given name.

Args hook:

The type of the hook in it.

shadybackend.api_tools.collect_apis(path: Path) → None

Loads the API objects described in the given file.

Args path:

The path to load the ‘api.py’ from.

shadybackend.api_tools.define_hook(hook: HookTypes) → Callable

Wrapper that registers a hook for the given hook type.

Args hook:

The hook type to bind two

shadybackend.api_tools.defnine_API(baseline: Dict[str, Any]) → Callable

Warper function to define an api call.

Args call_function:

The function to wrap as an API

shadybackend.api_tools.process_request(request: Request) → None

Takes a request, finds the necessary api and process the given data with said api.

Args request:

The request object to process.

Contributions

Pull requests are always welcome. Features can be requested on the repo issue page. Issues should also be reported there.

License

Everyone is welcome to build on the shady stack / swag stack concepts as they wish. The code located in the repo (https://github.com/user-1103/shady-stack) however is licensed under the following terms:

1. By using the code you irrevocably acknowledge that:

   • Trans rights are human rights.

   • The sovereign nations of Tibet and Hong Kong are being unjustly occupied by People’s Republic of China.

   • The software provided in the repo and any related services are provided on an “as is” and “as available” basis, without warranty of any kind, whether written or oral, express or implied.

2. For all issues not covered in the above bullet point, the code in repo is to be considered as licensed under GNU Affero General Public License v3.0 (https://choosealicense.com/licenses/agpl-3.0/).

By using and / or copying the code in the repo you acknowledge that you understand these rules and will abide by them.

Acknowledgments

This project would not have been possible without the following people.

• Rapptz’s discord.py

Change Log

V0.1.0

Initial release.

TODO

• PYPI package

• Tests

• Bridges for:

  • Slack

  • GitHub Actions

  • FIFO

• Utility bindings for:

  • Github

Indices and tables

• Index

• Module Index

• Search Page