Installing Fiber Flow

Technical Document to cover useful pre/post steps when installing Fiber Flow.

Flow is installed using Flow CLI, which is retrieved from the the Bitvis Artifact management system found at https://bitvis.jfrog.io/artifactory.

Installation

It is suggested that you set the credentials for Bitvis Artifactory in your env for ease of use, both in fetching the Flow CLI and logging into docker in later steps.

The credentials are retrieved from Bitvis.

export ARTIFACTORY_USER=your-user-name
export ARTIFACTORY_PASSWORD=IdentityToken
export ARTIFACTORY_URL="https://bitvis.jfrog.io/artifactory"

Use eg. wget to download the latest version of the Flow CLI:

export BRANCH=master
wget --user=$ARTIFACTORY_USER --password=$ARTIFACTORY_PASSWORD \
    https://bitvis.jfrog.io/artifactory/generic-public/flow-cli/$BRANCH/latest.tar.gz

For a specific release of flow-cli, set BRANCH to eg. release/2023.5.0.

Create a folder, unpack the CLI into it, and run the installer.

mkdir flow-cli
tar -xzvf latest.tar.gz -C flow-cli/
cd flow-cli

Custom hooks for the installer

There are four types of hooks that can be used to customize the installation. Hooks are placed in the src/hooks directory in the flow-cli folder. All files in the hooks directory are sourced by the flow script, so if multiple hooks are present in different files, only the last one sourced will be used.

The following types of hooks are available:

prerun_hook The prerun hook is called very early in the installation process, right after the flow script has been launched and before any other actions have been taken.
docker_login_hook The docker login hook is called before the first call to docker during installation and is used to log in to a docker registry if needed.
install_hook The install hook is called right before the install command is executed. It is only called if the command is install.
update_hook The install hook is called right before the update command is executed. It is only called if the command is update.
verify_hook The verify hook is called right after the install or update commands have finished. It is only called if the command is install or update.

Running the installer

Make sure you're logged in to docker before starting the installation, for example using the docker_login_hook.sh.template in the hooks directory, otherwise the installation will fail rather quietly from not being able to pull the images from Artifactory. Rename the file to docker_login_hook.sh and modify it to use your credentials.

Run the installer.

sudo ./flow install 2023.5.0

Configuration example

HOST_IP=172.31.22.50 # IPv4 address of the host
DOCKER_REGISTRY=bitvis.jfrog.io/docker-release # For legacy compat, use install.maintrac.se/release
HOST_TIMEZONE=Europe/Stockholm
COMMON_NAME=ec2-13-50-4-133.eu-north-1.compute.amazonaws.com
CUSTOMER_NAME=releasetest
FLOW_LICENSE=
DATABASE_MASTER_PASSWORD=cwRmD-Sgfw5-rXw6A-aa2fX-VvaFr
DATABASE_HOST=postgresql
DATABASE_INSTALLATION_HOST=172.31.22.50
DATABASE_NAME=flow
DATABASE_USER=flow
DATABASE_PASSWORD=nYKAb-pCXV1-Op9HL-AFFD4-TcF0T
DEEPSTREAM_HOST=deepstream
DEEPSTREAM_USER=flow@maintrac.se
DEEPSTREAM_PASSWORD=otDpc-xKIT2-BTlL0-zgkaU-GxKp7
DEEPSTREAM_ADMIN_USER=support@maintrac.se
DEEPSTREAM_ADMIN_PASSWORD=cpBUh-83iKV-zxISQ-rmS9c-ij7B0
AUTH_SECRET=pzjappKOReGvjxeDH1lEgwsHLH9yTOgwUi3sQYNOQQBz2kuIkziUCTDJ9g5SWeoK
ENCRYPTION_KEY=wwwX7lVNNximL7Gsv0hQHXY2CNwuxskXJpfSbIgpXUxDAf0gWjy72UTIE62vjr3F
TICKET_SECRET=yumAU9fJdOL86wHgadqS2S3kDC6Ui9N1ET0MyYzN48HkOX0SRlc8ZBl1L2JutRWd
ELASTIC_PASSWORD=oVjjy-1o1Uh-cLp8w-cSrdz-gtSqd
PREFIX=
INSTALL_DOCKER=y
INSTALL_OPENVPN=y
INSTALL_PSQL=y
INSTALL_SYSLOG=y
INSTALL_SYSCONF=y
DISABLE_IPV6=y
ENABLE_TRAEFIK=n

A note on database hostnames: The DATABASE_INSTALLATION_HOST is the host used by the installer during installation, updates, backups, etc. DATABASE_HOST is the hostname used in run-time by Flow. If the database is on the same host as Flow, the DATABASE_HOST should be set to postgresql (as available within the Docker Swarm network) and DATABASE_INSTALLATION_HOST should be set to the IP of the host (so that the installer can connect to the database from outside the Docker Swarm).

For external database setups, the DATABASE_INSTALLATION_HOST should be set to the same value as DATABASE_HOST, eg. the IP address of the database server.

The DATABASE_MASTER_PASSWORD is the superuser password for the database and can be configured during installation when eg. an external database is used. The superuser username is always assumed to be postgres.

The Flow database user credentials are set by DATABASE_USER and DATABASE_PASSWORD variables.

Deepstream compatibility

Flow Version	Deepstream Image
2024.1.0 and later	bitvis.jfrog.io/docker-release/bitvis.io-deepstream:2.3.6
2023.5.0 and earlier	deepstreamio/deepstream.io:2.3.6

The flow-cli installer is installing the bitvis.io-deepstream:2.3.6 image by default, since Flow 2024.2.0. For 2024.1.0, a manual upgrade according to the table above is needed. When downgrading to an earlier version of Flow, the deepstream image should be changed to deepstreamio/deepstream.io:2.3.6 using eg. Portainer.

Prometheus alertmanager compatibility

The Prometheus alertmanager project has dropped its support for the v1 API in v0.27.0. Flow releases prior to 2024.3.0 is using the prom/alertmanager:latest image, making it incompatible since the release of prom/alertmanager:v0.27.0. To mitigate this, please pin downgrade the version of alertmanager to prom/alertmanager:v0.26.0. This is resolved in Flow 2024.3.0.

Downgrading Flow

Flow support downgrading to the minor version previously released. Downgrading is performed by running the flow-cli install to the desired version. Database migrations are only done when upgrading, but the minor version prior to a release is always supported with the schema of a latter release.

DEEPSTREAM_USER=flow@maintrac.se
DEEPSTREAM_PASSWORD=otDpc-xKIT2-BTlL0-zgkaU-GxKp7

Traefik

Rate Limiting

This chapter describes how to use Traefik to rate limit a specific URL. To rate limit an entire container, do not add the second router and apply the middleware to the existing router. Values presented will be Key/Value label-pairs. Interpret these as you see fit depending on if you're changing these by CLI, by file or by portainer label settings.

Default as of date the labels for Flow-Auth Docker Service will look something like this:

traefik.enable = true
traefik.http.services.Flow-Auth.loadbalance.servicer.port = 80
traefik.http.routers.Flow-Auth.entrypoints = https
traefik.http.routers.Flow-Auth.rule = PathPrefix(`/api/auth`)
traefik.http.routers.Flow-Auth.tls = true

To enable rate limiting to the login-endpoint, which is recommended for all Flow Installations, add the following labels. Note that the Flow-Auth router could be called something else, and that the new router and the middleware could be called anything.

traefik.http.middleware.Limit-Flow-Auth.ratelimit.period = 1m
traefik.http.middleware.Limit-Flow-Auth.ratelimit.average = 10
traefik.http.middleware.Limit-Flow-Auth.ratelimit.burst = 5
traefik.http.routers.Login-Flow-Auth.entrypoints = https
traefik.http.routers.Login-Flow-Auth.tls = true
traefik.http.routers.Login-Flow-Auth.rule = Path(`/api/auth/login`)
traefik.http.routers.Login-Flow-Auth.middlewares = Limit-Flow-Auth
traefik.http.routers.Login-Flow-Auth.priority = 1
traefik.http.routers.Flow-Auth.priority = 2

Installation​

Custom hooks for the installer​

Running the installer​

Configuration example​

Deepstream compatibility​

Prometheus alertmanager compatibility​

Downgrading Flow​

Login​

Traefik​

Rate Limiting​