Lotus Farcaster

The visualization and analytics tool for Lotus Filecoin node.

Version 2 is out now ! For V1 users upgrading to V2:
- Don't forget to update Grafana to version 7.4.0
- Reimport the new Grafana Dashboard
- Install the new lotus-exporter-farcaster (instructions below)



  • Advanced analytics: Like average sealing time, historical
  • Realtime monitoring: Visualize all the different aspect of your miner at a glance
  • Designed to replace your terminal monitoring
  • Farcaster is developped in cooperation with Protocol Labs.
  • Leverages Prometheus, Grafana and Python.

Dashboard Features

Feature Implementation Comments
Sectors
Mpool
Storage Info
Workers Info
Sealing
Power
Wallets
Chain
Deadlines
Deals Partial
Fil+ Visualize granted datacap
Address lookup View friendly names instead of boring addresses

Architecture

 --------------------------          ------------          ---------------------          ------------
| Lotus Miner              | <----- | Prometheus | <----- | Grafana             | <----- | Web Client |
| lotus-exporter-farcaster |         ------------         | farcaster-dashboard |         ------------
 --------------------------                                ---------------------
                                      
Lotus farcaster comes with 2 Components :
  • A Grafana Dashboard
  • Lotus-exporter-farcaster : A standalone and configuration-less python script executed every minyte by crontab

Security

The design and choice of these components, was made to keep a high level of security :
  • metrics are pulled from the miner to avoid compromising network or management servers
  • lotus-exporter-farcaster doesn’t bind any ports to avoid exposing the miner to external threat
  • lotus-exporter-farcaster run under unprivileged user (ideally lotus user) to avoid system being compromised
  • The global solution doesn’t rely on any cloud or remote component. To avoid data leak.
lotus-exporter-farcaster is running locally on the miner without any specific privilege nor access to other servers.

It will follow prometheus guidelines :
  • configuration-less
  • ressources inexpensive
  • standard prometheus metrics naming
  • compatible with 3rd parties dashboards and exporters

Benefits

  • Easy to deploy
  • Configuration less
  • Small footprint
  • Collect lotus node and miner data
  • Only rely on API
  • Data are pulled from the Prometheus (increase security)
  • Deploy on the miner node only
  • Run under Unprivileged user

Tutorials

Deep dive in Lotus farcaster features


Step by step installation tutorial


Installation

Requirements

A Grafana / Prometheus working environement (local or remote)

Locally installed on the miner node only

  • prometheus-node-exporter
  • python3-toml
  • python3-aiohttp
  • py-multibase
  • lotus-exporter-farcaster

Tested environment

  • Grafana : 7.4.0
  • Prometheus : 2.20.1
  • Ubuntu : 20.04.1 LT
  • Python : 3.8

Install lotus-exporter-farcaster - Standalone

                                  git clone https://github.com/s0nik42/lotus-farcaster.git
                                  cd lotus-farcaster/lotus-exporter-farcaster
                                  chmod +x install.sh
                                  ./install.sh LOTUS_USER_USERNAME
                                

Install lotus-exporter-farcaster - Docker (alternative)

Farcaster can also run as a Docker container.

The container that corresponds to this repository will run the `docker_run_script.sh` script which just loops over calling lotus-farcaster code at a specific frequency (default: every minute)

This can be overriden by editing the Dockerfile. In case execution exceed the set requency, the execution restart after 10 seconds.

The output of the lotus-farcaster is written to `/data/farcaster.prom` inside the container which should be a bind mount in prometheus node exporter path.

Optional : This could be use in conjunction with a dockerised version of prometheus node_exporter (instructions to get the node_exporter container going


Building the container (as root)
                                  apt install docker.io
                                  docker build -t lotus-farcaster:latest -f dockerfiles/Dockerfile .
                                

Running the container (as root)

Set the 3 variables below (LOTUS_PATH, LOTUS_MINER_PATH, PROMETHEUS_NODE_EXPORTER_PATH), accordingly to your setup and simply copy paste the run command below.

                                  export LOTUS_PATH="/opt/lotus/.lotus/"
                                  export LOTUS_MINER_PATH="/opt/lotus/.lotusminer/"
                                  export LOTUS_FARCASTER_PATH="/opt/lotus/.lotus-exporter-farcaster/"
                                  export PROMETHEUS_NODE_EXPORTER_PATH="/var/lib/prometheus/node-exporter/"

                                  docker run --name lotus-farcaster -d \
                                    --mount type=bind,source=$LOTUS_PATH,target=/root/.lotus,readonly \
                                    --mount type=bind,source=$LOTUS_MINER_PATH,target=/root/.lotusminer,readonly \
                                    --mount type=bind,source=$LOTUS_FARCASTER_PATH,target=/root/.lotus-exporter-farcaster,readonly \
                                    --mount type=bind,source=$PROMETHEUS_NODE_EXPORTER_PATH,target=/data \
                                    --network=host \
                                    lotus-farcaster
                                

Docker Debug (as root)
                                  docker ps [-a]
                                  docker logs lotus-farcaster
                                  docker exec -it lotus-farcaster bash
                                

Uninstall docker (as root)
                                  docker stop lotus-farcaster
                                  docker rm lotus-farcaster
                                  docker image rm lotus-farcaster
                                

Install the Grafana Dashboard

Import in Grafana the relevent dashboard file from ./lotus-farcaster/grafana-dashboard

Contributing

If you're interested in contributing to the Grafana project:

  • Translate : help us by translating our dashboard to other languages.
  • Panels : submit your customized panels
  • Donate : Filcoin Address : f3v3lj5jrsvv3nwmsvvj57yyty6ndb27oyi4yaqhwzst3emdv25hefna6vxhtpjb5pytwahdod67syxjyzba3q

Sponsors

Protcol Labs

Contact

  • * Filecoin Slack : @Julien_NOEL_-_Twin_Quasar_(s0nik42)