Skip to content

Loki

Loki alerting

How to alert when there's an error in the logs

Configure grafana

https://grafana.com/docs/grafana/latest/features/datasources/loki/

URL: http://loki-stack.logging.svc.cluster.local:3100

Test query:

{job=~".*nextcloud.*"}

Logcli

https://grafana.com/docs/loki/latest/getting-started/logcli/ https://grafana.com/docs/loki/latest/logql/

Install

sudo pacman -S logcli

or via GitHub releases:

eget grafana/loki
mv ~/bin/download/logcli-linux-amd64 ~/bin/download/logcli

Setup

kubectl -n stackspin port-forward pod/loki-0 3100

Optional: If loki is username/password protected export the grafana/loki admin credentials:

export LOKI_USERNAME=admin
export LOKI_PASSWORD=...

Query exaples

logcli labels
logcli labels job
logcli labels filename

Show all logs:

From https://stackoverflow.com/a/70904375:

Due to Loki’s design, all LogQL queries must contain a log stream selector.

logcli query '{job=~".+"}'

Search for regex:

logcli query '{job=~".+"} |~ "(error|warn)"'

Show log entries over time:

sum(rate({app="mastodon"}[5m])) by (pod)