Danger

This article is not yet released. If you’re seeing it, please note that it is not complete.

Lessons in Grafana - Part Seven: Plotting Prospective Plans

2026-04-06

This project has been wonderful to work on. I’ve learned a lot, and have a lot more ideas for how this could be expanded. Many of them are difficult. I want to outline them below in the hopes that someone has suggestions on how to go about them, and what should be prioritized. I plan to keep adding to this series as I do more.

Note

NetGuard??

Exporting Apps

The following apps are useful targets because they allow automated exports of data. In some cases this is as SQLite databases. In others it is JSON data. Regardless, syncing them into my Postgres database is likely to be very useful.

AntennaPod

Plays podcasts. Ideally could give state timeline. Certainly can give listen time, maybe even total listen time. Certainly can break that down by series.

DroidShows

Tracks TV shows. Can make state timeline, but shouldn’t. Dates are an upper bound, so hard to know how it’s useful.

Flexify

Strength training app. Spits out sqlite

GNUCash

Oh god the schema is complicated

MediTrak

Exports a bunch of JSON, but was unstable when I tried it

Tachiyomi

Presumably could give time reading, pages read, etc. Weird file extension might indicate weird format. Must investigate.

Non-Automated

The following apps can manually export databases. I could presumably automate this export through Tasker, so it’s worth looking at how these could be useful.

NewPipe / Tubular

Low priority. Could give me watch time, maybe watch mode? Unclear if data would work for State Timeline.

RxDroid

Current medicine app, known stable, would be quite useful

APIs

Duolingo

This gives a lot more data than I currently track, but its API is shit and undocumented. Wish I could get streak / xp / time per-language

Fediverse Stats

This is kinda dumb, but it would be useful to have in my web analytics dashboard. Kinda trying to network and self-advertise, so it’s kinda important I guess

GoatCounter

Goatcounter exposes a lot more data than I am currently fetching. Geo maps, referrers, etc might be available

Libby

Ideally could give me a state timeline. Would need to take some effort to only get my listen time, excluding Lydia’s. Could combine with front timeline to get time per headmate

OpenStreeMap

hdyc reimplementation, but open source, hopefully

PetLibro

Water fountain + kibble dispenser. Need to investigate.

Sensi

Thermostat. Seems like I am stuck between a proprietary SDK or homeassistant. Sucks.

Sleep Number

Bed. Could get timeline of when bed is occupied. Maybe pressures and angles. It claims to collect biometrics. I should either collect that or turn it off.

StoryGraph

Doesn’t have an API but does have a semi-functional scraper. Might be able to use browser-based automation or similar.

Steam

Needs investigation

Weather

OpenWeatherMap + a local station? Thermostat sensors? Humidistat?

Various Utilities

Candidates:

  • Mint

  • Xperia

  • ComEd

  • water

  • gas

  • electricity at specific outlets

Automation Development

With some effort, I should be able to collect the following data using some combination of Tasker and user scripts.

App Usage

Could monitor with Tasker. Wish that Digital Wellbeing would export this. Maybe I can grab data with intents?

Email

This could be used to get inbox size over time. Could show number of important, or maybe journeys. Percent volume, number sent, number replied, size etc

Internet Speed Monitoring

Would be useful to know if Xperia is fucking us. Ideally as measured from the router or something on the WiFi as well.

Screen Time

Could monitor via Tasker, and either systemd or cron. Sync it all to host, which extracts it to the db

Bot Improvements

Alerting

It would be lovely if I could monitor for things to alert on. One example could be to notify when the Litterbot’s drawer gets full.

Bot Isolation

Right now my scraping bot is running in a docker container. This is nice, and it’s as locked down as I can reasonably make it. The problem is that I can’t get rid of the network_mode=host line because the standard Joplin app listens only on localhost. I am very unhappy with that. It’s a rather glaring hole in my isolation scheme. I’ve considered and/or tried the following:

  1. Using Joplin in a docker container + web interface, so that it doesn’t leave Docker’s networking (more cumbersome and heavyhanded than I’d like)

  2. Using Joplin Server in a docker container (already doing), with E2EE turned off (not doing at the moment)

  3. Using socat to broadcast the port specifically to this container (haven’t found a way to expose to only the one container / subnet reliably)

  4. Using iptables or similar to broadcast the port specifically to this container (I want it to be able to run without reconfiguring the host or requiring root access)

CalDav Integration

It would be wonderful if I could find a way to display my to-do list in a dashboard.

Dealing With Technical Debt

Because this project has been a series of minimum-viable changes, there are a lot of things to clean up

Packaging for Others

Because this has been for personal use, there is a fair amount of Personally Identifiable Information (PII) in there. Removing that is a slow process, and I can’t be certain that it’s ever complete. This is especially hard if I want to distribute the Grafana dashboards I have generated, as many of them rely on transformations that have that information hardcoded into them. This is something that I intend to work on in the long term.

Utilizing Other Messaging Backends

The other major blocker for packaging is creating alternative messaging backends. Right now, all manual data collection is done through Joplin (for time tracking) or Telegram (for surveys and events). That’s not very flexible. There are other time tracking tools out there, and there are other messaging services. It is difficult to decouple this setup from Telegram, as we are using the event loop there for everything. Removing that dependency will take time, and it’s not obvious what service to target next. Before recent news, I would have said that Discord is an ideal next target. But now? Maybe XMPP. We’ll see.

Wrapping Things Up

Citations

Cite As

Click here to expand the bibtex entry.
@online{appleton_blog_0007,
  title    = { Lessons in Grafana - Part Seven: Plotting Prospective Plans },
  author   = { Olivia Appleton-Crocker },
  editor   = { Ultralee0 and Ruby },
  language = { English },
  version  = { 1 },
  date     = { 2026-04-06 },
  url      = { https://blog.oliviaappleton.com/posts/0013-lessons-in-grafana-07 }
}

Tags: grafana, data-collection, data-visualization, databases, scraping, postgres, postgresQL, python

Comments Boosts , Likes