C# - How to maintain InfluxDB disk usage?

18 Views Asked by At

I have a local influxdb run using Docker

version: InfluxDB 2.0.9 (git: d1233b7951) build_date: 2021-10-01T21:09:53Z

and I need to maintain the storage used by this db

as in if I put a limit of 400KB per bucket, and the user attempt to add more data when its already 400KB, I would need to delete the old data before inserting the new one so the disk usage is 400KB or less

The program I build is in C# using the official .NET Client

The things I imagine:

  1. monitor the disk usage through du -sh
  2. if the disk is larger than the limit, delete data worth of a week or a day (depends on the user setting): using c# DeleteAPI
await deleteapi.Delete(startdate, enddate, "", "c9228e2fd410c790", "myown");

The Question:

  1. I noticed when I initially create a bucket and put a data to it, the disk size is around 150-160K. Then I clean up the data (delete all data in the bucket) and the bucket size turns to 180K. How did it increase the size? when i'm deleting
  2. at disk 180K, I add some data until it gets to like 224K-ish, and perform the delete (delete all data), the bucket size turn to 212K... does that mean all the data actually only worth 12K? so what are the rest?
# initial du
16      ./autogen/1/index/1
16      ./autogen/1/index/0
12      ./autogen/1/index/2
16      ./autogen/1/index/5
12      ./autogen/1/index/7
12      ./autogen/1/index/4
12      ./autogen/1/index/3
16      ./autogen/1/index/6
116     ./autogen/1/index
148     ./autogen/1
152     ./autogen
8       ./_series/05
8       ./_series/03
8       ./_series/02
8       ./_series/04
8       ./_series/06
8       ./_series/01
8       ./_series/07
8       ./_series/00
68      ./_series
224     .
# after deletion
16      ./autogen/1/index/1
16      ./autogen/1/index/0
16      ./autogen/1/index/2
16      ./autogen/1/index/5
16      ./autogen/1/index/7
16      ./autogen/1/index/4
16      ./autogen/1/index/3
16      ./autogen/1/index/6
132     ./autogen/1/index
136     ./autogen/1
140     ./autogen
8       ./_series/05
8       ./_series/03
8       ./_series/02
8       ./_series/04
8       ./_series/06
8       ./_series/01
8       ./_series/07
8       ./_series/00
68      ./_series
212     .
  1. I know my approach is far less than ideal, so wondering if there is actually a better way to keep the disk usage in place
  2. There's also this thing that I read, where the deletion actually happens when the compacting happen. Can I force this? Like how I force vacuum on SQL

I might not be so familiar with InfluxDB, so any help would be appreciated

0

There are 0 best solutions below