Deleting logrange logs

We’re seeing this on some of our clusters (this is 6.1.12) where the logrange folder in /var/lib/gravity is continuously growing, with tens of GBs in there. We don’t really need to retain this information historically (nor is it being sent anywhere), but I’m not quite sure how to delete it.

Is there a way to clean this (or to tell Logrange to not store it for so long altogether)?

It does not seem like logrange has implemented the configuration for limiting the record size (or generally, space-relevant configuration) but I will look into this.

Thanks @Dmitri_Shelenin. Is there any way to manually delete it?

I looked into the logging-app in more detail regarding the cleanup. There is a log-collector deployment running a gravity-specific logrange adapter that executes a clean up job (using truncate logrange statement) which caps the DB size at 10gb by default. So if you’re having an issue with this growing beyond the 10gb, we should look at the possible errors that the collector might be experiencing.
If you could share the logs of the log-collector and lr-aggregator Pods, this might shed some light.

@Dmitri_Shelenin this is still happening, even on 6.1.39. Do you want this as gravity report or something else?

@itay,
there were improvements introduced to logrange regarding this issue in the last couple of days: https://github.com/gravitational/logging-app/pull/72. I will see that we backport them into older branches if necessary and release a new 6.1.x.

Thanks @Dmitri_Shelenin that would be a big help.

Hi @itay, just following up on this, the related PR Dmitri tagged was also released in 6.1.44.