2 years ago

#60170

test-img

Zvi Mints

Rate Limiter for Logback (i.e BurstFilter)

I have a lot of warn logs which are potentially logged on each API request, which can make huge overload due to big traffic.

I'm looking for a declarative solution to sample logs on logback, i.e 10% of the logs will be printed or even better for logging mechanism that can limit the number of logs printed each time unit (rate)

I created the following custom filter, but I'm looking for something which exists like BurstFilter on log4j (which make rate-limiter and not threshold-limiter)


object ThresholdFilter {
  private final val LIMIT_NUM_OF_LOGS = 100
  private final val EXPIRE_AFTER_DURATION_DEFAULT: Duration = Duration.ofMinutes(1)
}

class ThresholdFilter extends Filter[ILoggingEvent] {

  final private val cache: Cache[String, AtomicInteger] = CacheBuilder.newBuilder()
                                                                      .expireAfterWrite(EXPIRE_AFTER_DURATION_DEFAULT)
                                                                      .build()

  def decide(event: ILoggingEvent): FilterReply = {
    if (isStarted && event.getLevel == Level.WARN && getLimit(event.getLoggerName) > LIMIT_NUM_OF_LOGS) {
      FilterReply.DENY
    }
    else {
      FilterReply.NEUTRAL
    }
  }

  private def getLimit(key: String): Int = {
    Try(cache.get(key, () => new AtomicInteger(0)).incrementAndGet()).getOrElse(0)
  }
}

scala

log4j

logback

0 Answers

Your Answer

Accepted video resources