Add passiveExpire stats (#43)

Add passiveExpire statistics to redis info.

README-CN.md

@@ -27,7 +27,6 @@
 - 不对TairHash进行全局排序（可以节省内存）
 - 每个TairHash内部依然会使用一个排序索引对fields进行排序（加速每个key内部的field查找）
 - 内置定时器会周期使用SCAN命令找到包含过期field的TairHash，然后检查TairHash内部的排序索引，进行field的淘汰
-- 每一次读写field，也会触发对这个field自身的过期淘汰操作
 - 排序中所有的key和field都是指针引用，无内存拷贝，无内存膨胀问题
 
 **支持的redis版本**: redis >= 5.0  
@@ -40,8 +39,6 @@
 - 使用两级排序索引，第一级对tairhash主key进行排序，第二级针对每个tairhash内部的field进行排序
 - 第一级排序使用第二级排序里最小的ttl进行排序，因此主key是按照ttl全局有序的
 - 内置定时器会周期扫描第一级索引，找出一部分已经过期的key，然后分别再检查这些key的二级索引，进行field的淘汰，也就是active expire
-- 每一次对tairhash的写操作，也会先检查第一级索引，并最多过期三个field，这些field不一定属于当前正在操作的key，因此理论上写的越快淘汰速度也就越快
-- 每一次读写field，也会触发对这个field自身的过期淘汰操作
 - 排序中所有的key和field都是指针引用，无内存拷贝，无内存膨胀问题
 
 **支持的redis版本**: redis >= 7.0  
@@ -63,6 +60,9 @@
 
 **使用方式**：cmake的时候加上`-DSLAB_MODE=yes`选项，并重新编译
 
+## 主动过期
+- 每一次读写field，会触发对这个field自身的过期淘汰操作  
+- 每次写一个field时，TairHash也会检查其它field（可能属于其它的key）是否已经过期（每次最多检查3个），因为field是按照TTL排序的，因此这个检查会很高效 (注意: SLAB_MODE暂时不支持这个功能)
 
 ## 事件通知  
 

README.md

@@ -27,7 +27,6 @@
 - Do not sort TairHash globally (Smaller memory overhead)
 - Each TairHash will still use a sort index to sort the fields internally (For expiration efficiency)
 - The built-in timer will periodically use the SCAN command to find the TairHash that contains the expired field, and then check the sort index inside the TairHash to eliminate the field
-- Every time you read or write a field, it will also trigger the expiration of the field itself
 - All keys and fields in the sorting index are pointer references, no memory copy, no memory expansion problem
 
 **Supported redis version**: redis >= 5.0
@@ -42,8 +41,6 @@
 - Use a two-level sort index, the first level sorts the main key of tairhash, and the second level sorts the fields inside each tairhash
 - The first-level uses the smallest ttl in the second-level for sorting, so the main key is globally ordered
 - The built-in timer will periodically scan the first-level index to find out a key that has expired, and then check the secondary index of these keys to eliminate the expired fields. This is the active expire
-- Every time a write operation to tairhash, the first level index will be checked first, and at most three fields will be expired, Note these fields may not belong to the key currently being operated, so in theory, the faster you write, the faster the elimination
-- Every time you read or write a field, it will also trigger the expiration of the field itself
 - All keys and fields in the sorting index are pointer references, no memory copy, no memory expansion problem
 
 **Supported redis version**: redis >= 7.0
@@ -54,8 +51,7 @@
 
 **Usag**e: cmake with `-DSORT_MODE=yes` option, and recompile
 
-### SLAB_MODE：
-
+### SLAB_MODE：  
 - Slab mode is a low memory usage (compared with SORT mode), cache-friendly, high-performance expiration algorithm
 - Like SORT mode, keys are globally sorted to ensure that keys that need to be expired can be found faster. Unlike SORT mode, SLAB does not sort the fields inside the key, which saves memory overhead. 
 - The SLAB expiration algorithm uses SIMD instructions (when supported by the hardware) to speed up the search for expired fields
@@ -68,6 +64,11 @@
 
 **Usage**: cmake with `-DSLAB_MODE=yes` option, and recompile
 
+## Passivity expiration  
+- Every time you read or write a field, it will also trigger the expiration of the field itself  
+- Every time you write a field, tairhash also checks whether other fields (may belong to other keys) are expired (currently up to 3 at a time), because fields are sorted by TTL, so this check will be very efficient (Note: SLAB_MODE does not support this feature)
+
+
 ## Event notification   
 
 tairhash will send an event notification when the field expires (triggered by active or passive expiration). The notification is sent in pubsub mode. The format of the channel is: `tairhash@<db>@<key>__:<event>` , currently only supports expired event type, so

src/scan_algorithm.c

@@ -199,7 +199,8 @@ void passiveExpire(RedisModuleCtx *ctx, int dbid, RedisModuleString *key) {
         ln = tair_hash_obj->expire_index->header->level[0].forward;
         while (ln && keys_per_loop) {
             field = ln->member;
-            if (fieldExpireIfNeeded(ctx, dbid, key, tair_hash_obj, field, 1)) {
+            if (fieldExpireIfNeeded(ctx, dbid, key, tair_hash_obj, field, 0)) {
+                g_expire_algorithm.stat_passive_expired_field[dbid]++;
                 start_index++;
                 keys_per_loop--;
             } else {

src/sort_algorithm.c

@@ -231,7 +231,8 @@ void passiveExpire(RedisModuleCtx *ctx, int dbid, RedisModuleString *up_key) {
         ln = tair_hash_obj->expire_index->header->level[0].forward;
         while (ln && keys_per_loop) {
             field = ln->member;
-            if (fieldExpireIfNeeded(ctx, dbid, key, tair_hash_obj, field, 0)) {
+            if (fieldExpireIfNeeded(ctx, dbid, key, tair_hash_obj, field, 1)) {
+                g_expire_algorithm.stat_passive_expired_field[dbid]++;
                 start_index++;
                 keys_per_loop--;
             } else {

src/tairhash.c

@@ -272,6 +272,7 @@ int fieldExpireIfNeeded(RedisModuleCtx *ctx, int dbid, RedisModuleString *key, t
     if (tair_hash_val == NULL) {
         return 0;
     }
+
     long long when = tair_hash_val->expire;
     if (when == 0) {
         return 0;
@@ -310,6 +311,10 @@ void swapDbCallback(RedisModuleCtx *ctx, RedisModuleEvent e, uint64_t sub, void
     uint64_t tmp_stat = g_expire_algorithm.stat_active_expired_field[from_dbid];
     g_expire_algorithm.stat_active_expired_field[from_dbid] = g_expire_algorithm.stat_active_expired_field[to_dbid];
     g_expire_algorithm.stat_active_expired_field[to_dbid] = tmp_stat;
+
+    tmp_stat = g_expire_algorithm.stat_passive_expired_field[from_dbid];
+    g_expire_algorithm.stat_passive_expired_field[from_dbid] = g_expire_algorithm.stat_passive_expired_field[to_dbid];
+    g_expire_algorithm.stat_passive_expired_field[to_dbid] = tmp_stat;
 }
 
 void flushDbCallback(RedisModuleCtx *ctx, RedisModuleEvent e, uint64_t sub, void *data) {
@@ -444,6 +449,15 @@ void infoFunc(RedisModuleInfoCtx *ctx, int for_crash_report) {
         snprintf(buf, sizeof(buf), "db%d", i);
         RedisModule_InfoAddFieldLongLong(ctx, buf, g_expire_algorithm.stat_active_expired_field[i]);
     }
+
+    RedisModule_InfoAddSection(ctx, "PassiveExpiredFields");
+    for (int i = 0; i < DB_NUM; ++i) {
+        if (g_expire_index[i]->length == 0 && g_expire_algorithm.stat_passive_expired_field[i] == 0) {
+            continue;
+        }
+        snprintf(buf, sizeof(buf), "db%d", i);
+        RedisModule_InfoAddFieldLongLong(ctx, buf, g_expire_algorithm.stat_passive_expired_field[i]);
+    }
 }
 
 #endif
@@ -2681,7 +2695,7 @@ int TairHashTypeActiveExpireInfo_RedisCommand(RedisModuleCtx *ctx, RedisModuleSt
     t_size += d_len;
 
     for (int i = 0; i < DB_NUM; ++i) {
-        if (g_expire_algorithm.stat_active_expired_field[i] == 0) {
+        if (g_expire_algorithm.stat_active_expired_field[i] == 0 && g_expire_algorithm.stat_passive_expired_field[i] == 0) {
             continue;
         }
         RedisModuleString *info_d = RedisModule_CreateStringPrintf(ctx, "db: %d, active_expired_fields: %ld, passive_expired_fields: %ld\r\n", i,