Calculation problem of fp and fn markers in pr_mep.py file

[wangzika]

Thank you for your excellent work and previous prompt responses. But I encountered a problem while reading your PR curve calculation code. Please correct me. Handling of false positives (fp): `I saw in your code module that the true examples (tp) are correctly marked in est_line[1]=1, but the false positives (fp) are not marked directly in est_line. Will this affect the calculation of fp in subsequent calculations, because fp should be those predictions that are predicted to be positive but are actually negative.

     est_line = [eval(line_info[2]), 0, 0, idx_curr]
           if pairing[1] != 'x':    #描述子预测到的回环帧
               idx_best = int(pairing[1])
               if np.linalg.norm(gt_pose[idx_curr].reshape(3, 4)[:, 3] -
                                 gt_pose[idx_best].reshape(3, 4)[:, 3]) < thres_dist:    #判断描述子的是否为真回环   
                   est_line[1] = 1   #如果描述子得到的是真回环则 :est[1]=1   

               # 3. if the overall is P
           est_line[2] = gt_positive[idx_curr]     #真实的回环标志赋值给：est[2]`

And in the subsequent processing, I saw that the code directly predicts all non-tp as fp, but there is no clear logic to exclude true negative examples (tn) when calculating fp.

        for i in range(est.shape[0]):
            if est[i, 1]:
                tp += 1
            else:
                fp += 1    
            fn = 0
            for j in range(i, est.shape[0]):
                if est[j, 2]:
                    fn += 1

            pr_points.append([tp / (tp + fn), tp / (tp + fp), est[i, 3]])

I hope you can help me answer it

[lewisjiang]

est_line=[x0, x1, x2, x3], where x0: estimated sim score, x1: if the gt pose of the best loop candidate is close enough, x2: if has gt loop, x3: current pose index. This implies if x1=1, then x2=1; if x2=0, then x1=0.

Definition of FP: reports a loop connection between two poses, but actually the connection is wrong, either connects to a wrong pose, or there are actually no loops at all.

In practice, we require every pose relates to one previous pose. And whether to report as Positive is controlled by a moving sim score threshold, which is exactly the currect x0 in the sorted est. Then if x1=0, it is estimated as positive but has wrong connection, aka FP. And TP and FP counts can be accumulated since we are sliding along the sorted list. A perfect loop detector is still possible (p=1.0, r=1.0), as long as the sim scores for poses without actual loops are low enough.

There is no need to record TN when calculating precision-recall. FN is all those loop pairs that have a true loop (x2_i=1), but the sim score is lower than the current moving threshold (x1_i<current x1) therefore not reported as positive.

[wangzika]

est_line=[x0, x1, x2, x3], where x0: estimated sim score, x1: if the gt pose of the best loop candidate is close enough, x2: if has gt loop, x3: current pose index. This implies if x1=1, then x2=1; if x2=0, then x1=0.

Definition of FP: reports a loop connection between two poses, but actually the connection is wrong, either connects to a wrong pose, or there are actually no loops at all.

In practice, we require every pose relates to one previous pose. And whether to report as Positive is controlled by a moving sim score threshold, which is exactly the currect x0 in the sorted est. Then if x1=0, it is estimated as positive but has wrong connection, aka FP. And TP and FP counts can be accumulated since we are sliding along the sorted list. A perfect loop detector is still possible (p=1.0, r=1.0), as long as the sim scores for poses without actual loops are low enough.

There is no need to record TN when calculating precision-recall. FN is all those loop pairs that have a true loop (x2_i=1), but the sim score is lower than the current moving threshold (x1_i<current x1) therefore not reported as positive.

Thank you for your answer. I never knew the function of x0, but now I seem to understand it a little bit.