Skip to content

v8.0 - Tracking experiment platform, MOT17 & MOT20 evaluation
Compare
Choose a tag to compare
@mikel-brostrom mikel-brostrom released this 30 Nov 11:42
· 2717 commits to master since this release
v8.0
78968a0
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

The goal of this release is to transform the repo into a user-friendly tracking experiment platform by adding different tracking methods and evaluation support for different MOT datasets. I will continue adding SOTA tracking methods as they come out.

Important updates

  • Tracking method selection
    • OCSORT added as a tracking option
    • ByteTrack added as a tracking option
  • Added evaluation support for MOT17 & MOT20
  • Added custom tracking dataset evaluation tutorial
  • Added to CI:
    • Model export testing
    • Tracking with exported model testing
    • Tracking with OCSORT testing
    • Tracking with ByteTrack testing
  • Less bloated README
  • Evaluation on specific GPUs and CPU
  • Update to Yolov5 release v7
  • --vid-stride to process every nth frame now available
  • Experiment results:

MOT17-train evaluation results

The hyperparameters used for evaluation can be found under val.py. Notice that
none of the models used during the evaluation has ever seen any of the MOT17 data and that our object detection model is a modest Yolov5m.

HOTA: exp105-pedestrian            HOTA      DetA      AssA      DetRe     DetPr     AssRe     AssPr     LocA      RHOTA     HOTA(0)   LocA(0)   HOTALocA(0)
MOT17-04-ss                        60.908    59        63.405    64.121    76.02     68.898    78.923    80.981    63.714    81.25     75.158    61.066    
MOT17-05-ss                        40.252    39.213    41.436    42.163    74.171    52.461    63.544    81.546    41.789    51.945    76.574    39.777    
MOT17-09-ss                        56.907    60.309    53.739    65.712    80.477    59.512    79.408    85.724    59.415    70.643    81.881    57.843    
MOT17-10-ss                        50.853    50.201    51.696    54.009    74.875    56.683    76.058    80.464    52.834    67.831    75.599    51.28     
MOT17-11-ss                        62.797    58.699    67.396    70.483    72.798    74.55     82.031    86.688    68.912    75.266    83.145    62.58     
MOT17-13-ss                        46.722    43.064    51.08     46.897    72.133    56.866    74.246    80.146    48.893    61.802    74.62     46.116    
COMBINED                           56.852    54.364    60.008    59.706    75.249    66.14     78.487    81.823    59.774    74.144    76.496    56.717    

CLEAR: exp105-pedestrian           MOTA      MOTP      MODA      CLR_Re    CLR_Pr    MTR       PTR       MLR       sMOTA     CLR_TP    CLR_FN    CLR_FP    IDSW      MT        PT        ML        Frag      
MOT17-04-ss                        69.718    78.487    69.847    77.097    91.404    50.602    33.735    15.663    53.133    36665     10892     3448      61        42        28        13        549       
MOT17-05-ss                        45.757    78.793    46.668    51.757    91.048    19.549    63.158    17.293    34.781    3580      3337      352       63        26        84        23        249       
MOT17-09-ss                        67.925    84.484    68.732    75.192    92.088    50        50        0         56.258    4004      1321      344       43        13        13        0         141       
MOT17-10-ss                        62.575    77.025    63.206    67.669    93.813    28.07     61.404    10.526    47.028    8688      4151      573       81        16        35        6         663       
MOT17-11-ss                        61.753    85.318    62.081    79.451    82.06     46.667    40        13.333    50.088    7497      1939      1639      31        35        30        10        200       
MOT17-13-ss                        52.534    76.716    53.247    59.131    90.95     30        41.818    28.182    38.766    6884      4758      685       83        33        46        31        321       
COMBINED                           63.933    79.251    64.319    71.832    90.531    34.091    48.76     17.149    49.028    67318     26398     7041      362       165       236       83        2123      

Identity: exp105-pedestrian        IDF1      IDR       IDP       IDTP      IDFN      IDFP      
MOT17-04-ss                        77.326    71.274    84.501    33896     13661     6217      
MOT17-05-ss                        54.125    42.446    74.669    2936      3981      996       
MOT17-09-ss                        70.485    64.019    78.404    3409      1916      939       
MOT17-10-ss                        69.701    59.989    83.166    7702      5137      1559      
MOT17-11-ss                        74.445    73.262    75.668    6913      2523      2223      
MOT17-13-ss                        63.099    52.062    80.077    6061      5581      1508      
COMBINED                           72.488    65.002    81.923    60917     32799     13442     

Count: exp105-pedestrian           Dets      GT_Dets   IDs       GT_IDs    
MOT17-04-ss                        40113     47557     131       83        
MOT17-05-ss                        3932      6917      103       133       
MOT17-09-ss                        4348      5325      52        26        
MOT17-10-ss                        9261      12839     96        57        
MOT17-11-ss                        9136      9436      152       75        
MOT17-13-ss                        7569      11642     133       110       
COMBINED                           74359     93716     667       484

No performance boost this time, only that we started evaluating on MOT17

Assets 2
Loading