Skip to content

A system for Human-Object Interaction (HOI) in real scene.

Notifications You must be signed in to change notification settings

birlrobotics/hoi-det-online

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Detecting Human-Object Interaction in Real.

In this repository, we offer a system for detecting Human-Object Interaction (HOI) in real scene.

Overview of the system

This following diagram illustrates the main components of our system. For the object or pose detector, we directly leverage the off-the-shelf implements in Pytorch. As for the subsequent HOI inference network, we use our VS-GATs and PMN. Among them, is the operation to construct the proper input features for HOI inference network. In the end, we visualize the results in the original image.
dragram of the system

Quick start

  • Download the pretrained checkpoints from here and put them in ./checkpoints/ file.

  • Run the code

    python main.py --act_threshold=0.3 --camera --show_line

    Explanation of some options:

    --camera : use the camera to detect in real.

    --use_pmn : use the PMN model or not. (Stay tuned)

    --act_threshold : only the inferred action score exceed this threshold will be shown.

    --show_line : visualize the linked line between the human and the corresponding interacted object.

    --show_pose : visualize the human pose. (Stay tuned)

    --save_img : save the original and processed images in ./results/ file.

    --save_video : save the recorded video in ./results/ file.

    --obj_det_only : just detect the objects, without HOI inference.

  • Results will be saved in ./results/ file.

Demo

This is a naive demo. det_obj shows the total numbers of detected objects, where the number in the bracket is for detected human. det_hoi means the number of detected HOIs.

demo

About

A system for Human-Object Interaction (HOI) in real scene.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages