Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Support for YOLOv3 #451

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

AIWintermuteAI
Copy link

This adds support for running multi-branch YOLO v3 object detection network on MaixPy.
Number of branches can be set to 1 or 2. Support for larger number of branches is possible, but will likely bring sub-optimal performance (low FPS).
The YOLO v3 model was trained using aXeleRate framework. It has number of advantages if compared to YOLO v2 models, that I was using before:

  • multi-branch structure allows detecting small and large objects
  • training is more stable, the model converges to lower loss
  • YOLO v3 loss works better for non-square images

Below is the best results I was able to achieve with YOLO v2 224x224 MobileNet 0.75 backbone
IMG_20211210_181030
And here is (probably) not the best YOLO v3, trained for only 50 epochs 320x240 MobileNet 0.5 backbone
IMG_20211210_181050

The out comparison for YOLOv2 and YOLOv3

"""
a = kpu.init_yolo2(task, 0.2, 0.1, 5, anchor)
>>> {"x":20, "y":30, "w":10, "h":21, "value":0.204172, "classid":1, "index":0, "objnum":6}
{"x":195, "y":38, "w":25, "h":33, "value":0.388860, "classid":0, "index":1, "objnum":6}
{"x":96, "y":65, "w":24, "h":52, "value":0.280810, "classid":1, "index":2, "objnum":6}
{"x":60, "y":95, "w":25, "h":33, "value":0.388278, "classid":0, "index":3, "objnum":6}
{"x":150, "y":125, "w":25, "h":51, "value":0.287573, "classid":0, "index":4, "objnum":6}
{"x":17, "y":174, "w":25, "h":33, "value":0.388692, "classid":0, "index":5, "objnum":6}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
"""
"""
a = kpu.init_yolo3(task, 0.5, 0.3, 3, 2, anchor)
>>> kpu_net __del__
YOLO v3
{"x":205, "y":135, "w":47, "h":68, "value":0.935626, "classid":0, "index":0, "objnum":11}
{"x":101, "y":8, "w":15, "h":17, "value":0.732297, "classid":0, "index":1, "objnum":11}
{"x":160, "y":14, "w":16, "h":24, "value":0.980218, "classid":0, "index":2, "objnum":11}
{"x":32, "y":28, "w":19, "h":24, "value":0.969574, "classid":1, "index":3, "objnum":11}
{"x":110, "y":36, "w":20, "h":22, "value":0.635944, "classid":1, "index":4, "objnum":11}
{"x":202, "y":46, "w":22, "h":30, "value":0.978948, "classid":0, "index":5, "objnum":11}
{"x":261, "y":51, "w":28, "h":33, "value":0.983265, "classid":0, "index":6, "objnum":11}
{"x":103, "y":68, "w":21, "h":32, "value":0.620853, "classid":0, "index":7, "objnum":11}
{"x":131, "y":78, "w":38, "h":50, "value":0.831876, "classid":1, "index":8, "objnum":11}
{"x":83, "y":96, "w":30, "h":40, "value":0.648528, "classid":1, "index":9, "objnum":11}
{"x":21, "y":182, "w":30, "h":32, "value":0.926121, "classid":0, "index":10, "objnum":11}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""

To maintain backwards compatibility, I have decided to preserve init_yolov2 and run_yolov2 functions and add init_yolov3 and run_yolov3 functions - this means code is somewhat clumsy, with nearly identical functions. Perhaps you have some suggestions or can make some edits yourself, @junhuanchen

I have compared the outputs for YOLOv2 inference on latest stable build of MaixPy and my fork, they are identical. Although more testing never hurts, if you'd like to do that.

"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)

kpu_net __del__
{"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-72-g22a8555b5 on 2021-11-01; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)
>>> {"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""

@AIWintermuteAI
Copy link
Author

@junhuanchen
Copy link
Contributor

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

2 similar comments
@Abandon-ht
Copy link

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

@Abandon-ht
Copy link

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

@junhuanchen
Copy link
Contributor

Do not merge for now, pending attention

@AIWintermuteAI
Copy link
Author

@junhuanchen any progress? :)

@alexjones85
Copy link

I am trying to upload a YOLOv3 script via MaixPy IDE to the M.A.R.K vehicle with k210 module but it seems it is still only compatible with YOLOv2. Will the master branch of the repo be merged upgraded to YOLOv3 anytime soon?

@junhuanchen
Copy link
Contributor

@junhuanchen any progress? :)

I'm watching, waiting for a good test.

@TheMrViper
Copy link

@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....

@junhuanchen
Copy link
Contributor

@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....

Dear, I’m very sorry that this project has stopped updating from the perspective of the enterprise and will be handed over to community users to take care of and maintain in the future. If you need it or are responsible for this submission, you can fork and test it to give reasons for the need to be merged.

The reason why I can‘t merge is not that it can’t work, but that I can‘t be responsible for this submission, because it also needs corresponding training scripts and document tutorials behind it to communicate to community users that this function is feasible, including API descriptions and sample code.

Forgive me for not being able to merge. In fact, I knew it was available for a long time, but I didn’t have the conditions to deal with what needs to be completed after merging it.

You know, we can‘t make money from this open source project. We can only earn income by selling hardware. We have gradually migrated to Linux AI chips in China. You can find this fact on our wiki.sipeed.com.

Thank you very much for your attention. If necessary, I can add you to the warehouse administrator to handle these PRs. If possible, please inform me of your reasons and expectations. I very much welcome our open source users to take the initiative to maintain and update the open source project.

Love you, junhuanchen.

@Abandon-ht
Copy link

Hi! Very thanks u pull requests it.I want to know how to use aXeleRate train my custom yolov3 model by this Repositories config list only have yolov2 configs.How can i modified the config just like detector.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants