Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ISP1 Pipeline error #32

Open
Mango-kid opened this issue Apr 19, 2021 · 20 comments
Open

ISP1 Pipeline error #32

Mango-kid opened this issue Apr 19, 2021 · 20 comments

Comments

@Mango-kid
Copy link

When testing out the camera I get some errors in kernel log. I am working with 5.11

I have an ov4689 connected to CSI1

media-ctl --device /dev/media2 --reset
media-ctl --device /dev/media2 --links "'ov4689 2-0036':0 -> 'rkisp1_isp':0 [1]"
media-ctl --device /dev/media2 --links "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [0]"
media-ctl --device /dev/media2 --links "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [1]"
media-ctl --device /dev/media2 --set-v4l2 '"ov4689 2-0036":0 [fmt:SBGGR10_1X10/2112x1568]'
media-ctl --device /dev/media2 --set-v4l2 '"rkisp1_isp":0 [fmt:SBGGR10_1X10/2112x1568 crop: (0,0)/1920x1080]'
media-ctl --device /dev/media2 --set-v4l2 '"rkisp1_isp":2 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'
media-ctl --device /dev/media2 --set-v4l2 '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1920x1080]'
media-ctl --device /dev/media2 --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1280x720]'

and the error I see is

[ 5500.322202] rkisp1 ff920000.isp1: start pipeline failed -32
[ 5509.280513] rkisp1 ff920000.isp1: start pipeline failed -67
@avafinger
Copy link
Owner

What is the output:
ls /dev/media*
media-ctl -p -d 1 |grep ENABLED |grep -v "rkisp"|grep -o "\".*\""|tr -d \"

where
-d mediaX

I think it should be: ov4689 1-0036

@Mango-kid
Copy link
Author

/dev/media0 /dev/media1 /dev/media2 /dev/media3

&

ov4689 2-0036

I have been digging through the isp drivers to figure out what these errors could mean. The camera + isp gets routed in this case to video6 and video7. When trying to start a stream from either of these I get -32 for video6 and -67 for video7

@Mango-kid
Copy link
Author

Here is the device tree.

device-tree.txt

@avafinger
Copy link
Owner

Do you refer CSI0 and CSI1 or CSI1 and CSI2?

The camera should be in this slot:
image

and:
sudo ln -sf nanopi-m4.dtb_5.11.0_csi1-ov4689 dtb

error -67 means your "links" are incorrect.

Can you confirm your camera is with IR cut?
Later I will get to the board and check some info.

@Mango-kid
Copy link
Author

Mango-kid commented Apr 20, 2021 •
edited
Loading

error -67 means your "links" are incorrect. This is very helpful? where did you find this. At first I was thinking that the resolution i set on the camera was incorrect, i know that can cause problems.

I have verified the camera connection using an v4.4 image and it is working correctly.

I am referring to things as they are found in the schematic. So my camera is going to CSI_1 / MIPI_TX1/RX1

Also the ov4689 is detected properly in the bootlog.

@avafinger
Copy link
Owner

media-ctl -d /dev/media2 -p
media-ctl -d /dev/media2 -p|grep enti

@Mango-kid
Copy link
Author

Mango-kid commented Apr 20, 2021 •
edited
Loading

On reboot it moves sometimes.

Media controller API version 5.11.15

Media device information
------------------------
driver          rkisp1
model           rkisp1
serial
bus info        platform:rkisp1
hw revision     0xa
driver version  5.11.15

Device topology
- entity 1: rkisp1_isp (4 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SRGGB10_1X10/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                <- "ov4689 2-0036":0 [ENABLED]
        pad1: Sink
                [fmt:unknown/0x0 field:none]
                <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
        pad2: Source
                [fmt:YUYV8_2X8/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                -> "rkisp1_resizer_mainpath":0 [ENABLED]
                -> "rkisp1_resizer_selfpath":0 [ENABLED]
        pad3: Source
                [fmt:unknown/0x0 field:none]
                -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]

- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:YUYV8_2X8/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                <- "rkisp1_isp":2 [ENABLED]
        pad1: Source
                [fmt:YUYV8_2X8/800x600 field:none]
                -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]

- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Sink
                [fmt:YUYV8_2X8/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                <- "rkisp1_isp":2 [ENABLED]
        pad1: Source
                [fmt:YUYV8_2X8/800x600 field:none]
                -> "rkisp1_selfpath":0 [ENABLED,IMMUTABLE]

- entity 12: rkisp1_mainpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video8
        pad0: Sink
                <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]

- entity 16: rkisp1_selfpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video9
        pad0: Sink
                <- "rkisp1_resizer_selfpath":1 [ENABLED,IMMUTABLE]

- entity 20: rkisp1_stats (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video10
        pad0: Sink
                <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]

- entity 24: rkisp1_params (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video11
        pad0: Source
                -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]

- entity 28: ov4689 2-0036 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [fmt:SBGGR10_1X10/2688x1520@10000/300000 field:none]
                -> "rkisp1_isp":0 [ENABLED]

&

- entity 1: rkisp1_isp (4 pads, 5 links)
- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
- entity 12: rkisp1_mainpath (1 pad, 1 link)
- entity 16: rkisp1_selfpath (1 pad, 1 link)
- entity 20: rkisp1_stats (1 pad, 1 link)
- entity 24: rkisp1_params (1 pad, 1 link)
- entity 28: ov4689 2-0036 (1 pad, 1 link)

@avafinger
Copy link
Owner

I don't see anything wrong, despite the 2-0036. I can only suggest you try this one:

media-ctl --device "platform:rkisp1" --reset
media-ctl --device "platform:rkisp1" --links "'ov4689 2-0036':0 -> 'rkisp1_isp':0 [1]"
media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [0]"
media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [1]"
media-ctl --device "platform:rkisp1" --set-v4l2 '"ov4689 2-0036":0 [fmt:SBGGR10_1X10/2688x1520]'
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":0 [fmt:SBGGR10_1X10/2688x1520 crop: (0,0)/1920x1080]'
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":2 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1920x1080]'
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]'
v4l2-ctl --media-bus-info "platform:rkisp1" --device "rkisp1_mainpath" --set-fmt-video "width=1920,height=1080,pixelformat=YU12"

Or use the pre-built image I provided in "Releases" if you have nanopim4 (not the v2).
In this Image you have:

[    0.992653] rockchip-drm display-subsystem: bound ff8f0000.vop (ops vop_component_ops)
[    0.996340] rockchip-drm display-subsystem: bound ff900000.vop (ops vop_component_ops)
[    1.431783] rockchip-drm display-subsystem: bound ff8f0000.vop (ops vop_component_ops)
[    1.433822] rockchip-drm display-subsystem: bound ff900000.vop (ops vop_component_ops)
[    1.436186] rockchip-drm display-subsystem: bound ff940000.hdmi (ops dw_hdmi_rockchip_ops)
[    1.580347] rockchip-drm display-subsystem: [drm] fb0: rockchipdrmfb frame buffer device
[    1.581771] [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0
[    4.656008] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[    9.113914] rkisp1 ff910000.isp0: Adding to iommu group 4
[    9.114460] rkisp1: registered rkisp1_mainpath as /dev/video0
[    9.114565] rkisp1: registered rkisp1_selfpath as /dev/video1

and if the sensor is connected to CSI-1:

[    0.000000] Movable zone start for each node
[    0.320999] thermal_sys: Registered thermal governor 'step_wise'
[    0.322175] cpuidle: using governor menu
[    1.406907] mmc_host mmc0: card is non-removable.
[    7.925134] ov4689 1-0036: driver version: 00.01.01
[    7.925158] ov4689 1-0036: could not get module information!
[    7.927904] ov4689 1-0036: Detected OV004688 sensor

@Mango-kid
Copy link
Author

Okay, ill try it out and report back.

I tried booting this image in the past on a platform that is identical to the nanopi-m4 however it has emmc attached. For some reason it would always boot from emmc anyway. I can probably sort that out.

@avafinger
Copy link
Owner

For some reason it would always boot from emmc anyway. I can probably sort that out.

You can remove the eMMC and give it a try. Or, edit boot.cmd and place the correct device number to boot in:

#
# Mainline Kernel 5.x
# 0 = eMMC
# 1 = SD card
setenv boot_from_device 1
# 0 = eMMC
# 1 = SD card
setenv boot_blk 1

Maybe with eMMC the device number changed...

@avafinger
Copy link
Owner

avafinger commented Apr 20, 2021 •
edited
Loading

PS: Rockchip gives priority to eMMC over microSD when booting... something you should know.
*edited:
I mean, to boot from SD card with eMMC attached you should change the boot order in eMMC, I don't think is a good thing to mess with.

@Mango-kid
Copy link
Author

I was able to boot the image and correctly link the device trees. I inspected the user space after reboot as well to make sure it loaded the correct device tree. This error still exits when using the camera on either CSI1 or CSI0. The attached log is testing with CSI0 and ISP0.

I also verified that the camera does work with 4.4.y and 4.19 so im guessing the hardware is okay. My next line of thinking could be that the clocks maybe
are incorrect.

boot.log

@Mango-kid
Copy link
Author

Mango-kid commented Apr 20, 2021 •
edited
Loading

Is there a script for writing the sd image to emmc? I saw the Kodi one but it didnt look like it would work for this image.

Thanks.

@avafinger
Copy link
Owner

Are you trying with dual camera?

[    9.174840] ov4689 1-0036: driver version: 00.01.01
[    9.174864] ov4689 1-0036: could not get module information!
[    9.177473] ov4689 1-0036: Detected OV004688 sensor
[    9.178053] ov4689 2-0036: driver version: 00.01.01
[    9.178071] ov4689 2-0036: could not get module information!
[    9.180480] ov4689 2-0036: Unexpected sensor id(000000), ret(-5)

Again, is your camera w/IR cut?

@Mango-kid
Copy link
Author

I was using the dual camera dts but only ever had one camera attached. I have a development board that gives me access to CSI0 but I am trying to get CSI1 working with the sensor.

Could it be that if ISP0 and CSI0 are enabled but unused that CSI1 will not work?

@avafinger
Copy link
Owner

It only works with CSI1 (single camera) and CSI1 slot is your CSI0 (in DTS). In the provided Image you should have camera attached to CSI1, exactly as the Image. And: sudo ln -sf nanopi-m4.dtb_5.11.0_csi1-ov4689 dtb

The instructions to write to the eMMC are the same as 4.4, only change would be the boot.cmd (device number changed from 4.4) due to u-boot version.

@Mango-kid
Copy link
Author

Thanks again for all the help so far. I am still working on getting it to boot eMMC so I can test a the correct camera lane on my platform. I was looking at the device tree for CSI1 and I noticed that the MIPI interfaces are disabled. I am wondering if some of my problems earlier could have been routed here. If using ISP do these need to be disabled? I know with the newer rkisp structure you dont need to create the camera->mipi link anymore, maybe this is related.

@avafinger
Copy link
Owner

Must be enabled:

&isp0_mmu {
status = "okay";
};

&isp0 {
status = "okay";
};

@avafinger
Copy link
Owner

I know with the newer rkisp structure you dont need to create the camera->mipi link anymore, maybe this is related.

Can you link where you found this info for the mainline kernel? Kernel 4.4 and 4.19 this may be true, you can use v4l2 straight.

@Mango-kid
Copy link
Author

https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html

It would be very weird if this was the case that ISP1 could only be used if ISP0 was already used. Also I should clarify my setup as well.

ov4689 -> CSI_1 (CSI1 on the chip level not the board labeling). -> ISP1

Also for some background I am working on putting support into Armbian for the second camera sensor on the RK3399 boards for when Armbian moves to 5.11. I'm using this build as the reference and 4.4 to check that the hardware is okay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Mango-kid @avafinger