Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP/RFC extract framerate #62

Open
wants to merge 4 commits into
base: release
Choose a base branch
from
+41 −14
Open

WIP/RFC extract framerate #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
07870cd
Fix extraction of U32 values from SPS header
Oct 2, 2018
7dd4614
WIP h264: extract framerate from SPS
Oct 1, 2018
eeeb337
[h265d] fix framerate num/den computing
Dec 17, 2018
c292648
[h265d] extract framerate from SPS
Dec 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions inc/mpp_frame.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,10 @@ RK_U32 mpp_frame_get_hor_stride(const MppFrame frame);
void mpp_frame_set_hor_stride(MppFrame frame, RK_U32 hor_stride);
RK_U32 mpp_frame_get_ver_stride(const MppFrame frame);
void mpp_frame_set_ver_stride(MppFrame frame, RK_U32 ver_stride);
RK_U32 mpp_frame_get_time_scale(const MppFrame frame);
void mpp_frame_set_time_scale(MppFrame frame, RK_U32 time_scale);
RK_U32 mpp_frame_get_num_units_in_tick(const MppFrame frame);
void mpp_frame_set_num_units_in_tick(MppFrame frame, RK_U32 num_units_in_tick);
RK_U32 mpp_frame_get_mode(const MppFrame frame);
void mpp_frame_set_mode(MppFrame frame, RK_U32 mode);
RK_U32 mpp_frame_get_discard(const MppFrame frame);
Expand Down
6 changes: 6 additions & 0 deletions mpp/base/inc/mpp_frame_impl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ struct MppFrameImpl_t {
* pointer for multiple frame output at one time
*/
MppFrameImpl *next;

/*
* timing information
*/
RK_U32 time_scale;
RK_U32 num_units_in_tick;
};


Expand Down
2 changes: 2 additions & 0 deletions mpp/base/mpp_buf_slot.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,8 @@ static void generate_info_set(MppBufSlotsImpl *impl, MppFrame frame, RK_U32 forc
mpp_frame_set_fmt(impl->info_set, fmt);
mpp_frame_set_hor_stride(impl->info_set, hal_hor_stride);
mpp_frame_set_ver_stride(impl->info_set, hal_ver_stride);
mpp_frame_set_time_scale(impl->info_set, mpp_frame_get_time_scale(frame));
mpp_frame_set_num_units_in_tick(impl->info_set, mpp_frame_get_num_units_in_tick(frame));
mpp_frame_set_buf_size(impl->info_set, size);
mpp_frame_set_buf_size(frame, size);
impl->buf_size = size;
Expand Down
4 changes: 4 additions & 0 deletions mpp/base/mpp_frame.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ MPP_RET mpp_frame_info_cmp(MppFrame frame0, MppFrame frame1)
(f0->height == f1->height) &&
(f0->hor_stride == f1->hor_stride) &&
(f0->ver_stride == f1->ver_stride) &&
(f0->time_scale == f1->time_scale) &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no necessary to add info change test. For framerate change will reflect on pts.
The info change judgement will report resolution / bitdepth / format changes which will affect buffer allocation and display setup.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I had doubts about where to put the information and how to have the parser notify the app when those values change. What do you think would be the best way ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be demux before mpp ( codec ). So demuxer can know the change.
When use mpp alone without demux embeded the frame rate change can be detect by pts gap.
These frame rate information looks like ffmpeg style : )

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, we're already in the process moving away from PTS-based detection of framerate changes, and on our side the need for this patch is temporary. If you feel it's not the right place for the feature, let's just close this PR, and we'll just keep this patch on our side until our need for it is gone :)

(f0->num_units_in_tick == f1->num_units_in_tick) &&
(f0->fmt == f1->fmt) &&
(f0->buf_size == f1->buf_size)) {
return MPP_OK;
Expand All @@ -170,6 +172,8 @@ MPP_FRAME_ACCESSORS(RK_U32, width)
MPP_FRAME_ACCESSORS(RK_U32, height)
MPP_FRAME_ACCESSORS(RK_U32, hor_stride)
MPP_FRAME_ACCESSORS(RK_U32, ver_stride)
MPP_FRAME_ACCESSORS(RK_U32, time_scale)
MPP_FRAME_ACCESSORS(RK_U32, num_units_in_tick)
MPP_FRAME_ACCESSORS(RK_U32, mode)
MPP_FRAME_ACCESSORS(RK_U32, discard)
MPP_FRAME_ACCESSORS(RK_U32, viewid)
Expand Down
9 changes: 9 additions & 0 deletions mpp/codec/dec/h264/h264d_init.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,15 @@ static MPP_RET dpb_mark_malloc(H264dVideoCtx_t *p_Vid, H264_StorePic_t *dec_pic)
mpp_frame_set_color_trc(mframe, MPP_FRAME_PRI_UNSPECIFIED);
mpp_frame_set_colorspace(mframe, MPP_FRAME_SPC_UNSPECIFIED);
}

if (p->timing_info_present_flag) {
mpp_frame_set_time_scale(mframe, p->time_scale);
mpp_frame_set_num_units_in_tick(mframe, p->num_units_in_tick);
} else {
// unspecified
mpp_frame_set_time_scale(mframe, 0);
mpp_frame_set_num_units_in_tick(mframe, 0);
}
}
mpp_buf_slot_set_prop(p_Dec->frame_slots, cur_mark->slot_idx, SLOT_FRAME, mframe);
mpp_frame_deinit(&mframe);
Expand Down
6 changes: 2 additions & 4 deletions mpp/codec/dec/h264/h264d_sps.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,8 @@ static MPP_RET read_VUI(BitReadCtx_t *p_bitctx, H264_VUI_t *vui)
}
READ_ONEBIT(p_bitctx, &vui->timing_info_present_flag);
if (vui->timing_info_present_flag) {
READ_BITS(p_bitctx, 16, &vui->num_units_in_tick); //!< num_units_in_tick(high 16bit)
READ_BITS(p_bitctx, 16, &vui->num_units_in_tick); //!< num_units_in_tick(low 16bit)
READ_BITS(p_bitctx, 16, &vui->time_scale); //!< time_scale(high 16bit)
READ_BITS(p_bitctx, 16, &vui->time_scale); //!< time_scale(low 16bit)
READ_BITS_LONG(p_bitctx, 32, &vui->num_units_in_tick);
READ_BITS_LONG(p_bitctx, 32, &vui->time_scale);
READ_ONEBIT(p_bitctx, &vui->fixed_frame_rate_flag);
}
READ_ONEBIT(p_bitctx, &vui->nal_hrd_parameters_present_flag);
Expand Down
5 changes: 5 additions & 0 deletions mpp/codec/dec/h265/h265d_codec.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,11 @@ typedef struct H265dContext {
**/
RK_S32 coded_width, coded_height;

/**
* framerate
**/
RK_U32 framerate_den, framerate_num;

RK_U8 *extradata;

RK_U32 extradata_size;
Expand Down
17 changes: 7 additions & 10 deletions mpp/codec/dec/h265/h265d_parser.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -504,19 +504,16 @@ static RK_S32 set_sps(HEVCContext *s, const HEVCSPS *sps)
s->vps = (HEVCVPS*) s->vps_list[s->sps->vps_id];

if (s->vps->vps_timing_info_present_flag) {
num = s->vps->vps_num_units_in_tick;
den = s->vps->vps_time_scale;
num = 2 * s->vps->vps_time_scale;
den = s->vps->vps_num_units_in_tick;
} else if (sps->vui.vui_timing_info_present_flag) {
num = sps->vui.vui_num_units_in_tick;
den = sps->vui.vui_time_scale;
num = 2 * sps->vui.vui_time_scale;
den = sps->vui.vui_num_units_in_tick;
}

if (num != 0 && den != 0) {
// s->h265dctx->time_base.num = num;
// s->h265dctx->time_base.den = den;
// av_reduce(&s->h265dctx->time_base.num, &s->h265dctx->time_base.den,
// num, den, 1 << 30);
}
// 0 if no timings
s->h265dctx->framerate_num = num;
s->h265dctx->framerate_den = den;

return 0;

Expand Down
2 changes: 2 additions & 0 deletions mpp/codec/dec/h265/h265d_refs.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ static HEVCFrame *alloc_frame(HEVCContext *s)
mpp_frame_set_poc(frame->frame, s->poc);
mpp_frame_set_color_primaries(frame->frame, s->sps->vui.colour_primaries);
mpp_frame_set_color_trc(frame->frame, s->sps->vui.transfer_characteristic);
mpp_frame_set_time_scale(frame->frame, s->h265dctx->framerate_num);
mpp_frame_set_num_units_in_tick(frame->frame, s->h265dctx->framerate_den);
h265d_dbg(H265D_DBG_GLOBAL, "w_stride %d h_stride %d\n", s->h265dctx->coded_width, s->h265dctx->coded_height);
ret = mpp_buf_slot_get_unused(s->slots, &frame->slot_index);
mpp_assert(ret == MPP_OK);
Expand Down