linux-kernel - [PATCH 5.4 151/203] media: hantro: h264: Fix the frame

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20200116231758.077259817@linuxfoundation.org>
Date:   Fri, 17 Jan 2020 00:17:48 +0100
From:   Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:     linux-kernel@...r.kernel.org
Cc:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        stable@...r.kernel.org,
        Francois Buergisser <fbuergisser@...gle.com>,
        Boris Brezillon <boris.brezillon@...labora.com>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        Francois Buergisser <fbuergisser@...omium.org>,
        Tomasz Figa <tfiga@...omium.org>,
        Hans Verkuil <hverkuil-cisco@...all.nl>,
        Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
Subject: [PATCH 5.4 151/203] media: hantro: h264: Fix the frame_num wraparound case

From: Boris Brezillon <boris.brezillon@...labora.com>

commit 9db5f87f6723678a7e7e5e3165439c5c4378edbb upstream.

Step '8.2.4.1 Decoding process for picture numbers' was missing in the
reflist creation logic, leading to invalid P reflists when a
->frame_num wraparound happens.

Fixes: a9471e25629b ("media: hantro: Add core bits to support H264 decoding")
Reported-by: Francois Buergisser <fbuergisser@...gle.com>
Signed-off-by: Boris Brezillon <boris.brezillon@...labora.com>
Reviewed-by: Philipp Zabel <p.zabel@...gutronix.de>
Tested-by: Philipp Zabel <p.zabel@...gutronix.de>
Tested-by: Francois Buergisser <fbuergisser@...omium.org>
Reviewed-by: Tomasz Figa <tfiga@...omium.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@...all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@...uxfoundation.org>

---
 drivers/staging/media/hantro/hantro_h264.c |   22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -271,6 +271,7 @@ struct hantro_h264_reflist_builder {
 	const struct v4l2_h264_dpb_entry *dpb;
 	s32 pocs[HANTRO_H264_DPB_SIZE];
 	u8 unordered_reflist[HANTRO_H264_DPB_SIZE];
+	int frame_nums[HANTRO_H264_DPB_SIZE];
 	s32 curpoc;
 	u8 num_valid;
 };
@@ -294,13 +295,20 @@ static void
 init_reflist_builder(struct hantro_ctx *ctx,
 		     struct hantro_h264_reflist_builder *b)
 {
+	const struct v4l2_ctrl_h264_slice_params *slice_params;
 	const struct v4l2_ctrl_h264_decode_params *dec_param;
+	const struct v4l2_ctrl_h264_sps *sps;
 	struct vb2_v4l2_buffer *buf = hantro_get_dst_buf(ctx);
 	const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
 	struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q;
+	int cur_frame_num, max_frame_num;
 	unsigned int i;
 
 	dec_param = ctx->h264_dec.ctrls.decode;
+	slice_params = &ctx->h264_dec.ctrls.slices[0];
+	sps = ctx->h264_dec.ctrls.sps;
+	max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4);
+	cur_frame_num = slice_params->frame_num;
 
 	memset(b, 0, sizeof(*b));
 	b->dpb = dpb;
@@ -318,6 +326,18 @@ init_reflist_builder(struct hantro_ctx *
 			continue;
 
 		buf = to_vb2_v4l2_buffer(vb2_get_buffer(cap_q, buf_idx));
+
+		/*
+		 * Handle frame_num wraparound as described in section
+		 * '8.2.4.1 Decoding process for picture numbers' of the spec.
+		 * TODO: This logic will have to be adjusted when we start
+		 * supporting interlaced content.
+		 */
+		if (dpb[i].frame_num > cur_frame_num)
+			b->frame_nums[i] = (int)dpb[i].frame_num - max_frame_num;
+		else
+			b->frame_nums[i] = dpb[i].frame_num;
+
 		b->pocs[i] = get_poc(buf->field, dpb[i].top_field_order_cnt,
 				     dpb[i].bottom_field_order_cnt);
 		b->unordered_reflist[b->num_valid] = i;
@@ -353,7 +373,7 @@ static int p_ref_list_cmp(const void *pt
 	 * ascending order.
 	 */
 	if (!(a->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM))
-		return b->frame_num - a->frame_num;
+		return builder->frame_nums[idxb] - builder->frame_nums[idxa];
 
 	return a->pic_num - b->pic_num;
 }