ipc4: mixin/mixout: Switch to use sink/source API #8537

serhiy-katsyuba-intel · 2023-11-27T20:07:19Z

Modifications to mixin/mixout implementation to switch to use sink/source API.

btian1

later, we can try with hifi4 instructions to optimize those buffer wrappers.

lyakh · 2023-11-29T13:30:11Z

src/audio/mixin_mixout/mixin_mixout.c

 		break;
 	default:
 		comp_err(dev, "unsupported data format %d", fmt);
 		return -EINVAL;
 	}

-	if (!md->normal_mix_channel || !md->mute_channel) {
+	if (!md->normal_mix_channel) {


This suggests that md->mute_channel was never NULL, which is probably correct

lyakh · 2023-11-29T13:32:16Z

src/audio/mixin_mixout/mixin_mixout.c

-			 uint16_t sink_index, struct audio_stream *sink,
-			 uint32_t start_frame, uint32_t mixed_frames,
-			 const struct audio_stream *source, uint32_t frame_count)
+static int mix(struct comp_dev *dev, const struct mixin_data *mixin_data,


not a blocker, but if you do end up doing another round - maybe you could come up with a longer and more unique name, I guess grepping for "mix" might be a bit difficult...

lyakh · 2023-11-29T13:52:39Z

src/audio/mixin_mixout/mixin_mixout.c

+		if (!mixout_data->acquired_buf.ptr) {
+			struct sof_sink *sink = mixout_mod->sinks[0];
+			uint32_t free_bytes = sink_get_free_size(sink);
+			uint32_t buf_size;


all these byte counts should really be size_t - we're getting 64-bit architectures already, no need to punish them with 32-bit arithmetics. Throughout the code of course, not just here

lyakh · 2023-11-29T14:01:36Z

src/include/sof/audio/audio_stream.h

+static inline void cir_buf_set_zero(void *ptr, void *buf_addr, void *buf_end, uint32_t bytes)
+{
+	uint32_t head_size = bytes;
+	uint32_t tail_size = 0;


here too of course - size_t

Should I replace those uint32_t with size_t in this PR or that will be done later as part of global update through all code base? That new cir_buf_set_zero() function is mostly a copy-paste of audio_stream_set_zero() located just above it. Those uint32_t came as a copy-paste from audio_stream_set_zero().

This isn't critical either, but it's my personal opinion - I do think that size_t is a better type for these variables. I won't block this PR if you don't and probably nobody else would (but I might be wrong), but I think it would be a good idea to do the change at least in locations where it wouldn't mean changing any global APIs, like for local variables and parameters of static functions.

src/audio/mixin_mixout/mixin_mixout.c

marcinszkudlinski

mixin_mixout.c line 593:
list_for_item(blist, &dev->bsink_list) {

don't use dev->bsink_list and dev->bsource_lists
don't assume comp_buffer is used

In general the module should not use any API od audio_stream or comp_buffer because it may be connecxted to something else.
All format settings like audio_stream_set_channels should be changed to sink/src functions (like sink_set_channels/source_set_channels)

serhiy-katsyuba-intel · 2023-12-01T18:24:37Z

mixin_mixout.c line 593: list_for_item(blist, &dev->bsink_list) {

don't use dev->bsink_list and dev->bsource_lists don't assume comp_buffer is used

In general the module should not use any API od audio_stream or comp_buffer because it may be connecxted to something else. All format settings like audio_stream_set_channels should be changed to sink/src functions (like sink_set_channels/source_set_channels)

Done. Please re-review.

serhiy-katsyuba-intel · 2023-12-01T18:46:08Z

src/audio/mixin_mixout/mixin_mixout.c


 	/* comp_verify_params() does not modify valid_sample_fmt (a BUG?), let's do this here */
 	audio_stream_fmt_conversion(mod->priv.cfg.base_cfg.audio_fmt.depth,
 				    mod->priv.cfg.base_cfg.audio_fmt.valid_bit_depth,
 				    &frame_fmt, &valid_fmt,
 				    mod->priv.cfg.base_cfg.audio_fmt.s_type);

-	audio_stream_set_valid_fmt(&sink->stream, valid_fmt);
-	audio_stream_set_channels(&sink->stream, params->channels);


Just in case, for eagle-eye reviewers :) , the removal of line with audio_stream_set_channels() is intentional. Channels number is already set above by comp_verify_params(), no need to set it again.

kv2019i

This is a pretty massive change. Codewise this seems pretty ok,.

I do note a couple of fails in CI:

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

kv2019i · 2023-12-04T13:02:26Z

src/include/ipc4/mixin_mixout.h

+	void *buf_end;
+	void *ptr;
+};
+


We have cir_buf_wrap() in audio_stream.h, should we have the "cir_" helpers in some common header as well. Seems pretty generic and not limited to mixinmixout.

best place is https://github.com/thesofproject/sof/blob/main/src/audio/sink_source_utils.c however we should use it also in loadable modules.

kv2019i · 2023-12-04T13:19:08Z

src/audio/mixin_mixout/mixin_mixout.c

+	 * enum sof_ipc_frame describes both container and sample size and so having one
+	 * variable of this type is enough. frame_fmt is set by comp_verify_params(), however,
+	 * valid_sample_fmt does not. Hence valid_sample_fmt is set below in a loop. If mess
+	 * with having both frame_fmt and valid_sample_fmt in SOF is solved when this loop can


This is a bit scary comment :) but yeah, this is is true. The distinction of container and valid bits was added afterwards to "enum sof_ipc_frame" and as the original definition is part of the IPC interface, IPC4 concepts were added on top. I don't think we need to drag this along all over SOF codebase though. So it should be possible to set/fmt fmt with a single interface. But alas, this is a cleanup for another PR.

src/audio/mixin_mixout/mixin_mixout_generic.c

pjdobrowolski · 2023-12-06T12:17:06Z

src/include/ipc4/mixin_mixout.h

+	void *buf_end;
+	void *ptr;
+};
+


best place is https://github.com/thesofproject/sof/blob/main/src/audio/sink_source_utils.c however we should use it also in loadable modules.

serhiy-katsyuba-intel · 2023-12-06T14:59:57Z

I do note a couple of fails in CI:

https://sof-ci.01.org/sofpr/PR8537/build638/devicetest/index.html

https://sof-ci.01.org/sofpr/PR8537/build637/devicetest/index.html

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

This @RanderWang commit 1855581 fixes the problems reported by CI. Verified on this draft PR #8576.

So let's wait for Rander's PR #8580 to be merged (or that commit moved to separate PR if that speeds up merging).

lgirdwood · 2023-12-06T15:53:45Z

I do note a couple of fails in CI:

https://sof-ci.01.org/sofpr/PR8537/build638/devicetest/index.html

https://sof-ci.01.org/sofpr/PR8537/build637/devicetest/index.html

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

This @RanderWang commit 1855581 fixes the problems reported by CI. Verified on this draft PR #8576.

So let's wait for Rander's PR #8580 to be merged (or that commit moved to separate PR if that speeds up merging).

@RanderWang probably best to make the fix a new PR (without smart amp) as it will merge faster.

RanderWang · 2023-12-08T07:42:50Z

@lgirdwood @serhiy-katsyuba-intel can you help to check this PR ? #8594. Thanks!

src/audio/mixin_mixout/mixin_mixout.c

serhiy-katsyuba-intel · 2023-12-08T13:58:18Z

@lgirdwood @serhiy-katsyuba-intel can you help to check this PR ? #8594. Thanks!

I've cherry-picked commits from your PR and added to this draft PR https://github.com/thesofproject/sof/pull/8576/commits to be verified by CI. There is mixin/mixout re-implementation there which uses sink/source API and so CI will test mixout with multiple sources case (previously it failed without your fix).
Your previous versions of that fix was tested on that draft PR and it worked fine.

serhiy-katsyuba-intel · 2023-12-08T17:31:19Z

Rebased to latest main.

lgirdwood · 2023-12-11T16:11:28Z

@serhiy-katsyuba-intel some build errors with stricter host compilation, it picked out some pointer mismatches. See ALSA plugin build in CI.

/usr/bin/cc -DBLD_COUNTERS=0 -Dsof_mixer_EXPORTS -DRELATIVE_FILE=\"src/audio/mixin_mixout/mixin_mixout.c\" -I/home/runner/work/sof/sof/tools/rimage/src/include -I/home/runner/work/sof/sof/src/include -I"/home/runner/work/sof/sof/build plugin/sof_ep/build/generated/include" -I/home/runner/work/sof/sof/src/arch/host/include -I/home/runner/work/sof/sof/src/platform/library/include -I/home/runner/work/sof/sof/posix/include -fPIC -g -O3 -fPIC -DPIC -std=c99 -std=gnu99 -fgnu89-inline -Wall -Werror -Wmissing-prototypes -Wimplicit-fallthrough -Wno-pointer-to-int-cast -Wno-int-to-pointer-cast -Wpointer-arith -DCONFIG_LIBRARY "-imacros/home/runner/work/sof/sof/build plugin/sof_ep/build/library_autoconfig.h" -MD -MT src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o -MF src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o.d -o src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o -c /home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c
/home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c: In function ‘mixin_process’:
/home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c:415:79: error: passing argument 5 of ‘sink_get_buffer’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  415 |                                         &mixout_data->acquired_buf.buf_start, &buf_size);
      |                                                                               ^~~~~~~~~
      |                                                                               |
      |                                                                               uint32_t * {aka unsigned int *}
In file included from /home/runner/work/sof/sof/src/include/sof/audio/sink_api.h:10,
                 from /home/runner/work/sof/sof/src/include/sof/audio/audio_stream.h:18,
                 from /home/runner/work/sof/sof/src/include/sof/audio/buffer.h:11,
                 from /home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c:5:
/home/runner/work/sof/sof/src/include/module/audio/sink_api.h:171:29: note: expected ‘size_t *’ {aka ‘long unsigned int *’} but argument is of type ‘uint32_t *’ {aka ‘unsigned int *’}
  171 |                     size_t *buffer_size);
      |                     ~~~~~~~~^~~~~~~~~~~
cc1: all warnings being treated as errors

serhiy-katsyuba-intel · 2023-12-18T10:03:42Z

@serhiy-katsyuba-intel some build errors with stricter host compilation, it picked out some pointer mismatches. See ALSA plugin build in CI.

Fixed.

serhiy-katsyuba-intel · 2023-12-19T08:57:16Z

@wszypelt , Internal Intel CI seems stuck.

wszypelt · 2023-12-19T09:25:12Z

@serhiy-katsyuba-intel Thanks for the info, I added it to the queue, the results should be available within 40 minutes

serhiy-katsyuba-intel · 2023-12-19T13:12:53Z

@wszypelt , when I click on a failed Internal Intel CI System/merge/build -- blank page is shown. How can I find out what (tests?) have failed?

wszypelt · 2023-12-19T14:41:33Z

@serhiy-katsyuba-intel Unfortunately, at the moment, the results are only available at the internal service
failed are in two tests on MTL and LNL
host_loop_pipeline_set_states_stress[PipelineSetState and host_loop_pipeline_set_states_stress[MultiPipelineSetState]
after Setting pipeline 0 state to RUNNING
DSP PANIC occured

lgirdwood · 2023-12-19T18:27:03Z

@serhiy-katsyuba-intel Unfortunately, at the moment, the results are only available at the internal service failed are in two tests on MTL and LNL host_loop_pipeline_set_states_stress[PipelineSetState and host_loop_pipeline_set_states_stress[MultiPipelineSetState] after Setting pipeline 0 state to RUNNING DSP PANIC occured

@serhiy-katsyuba-intel btw, if it makes it easier, we could split the PR and merge the simpler patches 1st that pass CI whilst the larger more complex patches are being fixed to pass CI ?

serhiy-katsyuba-intel · 2023-12-19T22:16:08Z

SOFCI TEST

serhiy-katsyuba-intel · 2023-12-20T09:58:43Z

Temporary put to draft to investigate CI failure. Extracted cleanup commits to separate PR #8655.

serhiy-katsyuba-intel · 2023-12-20T12:15:53Z

CI failure is caused by a recent regression introduced by #8594

Previously sinks/sources were setup in .prepare(), after that PR they are setup in .bind() and .unbind(). However, module_adapter_reset() also cleans sinks and sources arrays and set num_of_sources = num_of_sinks = 0. The failed test is a stress test which does many iterations of PAUSE -> RESET -> RUN. First iteration works fine, second iteration leads to DSP panic, as sinks and sources array are cleared on RESET and were never setup back (as they can only be setup on .bind() and .unbind()).

ACHTUNG: So if topology uses some module which utilizes sink/source API, after RESET such module will not work properly (or just crash)!

cc: @RanderWang , @marcinszkudlinski, @lgirdwood.

serhiy-katsyuba-intel · 2023-12-20T13:11:28Z

PR with the fix for CI problem: #8657.

Modifications to mixin/mixout .process() and .reset() implementations to switch to use sink/source API. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

In channel remapping mode mixin sinks could have different number of channels. Since channel remapping mode has been removed, no need to support individual channel number setup for each sink. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

The removed code may falsely fail for freq like 44.1 kHz. Also, we generally do not check for sufficient buffer size in module .prepare() handler. That should be/is done elsewhere. No need to do the exception for mixout component. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

Modification of mixin/mixout .prepare() handler to use sink/source API. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

lgirdwood · 2023-12-21T16:04:28Z

PR with the fix for CI problem: #8657.

@serhiy-katsyuba-intel now merged, can you rebase. Thanks !

serhiy-katsyuba-intel · 2023-12-21T18:29:56Z

@serhiy-katsyuba-intel now merged, can you rebase. Thanks !

Rebased. CI has finished with kind of success: timeout on 2 cavs tests.

kv2019i · 2024-01-03T11:09:16Z

The system PM timeouts in https://sof-ci.01.org/sofpr/PR8537/build1400/devicetest/index.html can be ignored, not related to this PR. Proceeding with merge.

serhiy-katsyuba-intel requested review from RanderWang, marcinszkudlinski, pblaszko, lgirdwood, plbossart, mmaka1, lbetlej, dbaluta and kv2019i as code owners November 27, 2023 20:17

serhiy-katsyuba-intel force-pushed the mixer_ssapi2 branch from 68acb14 to 58ca058 Compare November 28, 2023 08:33

btian1 approved these changes Nov 29, 2023

View reviewed changes

lyakh reviewed Nov 29, 2023

View reviewed changes

marcinszkudlinski requested changes Nov 29, 2023

View reviewed changes

src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved

src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved

src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved

marcinszkudlinski requested changes Nov 30, 2023

View reviewed changes

serhiy-katsyuba-intel force-pushed the mixer_ssapi2 branch from 58ca058 to ac78e05 Compare December 1, 2023 18:15

serhiy-katsyuba-intel commented Dec 1, 2023

View reviewed changes

marcinszkudlinski approved these changes Dec 4, 2023

View reviewed changes

kv2019i reviewed Dec 4, 2023

View reviewed changes

kv2019i mentioned this pull request Dec 5, 2023

Audio:mixin_mixout: move mixin_mixouot header file to module folder #8553

Merged

pjdobrowolski requested changes Dec 6, 2023

View reviewed changes

serhiy-katsyuba-intel mentioned this pull request Dec 6, 2023

Add dummy Loadable smart amp support #8580

Merged

pjdobrowolski approved these changes Dec 8, 2023

View reviewed changes

lyakh reviewed Dec 8, 2023

View reviewed changes

src/audio/mixin_mixout/mixin_mixout.c Outdated Show resolved Hide resolved

src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved

src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved

serhiy-katsyuba-intel force-pushed the mixer_ssapi2 branch from ac78e05 to 0230d97 Compare December 8, 2023 17:17

lgirdwood approved these changes Dec 11, 2023

View reviewed changes

serhiy-katsyuba-intel force-pushed the mixer_ssapi2 branch from 0230d97 to f45ef06 Compare December 18, 2023 09:30

serhiy-katsyuba-intel mentioned this pull request Dec 20, 2023

ipc4: mixin/mixout cleanup after channel remapping mode been removed #8655

Merged

serhiy-katsyuba-intel added 4 commits December 21, 2023 12:28

ipc4: mixin/mixout: Modify .process() impl to use sink/source API

8c3d605

Modifications to mixin/mixout .process() and .reset() implementations to switch to use sink/source API. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

ipc4: mixin/mixout: Modify .prepare() to use sink/source API

844cc62

Modification of mixin/mixout .prepare() handler to use sink/source API. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>

serhiy-katsyuba-intel force-pushed the mixer_ssapi2 branch from f45ef06 to 844cc62 Compare December 21, 2023 16:06

kv2019i merged commit 9315ada into thesofproject:main Jan 3, 2024
43 of 44 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ipc4: mixin/mixout: Switch to use sink/source API #8537

ipc4: mixin/mixout: Switch to use sink/source API #8537

serhiy-katsyuba-intel commented Nov 27, 2023

btian1 left a comment

lyakh Nov 29, 2023

lyakh Nov 29, 2023

lyakh Nov 29, 2023

lyakh Nov 29, 2023

serhiy-katsyuba-intel Dec 1, 2023

lyakh Dec 4, 2023

marcinszkudlinski left a comment

serhiy-katsyuba-intel commented Dec 1, 2023

serhiy-katsyuba-intel Dec 1, 2023

kv2019i left a comment

kv2019i Dec 4, 2023

pjdobrowolski Dec 6, 2023

kv2019i Dec 4, 2023

pjdobrowolski Dec 6, 2023

serhiy-katsyuba-intel commented Dec 6, 2023

lgirdwood commented Dec 6, 2023

RanderWang commented Dec 8, 2023

serhiy-katsyuba-intel commented Dec 8, 2023

serhiy-katsyuba-intel commented Dec 8, 2023

lgirdwood commented Dec 11, 2023 •

edited

Loading

serhiy-katsyuba-intel commented Dec 18, 2023 •

edited

Loading

serhiy-katsyuba-intel commented Dec 19, 2023

wszypelt commented Dec 19, 2023

serhiy-katsyuba-intel commented Dec 19, 2023

wszypelt commented Dec 19, 2023 •

edited

Loading

lgirdwood commented Dec 19, 2023 •

edited

Loading

serhiy-katsyuba-intel commented Dec 19, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

lgirdwood commented Dec 21, 2023

serhiy-katsyuba-intel commented Dec 21, 2023

kv2019i commented Jan 3, 2024

ipc4: mixin/mixout: Switch to use sink/source API #8537

ipc4: mixin/mixout: Switch to use sink/source API #8537

Conversation

serhiy-katsyuba-intel commented Nov 27, 2023

btian1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcinszkudlinski left a comment

Choose a reason for hiding this comment

serhiy-katsyuba-intel commented Dec 1, 2023

Choose a reason for hiding this comment

kv2019i left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

serhiy-katsyuba-intel commented Dec 6, 2023

lgirdwood commented Dec 6, 2023

RanderWang commented Dec 8, 2023

serhiy-katsyuba-intel commented Dec 8, 2023

serhiy-katsyuba-intel commented Dec 8, 2023

lgirdwood commented Dec 11, 2023 • edited Loading

serhiy-katsyuba-intel commented Dec 18, 2023 • edited Loading

serhiy-katsyuba-intel commented Dec 19, 2023

wszypelt commented Dec 19, 2023

serhiy-katsyuba-intel commented Dec 19, 2023

wszypelt commented Dec 19, 2023 • edited Loading

lgirdwood commented Dec 19, 2023 • edited Loading

serhiy-katsyuba-intel commented Dec 19, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

serhiy-katsyuba-intel commented Dec 20, 2023

lgirdwood commented Dec 21, 2023

serhiy-katsyuba-intel commented Dec 21, 2023

kv2019i commented Jan 3, 2024

lgirdwood commented Dec 11, 2023 •

edited

Loading

serhiy-katsyuba-intel commented Dec 18, 2023 •

edited

Loading

wszypelt commented Dec 19, 2023 •

edited

Loading

lgirdwood commented Dec 19, 2023 •

edited

Loading