Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipc4: mixin/mixout: Switch to use sink/source API #8537

Merged
merged 4 commits into from
Jan 3, 2024

Conversation

serhiy-katsyuba-intel
Copy link
Contributor

Modifications to mixin/mixout implementation to switch to use sink/source API.

Copy link
Contributor

@btian1 btian1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

later, we can try with hifi4 instructions to optimize those buffer wrappers.

break;
default:
comp_err(dev, "unsupported data format %d", fmt);
return -EINVAL;
}

if (!md->normal_mix_channel || !md->mute_channel) {
if (!md->normal_mix_channel) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suggests that md->mute_channel was never NULL, which is probably correct

uint16_t sink_index, struct audio_stream *sink,
uint32_t start_frame, uint32_t mixed_frames,
const struct audio_stream *source, uint32_t frame_count)
static int mix(struct comp_dev *dev, const struct mixin_data *mixin_data,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a blocker, but if you do end up doing another round - maybe you could come up with a longer and more unique name, I guess grepping for "mix" might be a bit difficult...

if (!mixout_data->acquired_buf.ptr) {
struct sof_sink *sink = mixout_mod->sinks[0];
uint32_t free_bytes = sink_get_free_size(sink);
uint32_t buf_size;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all these byte counts should really be size_t - we're getting 64-bit architectures already, no need to punish them with 32-bit arithmetics. Throughout the code of course, not just here

static inline void cir_buf_set_zero(void *ptr, void *buf_addr, void *buf_end, uint32_t bytes)
{
uint32_t head_size = bytes;
uint32_t tail_size = 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too of course - size_t

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I replace those uint32_t with size_t in this PR or that will be done later as part of global update through all code base? That new cir_buf_set_zero() function is mostly a copy-paste of audio_stream_set_zero() located just above it. Those uint32_t came as a copy-paste from audio_stream_set_zero().

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't critical either, but it's my personal opinion - I do think that size_t is a better type for these variables. I won't block this PR if you don't and probably nobody else would (but I might be wrong), but I think it would be a good idea to do the change at least in locations where it wouldn't mean changing any global APIs, like for local variables and parameters of static functions.

Copy link
Contributor

@marcinszkudlinski marcinszkudlinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mixin_mixout.c line 593:
list_for_item(blist, &dev->bsink_list) {

don't use dev->bsink_list and dev->bsource_lists
don't assume comp_buffer is used

In general the module should not use any API od audio_stream or comp_buffer because it may be connecxted to something else.
All format settings like audio_stream_set_channels should be changed to sink/src functions (like sink_set_channels/source_set_channels)

@serhiy-katsyuba-intel
Copy link
Contributor Author

mixin_mixout.c line 593: list_for_item(blist, &dev->bsink_list) {

don't use dev->bsink_list and dev->bsource_lists don't assume comp_buffer is used

In general the module should not use any API od audio_stream or comp_buffer because it may be connecxted to something else. All format settings like audio_stream_set_channels should be changed to sink/src functions (like sink_set_channels/source_set_channels)

Done. Please re-review.


/* comp_verify_params() does not modify valid_sample_fmt (a BUG?), let's do this here */
audio_stream_fmt_conversion(mod->priv.cfg.base_cfg.audio_fmt.depth,
mod->priv.cfg.base_cfg.audio_fmt.valid_bit_depth,
&frame_fmt, &valid_fmt,
mod->priv.cfg.base_cfg.audio_fmt.s_type);

audio_stream_set_valid_fmt(&sink->stream, valid_fmt);
audio_stream_set_channels(&sink->stream, params->channels);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case, for eagle-eye reviewers :) , the removal of line with audio_stream_set_channels() is intentional. Channels number is already set above by comp_verify_params(), no need to set it again.

Copy link
Collaborator

@kv2019i kv2019i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty massive change. Codewise this seems pretty ok,.

I do note a couple of fails in CI:

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

void *buf_end;
void *ptr;
};

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have cir_buf_wrap() in audio_stream.h, should we have the "cir_" helpers in some common header as well. Seems pretty generic and not limited to mixinmixout.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best place is https://github.com/thesofproject/sof/blob/main/src/audio/sink_source_utils.c however we should use it also in loadable modules.

* enum sof_ipc_frame describes both container and sample size and so having one
* variable of this type is enough. frame_fmt is set by comp_verify_params(), however,
* valid_sample_fmt does not. Hence valid_sample_fmt is set below in a loop. If mess
* with having both frame_fmt and valid_sample_fmt in SOF is solved when this loop can
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit scary comment :) but yeah, this is is true. The distinction of container and valid bits was added afterwards to "enum sof_ipc_frame" and as the original definition is part of the IPC interface, IPC4 concepts were added on top. I don't think we need to drag this along all over SOF codebase though. So it should be possible to set/fmt fmt with a single interface. But alas, this is a cleanup for another PR.

src/audio/mixin_mixout/mixin_mixout_generic.c Show resolved Hide resolved
void *buf_end;
void *ptr;
};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best place is https://github.com/thesofproject/sof/blob/main/src/audio/sink_source_utils.c however we should use it also in loadable modules.

@serhiy-katsyuba-intel
Copy link
Contributor Author

I do note a couple of fails in CI:

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

This @RanderWang commit 1855581 fixes the problems reported by CI. Verified on this draft PR #8576.

So let's wait for Rander's PR #8580 to be merged (or that commit moved to separate PR if that speeds up merging).

@lgirdwood
Copy link
Member

I do note a couple of fails in CI:

Multiple pipeline checks seem to be failing, but in quite strange way (no errors in FW log, just wrong status reported at end of tests). But definitely fails across large set of DUTs, so needs to be clarified before can be merged.

This @RanderWang commit 1855581 fixes the problems reported by CI. Verified on this draft PR #8576.

So let's wait for Rander's PR #8580 to be merged (or that commit moved to separate PR if that speeds up merging).

@RanderWang probably best to make the fix a new PR (without smart amp) as it will merge faster.

@RanderWang
Copy link
Collaborator

@lgirdwood @serhiy-katsyuba-intel can you help to check this PR ? #8594. Thanks!

src/audio/mixin_mixout/mixin_mixout.c Outdated Show resolved Hide resolved
src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved
src/audio/mixin_mixout/mixin_mixout.c Show resolved Hide resolved
@serhiy-katsyuba-intel
Copy link
Contributor Author

@lgirdwood @serhiy-katsyuba-intel can you help to check this PR ? #8594. Thanks!

I've cherry-picked commits from your PR and added to this draft PR https://github.com/thesofproject/sof/pull/8576/commits to be verified by CI. There is mixin/mixout re-implementation there which uses sink/source API and so CI will test mixout with multiple sources case (previously it failed without your fix).
Your previous versions of that fix was tested on that draft PR and it worked fine.

@serhiy-katsyuba-intel
Copy link
Contributor Author

Rebased to latest main.

@lgirdwood
Copy link
Member

lgirdwood commented Dec 11, 2023

@serhiy-katsyuba-intel some build errors with stricter host compilation, it picked out some pointer mismatches. See ALSA plugin build in CI.

/usr/bin/cc -DBLD_COUNTERS=0 -Dsof_mixer_EXPORTS -DRELATIVE_FILE=\"src/audio/mixin_mixout/mixin_mixout.c\" -I/home/runner/work/sof/sof/tools/rimage/src/include -I/home/runner/work/sof/sof/src/include -I"/home/runner/work/sof/sof/build plugin/sof_ep/build/generated/include" -I/home/runner/work/sof/sof/src/arch/host/include -I/home/runner/work/sof/sof/src/platform/library/include -I/home/runner/work/sof/sof/posix/include -fPIC -g -O3 -fPIC -DPIC -std=c99 -std=gnu99 -fgnu89-inline -Wall -Werror -Wmissing-prototypes -Wimplicit-fallthrough -Wno-pointer-to-int-cast -Wno-int-to-pointer-cast -Wpointer-arith -DCONFIG_LIBRARY "-imacros/home/runner/work/sof/sof/build plugin/sof_ep/build/library_autoconfig.h" -MD -MT src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o -MF src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o.d -o src/audio/CMakeFiles/sof_mixer.dir/mixin_mixout/mixin_mixout.c.o -c /home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c
/home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c: In function ‘mixin_process’:
/home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c:415:79: error: passing argument 5 of ‘sink_get_buffer’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  415 |                                         &mixout_data->acquired_buf.buf_start, &buf_size);
      |                                                                               ^~~~~~~~~
      |                                                                               |
      |                                                                               uint32_t * {aka unsigned int *}
In file included from /home/runner/work/sof/sof/src/include/sof/audio/sink_api.h:10,
                 from /home/runner/work/sof/sof/src/include/sof/audio/audio_stream.h:18,
                 from /home/runner/work/sof/sof/src/include/sof/audio/buffer.h:11,
                 from /home/runner/work/sof/sof/src/audio/mixin_mixout/mixin_mixout.c:5:
/home/runner/work/sof/sof/src/include/module/audio/sink_api.h:171:29: note: expected ‘size_t *’ {aka ‘long unsigned int *’} but argument is of type ‘uint32_t *’ {aka ‘unsigned int *’}
  171 |                     size_t *buffer_size);
      |                     ~~~~~~~~^~~~~~~~~~~
cc1: all warnings being treated as errors

@serhiy-katsyuba-intel
Copy link
Contributor Author

serhiy-katsyuba-intel commented Dec 18, 2023

@serhiy-katsyuba-intel some build errors with stricter host compilation, it picked out some pointer mismatches. See ALSA plugin build in CI.

Fixed.

@serhiy-katsyuba-intel
Copy link
Contributor Author

@wszypelt , Internal Intel CI seems stuck.

@wszypelt
Copy link

@serhiy-katsyuba-intel Thanks for the info, I added it to the queue, the results should be available within 40 minutes

@serhiy-katsyuba-intel
Copy link
Contributor Author

@wszypelt , when I click on a failed Internal Intel CI System/merge/build -- blank page is shown. How can I find out what (tests?) have failed?

@wszypelt
Copy link

wszypelt commented Dec 19, 2023

@serhiy-katsyuba-intel Unfortunately, at the moment, the results are only available at the internal service
failed are in two tests on MTL and LNL
host_loop_pipeline_set_states_stress[PipelineSetState and host_loop_pipeline_set_states_stress[MultiPipelineSetState]
after Setting pipeline 0 state to RUNNING
DSP PANIC occured

@lgirdwood
Copy link
Member

lgirdwood commented Dec 19, 2023

@serhiy-katsyuba-intel Unfortunately, at the moment, the results are only available at the internal service failed are in two tests on MTL and LNL host_loop_pipeline_set_states_stress[PipelineSetState and host_loop_pipeline_set_states_stress[MultiPipelineSetState] after Setting pipeline 0 state to RUNNING DSP PANIC occured

@serhiy-katsyuba-intel btw, if it makes it easier, we could split the PR and merge the simpler patches 1st that pass CI whilst the larger more complex patches are being fixed to pass CI ?

@serhiy-katsyuba-intel
Copy link
Contributor Author

SOFCI TEST

@serhiy-katsyuba-intel
Copy link
Contributor Author

Temporary put to draft to investigate CI failure. Extracted cleanup commits to separate PR #8655.

@serhiy-katsyuba-intel
Copy link
Contributor Author

CI failure is caused by a recent regression introduced by #8594

Previously sinks/sources were setup in .prepare(), after that PR they are setup in .bind() and .unbind(). However, module_adapter_reset() also cleans sinks and sources arrays and set num_of_sources = num_of_sinks = 0. The failed test is a stress test which does many iterations of PAUSE -> RESET -> RUN. First iteration works fine, second iteration leads to DSP panic, as sinks and sources array are cleared on RESET and were never setup back (as they can only be setup on .bind() and .unbind()).

ACHTUNG: So if topology uses some module which utilizes sink/source API, after RESET such module will not work properly (or just crash)!

cc: @RanderWang , @marcinszkudlinski, @lgirdwood.

@serhiy-katsyuba-intel
Copy link
Contributor Author

PR with the fix for CI problem: #8657.

Modifications to mixin/mixout .process() and .reset() implementations
to switch to use sink/source API.

Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
In channel remapping mode mixin sinks could have different number of
channels. Since channel remapping mode has been removed, no need to
support individual channel number setup for each sink.

Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
The removed code may falsely fail for freq like 44.1 kHz.

Also, we generally do not check for sufficient buffer size in module
.prepare() handler. That should be/is done elsewhere. No need to do
the exception for mixout component.

Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
Modification of mixin/mixout .prepare() handler to use sink/source API.

Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
@lgirdwood
Copy link
Member

PR with the fix for CI problem: #8657.

@serhiy-katsyuba-intel now merged, can you rebase. Thanks !

@serhiy-katsyuba-intel
Copy link
Contributor Author

@serhiy-katsyuba-intel now merged, can you rebase. Thanks !

Rebased. CI has finished with kind of success: timeout on 2 cavs tests.

@kv2019i
Copy link
Collaborator

kv2019i commented Jan 3, 2024

The system PM timeouts in https://sof-ci.01.org/sofpr/PR8537/build1400/devicetest/index.html can be ignored, not related to this PR. Proceeding with merge.

@kv2019i kv2019i merged commit 9315ada into thesofproject:main Jan 3, 2024
43 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants