Skip to content

ASoC: SOF: ipc4: Make multi-pin format selection and propagation pin-aware - #6030

Open
ujfalusi wants to merge 19 commits into
thesofproject:topic/sof-devfrom
ujfalusi:peter/sof/pr/multi-pin-multi-format
Open

ujfalusi wants to merge 19 commits into
thesofproject:topic/sof-devfrom
ujfalusi:peter/sof/pr/multi-pin-multi-format

Conversation

@ujfalusi

@ujfalusi ujfalusi commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Separate module format configuration from the format propagated along each connection. Previously, format lookup could consider formats belonging to unrelated pins, while already-prepared modules propagated the pin 0 format regardless of the output feeding the path.

This series:

Passes the connection’s output pin to widget preparation.
Preserves independent parameters across branching paths, including Multi-WOV branches that reconverge.
Configures process modules using pin 0 and propagates the configured format of the actual output pin.
Handles copier secondary outputs separately, preserving bind-time format configuration.
Supports paths entering modules through secondary input pins.
Restricts parameter-change detection to pin 0 formats and includes sample type in format-equivalence checks.
Validates complete pin-binding arrays and warns about ambiguous process-module pin formats.
Also includes the upstream process-widget pin-format leak fix.

Validation
Focused W=1 builds and strict checkpatch checks.
DUT: ECNS capture and silent speaker/deepbuffer playback passed.
Multi-WOV PCM12 prepares and reaches RUNNING without kernel warnings/errors; frame delivery remains unverified because capture stayed in the listening state.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The topology changes can propagate incorrect formats across fanout connections, inflate pipeline resource accounting during repeated preparation, and accept formats incompatible with the bound input pin.

3 open issues
What changed in this PR

This PR makes SOF IPC4 format selection and propagation pin-aware, separating module configuration from the format passed along each connection.

Changes:

  • Passes output-pin identity through widget preparation and isolates parameters across branching paths.
  • Selects and propagates process and copier formats per pin, including secondary-pin handling.
  • Validates complete pin bindings, checks ambiguous formats, and fixes process-format cleanup.
File Description
sound/​soc/​sof/​topology.c Rejects partially defined pin-binding arrays.
sound/​soc/​sof/​sof-audio.h Adds output-pin identifiers and extends the preparation interface.
sound/​soc/​sof/​sof-audio.c Prepares each connection with pin identity and independent branch parameters.
sound/​soc/​sof/​ipc4-topology.h Clarifies that parameter-change detection concerns pin 0.
sound/​soc/​sof/​ipc4-topology.c Implements pin-aware format handling, validation, and error-path cleanup.

Review effort: Balanced

Comment thread sound/soc/sof/ipc4-topology.c Resolved
Comment thread sound/soc/sof/ipc4-topology.c Resolved
Comment thread sound/soc/sof/ipc4-topology.c Resolved

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Complicated series, but commit messages look good and explain the changes. There is unfortunately lot of special logic around pin0, but that is fundamental of the firmware interface and kernel can do nothing about it. Minor comment(s) inline.

}

copier_data->gtw_cfg.node_id &= ~SOF_IPC4_NODE_INDEX_MASK;
copier_data->gtw_cfg.node_id |= SOF_IPC4_NODE_INDEX(host_dma_id);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to explain this simpler, but I think this commit has complexities:
""Select the copier output format among the pin 0 formats only and modify
the params for the next widget with the format of the output pin feeding
the path. The format of a pin above 0 is selected from its formats in
topology""
This seems selfcontracticting. First it is said selecting format amont pin 0 formats only, but then in next sentence, format selection is done based on other factors. Was the intention that for pin 0, only formats in pin 0 are considered?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly, the pin 0 restriction applies to the copier’s initialization format. Downstream parameter propagation separately uses the format of the pin feeding that connection. I’ll clarify it.

Copilot AI balanced review requested due to automatic review settings November 10, 2026 13:37
@ujfalusi
ujfalusi force-pushed the peter/sof/pr/multi-pin-multi-format branch from aa84a33 to ba430ff Compare November 10, 2026 13:37
@ujfalusi

Copy link
Copy Markdown
Collaborator Author

Changes since v1:

  • moved the process-module prepared check outside the input-format condition, so it also covers modules without input formats.
  • clarified copier initialization versus downstream format propagation commit message

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The secondary copier output lookup can reject a bind-compatible mono connection by requiring the output to match the four-channel input, rather than the connected sink’s constraints.

1 open issue
3 resolved since last review

Review effort: Balanced

Comment thread sound/soc/sof/ipc4-topology.c Resolved
kv2019i
kv2019i previously approved these changes Nov 10, 2026
Copilot AI balanced review requested due to automatic review settings November 11, 2026 12:04
@ujfalusi
ujfalusi force-pushed the peter/sof/pr/multi-pin-multi-format branch from ba430ff to 9fe415d Compare November 11, 2026 12:04
@ujfalusi

Copy link
Copy Markdown
Collaborator Author

Changes since v2:

  • sample-type matching: treats MSB and LSB integers as equivalent only for full-width 32/32 formats. The old branch compared raw types there.
  • Update copier and multi-input patches to use the refined type helper consistently.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Firmware-visible format changes across reconverging paths require DUT validation of actual Multi-WOV frame delivery, since reported validation confirms only the RUNNING state.

1 open issue

Review effort: Balanced

@ujfalusi
ujfalusi force-pushed the peter/sof/pr/multi-pin-multi-format branch from 9fe415d to 8d27499 Compare November 16, 2026 11:12
Copilot AI balanced review requested due to automatic review settings November 16, 2026 11:12
@ujfalusi

Copy link
Copy Markdown
Collaborator Author

Changes since v3:
Nit strictly related, but only applicable for multi pin/format...

  • clear the prepared flag on error for modules w/o unprepare callback
  • reject consecutive prepare for mixers if the new format is not exactly the same as the format which the mixer is configured for initially

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new ALH error path leaks allocated group IDs, risking pool exhaustion and blocked streams, while repeated SRC preparation can incorrectly reject valid reconverging pipelines.

3 open issues

Review effort: Balanced

Comment thread sound/soc/sof/ipc4-topology.c Resolved
Comment thread sound/soc/sof/ipc4-topology.c Resolved
@ujfalusi
ujfalusi force-pushed the peter/sof/pr/multi-pin-multi-format branch from 8d27499 to 0752819 Compare November 18, 2026 09:36
ujfalusi added a commit to ujfalusi/sof-linux that referenced this pull request Nov 18, 2026
An SRC or ASRC can be visited again when paths reconverge or a second
PCM uses a shared downstream path. Its prepared fast path currently
returns without updating the pipeline parameters, leaving downstream
widgets with the incoming rate instead of the configured output rate.

Derive the prepared output format from the configured base format and
the stored SRC sink rate or ASRC output frequency. Propagate it without
reconfiguring the converter or accounting its resources again. Preserve
the format, channel and rate update mask used by initial preparation.

Implemented with GitHub Copilot assistance.

Link: thesofproject#6030 (comment)
Copilot AI balanced review requested due to automatic review settings November 18, 2026 09:36
@ujfalusi

Copy link
Copy Markdown
Collaborator Author

Changes since v4:

  • Reworked copier output selection and per-connection propagation; added bind-time format validation.
  • Fixed SRC rate-only propagation, including already-prepared instances.
  • Normalized equivalent signed 32-bit sample formats when comparing formats.
  • Fixed rollback across completed branches, pin-binding cleanup on setup errors, and ALH allocation ordering.
  • Split copier changes into topic-based patches.

ujfalusi added a commit to ujfalusi/sof-linux that referenced this pull request Nov 18, 2026
An SRC or ASRC can be visited again when paths reconverge or a second
PCM uses a shared downstream path. Its prepared fast path currently
returns without updating the pipeline parameters, leaving downstream
widgets with the incoming rate instead of the configured output rate.

Derive the prepared output format from the configured base format and
the stored SRC sink rate or ASRC output frequency. Propagate it without
reconfiguring the converter or accounting its resources again. Preserve
the format, channel and rate update mask used by initial preparation.

Implemented with GitHub Copilot assistance.

Link: thesofproject#6030 (comment)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two critical issues remain: valid secondary-output format conversion is rejected, and ALH allocation failure leaves an incorrect configuration length that can cause an out-of-bounds read on retry.

3 open issues
2 resolved since last review

Review effort: Balanced

Comment thread sound/soc/sof/ipc4-topology.c Resolved
Comment thread sound/soc/sof/ipc4-topology.c Resolved
Baoli Zhang and others added 19 commits November 18, 2026 13:59
sof_ipc4_widget_setup_comp_process() allocates the input and output pin
format arrays via sof_ipc4_get_audio_fmt(). When the following
sof_ipc4_widget_setup_msg() fails, e.g. because the firmware does not
provide a module matching the widget UUID, the error path jumps to the
'err' label, which frees the process struct and clears swidget->private
but skips sof_ipc4_free_audio_fmt(). The arrays are then unreachable and
are reported by kmemleak:

    sof_ipc4_get_audio_fmt+0xc9/0x420 [snd_sof]
    sof_ipc4_widget_setup_comp_process+0x8e/0x200 [snd_sof]
    snd_sof_load_topology+0x71a/0x840 [snd_sof]
    sof_pcm_probe+0x77/0x120 [snd_sof]
    snd_soc_component_probe+0x21/0x60 [snd_soc_core]
    ...

Jump to 'free_available_fmt' instead, as the other process widget error
paths after sof_ipc4_get_audio_fmt() already do.

The other IPC4 widget setup functions (pga, mixer, src, asrc) free the
audio formats in their common error label and are not affected.

Fixes: f9efae9 ("ASoC: SOF: ipc4-topology: Add support for base config extension")
Cc: [email protected]
Signed-off-by: Baoli Zhang <[email protected]>
SRC/ASRC can only change rate, there is no reason to re-set the
format and channels to the same - as they are not changed by the module.

Signed-off-by: Peter Ujfalusi <[email protected]>
An SRC or ASRC can be visited again when paths reconverge or a second
PCM uses a shared downstream path. Its prepared fast path currently
returns without updating the pipeline parameters, leaving downstream
widgets with the incoming rate instead of the configured output rate.

Derive the prepared output format from the configured base format and
the stored SRC sink rate or ASRC output frequency. Propagate it without
reconfiguring the converter or accounting its resources again. Preserve
the format, channel and rate update mask used by initial preparation.

Implemented with GitHub Copilot assistance.

Link: thesofproject#6030 (comment)
When widget preparation fails, the error rollback clears the prepared
flag only if the widget has an ipc_unprepare callback. A module that
needs no cleanup callback therefore remains marked prepared even though
its pipeline was never committed. Later preparation attempts can fail
with stale format state.

Clear the prepared flag for unused widgets during rollback regardless
of whether they need an ipc_unprepare callback. Keep the use_count
check to preserve widgets shared by active paths.

Implemented and build-tested with GitHub Copilot assistance.

Fixes: 6f2a43e ("ASoC: SOF: sof-audio: Skip unprepare for in-use widgets on error rollback")
Cc: [email protected]
Signed-off-by: Peter Ujfalusi <[email protected]>
A mixout shared by several playback paths is configured by the first
path prepared. Further paths skip mixer preparation without checking
whether their input format matches the configured mixer.

With dynamic formats, a second frontend can consequently feed a 96 kHz
mixin into a mixout already running at 48 kHz. The stream is accepted
without rate conversion, leading to incorrect playback and delay
accounting.

Validate the propagated pipeline rate, channel count, valid bit depth
and sample type against the configured mixer format before reusing a
prepared mixer. Reject mismatches with -EINVAL without changing the
active configuration. Compare the format at the mixer rather than the
original frontend format so host copier format conversion remains valid.

Implemented and build-tested with GitHub Copilot assistance.

Fixes: 94648dc ("ASoC: SOF: ipc4-topology: Update the pipeline_params of prepared modules")
Cc: [email protected]
Signed-off-by: Peter Ujfalusi <[email protected]>
The pin binding array is only stored if the binding of pin 0 is defined
in topology. The bindings of the other pins are assumed to be present
and they are compared with strcmp() when looking for the pin a widget is
connected to, which results in a NULL pointer dereference if the topology
does not define the binding for all pins.
If the binding of pin 0 is not defined, the parsed bindings of the other
pins are leaked.

The pin binding must be defined for all pins, with "NotConnected" for the
unused pins as documented in struct snd_sof_widget. Reject the topology
if the binding is only defined for some of the pins.

Fixes: 3b3aced ("ASoC: SOF: Add support to parse pin binding array from topology")
Signed-off-by: Peter Ujfalusi <[email protected]>
Move the output format selection of process modules to
sof_ipc4_process_select_output_fmt(). Pass the selected input format,
or NULL for modules without input formats.

Keep the prepared-widget shortcut and pin 0-only parameter propagation
unchanged. This prepares for separating module configuration from the
output format propagated along each path.

No functional change.

Signed-off-by: Peter Ujfalusi <[email protected]>
…repare

The ipc_prepare op of a widget is called once, before its sinks are
walked, without information about which output pin the path continues
on. A widget with multiple output pins, each with its own format, cannot
tell which pin format to use as the input params of the next widget.

The pin cannot be looked up once for the whole PCM walk either: a widget
can feed several sinks of the same PCM graph via different pins, like
the KPB in the Multi-WOV topology, with its two outputs reconverging at
the WOV arbiter.

Call ipc_prepare once per outgoing connection in the PCM widget list and
pass the output pin feeding that sink:
- 0 for widgets with a single output pin,
- the pin bound to the sink in topology for widgets with multiple pins,
- SOF_OUTPUT_PIN_UNKNOWN if the widget has no output pin bindings,
- SOF_OUTPUT_PIN_NONE for widgets without a downstream connection in the
  PCM, which still receive one call.

A connection from a widget without output pins or to a sink missing from
the bindings is rejected. Virtual widgets (input, output, out_drv) are not
connected in the firmware, the walk does not continue to them and they
are not counted as connections.

The first call initializes the widget and marks it prepared, subsequent
calls only propagate the format of the given pin. When the widget fans
out to multiple sinks each branch is walked with its own copy of the
incoming params, so that the params propagated along one branch do not
leak into the others. The source is unprepared if any of its connections
fails.

The IPC4 prepare callbacks are updated for the new parameter, the output
pin is not used yet.

Signed-off-by: Peter Ujfalusi <[email protected]>
The output format lookup selects among the formats of all output pins of
a module. Modules with multiple output pins need to select the format
among the formats of a given pin.

Add a pin parameter to sof_ipc4_init_output_audio_fmt() and
sof_ipc4_is_single_format() to only consider the formats of that pin.
SOF_OUTPUT_PIN_ANY considers the formats of all pins as before. Also
allow selecting an output format without updating the obs of a base
config, for pins which are not configured via the base config.

Factor out the format comparison to sof_ipc4_is_same_fmt() and move
the pin format helpers next to sof_ipc4_fmt_cfg_to_type(), so that they
can also be used while the topology is parsed.

All callers pass SOF_OUTPUT_PIN_ANY, no functional change.

Signed-off-by: Peter Ujfalusi <[email protected]>
… single format

The formats of a pin are treated as a single format if they have the same
rate, channel count and valid bit depth. In that case the first format is
used without matching the formats against the params.

Formats which only differ in their sample type, like 32-bit integer and
32-bit float, or unsigned 8-bit and A-law/mu-law, are not the same format:
the first one is used even if the params ask for the other one.

Compare the sample type as well, the same way as the formats are matched
against the params. Treat signed integer sample types equivalently when
32 valid bits fill a 32-bit container.

Signed-off-by: Peter Ujfalusi <[email protected]>
…mats

The formats of the pins of a process module are fixed when the module is
initialized: pin 0 via the base config, the other pins via the base
config extension or by the module itself. Unlike the copier, which can
change the format of its output pins with SET_SINK_FORMAT, there is no
message to change them afterwards.

On a side with multiple pins the module can be initialized via a path
using any of the pins. The format of the other pins can only be known if
they have a single format, there are no params to select one of their
formats from.

Warn at topology load if a pin on a side with multiple pins has more than
one distinct format.

Signed-off-by: Peter Ujfalusi <[email protected]>
…ocess modules

A process module with multiple output pins can be shared by paths of
several PCMs. Selecting among all output formats with the FE params
guesses the pin used by the path. The prepared-widget shortcut always
propagates process->output_format, which describes pin 0 and may not
have been set if the first path selected another pin.

The module is configured with the output pin 0 format and buffer size
in the base config. Other pins use their topology format via the base
config extension or are configured by the module itself. Unlike a
copier, a process module cannot change pin formats after initialization.

Use the output pin feeding the path to separate module configuration
from downstream propagation. Select the base config format only among
pin 0 formats. If the path leaves on another pin, pin 0 must have a
single format since the path params cannot select it.

Propagate the configured format of the pin feeding the path on first
and subsequent preparation. Pins without topology formats use the pin 0
format. Reject ambiguous secondary pin formats, including when building
the base config extension. Modules with input formats are configured
once, the pin formats are not selected again with the FE params of a
subsequent path.

Without output pin bindings the output pin of a sink is only allocated
at bind time and the pin 0 format is propagated to it. Accept this only
if all output pins are configured with the pin 0 format: pins without
formats in topology use it, the others must have it as their single
format.

When there is no downstream connection, configure pin 0 without
propagating an output format. Zero-output consumers skip output format
selection entirely.

Signed-off-by: Peter Ujfalusi <[email protected]>
…n > 0

The input format of a module is looked up among its pin 0 input formats
with the params of the source widget in the path being prepared. This
assumes that the path enters the module on input pin 0.

A module with multiple input pins can be reached first via another pin,
depending on the order of the widgets in the widget list. For example
the ECNS module receives 16kHz/32bit/4ch from dmic16k on pin 0 and
48kHz/32bit/4ch from dmic01 on pin 1. If the path from dmic01 is walked
first the 48kHz params are looked up among the pin 0 formats and the
prepare fails, even though they are valid for pin 1. It only works today
because the dmic16k path happens to be walked first.

The base config of the module is always set from the pin 0 input format.
If the params do not match any pin 0 format but match the format of
another input pin, the path enters the module on that pin: use the pin 0
format for the base config.

This is only possible if pin 0 has a single format, otherwise the format
to configure is unknown: modules with multiple pin 0 formats must be
prepared first via the path entering on pin 0. Fail the prepare with an
explicit error if this is not the case.

The fallback is only taken where the lookup used to fail, the selection
for paths entering on pin 0 is not changed.

Signed-off-by: Peter Ujfalusi <[email protected]>
…nput process modules

The pipeline params are updated with the output format of a process module
based on the changed_params mask, which is evaluated between the input and
output formats of the module.

When the path enters a multi-input module on an input pin other than pin 0
the pipeline params carry the format of that pin while the output format is
derived from the pin 0 format of the module. The params not flagged as
changed would be propagated from the wrong pin.

This only worked because the changed_params is evaluated over the formats
of all pins and the format of the other input pin usually differs from the
output format.

Update all params with the output format for modules with multiple input
pins.

Signed-off-by: Peter Ujfalusi <[email protected]>
…s only

The changed_params mask is evaluated by comparing all input formats with
all output formats of a module regardless of the pin they belong to.

For modules with multiple pins this results in params flagged as changed
which are not changed by the module on its data path. For example the
google-rtc-aec has the 2 channel echo reference on input pin 1 and with
4 channel microphones it has a 4 channel format on input pin 0 and on the
output, but the channels are flagged as changed.

Only compare the formats of input pin 0 and output pin 0. The params of a
path leaving a multi-output module on other pins or entering a
multi-input module are updated with the full output format, they are not
affected by the mask.

Signed-off-by: Peter Ujfalusi <[email protected]>
The copier is initialized with the format of output pin 0, but its
output format lookup currently searches formats from all output pins.
A format declared for another pin can therefore be used for pin 0.

Restrict the lookup to pin 0. For module copiers, select at the input
rate: use a fixed channel count or sample representation when the pin
has only one, otherwise preserve the corresponding input parameter.
Require the resulting combination to be declared.

Keep the endpoint-specific selection for gateway copiers.

Signed-off-by: Peter Ujfalusi <[email protected]>
…ailure

The DMA configuration TLV size is temporarily added to the gateway
configuration length before allocating the IPC payload. If allocation
fails, the original length is not restored. On retry, the TLV size is
counted again and the copier configuration blob can be read beyond its
allocation.

Defer updating the gateway configuration length until the IPC payload is
ready to be copied, after the fallible preparation steps. Keep the existing
restoration after the payload is assembled.

Fixes: a0659f8 ("ASoC: SOF: ipc4-topology: add DMA config TLV to IPC data")
Signed-off-by: Peter Ujfalusi <[email protected]>
Copier preparation always propagates the output pin 0 format to the
next widget, even when the path is fed by another output pin.

Use the connected output pin to select and propagate its format. When
that pin has no declared formats, or its identity is not known until
bind, continue propagating the pin 0 format. Skip propagation when
there is no downstream connection.

Move ALH group allocation after the parameter propagation, so that a
failure during preparation does not leave an allocated group behind.

Signed-off-by: Peter Ujfalusi <[email protected]>
The sink determines the format configured for a copier output pin above
pin 0. Check that the sink format is declared for that pin when the
topology provides a format list.

Also check that it matches the output format propagated during prepare.
This prevents the sink from being prepared with one format while the
copier is configured with a different one at bind time.

Signed-off-by: Peter Ujfalusi <[email protected]>
@ujfalusi
ujfalusi force-pushed the peter/sof/pr/multi-pin-multi-format branch from 0752819 to 3317909 Compare November 18, 2026 12:00
Copilot AI balanced review requested due to automatic review settings November 18, 2026 12:00
@ujfalusi

Copy link
Copy Markdown
Collaborator Author

Changes since v5:

  • Added a standalone fix preserving gateway configuration length on prepare allocation failures.
  • Restored sink-derived format conversion for dynamically assigned copier secondary outputs, while retaining topology-format validation and prepared-format checks for explicitly bound pins.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Firmware-facing pin-format propagation across branching and reconvergent pipelines needs SOF maintainer validation of end-to-end capture, as Multi-WOV reaches RUNNING but frame delivery remains unverified.

No open issues 🎉

3 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants