Skip to content

ASoC: SOF: ipc4: support runtime codec capability discovery - #6044

Draft
lgirdwood wants to merge 2 commits into
thesofproject:topic/sof-devfrom
lgirdwood:topic/sof-runtime-codec-info
Draft

lgirdwood wants to merge 2 commits into
thesofproject:topic/sof-devfrom
lgirdwood:topic/sof-runtime-codec-info

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Upgrade IPC4 codec discovery in the Linux SOF driver to support runtime codec queries (SOF_IPC4_FW_PARAM_SOF_INFO_GET), idle dynamic library unloading/purging (SOF_IPC4_FW_PARAM_SOF_LIBRARY_UNLOAD), and topology media_decoder format-to-UUID lookup with on-demand LLEXT loading — while preserving boot-time FW_CONFIG fallback for older firmware.

Companion SOF firmware PR: thesofproject/sof#11472

Changes

  1. Runtime SOF_INFO discovery & library unload (header.h, ipc4-priv.h, ipc4.c, ipc4-loader.c)

    • Added SOF_IPC4_FW_PARAM_SOF_INFO_GET = 37, SOF_IPC4_FW_PARAM_SOF_LIBRARY_UNLOAD = 38, SOF_IPC4_SOF_CAP_RUNTIME_SOF_INFO, and SOF_IPC4_SOF_CAP_LIBRARY_UNLOAD.
    • Protected cached codec_info updates/reads with codec_info_mutex and rejected duplicate SOF_IPC4_SOF_CODEC_INFO tuples in a single payload.
    • Refreshes SOF_INFO on library load and non-IMR library reloads (avoiding redundant queries after FW_CONFIG or IMR restore).
    • Added gap-safe library ID allocation, sof_ipc4_unload_library(), and automatic purging of idle unloadable UUID libraries when slots or memory are exhausted.
  2. media_decoder format-to-UUID lookup & on-demand load (tokens.h, sof-audio.h, topology.c, ipc4-topology.h, ipc4-topology.c, ipc4-compress.c)

    • Parses SOF_TKN_MEDIA_CODEC_* (2200..2205) tokens into swidget->media_formats[] and allows media_decoder widgets to defer module binding when their default UUID is not resident at topology load.
    • Reports both cached firmware codec_info (via scoped_guard(mutex, ...)) and media_formats[] in sof_ipc4_compr_get_caps() without waking or booting a suspended DSP.
    • In sof_ipc4_compr_set_params(), tears down any previous prepared stream first (freeing prior LLEXT references for purge), then resolves the best matching media_formats[] module UUID (resident module first, else boot DSP + demand-load/purge via sof_ipc4_find_module_by_uuid()) and binds it via sof_ipc4_process_bind_module().

Verification

  • scripts/checkpatch.pl: 0 errors, 0 warnings on both commits.
  • make M=sound/soc/sof: compiled and linked all sound/soc/sof modules with 0 warnings and 0 errors.
  • Live hardware testing on DSP targets has not yet been run.

Comment thread sound/soc/sof/ipc4-loader.c Outdated
if (unlikely(ret))
goto release;

if (sdev->fw_state == SOF_FW_BOOT_COMPLETE) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the DSP is not booted at this point then we would not be able to load the library to fw in the first place, this check is redundant.
If the loading crashes the fw, then this would be skipped anyways.

Comment thread sound/soc/sof/ipc4-compress.c Outdated
}
ret = sof_ipc4_compr_refresh_codec_info(sdev);
if (ret)
return ret;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Booting up the DSP and re-reading the info will not change anything, the info refresh after a library loading is the only place where it can change.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we need an API call here that can do 2 things:

  1. Check the kernel info (refreshed after library loads) of what modules are in FW and return with a match/context in order to proceed here.

  2. If no info match found, then we look at the FS for a match.

Now for 2) need a way for 3) kernel to know where to look on the FS for modules.

Re 3) I'm thinking our feature/function topology would be like NXP ipc3 decoder where we can have a container module and we could list UUIDs supported for this configuration. This list would allow the kernel to know which UUIDs would support 44.1k S16le MP3 or whicg UUIDs support 48k 24bit AAC etc.

i.e. kernel would know where to find UUID and hence llext module that would align with request.

Comment thread sound/soc/sof/ipc4-compress.c Outdated
for (i = 0; i < codec_info->count; i++) {
int _dir = SOF_IPC4_CODEC_INFO_GET_DIR(codec_info->items[i]);
{
guard(mutex)(&ipc4_data->codec_info_mutex);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use scoped_guard() instead if must

Comment thread sound/soc/sof/ipc4-compress.c Outdated

ret = sof_ipc4_compr_refresh_codec_info(sdev);
if (ret)
return ret;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not needed, just booting up will not change anything in configuration, keep the existing flow.

if (count) {
kfree(new_codec_info);
new_codec_info = kmemdup(tuple->value, tuple->size, GFP_KERNEL);
if (!new_codec_info) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you expect multiple CODEC_INFO in the sof_info block and ignore all, but the last one?
When a module is loaded to the firmware, they cannot be unloaded, so the last codec_info should contain the cumulative list of all codecs, including if there are cadence codecs in basefw, ffmpeg codecs or any other codecs loaded as library via UUID.
Loading new codecs will not remove the existing ones.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually we do need to be able to purge modules. - we need to consider that non core modules can come and go (and this will also include models too).

Comment thread sound/soc/sof/ipc4-loader.c Outdated
}

if (ipc4_data->sof_info_runtime_supported) {
ret = sof_ipc4_query_sof_info(sdev);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we just parsed the sof_info from fw_config, the separate sof_info_get will query different sof_info than the sof_info we just parsed?

Comment thread sound/soc/sof/ipc4-loader.c Outdated
}
}

if (!ret && ipc4_data->sof_info_runtime_supported)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, this code does not know if the libraries are indeed reloaded or it was a NOP due to the fact that on Intel hardware IMR stored the libraries.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The GET_CAPS refresh lacks a runtime-PM reference across firmware boot and capability querying, so it can access powered-down hardware and fail when SOF is suspended.

1 open issue
What changed in this PR

Adds runtime codec capability discovery to the SOF IPC4 driver, complementing the firmware changes in thesofproject/sof#11472 while preserving boot-time discovery for older firmware.

Changes:

  • Defines the runtime SOF-info query and capability flag.
  • Refreshes codec information after library loads and reloads, with tuple validation and mutex-protected cache replacement.
  • Refreshes capabilities before compressed-stream queries and codec selection, and frees cached information at exit.
File Description
sound/​soc/​sof/​ipc4.c Initializes the codec-info mutex and frees cached data at exit.
sound/​soc/​sof/​ipc4-priv.h Declares runtime capability state, locking, and the query helper.
sound/​soc/​sof/​ipc4-loader.c Validates SOF-info tuples and refreshes capabilities during firmware and library setup.
sound/​soc/​sof/​ipc4-compress.c Refreshes and locks codec information used by compressed streams.
include/​sound/​sof/​ipc4/​header.h Adds runtime SOF-info query and capability constants.

Review effort: Balanced

Comment thread sound/soc/sof/ipc4-compress.c Outdated
* Ensure the DSP is booted (on-demand boot) and runtime codec
* capabilities are synchronized before consulting codec_info.
*/
ret = snd_sof_boot_dsp_firmware(sdev);
@lgirdwood

Copy link
Copy Markdown
Member Author

@ujfalusi I'm going to work on 3) below, so we have something similar to NXP in IPC4 - lets see if we can get a simple method for kernel to get UUIDs for needed modules

@lgirdwood
lgirdwood force-pushed the topic/sof-runtime-codec-info branch from cd8df21 to 42d44e9 Compare November 17, 2026 20:27
@lgirdwood

Copy link
Copy Markdown
Member Author

Addressed review feedback and added module purge + topology format-to-UUID lookup:

  • Removed redundant sof_ipc4_query_sof_info() after FW_CONFIG, removed redundant fw_state check in sof_ipc4_load_library(), and gated reload refresh on non-IMR reloads with external libraries.
  • Removed the extra snd_sof_boot_dsp_firmware() / sof_ipc4_query_sof_info() polling from sof_ipc4_compr_get_caps() and early sof_ipc4_compr_set_params(), resolving both the redundant IPC and suspended-DSP PM concerns, and switched to scoped_guard(mutex, ...).
  • Rejected duplicate SOF_IPC4_SOF_CODEC_INFO tuples in sof_ipc4_parse_sof_info().
  • Added idle dynamic library purge (sof_ipc4_unload_library(), SOF_IPC4_FW_PARAM_SOF_LIBRARY_UNLOAD) and media_decoder SOF_TKN_MEDIA_CODEC_* parsing with 2-tier UUID lookup and demand loading in sof_ipc4_compr_set_params().

Add IPC4 runtime SOF info discovery (SOF_IPC4_FW_PARAM_SOF_INFO_GET)
and idle library unload (SOF_IPC4_FW_PARAM_SOF_LIBRARY_UNLOAD) gated by
SOF_IPC4_SOF_CAPS capability flags, while keeping boot-time FW_CONFIG
SOF_IPC4_SOF_CODEC_INFO parsing for backwards compatibility:

- Protect cached codec_info with codec_info_mutex and reject duplicate
  SOF_IPC4_SOF_CODEC_INFO tuples in a single SOF_INFO payload.
- Refresh SOF info on library load and non-IMR library reloads (avoiding
  redundant queries after FW_CONFIG or IMR restore).
- Implement gap-safe library ID allocation, sof_ipc4_unload_library(),
  and automatic purging of idle unloadable UUID libraries when library
  slots or memory are exhausted.

Signed-off-by: Liam Girdwood <[email protected]>
…d load

Parse SOF_TKN_MEDIA_CODEC_* topology tokens on decoder widgets into
swidget->media_formats[] mapping codec IDs and stream constraints to
priority-ordered candidate module UUIDs:

- Allow media_decoder widgets with media_formats[] to defer module
  binding at topology load time when their default UUID is not resident.
- Report both cached firmware codec_info (under scoped_guard mutex) and
  topology media_formats[] in sof_ipc4_compr_get_caps() without waking
  or booting a suspended DSP.
- In sof_ipc4_compr_set_params(), free any prior prepared stream first,
  then perform a two-tier lookup across matching media_formats[]
  (resident module first, else boot DSP and demand-load/purge UUID
  library) and bind the selected module before pipeline setup.

Signed-off-by: Liam Girdwood <[email protected]>
@lgirdwood
lgirdwood force-pushed the topic/sof-runtime-codec-info branch from 42d44e9 to c49efc5 Compare November 17, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants