Skip to content

ipc4: base_fw: runtime codec capability discovery for built-in and LLEXT modules - #11472

Draft
lgirdwood wants to merge 4 commits into
thesofproject:mainfrom
lgirdwood:topic/runtime-codec-info
Draft

lgirdwood wants to merge 4 commits into
thesofproject:mainfrom
lgirdwood:topic/runtime-codec-info

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Remove build-time hardcoded per-codec #ifdef CONFIG_* capability entries from src/kernel/audio/base_fw.c, add IPC4 runtime library unload (IPC4_SOF_LIBRARY_UNLOAD) + SOF info query (IPC4_SOF_INFO_GET), and add the media_decoder topology2 component and media-decode-feature feature topology mapping codec formats to priority-ordered candidate module UUIDs.

Companion Linux host driver PR: thesofproject/linux#6044

Changes

  1. Static build-time codec registry, runtime query, and unload (<sof/audio/codec_info.h>, ipc4/base_fw.h, src/kernel/audio/base_fw.c)

    • Replaced dynamic rmalloc()/rfree() list with a static pre-allocated codec_info_slots[SOF_IPC4_CODEC_INFO_MAX_ITEMS] array and 1-by-1 add/remove helpers (sof_codec_info_add(), sof_codec_info_remove()) to avoid heap fragmentation.
    • Added IPC4_SOF_INFO_GET (37) and IPC4_SOF_LIBRARY_UNLOAD (38) gated by IPC4_SOF_CAP_RUNTIME_SOF_INFO and IPC4_SOF_CAP_LIBRARY_UNLOAD under IPC4_SOF_CAPS.
    • Retained IPC4_FW_CONFIG emitting IPC4_SOF_CODEC_INFO and IPC4_SOF_CAPS for backwards compatibility with older host drivers.
  2. LLEXT discovery & safe library unload (lib_manager, llext_manager)

    • Discovers .sof_codec_info sections on LLEXT load and DRAM restore, adding items 1-by-1.
    • Added lib_manager_unload_library() and llext_manager_remove_library() (rejecting non-LLEXT libraries with -ENOTSUP, freeing retained inspection data via llext_free_inspection_data() before llext_unload(), and removing codec items).
  3. Per-module codec advertisement (ffmpeg_dec, mfcc, cadence_ipc4)

    • Emits .sof_codec_info in LLEXT builds and registers built-in codec items 1-by-1 in __maybe_unused SOF_MODULE_INIT handlers.
  4. Media decode feature topology & validator (tools/topology/)

    • Added media_codec vendor tokens (2200..2205), Class.Base.media_format, Class.Widget.media_decoder, Class.Pipeline.compr-playback-media, and media-decode-feature.conf (sof-media-decode-feature), plus updated validator schema (kernel_tokens.py) and engine (kernel_engine.py).

Verification

  • west build -b intel_adsp/ace30/wcl built cleanly (0 errors) for modular LLEXT (CONFIG_LLEXT=y, CONFIG_COMP_FFMPEG_DEC=m, CONFIG_COMP_MFCC=m) and built-in configurations (rimage not installed).
  • scripts/check_kernel_includes.py --strict passed (0 violations).
  • POSIX host library build (-DCONFIG_LIBRARY=ON -DINIT_CONFIG=library_defconfig --target sof) built cleanly (0 errors).
  • alsatplg -p + tools/topology/validator/tplg_validate.py validated sof-media-decode-feature.tplg with 0 errors (G001 waived for mixout.1.1 of base topology).
  • Runtime / on-device DSP verification has not yet been run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unload paths can free non-LLEXT libraries with active module instances, preventing later resource cleanup, and leak retained LLEXT inspection metadata.

2 open issues
What changed in this PR

This PR replaces hardcoded codec capability reporting with a runtime registry, allowing SOF hosts to discover capabilities from built-in modules and loaded LLEXT libraries while preserving boot-time reporting for older hosts.

Changes:

  • Adds a deduplicated codec registry and runtime SOF-information query IPC.
  • Discovers codec metadata in LLEXT sections and adds registration, restore, and library-unload handling.
  • Moves codec capability declarations into the FFmpeg, MFCC, and Cadence modules.
File Description
src/​kernel/​library_manager/​llext_manager.c Discovers codec sections and adds library registration and removal.
src/​kernel/​library_manager/​llext_manager_dram.c Restores codec registrations and cleans them up on failure.
src/​kernel/​library_manager/​lib_manager.c Adds the library-unload entry point.
src/​kernel/​audio/​base_fw.c Implements the codec registry and runtime capability responses.
src/​include/​sof/​llext_manager.h Declares LLEXT removal and codec-registration APIs.
src/​include/​sof/​lib_manager.h Declares the library-unload API.
src/​include/​sof/​audio/​codec_info.h Defines codec metadata encoding and registry APIs.
src/​include/​ipc4/​base_fw.h Defines the runtime query ID and capability flag.
src/​audio/​module_adapter/​module/​cadence_ipc4.c Registers built-in Cadence codec capabilities.
src/​audio/​mfcc/​mfcc.c Adds built-in and LLEXT MFCC capability metadata.
src/​audio/​ffmpeg_dec/​ffmpeg_dec.c Adds built-in and LLEXT FFmpeg capability metadata.

Review effort: Balanced

Comment on lines +1391 to +1396
if (ctx->load_type == SOF_MAN_MOD_TYPE_LLEXT ||
ctx->load_type == SOF_MAN_MOD_TYPE_LLEXT_AUX) {
ret = llext_manager_remove_library(module_id);
if (ret < 0)
return ret;
}
mctx->llext = NULL;
}
if (mctx->ebl) {
rfree(mctx->ebl);
Comment thread src/include/sof/audio/codec_info.h Outdated
* \param[in] count Number of items in @p items.
* \return 0 on success, negative error code on failure.
*/
int sof_codec_info_register(const void *owner, const uint32_t *items, uint32_t count);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for simplicity, it would be better to add and remove elements 1 by 1 due to fragmentation

Comment thread src/kernel/audio/base_fw.c Outdated
return -EINVAL;

items_bytes = count * sizeof(uint32_t);
entry = rmalloc(SOF_MEM_FLAG_KERNEL, sizeof(*entry) + items_bytes);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just pre-allocate this at build time since we have defined a max items macro

…try, query, and unload

Remove all build-time per-codec #ifdef checks from get_codec_info() in
base_fw.c and replace them with a build-time pre-allocated slot table
updated 1-by-1 via sof_codec_info_add() / sof_codec_info_remove() under
a spinlock (avoiding heap fragmentation across LLEXT load/unload cycles).

Add IPC4_SOF_INFO_GET (37) and IPC4_SOF_LIBRARY_UNLOAD (38) along with
IPC4_SOF_CAP_RUNTIME_SOF_INFO and IPC4_SOF_CAP_LIBRARY_UNLOAD under
IPC4_SOF_CAPS so hosts can query updated codecs after library load/unload
and purge idle dynamic libraries. Keep IPC4_FW_CONFIG emitting
IPC4_SOF_CODEC_INFO and IPC4_SOF_CAPS for backwards compatibility.

Signed-off-by: Liam Girdwood <[email protected]>
…brary unload

Parse the optional ".sof_codec_info" ELF section when loading or
DRAM-restoring an LLEXT library and register each codec item 1-by-1
with base_fw. Add lib_manager_unload_library() and
llext_manager_remove_library() to reject unsupported library types
(-ENOTSUP), free retained LLEXT inspection data before unloading, and
unregister the library's codec entries.

Signed-off-by: Liam Girdwood <[email protected]>
… LLEXT

Move codec capability declarations out of base_fw.c into ffmpeg_dec,
mfcc, and cadence_ipc4: embed ".sof_codec_info" tables in LLEXT builds
and register built-in codec items 1-by-1 during SOF_MODULE_INIT via
sof_codec_info_add_items(). Mark the init wrappers __maybe_unused for
host library builds where SOF_MODULE_INIT is a no-op.

Signed-off-by: Liam Girdwood <[email protected]>
…ure topology

Define the media_codec vendor token group (2200..2205), Class.Base.media_format,
Class.Widget.media_decoder, Class.Pipeline.compr-playback-media, and the
standalone media-decode-feature.conf topology mapping compressed codec
formats (MP3, AAC, FLAC, ALAC, Vorbis, Opus, PCM) to priority-ordered
module UUID lists. Update the topology validator schema and engine to
match.

Signed-off-by: Liam Girdwood <[email protected]>
@lgirdwood
lgirdwood force-pushed the topic/runtime-codec-info branch from a89e92f to 6699de0 Compare November 17, 2026 20:27
@lgirdwood

Copy link
Copy Markdown
Member Author

Addressed review and CI feedback:

  • Switched to a static build-time codec_info_slots array with 1-by-1 sof_codec_info_add() / sof_codec_info_remove() APIs (no dynamic heap allocations/fragmentation).
  • Rejected unsupported (non-LLEXT) library types in lib_manager_unload_library() with -ENOTSUP.
  • Freed LLEXT inspection metadata via llext_free_inspection_data() before llext_unload() / freeing ebl.
  • Marked sys_comp_*_init() with __maybe_unused to fix the POSIX host library -Werror=unused-function failure.
  • Added IPC4_SOF_LIBRARY_UNLOAD (38) and the media_decoder / media_format topology2 definitions + media-decode-feature.conf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants