Last month in Nice, active media developers came together for the annual Linux Media Summit to exchange insights and tackle ongoing challenges in the media subsystem. The event, co-located for the first time with Embedded Recipes, was co-sponsored by Cisco and Collabora.
Our largest Media Summit to date brought together around 20 engaged participants. Engagement was strong, marked by thoughtful questions and lively discussions. Here’s a brief summary of the key topics that were discussed, and upcoming areas of focus.
Stateless Video Encoders
We kicked of the summit with a discussion on codecs, and reviewed the progress so far on encoder development. We explored a hybrid approach—stateless encoding using compound controls and requests, paired with a stateful rate controller implemented in the kernel. The idea is to have a shared C implementation across drivers, with the potential to extend it using eBPF in the future, while always offering a fixed per-frame control to applications. The proposal sparked a lot of questions, but overall, it was well-received, with attendees sharing similar experiences and use cases involving eBPF.
Camera Support
There is definitely a lot of work to be done with the camera infrastructure, as it is overly complex, to say the least. Some stacks have over 50 different driver nodes and drivers! We discussed some of the challenges of coordinating all of this, including using dynamic array control to manage hardware links that have multiple streams in them (i.e. MIPI, FDP). There was also a proposal made for a userspace DB to identify camera modules. Lastly, some concerns were addressed over ambiguous APIs with inconsistent driver implementations.
Staging Drivers
Once again, we had a good discussion about the future of all these drivers that have been sitting in stagings for years. The staging area has always been a place to allow rapid development and collaboration. Unfortunately however, some projects end up being abandoned, and in practice, these drivers stay there for a long time, whether they are used or not. There are no simple black and white answers on how to handle these, so we went through the list and discussed how to proceed with each of them. Announcements will be made before any archiving or removal.
Memory Accounting
Maxime Ripard, who is usually mostly involved in the graphics side of things, joined us to discuss memory accounting to prevent infinite allocations in DRM/V4L2. Essentially, if you’re working with a graphics or V4L2 driver, there’s currently no real limit on memory allocation (so it’s “infinite”). Memory allocations aren’t properly tracked, and even if you set constraints using cgroups, they can be bypassed. To address this, he developed an implementation and presented the necessary changes for V4L. The proposed updates were well-received, and fairly straightforward. Much like in DRM, V4L centralizes most of its memory allocation, ultimately relying on the DMA API. That centralization makes the changes more manageable and helps improve both driver robustness and sandboxing safety. It’s great to see this moving forward.
We also discussed the idea of having PipeWire handle hardware decoding for sandboxed apps, however that approach is limited as it does not cover virtual machine use cases. We also need better Flatpak solutions for V4L2 decoders.
Multi-committer model
The last discussion of the day was a status update on the multi-committer model, which involves transitioning from the somewhat secretive PRs, to merge requests (MRs) for transparency and better scaling. For this, Hans, Mauro, and Ricardo have been adjusting the CI and testing it. It’s all in very good shape, just need to finish some documentation so everything is written down before formalizing the process.
Looking forward Overall, this summit showed a live and healthy subsystem with a clear focus on continuous improvement, especially in the areas of collaboration and scalability. Face-to-face meetings like this one help build connections and foster cohesion among developers, making it possible for different companies to work together toward compatible goals. A great example of that shared commitment this year is the ongoing, collaborative effort to support on chip camera ISP pipelines and video encoders.
