Open standards streamline interoperability between critical technologies, reducing product development costs and time to market while speeding industry innovation.
To learn of the latest progress being made in this area, IMVE recently picked the brain of Laurent Pinchart, Founder of Ideas on Board and a member of a new working group pursuing an embedded vision standard under the umbrella of the Khronos Group.
IMVE: What is the background of your organisation?
LP: As a non-profit standards body with close to 200 member companies, The Khronos Group is committed to developing and evolving open, royalty-free interoperability standards for the good of the industry. Our tagline ‘Connect Software to Silicon’ reflects our mission to enable key markets with timely and effective standard APIs that enable software applications, libraries, and engines to harness the power of silicon acceleration for demanding use cases such as 3D graphics, parallel computation, augmented and virtual reality, vision processing and inferencing.
Khronos Group member companies
We are a member-driven organisation that believes open standards not controlled by, or dependent on, any single company can often be the thread of continuity for industry progress as technologies, platforms, and market positions swirl and evolve.
IMVE: What are some of the existing standards developed by Khronos members?
LP: The family of open standards being actively developed by Khronos include 3D acceleration APIs such as OpenGL and the new generation, Vulkan GPU API; initiatives around 3D format standards including glTF for assets and KTX for textures; the OpenXR API standard for portable augmented and virtual reality; and a family of APIs and languages for parallel computation, vision acceleration and inferencing – including OpenCL, SYCL, SPIR-V and OpenVX— and now the new Kamaros embedded camera API currently in development.
IMVE: Are there alternate versions of any of these standards for different applications?
LP: Yes. Khronos also has a history of adapting mainstream acceleration APIs for safety-critical markets. Vulkan SC is derived from Vulkan API, to streamline safety certification of systems that employ GPU acceleration, by enabling system implementers to provide certification evidence packages with reduced cost and effort. OpenVX has a safety-critical profile that enables the rapid deployment of trained neural network models.
Finally, the newly created Khronos SYCL Safety-Critical Working Group is investigating industry requirements for a general parallel programming API for accelerated compute using SYCL’s standard C++ single source programming model in safety-critical markets. SYCL SC will bridge the gap between low-level APIs such as Vulkan SC and the C++ high-level language to streamline the development and safety certification of systems incorporating sophisticated parallel processing acceleration, including artificial intelligence and machine learning pipelines.
IMVE: What types of organisations get involved in Khronos working groups?
LP: To use the SYCL SC Working Group as an example, this working group has already gained support from industry leaders including AMD, Arm, Barcelona Supercomputer Center, Codeplay, CoreAVI, Intel, Intellias, Mercedes-Benz, and Qualcomm Technologies Inc. All Khronos members are able to participate in any working group, and membership in Khronos is open to all. Khronos warmly welcomes any company that wishes to participate.
Khronos active standardisation working groups
IMVE: Why is an embedded camera API Standard necessary?
LP: Sophisticated camera subsystems are increasingly critical in diverse markets such as robotics, autonomous driving, digital twins, and virtual and augmented reality where vision processing, often utilising machine learning, is used to understand surroundings, processes, objects, and users. Consequently, an increasing number and diversity of sensors and associated image signal processors (ISPs) are being tightly integrated with vision and inferencing accelerators in self-contained embedded systems.
The ongoing lack of interoperable camera API standards results in unnecessarily high integration costs for these new camera technologies – increasing application development time and maintenance costs while reducing software portability and opportunities for code reuse.
IMVE: What work are Khronos members doing on developing an Embedded Camera API Standard?
LP: During 2021, The Khronos Group and European Machine Vision Association (EMVA) worked together to convene an Exploratory Group that involved over seventy companies working together to explore industry consensus on the need, terminology, scope, requirements, and design methodology for a new open standard camera system API. The requirements generated by the Khronos/EMVA Exploratory Group were the motivation to create the Kamaros Working Group at Khronos in March 2022, which was to focus on the development of an open, cross-vendor standard for camera, sensor, and ISP control.
Kamaros is a cross-platform developer-facing camera API
As the design of the Kamaros API proceeds, Khronos and the EMVA continue their close collaboration and have entered into a Liaison Agreement, enabling ongoing close coordination between the two groups. Designated liaisons have been granted joint member privileges in both organisations. In addition, all EMVA members have a standing invitation to join the Kamaros Advisory Panel at no charge.
IMVE: What are the goals of the Kamaros Working Group?
LP: The design goals of Kamaros include portability of application code across diverse platforms, and multiple generations of cameras and sensors, while providing an effective API for the development of applications, frameworks and libraries that need access to sophisticated camera functionality. Kamaros will leverage many design best-practices from other established Khronos APIs including specification structure and tooling, flexible and powerful extensibility, installable drivers with debug and instrumentation layers, and schema-based generation of multiple API language bindings.
Additional key design directions currently being discussed at the Kamaros Working Group include: how best to balance exposing the full power of camera and ISP processing versus a simplified programming model – perhaps through a layered utility library; and to what extent to leverage the proven resource allocation, synchronisation, and work submission model of the new generation Vulkan API, perhaps leading to the option of seamless integration of sensor, ISP and GPU processing within a single runtime.
Laurent Pinchart is the Founder of Ideas on Board and a member of a new working group pursuing an embedded vision standard under the umbrella of the Khronos Group
Lead image: shutterstock/asharkyu
