The 141st MPEG meeting, held in person and online in the third week of January, covered many topics, including the establishment of an over-arching project that will cover all machine learning-related initiatives within MPEG and Video Coding for Machines (VCM).
At the meeting, experts initiated MPEG-AI, an umbrella initiative for all the AI-related activities within MPEG (VCM, FCVCM, NNC, etc.). The project will officially be launched at the April 2023 MPEG meeting. More information can be found on the project’s website, at https://www.mpeg.org/standards/MPEG-AI/.
Activities on the Video Coding for Machines (VCM) standard continued during the ad-hoc group and break-out group meetings.
As a reminder, the VCM call for proposals issued in July of 2022. Excerpt from the MPEG’s press release (https://www.mpeg.org/meetings/mpeg-140/) stated:
At the 140th MPEG meeting, MPEG Technical Requirements (WG 2) evaluated the responses to the Call for Proposals (CfP) for technologies and solutions enabling efficient video coding for machine vision tasks. A total of 17 responses to this CfP were received, with responses providing various technologies such as (i) learning-based video codecs, (ii) block-based video codecs, (iii) hybrid solutions combining (i) and (ii), and (iv) novel video coding architectures. Several proposals use a region of interest-based approach, where different areas of the frames are coded in varying qualities.
The responses to the CfP reported an improvement in compression efficiency of up to 57% on object tracking, up to 45% on instance segmentation, and up to 39% on object detection, respectively, in terms of bit rate reduction for equivalent task performance. Notably, all requirements defined by WG 2 were addressed by a variety of proposals.
Given the success of this call, MPEG will continue working on video compression methods for machine vision tasks. The work will continue in MPEG Video Coding (WG 4) within a new standardization project. A test model will be developed based on technologies from the responses to the CfP and results from the first round of core experiments in one or two meeting cycles. At the same time, the Joint Video Team with ITU-T SG 16 (WG 5) will study encoder optimization methods for machine vision tasks on top of existing MPEG video compression standards.”
Furthermore, in the publc document “CfP response report for Video Coding for Machines” (https://www.mpeg.org/wp-content/uploads/mpeg_meetings/140_Mainz/w22071.zip), MPEG expressed acknowledgment of the participating organizations:
The following organizations are thanked for responding to this CfP:
· Institute of Computing Technology, Chinese Academy of Sciences (CAS-ICT)
· China Telecom
· City University of Hong Kong
· Electronics and Telecommunications Research Institute (ETRI)
· Florida Atlantic University (FAU)
· Konkuk University
· Myongji University
· OP Solutions
· Poznan University of Technology (PUT)
· Wuhan University
· Zhejiang University
During the 141st meeting updated results from the proponents that responded to the Call for Proposals were reviewed, and the decision was made to continue the work on the reference software as well as five core experiments (CEs):
CE 1 – Region-of-interest based coding methods,
CE 2 – Neural network based inner coding,
CE 3 – Frame level spatial resampling,
CE 4 – Temporal resampling,
CE 5 – Post filtering.
OP Solutions, together with its partner institution, Florida Atlantic University, continues to participate in the development of the VCM standard, as a proponent of proposals directed to several core experiments. New and updated results of or proposed technology will be presented in the 142nd MPEG meeting in April.
In addition, the draft CfP was issued for the Feature Compression for Video Coding for Machines (FCVCM). In contrast to VCM, which as inputs takes the pixel domain picture or a frame of a video, the FCVCM takes as inputs the features from the arbitrary layer of the neural network processing the input picture. (We are planning to write additional blog posts explaining details of those technologies in the near future – stay tuned!).
The final CfP for FCVCM will issue in April. OP Solutions plans to respond to this CfP as well.
3. MPEG roadmap
MPEG’s roadmap emphasizes the importance of the VCM, FCVCM, and related. This is a short-term plan that is result of MPEG experts’ assessment of current status and near-term viability of the ongoing standardization efforts.
In the accompanying presentation, MPEG gives following rationale for producing and publicizing the roadmap:
MPEG has created, and still produces, media standards that enable huge markets to flourish
• MPEG works on requirements from industry.
• Many industries are represented in MPEG, but not all of MPEG’s customers can or need to participate in the process.
• MPEG wants to inform its customers about its long-term plans (~ 5 years out).
• MPEG collects feedback and requirements from these customers.
The roadmap is shaped by significant developments
• The relentless increase of IP-distributed and mobile media
• Higher quality media
• More immersion (UHD, VR, AR, Light Fields, Holography)
• The Internet of Media Things & Wearables
• Cloud-based media processing, storage and delivery
• New high-speed networks including fiber, 5G mobile, and cable 10G
• New emerging technologies (machine vision, AI)
The short-term plan for the MPEG’s roadmap, after the 141st meeting, is depicted in the picture accompanying this blog post.
We are glad to announce that OP Solutions will continue participating in the MPEG’s work on the exciting and promising new technologies.