The European COST211ter Group - Research on Redundancy Reduction Techniques and Content Analysis for Multimedia Services

 

Thomas Sikora1 and Helga Seguin2

1 Heinrich-Hertz-Institute (HHI) Berlin GmbH, Germany , Email: sikora@hhi.de
2 France Telekom, France, Email: helga.seguin@wanadoo.fr

 

Abstract

During the last ten years the work in the European COST211bis and COST211ter Action research groups has been instrumental in the definition and development of the ITU-T H.261 standard for ISDN video telephony aimed at bit rates of multiples of 64 kbit/s, as well as for the development of the H263 standard for very low bitrate video coding (<64 kbit/s). The group has also contributed significantly to the ISO MPEG-4 activities. At present a significant effort of the COST211ter group activities is further dedicated towards image and video sequence analysis and segmentation - an important technological aspect for the success of emerging object-based MPEG-4 multimedia applications. This paper provides an overview of the COST Telecommunication framework and discusses the technical Simulation Model approach taken by the COST211ter group.

 

For quick access:

What is COST?

Research on Redundancy Reduction Techniques and Content Analysis for Multimedia Services

Co-operation between COST211ter and RACE/ACTS Programs

Summary and Conclusion

References

 

1 What is COST?

The COST (Coopération européenne dans la recherche scientifique et technique) Telecommunications activities provide an open and flexible framework for R&D co-operation in Europe. COST Actions involve pre-competitive or basic research, or activities of public utility - in particular research topics which are for strategic importance to the development of the information society.

There are four basic principles underlying the COST mechanism:

In contrast to much EU research, this form of collaboration does not require an agreed overall research policy. It focuses on specific themes for which there is interest in particular COST countries, with the primary aim to raise intellectual property rights. A special feature of all COST Actions is the complete freedom of participation of each country involved. Any COST country can join any Action by signing the Memorandum of Understanding (MoU) which is the legal basis of the Action even though it in fact resembles an expression of good faith rather than a legally binding document. Each MoU governs the joint aims, the type of activity to be pursued, the terms of participation and compliance with intellectual property rights.

 

2 Research on Redundancy Reduction Techniques and Content Analysis for Multimedia Services

 

2.1 Background of COST211

 

The present COSTter action is a follow up project to Actions COST 211, 211bis, all dealing with redundancy reduction techniques applied to video signals [1]. The two preceding COST 211 projects allowed the creation and maintenance of a high level of expertise in the field of video coding in Europe and allowed very active European contribution to the standardization bodies in this field. The digital video standards adopted and deployed today are strongly influenced by the results of the COST 211 Actions and the label COST 211 is well known and respected worldwide in the field of video coding.

 

Action COST 211 started in 1976 at the very beginning of the development of digital video coding. The result of this Action was a video coding algorithm allowing the first digital videoconference system at 2 Mbit/s and this in turn led to the first ITU standard in this field (H.120). The follow up Action COST 211bis improved to a large extent the efficiency of the coding algorithm allowing videophone and videoconferencing at ISDN rates and futher ITU standards (in particular H.261). Finally, COST 211ter has been a major contributor to the standardization activity in MPEG (MPEG 1, 2 and 4) and ITU. The COST projects cover more than 20 years of cooperation and strengthened considerably the European position and influence in the field of video coding.

 

All COST 211 Actions have been and are following a flexible framework which is open to a large number of members and allows long period cooperation. The COST 211 Actions are considered to be complementary to other EU projects in this field.

 

 

2.2 Current Objectives of the COST211ter Research Group

 

The main objective of the current COST211ter Action group is to improve the efficiency of redundancy reduction and to develop content analysis techniques for video signals to assist future multimedia applications. In particular the group focuses today on content-oriented processing for emerging interactive multimedia services, such as the ongoing ISO MPEG-4 standardization phase [2] as well as the new ISO MPEG-7 initiative. The aim is to define and develop a set of tools assisting these new services in the analysis, characterisation and processing of their video and audio signals. In order to pursue the just started activity on content analysis, a new COST211 Action is presently forwarded for approval.

 

Interactive Multimedia services will strongly influence and even dominate the future of communications and telecommunications. Both, the flexibility and efficiency of the coding systems used as well as the ability to efficiently search for particular content of interest on distributed data-bases are essential for the success of these emerging services.

 

 

2.3 Scientific Program of COST211ter and proposed new COST211 Action on Content Analysis

 

General Objectives

 

The participants in the Action simultaneously undertake and coordinate research and development work with the aim of applying analysis and coding techniques to video and audio signals to assist emerging interactive multimedia applications. Particular emphasis is placed on services that envision the access to and interaction with audio-visual content in multimedia environments, where a high degree of interactivity between the end user (or an agent) and particular audio-visual content is required.

 

The service and application profiles envisioned are

In order to meet the objectives indicated above, the research and development work undertaken focuses, but is not restricted, on achieving the following functionalities:

  1. Audio-Visual Content Identification (e.g. the automatic detection of content of interest in video scenes, such as a person, car, etc.)
  2. Content and/or Feature Extraction (e.g. the automatic extraction of content of interest in complex video or audio scenes)
  3. Tracking of Content and/or Features (e.g. the tracking of content or features of interest over time in an audio or video scene allowing user interaction with the tracking process)
  4. Selective Coding of Objects (for the separate compression of content data to allow separate access and query to the content in a data-base)
  5. Improved Coding Efficiency (for the reconstruction of content with sufficient quality by means of sophisticated compression technology)
  6. Selective Error Protection (for a robust storage or transmission of the most important content in error prone environments)
  7. Content-Based Visual Database Query and Indexing (e.g. the visual query for content of interest)

Particular emphasis is placed on problems related to image and video segmentation, joint segmentation of audio, speech and video signals, image and video feature recognition, and parameteric description of content in images and video based on first, second and third generation computer vision techniques.

 

The focus items are researched by means of:

 

The action primarily focuses on studies applied to video signals, but in the future will associate as much as possible with equivalent studies applied to audio signals. As in the past, the present action also contributes very actively to the standardization bodies and fora in this field (ITU, ETSI, MPEG, DAVIC,..). Particular emphasis is dedicated towards the ISO MPEG-4 and MPEG-7 standardization activities which are directly relevant to the goals of COST211ter.

 

Collaborative Work

 

The work in the Cost211ter group progresses in a collaborative manner. To this end a "Test" or "Analysis" Model (AM) approach is adopted to investigate and optimize algorithms for Image and Video Analysis. The figure 1 below depicts the basic building blocks of the envisioned application scenario (KANT: Kernel for Analysis for New Multimedia Technologies), the input and output for the image analysis as well as provisions for user interaction and coding of the image sequences.

 

 

Figure 1: The European COST211ter KANT Model

 

The COST211ter meeting in Ankara, Turkey, in October 1996 witnessed the definition of the 1st Analysis Model - which consists of a full description of tools and algorithms for automatic and semi-automatic image sequence segmentation (object segmentation and tracking). This algorithm was adopted based on work described in [4] and [5]. The outline of this Analysis Model - which provides a first implementation of the „Analysis Kernel" in figure 1 - is depicted in figure 2.

 

The algorithm consists of a set of tools that can be flexibly combined to achieve fully automatic segmentation of moving objects within scenes with static textured background or background that can be described by means of global motion parameters (e.g. static textured background and moving, panning and zooming camera). After elimination of the global motion in a scene a number of input sensor parameters can be used by a decision block to merge parts of an image to a particular moving object - and to thus perform object detection and tracking. In general these parameters consist of a) color segmentation and b) motion field segmentation to identify objects and c) motion compensation and d) change detection mask to allow a temporally coherent identification and tracking of the objects. This algorithm can be readily used to perform segmentation or object tracking in a semi-automatic way with user interaction - for example when an object is marked by a user in any particular frame (e.g. the first frame).

 

The defined Analysis Model algorithms and tools have been and will be further refined and improved in successive meetings in a collaborative effort by software implementation of the Analysis Model, by exchange of software, and by performing joint experiments based on the AM.

 

 

 

Figure 2: Diagram of the basic building blocks of the Image Analysis Test Model

 

 

3 Co-operation between COST211ter and RACE/ACTS Programs

 

The COST211ter research group Action aims to promote basic applied scientific and technical research - and is in the nature of precompetitive research, falling somewhere between fundamental research and actual development work aimed at defining new products. During recent years it has been joined by RACE and ACTS activities funded by the European Commission, in particular the RACE-MAVT group, and most recently by the ACTS-MOMUSYS group. These groups are not seen as competitors but rather as complementary programs to the COST211ter work. It is clear that COST - RACE/ACTS interactions thrive on the unique attributes of one another. The complementary is aided at the very broad range of research subjects in the COST211ter group - in contrast to the focused, specific projects in RACE/ACTS. Although both, COST211ter on one hand and RACE/ACTS on the other, operate in the pre-competitive research domain, COST offers the specific advantage of providing a very flexible environment in which for example activities of particular commercial sensitivity can be sometimes discussed without formal commitments being required from the beginning.

 

4 Summary and Conclusion

 

This paper has provided an overview of the aim and the framework of the COST Telecommunication Action as well as of the European COST211ter research group activities. The Simulation Model Approach adopted by the COST211bis and COST211ter groups was discussed, covering the technical scope of the COST211ter Image Analysis Simulation Model.

 

The COST211 framework, dedicated to the research and development of redundancy reduction and content analysis for emerging multimedia applications, provides an active and important European forum for the definition of concepts and algorithms within the European Community and beyond. A new COST211 Action is presently forwarded for approval in order to pursue this major activity in Europe. As such, and especially in light of the contributions and impact of the COST211, COST211bis and COST211ter activities for the H.261, H.263 and MPEG-4 standards, the groups activities have been instrumental in raising intellectual property for the COST, RACE and ACTS member companies and institutions involved in the COST211 efforts.

 

References:

[1] T.Sikora, „The European COST211ter Activities - Research Towards Advanced Algorithms for Coding Video Signals at Very Low Bit Rates", IEEE International Conference on Image Processing, ICIP'96, Lausanne, September 1996.

[2] T.Sikora and L.Chiariglione, „Requirements and Algorithms for MPEG-4 Multimedia Applications", IEEE International Symposium on Circuits and Systems, ISCAS, „Special Session on "Networked Multimedia Technologies and Applications", June 1997.

[3] COST211ter Group Members, „COST211ter SIM Group Focus Document Vers. 2.0", Nov. 1996, http://www.teltec.dcu.ie/cost211ter/focus.html .

[4] A. Alatan, E. Tuncel and L. Onural, "Object Segmentation via Rule-Based Data Fusion", Bilkent University, Turkey, Proc. Workshop Image Analysis for Multimedia Interactive Services WIMIS’97, Belgium, June 1997.

[5] R. Mech and M. Wollborn, „A Noise Robust Method for 2D Shape Estimation of Moving Objects in Video Sequences Considering a Moving Camera", University of Hannover, Germany, Proc. Workshop Image Analysis for Multimedia Interactive Services WIMIS’97, Belgium, June 1997.

 

Back to Introduction to COST 211