This paper proposes a method of error detection and recovery by hiding specific information into video bitstream using fragile watermarking and checking it later. The proposed method requires no additional bits into compressed bitstream since it embeds a user-specific data pattern in the least significant bits of LEVEL's in VLC codewords. The decoder can extract the information to check whether there is an error in the received bitstream. We also propose to use this method to embed essential data such as motion vectors that can be used for error recovery. The proposed method can detect corrupted MB's that usually escape the conventional syntax-based error detection scheme. This proposed method is quite simple and of low complexity.

We propose a new motion estimation (ME) algorithm adequate for temporal error concealment. To increase spatial motion vector (MV) correlation for efficient temporal error concealment, the proposed algorithm utilizes an enlarged macroblock (MB) for ME, which includes pixels in the adjacent MBs in addition to an original MB. It is proved that the algorithm considerably improves error concealment performance, while minimizing the degradation of encoding performance.

This paper presents an embedded video compression with error resilience and error concealment using three dimensional SPIHT (3-D SPIHT) algorithm. We use a new method for partitioning the wavelet coefficients into spatio-temporal (s-t) blocks to get higher error resilience and to support error concealment. Instead of grouping adjacent coefficients, we group coefficients in some fixed intervals in the lowest subband to get interleaved trees. All of the sub-blocks correspond to a group of full frames within the image sequence, because each group of interleaved trees has coefficients dispersed over the entire frame group. Then we separate the stream into fixed length packets and encode every one with a channel code. Experiments show that our proposed method brings higher error resilience in noisy channels since the decoded coefficients associated with early decoding error would be spread out to the whole area along with the sequence, and we can conceal the lost coefficients with the surrounding coefficients even if some of substreams are totally missing. In addition to that, our proposed method gives higher coding performance in noseless channels than the conventional grouping method of grouping contiguous trees.

In this paper, a further study based on the results of is conducted to accomplish a near-optimum design of stream-shuffling and error concealment for error-resilient transmission of wireless video. In particular, a two-phase stream-shuffling scheme is systematically scheduled in conjunction with embedded channel codes, while a new error detection method and adaptive error concealment algorithm are contrived to effectively mitigate the effects of residual errors on visual quality. Comparison with the algorithms proposed in, the new methodology may provide better understanding of optimum design of stream-shuffling in a systematic way, which yielding improved performance in wireless video communication.

With the emerging of the third generation (3G) wireless technology, digital media, like image and video, over wireless channel becomes more and more demanding. In this paper, the measure metrics for the wireless image is proposed and a Qos-guarantee error control is presented, combining UEP with Forward Error Correction (FEC) and Automatic Repeat reQuest (ARQ), aiming to high quality image transmission with short delay and little energy. Simulation results show that our scheme can achieve good reconstructed image with few retransmission times and small bit budget under different channel conditions, which can reduce the energy consumed in the network interface.

An enhanced network service model for layered video multicast applications is presented for differentiated service (Diffserv) networks in this work, which consists of enhanced active queue management (AQM) and hierarchial priority marking. It is found from experiments that a straightforward application of the receiver-driven layer multicast (RLM) model to the Diffserv network may perform poorer than to the best-effort Internet, since RLM applications could behave like unresponsive flows and conflict with Diffserv active queues. To fix this problem, we present three sets of extended RED/RIO parameters as the enhanced AQM model, which is designed to be generally applicable to service protection as well as to support RLM traffic in Diffserv networks. Furthermore, to take advantage of the priority service in Diffserv networks, we present a priority marking model of RLM video layers that conform to the priority marking architecture of Dffserv networks. It is shown by simulation experiments that the proposed network service enhancement model improves the stability of the join experiment and reduces the packet loss rate to support RLM traffic in Diffserv networks.

In this research, a video delivery system is proposed by integrating the layered codec and the proposed corruption model. In the system, it is assumed that the base layer packets are delivered without loss since they are assigned the most reliable service level. Then, packets in the enhancement layer are analyzed with the corruption model and categorized for differentiated delivery services. The corruption model for generating the relative priority index (RPI) is extended to layered bitstreams. In addition to taking into account the initial error and the propagation error due to packet loss, the dependency between the same and different layers is reflected in the modified macroblock corruption model. In order to facilitate the prioritized delivery of the enhancement layer, the RPI-based corruption model for each packet as well as coordinated delivery of packetized video over QoS networks are investigated. Finally, a per-packet optimization framework with unequal error protection (UEP) is realized to consider both end-to-end video performance and pricing. The performance of the proposed solution is verified with extensive simulations.

This work proposes and implements a system to transmit in real-time slow video signals - in particular video conference signal, that is head and shoulder sequences - over the public network. The proposed codec is based on a multiple description approach that gives N equally important and independent flows. The advantage of this codec is an intrinsic robustness to the transmission errors and to the packet loss, as the simulation results have proved. This feature results very suitable not only for IP-network, but also for every packet network like the next generation mobile systems. The approach is important also for the end user scalability since each device can decide the information to receive according to the resolution of the display and the bandwidth of the connecting link using the same source data stream.

TCP-friendly layered video transmission for the Internet multicast application is investigated in this research. To adapt the available bandwidth in a TCP-friendly manner, a receiver-driven rate adaptation approach based on layered video is used. The receiver adjusts its number of subscription layer based upon the TCP-throughput estimation obtained from our previous work known as smooth and fast rate adaption mechansim(SFRAM). Since SFRAM mitigates unnecessary rate fluctuation and responds only to distinct network variations, a receiver can add or drop a certain number of video layers in a very stable manner. Furthermore, we apply the priority dropping mechanism (PDM) to protect the video layer selectively. To achieve fairness among different flows with PDM, we propose a new packet classification scheme (PCS) to assign the priority to each layer based upon the target rate. Since proposed PCS can also distinguish the priority for packets even within the same layer, it is suitable for Internet video multicast. With PCS, we implement PDM on the random early dropping (RED) queue to drop packets of the lowest priority layer so that end users may avoid instant error propagation. We conduct NS simulations to examine the proposed layer adaptation behavior and the effect of PDM. Our results show that SFRAM is effective enough to direct the right number of subscription layer stably while it behaves in a TCP-friendly manner. Furthermore, PDM on RED can protect layered video of the highest priority effectively with proposed PCS.

This paper presents an effective VOD transmission scheme under RCBR network that is suitable for the statistically non-stationary video traffic since it supports the bandwidth renegotiation during data transmission. The proposed system includes two parts: the VOD server architecture and traffic smoothing/admission control algorithms. The proposed VOD server system consists of video DB and metadata DB that includes compression history information including the quantization parameters versus output rate profile, encoded frame positions and encoding types, etc. And, the proposed traffic smoothing and admission control algorithms are performed based on the aforementioned metadata and control the traffic properties of the selected VOD through the communication between them to be accommodated under the current network situations. And, the experimental results are provided to demonstrate the efficiency of the proposed system.

In network delivery of compressed video, packets may be lost if the channel is unreliable. Such losses tend to occur in burst. In this paper, we develop an error resilient video encoding approach to help error concealment at the decoder. We introduce a new block shuffling scheme to isolate erroneous blocks caused by packet losses. And we apply data hiding to add additional protection for motion vectors. The incorporation of these scheme adds little complexity to the standard encoder. Experimental results suggest that our approach can achieve a reasonable quality for packet loss up to 30% over a wide range of video materials.

A synchronization scheme that can specify the start position of each macroblock in a compressed video bitstream for low data rate wireless applications is proposed by extending the Error Resilient Entropy Coding (EREC) method. Our scheme is implemented in the form of a transcoder, placed before and after the channel, of a MPEG-4 Simple Profile bitstream. We compare our proposed technique using reconstructed video quality (measured in PSNR) and the length of the extra redundancy bits incurred by the transcoder with the error resilient tools of H.263⁺. A simple syntax-based codeword repair method is also proposed so that the transcoder generates a MPEG-4 compliant bitstream which can then be decoded with a standard MPEG-4 decoder such as MoMuSys (FDIS V1.0).

In this paper we present a method of creating domain-based multiple descriptions of images and video. Descriptions are created by partitioning the domain of the signal into sets whose points are maximally separated from each other. This property enables simple error concealment methods to produce good estimates of lost signal samples. We present the approach in the context of Internet transmission of subband-coded images and scalable motion compensated 3-D subband-coded video, but applications are not limited to these scenarios.

A block-interlaced codewords realignment (BICR) scheme is proposed for error resilience enhancement video transmission over burst error channels. The video coder is compatible with the MPEG-4 coding standard. Specifically, based on the error propagation property of the variable length coding (VLC) and reversible variable length coding (RVLC), the proposed scheme realigns the codewords within a minimum data unit according to the loss probability and the source significance information. This realignment automatically achieves an unequal error protection for different codewords of the coded video bit-stream with different importance, increasing the error resilience of the video coding without any penalty in transmission efficiency. Simulation results confirm that this realignment can increase the quality of the reconstructed video frames remarkably over channels with burst errors in terms of PSNR criterion.

In this paper we proposed a temporal error concealment scheme utilizing the overlapping approach for the block-based video coder. In the proposed algorithm, each pixel of the lost block is recovered by overlapping from four nearest available neighboring blocks. The weighting coefficients used for overlapping have been carefully designed to address the suitable weights on different pixel locations inside the lost block. The weighted extended boundary match is proposed to replace the popularly used boundary match for recovering the lost motion vectors, such that the recovered motion vector can provide the best lost pixel prediction for the overlapping error concealment process. The experimental results were shown by integrating our algorithm into the H.263+ coder.

We present a low cost high performance scheme for video transcoding. Our scheme includes the information look-ahead structure and the transcoding algorithm. It is this look-ahead mechanism that greatly facilitates the use of information embedded in the bitstream for informative and intelligent transcoding. Furthermore, our transcoding algorithm has four new features: 1) picture complexity look-ahead for efficient rate allocation and control; 2) dynamic bit allocation weighting on picture types based on the scene complexity and bit rate; 3) adaptive nonlinear rate control on macroblocks based on complexity distribution in the picture; 4) Intra frame requantization adjustment to reduce the transcoding distortion. The software simulation results have proven that the performance of our transcoding scheme is significantly better than the reencoding scheme and comparable to the direct encoding scheme.

In video transcoding, useful coding statistics become available when the input video stream is decoded. It is possible to utilize these statistics to improve the transcoding performance. In this paper we investigate how to use these statistics to perform the picture bit-allocation for transcoding a pre-encoded video bit-stream. We propose a scheme to estimate the complexities of the pictures of the output video using the coding statistics computed from the input video stream. Based on the estimated picture complexity, we present a picture bit-allocation algorithm for the rate-control of the transcoding process. The algorithm is simple to compute, and effectively improves the video quality and reduces the picture quality variations.

In this paper, we work on the problem of inserting logo into part of a previously compressed video sequence while transcoding it to a video stream with lower rate. In most of the transcoding architecture, it has been widely accepted to reuse the motion information extracted from the original incoming bit stream. For our transcoding and logo insertion process, we preserve the same motion information as the incoming video stream for the parts that were unaffected by logo, and propose an efficient algorithm to code logo areas. We first find the proper range of pictures that might be affected by logo insertions, analyze logo-affected parts of those pictures, and perform corresponding algorithms depending on the picture types. In addition, we develop a method to achieve better bit allocation during logo insertion. The efficiency of our algorithm is demonstrated by simulation.

This paper discusses the problem of reduced-resolution transcoding of compressed video bitstreams. An analysis of drift errors is provided to identify the sources of quality degradation when transcoding to a lower spatial resolution. Two types of drift error are considered: a reference picture error, which has been identified in previous works, and error due to the non-commutative property of motion compensation and down-sampling, which is unique to this work. To overcome these sources of error, four novel architectures are presented. One architecture attempts to compensate for the reference picture error in the reduced resolution, while another architecture attempts to do the same in the original resolution. We present a third architecture that attempts to eliminate the second type of drift error and a final architecture that relies on an intra block refresh method to compensate all types of errors. In all these architectures, a variety of macroblock level conversions are required, such as motion vector mapping and texture down-sampling. These conversions are discussed in detail. Another important issue for the transcoder is rate control. This is especially important for the intra refresh architecture since it must find a balance between number of intra blocks used to compensate errors and the associated rate-distortion characteristics of the low-resolution signal. The complexity and quality of the architectures are compared. Based on the results, we find that the intra refresh architecture offers the best trade-off between quality and complexity, and is also the most flexible.

MPEG-2 and MPEG-4 are two most popular international video compression standards. They will coexist in different systems and networks for a long time. For compatibility of these two video standards, this paper put forward three transcoding algorithms on the basis of detailed analysis of computational complexity. In the paper, the principle and implementation of two efficient transcoding algorithms are emphasized. Both of them are tradeoff between the computational complexity and reconstructed video quality. One is motion-information reusing algorithm, which meets the highest video quality requirement of studio and post-processing environments. The other is real-time low complexity transcoding algorithm, which is proper for low delay requirement of the network server and user terminal with limited computational performance. The simulation results of many test sequences verified the high performance of the algorithms.

Current rate control schemes in video coding standards do not have efficient frame-level bit allocation due to the limitation of real-time encoding. In this paper, by taking advantage of offline video encoding, we proposed a two-pass video encoding scheme for low bit rate streaming applications. Specifically, in the first pass, we generate the feature information of video sequences, including rate- distortion (R-D) functions and scene change boundaries. Then, in the second pass, according to the available channel bandwidth, by exploiting the feature information, we are able to implement frame-level bit allocation in an optimal way so that video sequences can be coded at low bit rate with an improved quality. Experimental results show the proposed scheme is able to achieve not only the improved PSNR results but also much smoother visual quality.

Streaming media over heterogeneous lossy networks and time-varying communication channels is an active area of research. Several video coders that operate under the varying constraints of such environments have been proposed recently. Scalability has become a very desirable feature in these video coders. In this paper, we make use of a leaky-bucket rate allocation method (DBRC) that provides constant quality video under buffer constraints, and extend it in two advantageous directions. First, we present a rate control mechanism for 3D wavelet video coding using DBRC. Second, we enhance the DBRC so that it can be utilized when multiple sequences are multiplexed over a single communications channel. The goal is to allocate the capacity of the channel between sequences to achieve constant quality across all sequences.

Proxy cache located at base station is helpful for improving performance of wireless multimedia streaming. In this paper we propose a cooperative proxy cache architecture, MobileCache, for cell-based wireless environment. The proxies cooperate to provide seamless streaming for wireless client even when the client crosses the cell boundary. In order to improve the efficiency for MobileCache, we develop a cache cooperation management scheme, including cache replacement, cache migration and cache replication, which maximize the revenue for MobileCache. Simulation results confirm that, by employing the cache cooperation management, MobileCache achieves higher performance than other cache management schemes.

Streaming video over IP is studied by integrating the error resilient scalable source codec with constant quality rate adaptation and prioritized network transmission. The scalable codec is based on the standardized ISO/IEC MPEG-4 fine granular scalable (FGS) coding scheme, which allows arbitrary truncation according to the given rate budget. To maximize the end-to-end visual quality, constant quality video rate control (CQVRC) is proposed to preserve smooth video quality when there is a significant amount of variation in the video source, the available bandwidth or both. The proposed CQVRC attempts to provide smooth video quality for both the base layer (BL) and the enhancement layer (EL). The video stream is then prioritized for priority dropping, where rate adaptation is dynamically performed to meet the time-varying available bandwidth. This kind of prioritized stream can benefit from QoS networks such as the IP differentiated service (DiffServ). All key components, including FGS encoding, CQVRC based rate adaptation and packetization, error resilient decoding, and differentiated forwarding, are seamlessly integrated into one system. By focusing on the end-to-end quality, we set both source and network parameters properly to achieve a superior performance of FGS video streaming.

This paper proposes a new content-based rate shaping algorithm for Fine Granular Scalable scheme (FGS). FGS has been adopted by the MPEG-4 video standard as the core compression tool for video streaming over Internet, and has enabled a wide range of multimedia applications. However, most of current video streaming technologies only protect the video stream by rate allocation or CRC means, based on the bandwidth of the network. As a result, the video quality that user expected is unrelated with the content of video. Our approach is innovative in that it is based on video content analysis and extraction of the information of video content. Firstly, we evaluate the importance of the video sequence by using NFL (Nearest Feature Line) method. Then we drop the enhancement layer in term of the importance, which is decided by the bits sent out that meet the current bandwidth of network. The experimental results indicate that our layer dropping method not only improves the performance of FGS to 0.2 dB, but also enhances the subjective quality of video effectively.

In this paper, we investigate the object placement problem in distributed cooperative proxy systems and build a mathematics model. We prove that the problem is a NP-Hard problem and two novel greedy heuristic placement algorithms MWLC and MWGB are proposed to get the near-optimal answer in a polynomial complexity. We perform simulation and the results reported in this paper show that these two algorithms could achieve better performance than the traditional MFU algorithm (about 13.05% and 14.13% respectively). By the analysis of the experiment results, we find that the two algorithms can efficiently reduce the average request cost and balance the load of proxies. In general, the technologies proposed in this paper could be used to develop effective distributed cooperative proxy systems such as CDSP and digital library and so on.

Illumination variability has a considerable influence on the performance of computer vision algorithms or video coding methods. The efficiency and robustness of these algorithms can be significantly improved by removing the undesired effects of changing illumination. In this paper, we introduce a 3-D model-based technique for estimating and manipulating the lighting in an image sequence. The current scene lighting is estimated for each frame exploiting 3-D model information and by synthetic re-lighting of the original video frames. To provide the estimator with surface normal information, the objects in the scene are represented by 3-D shape models and their motion and deformation are tracked over time using a model-based estimation method. Given the normal information, the current lighting is estimated with a linear algorithm of low computational complexity using an orthogonal set of light maps. This results in a small set of parameters which efficiently represent the scene lighting. In our experiments, we demonstrate how this representation can be used to create video sequences with arbitrary new illumination. When encoding a video sequence for transmission over a network, significant coding gains can be achieved when removing the time varying lighting effects prior to encoding. Improvements of up to 3 dB in PSNR are observed for an illumination-compensated sequence.

A view-dependent progressive mesh coding algorithm is proposed in this work to facilitate interactive 3D graphic streaming and browsing. First, a 3D graphic model is split into several partitions. Second, each partition is simplified independently to generate a base model that can be efficiently encoded. Third, topological and geometrical data are reorganized to enable the view dependent transmission. Before the transmission, the server is informed of the viewing parameters. Then, the server can accordingly transmit visible parts in detail, while cutting off invisible parts. Experimental results demonstrate that the proposed algorithm reduces the required transmission bandwidth, and provides an acceptable visual quality even at low bit rates.

In the field of compression, the type of 3D models traditionally considered is that of polygonal meshes, for which several efficient compression techniques have been proposed in the recent years. Nowadays, an increasing proportion of 3D models are created by a synthesis or modeling process, instead of captured from the real world. Such models are most often given as parametric surfaces, which have several advantages over polygonal meshes, such as resolution independence and a more compact representation. This paper proposes a method to code parametric surfaces, given as Non-Uniform Rational B-Splines (NURBS). The coding scheme consists in coding the NURBS parameters (knots and control points) using a predictive scheme, coupled with uniform quantization and entropy coding. The multiplicity of knots is preserved by decomposing the knot vectors in a break vector (the values) and a multiplicity map. The rate-distortion of the proposed scheme is evaluated and compared against compressed triangular meshes. The results show that a considerable compression ratio is achievable under visually lossless conditions, that outperforms by far triangular meshes. In addition of having a better rate-distortion performance, the coding scheme enables the efficient transmission of synthesized 3D models retaining their resolution independence.

In this paper, we propose a new algorithm to code the animated three-dimensional (3-D) mesh model. After we classify the mesh frames in the animation model into intra- and inter-meshes, each mesh frame is decomposed into linear triangular strips, which in turn are partitioned into several fixed-length segments. The intra-mesh is compressed by differential coding of vertex coordinates. The inter-mesh is motion-compensated segment by segment from the previous meshes and residual errors are transformed by 1-D DCT. The transform coefficients are then entropy coded. We demonstrate that the proposed algorithm yields an improved coding gain than the MPEG-4 SNHC codec.

A jointly designed source-channel coding technique that protects video signals against transmission errors is proposed in this work. We develop a low-complexity source model which can estimate the bitrate and the quantization error for each packet. Also, the channel distortion is estimated based on the packet importance and the packet loss rate. The packet importance is measured as the mean square error between the error-free reconstruction and the concealed reconstruction, and the packet loss rate is determined by channel conditions. According to the informed channel condition, the encoder adaptively assigns the quantization parameter and the channel code rate to each packet so that the expected mean square error due to source and channel distortions is minimized subject to a constraint on the overall bitrate. Simulation results show that the proposed adaptive system provides acceptable image quality even in a high bit error rate environment.

Dynamic throughput estimation for wireless multimedia transmission is examined in this research. A novel dynamic mapping scheme from measurements of a real-world wireless environment to a discrete Markov model is first proposed to simulate wireless video packet-based transmission. Then, the available throughput is estimated by using the derived discrete Markov model together with the number of negative acknowledgments (NACK) that are fed back from the receiver to the transmitter. The estimated throughput is used to perform the adaptive region-of-interest (ROI) rate control based on the multiple constraint optimization. The performance of the channel mapping scheme, the throughput estimation, and the region-of-interest rate control is demonstrated by simulating a micro-cell environment and ITU-T H.263+ video standard. Performance variation due to different estimation of throughput is compared and the performance improvement due to the multiple constraint optimization is demonstrated both in objective and subjective quality.

Video transmission over the multi-path fading wireless channel has to overcome the inherent vulnerability of compressed video to the channel errors. To effectively prevent the corruption of video stream and its propagation in spatial and temporal domain, proactive error controls are widely being deployed. Among possible candidates, turbo code is known to exhibit superior error correction performance over fading channel. Ordinary turbo codes, however, are not suitable to support the variable-size segment of the video stream. A version of turbo code, byte-aligned variable-length turbo code, is thus proposed and applied for the robust video transmission system. Protection performance of the proposed turbo code is evaluated by applying it to GOB-based variable-size ITU-T H.263+ video packets, where the protection level is controlled based on the joint source-channel criteria. The resulting performance comparison with the conventional RCPC code clearly demonstrates the possibility of the proposed approach for the time-varying correlated Rayleigh-fading channel.

This article presents a novel unequal error protection (UEP) for efficient multimedia services over mobile cellular environments, in particular for the case that runs delay- sensitive (e.g., video streaming) and delay-insensitive (e.g., ftp, http) applications simultaneously. In cdma2000 1x system, for example, one Physical Layer Packer (PLP) can obtain multiple streams from multiple information/application sources. However, each PLP is error-protected evenly without consideration f different QoS attributes of different information/application sources. The key idea to be presented in this paper is that more protection is employed to the short delay-insensitive application packets to ensure very low error rate and less protection is given to relatively long delay-sensitive application packets to get almost the same error rate as the normal case - this is using the concept of inter-media QoS classification taking advantage of Turbo coding gain and penalty. By doing this, entire rate of successful transmission for delay-insensitive application is increased so that total packet connection time is reduced with comparable quality of delay-sensitive applications. The scope of this paper is to illustrate the proposed concept, its performance gain and some identified key future works. Section 1 contains some preliminary aspects of the recent mobile cellular systems for high-speed packet access applications. Section 2 describes the key concept of the proposed scheme and its associated simulation results are presented in section 3. Conclusions are drawn in section 4 followed by the final section to identify some possible future works.

A call admission control (CAC) scheme and a resource-reservation estimation (RRE) method suitable for the interference-based wireless system, such as wide-band code division multiple access (W-CDMA), are proposed in this work. The proposed CAC scheme gives preferential treatment to high priority handoff calls by pre-reserving a certain amount of interference margin called the interference guard margin (IGM). The amount of guard margin is determined by the measurement performed by the RRE module in base stations. Each RRE module dynamically adjusts the level of guard margin by considering traffic conditions in neighboring cells based upon handoff requests. A service model is adopted to support multiple services, which includes mobile terminal's data rate, different levels of priorities, mobility and rate adaptivity characteristics. Simulations are conducted with OPNET to study the performance of the proposed scheme in terms of the objective function, blocking probabilities and system utilization under different traffic conditions.

In this paper, we present a motion compensated two-link chain coding technique to effectively encode 2-D binary shape sequences for object-based video coding. This technique consists of a contour motion estimation and compensation algorithm and a two-link chain coding algorithm. The object contour is defined on a 6-connected contour lattice for a smoother contour representation. The contour in the current frame is first predicted by global motion and local motion based on the decoded contour in the previous frame; then, it is segmented into motion success segments, which can be predicted by the global motion or the local motion, and motion failure segments, which can not be predicted by the global and local motion. For each motion failure segment, a two-link chain code, which uses one chain code to represent two consecutive contour links, followed by an arithmetic coder is proposed for efficient coding. Each motion success segment can be represented by the motion vector and its length. For contour motion estimation and compensation, besides the translational motion model, an affine global motion model is proposed and investigated for complex global motion. We test the performance of the proposed technique by several MPEG-4 shape test sequences. The experimental results show that our proposed scheme is better than the CAE technique which is applied in the MPEG-4 verification model.

This paper presents a region-based scalable coding technique that can be used in interactive transmission of images over networks. This method has a capability of near lossless coding for a specific region of interest (ROI), while the rest of the region is coded with a high quality lossy codec. The enhancement layers add refinement to the quality of the images that have been reconstructed using the basic layer of the intra frame. The proposed coding technique uses multiple quantizers with thresholds (QT) for layering and it creates a bit plane for each layer of both the intra and residual frames. The bit plane is then partitioned into sets of small areas to be coded independently. Run-length and entropy coding are applied to each of the sets to provide scalability for the entire image sets resulting in high picture quality in the end user-specified ROI. We tested this technique by applying it to various test image sequences and we consistently achieved a high level of performance.

This paper describes a bit allocation algorithm that can achieve a constant bit rate when coding multiple video objects (MVOs), while improving the rate-distortion (R-D) performance over the reference method for MPEG-4 object-based rate control. In object-based coding, bit allocation is performed at the object level and temporal rates of different objects may vary. The proposed algorithm in this paper deals with these two issues. We pay particular attention to maintenance of buffer occupancy levels and propose a new method for spatio-temporal trades-offs for object-based coding. In order to improve the coding efficiency, we also consider several new R-D coding modes. These modes are available at the encoder to determine the best R-D performance under different coding conditions and are chosen automatically. Simulation results demonstrate moderate improvements at low and high bit rates. One important key aspect of the proposed algorithm is that the actual coded bits in the proposed algorithm are similar to the target bits over a wide range of bit rates. Consequently, the proposed algorithm has not experienced the buffer overflow/underflow over the entire range of bit rates.

In this paper, we propose a modified shape adaptive discrete wavelet transform (SA DWT). In conventional SA DWT method, after several levels of decomposition, the number of coefficients that has no parent in coarser scale is usually large and uncontrollable. If a tree-structure coding method is applied to the wavelet coefficients, the coding efficiency will be decreased by these coefficients. In our proposed method the boundary of object needs to be extrapolated first and then symmetric reflection is applied. By using this kind of procedure, the number of coefficients without parent can be reduced. The proposed SA DWT is also non-expansive. We establish a modified SPIHT coding algorithm to test the new method's contribution to coding efficiency. Experiment results show that the proposed method can produce more regular LL band figure and achieve a better rate-distortion curve than that of a conventional one.

Facial Animation Parameters (FAPs) are defined in MPEG-4 to animate a facial object. The algorithm proposed in this paper to extract these FAPs is applied to very low bit-rate video communication, in which the scene is composed of a head-and-shoulder object with complex background. This paper addresses the algorithm to automatically extract all FAPs needed to animate a generic facial model, estimate the 3D motion of head by points. The proposed algorithm extracts human facial region by color segmentation and intra-frame and inter-frame edge detection. Facial structure and edge distribution of facial feature such as vertical and horizontal gradient histograms are used to locate the facial feature region. Parabola and circle deformable templates are employed to fit facial feature and extract a part of FAPs. A special data structure is proposed to describe deformable templates to reduce time consumption for computing energy functions. Another part of FAPs, 3D rigid head motion vectors, are estimated by corresponding-points method. A 3D head wire-frame model provides facial semantic information for selection of proper corresponding points, which helps to increase accuracy of 3D rigid object motion estimation.

Efficient MPEG encoding begins with the selection of each frame's coding type. Depending on the constraints of a given application, the options available for individual frame types differ. These options are further limited by the encoding environment, in particular the frame buffer size and computational power available. In this work we assume an off-line encoding environment, with its implicit unlimited frame buffer and arbitrarily fast computations. Such an environment enables us to examine ideal frame types for a variety of situations, with changing emphasis between encoded bits and motion estimation error. We find that the optimal frame types vary dramatically as focus shifts from bit allocation to motion estimation error, even for ``natural'' video sequences. When additional video effects such as fades and scene cuts are introduced, the frame types become even more interesting, deviating significantly from the M and N fixed-spacing scheme used by many real-time MPEG encoders.

Common motion compensated hybrid video coding standards such as H.263, MPEG-1, MPEG-2, MPEG-4 are based on a fractional-pel displacement vector (DV) resolution of 1/2-pel. Recent approaches like MPEG-4 (ACE-profile) and H.26L use higher DV resolutions of 1/4-pel and 1/8-pel. In order to estimate and compensate fractional-pel displacements, the image signal has to be interpolated. Especially for higher DV resolutions an efficient interpolation filter is necessary. Thus, in MPEG-4 (ACE-profile) and H.26L 6 or 8-tap Wiener interpolation filters are applied. These Wiener filters were designed to interpolate the image signal while reducing spacial aliasing components that deteriorate the motion compensated prediction. In we presented an improved interpolation filter that allows to interpolate an aliasing affected image more accurate then the conventional Wiener filter. This filter is a combination of the Wiener filter and a motion compensated interpolation filter. Up to now these filters are based on invariant filter coefficients. The same coefficients are applied for all sequences and for all images of a sequence. In we presented an adaptive interpolation filter that improves the coding efficiency by reducing the impact of displacement estimation errors and by compensating the aliasing components. The interpolation scheme of is based on filter coefficients that are adapted once per frame to the non-stationary statistical features of the sequence. This paper introduces a motion compensated adaptive interpolation filter that combines the frame-adaptive approach of and the motion compensated interpolation approach of. Due to the motion compensated adaptive interpolation filter, the coding efficiency of an H.26L codec is improved up to 0.8 dB.

An encoder-based rate smoothing technique is developed to reduce the rate burstiness in video coding. At first, we propose a new source model to accurately estimate the rate-distortion (R-D) function of each video frame with very low computational complexity. Based on the estimated R-D function, in the second step, we propose a rate smoothing algorithm to make a tradeoff between the constant bit rate and the constant picture quality. The tradeoff forces the output video stream to have both a smoothed rate shape and a consistent video presentation quality, which is highly desirable in video coding and transportation.

Video decoding at reduced resolution with resizing embedded in the decoding loop saves computational resources such as memory, memory bandwidth and CPU cycles. Key to such embedded resizing is proper filtering/scaling of DCT data, and motion compensation at the reduced resolution. Although MPEG-2 video decoding with embedded resizing has been investigated in the past, little work has been reported on solving problems associated with interlaced video undergoing decoding with embedded resizing. In particular, annoying artifacts may occur in moving areas of interlaced video due to improper scaling or motion compensation. In this paper, we introduce the notion of the Local Interlacing Property for interlaced moving areas and propose algorithms to detect and process data with the Local Interlacing Property properly in the context of decoding with embedded resizing. Specifically, we demonstrate that 1) vertical high frequency in interlaced moving areas should be preserved during downscaling, and 2) phase shift must be added for motion compensation in interlaced moving areas under certain circumstances. Experimental results show that our method effectively removes artifacts in interlaced moving areas, making MPEG-2 video decoding with embedded resizing a practical tradeoff for interlaced video.

Complexity scalable algorithms are capable of trading resource usage for output quality in a near-optimal way. We present a complexity scalable motion estimation algorithm based on the 3-D recursive search block matcher. We introduce data prioritizing as a new approach to scalability. With this approach, we achieve a near-constant complexity and a continuous quality-resource distribution. While maintaining acceptable quality, it is possible to vary the resource usage from below 1 match-error calculation per block on the average to more than 5 match-error calculations per block on the average.

Existing fast block motion estimation algorithms, which reduce the computation by limiting the number of search points, utilize the motion vector (MV) characteristics of high spatial correlation as well as center-biased distribution in predicting an initial MV. Even though they provide good performance for slow motion sequences, they suffer from poor accuracy for fast or complex motion sequences. In this paper, a new fast and efficient block motion estimation algorithm is proposed. To find an initial search point, in addition to the predictors of zero MV and neighboring MVs, the algorithm utilizes another predictor obtained from one-dimensional feature matching using selective integral projections. This low complexity procedure enables the selection of a better initial search point so that a simple gradient descent search near this point may be enough to find the global minimum point. Compared to recent fast search algorithms, the proposed algorithm has lower computational complexity and provides better prediction performance, especially for fast or complex motion sequences.

A method for scalable motion compensated up-conversion of video sequences is proposed. By computing suitable objective quality estimates,(which show a high correlation with subjective quality scores), the method allows to predict the resources needed to up-convert a given input video sequence achieving a certain visual quality of the output video sequence. This method permits to dynamically change the resource utilization in a way which is optimal for the system, e.g. for a programmable platform for media processing.

Long term memory motion compensation (LTMC) is an approach to extend the temporal motion search range by using multiple decoded frames as reference frames. By employing multiple reference frames, LTMC reduces the residual frame energy significantly. However the computational complexity of motion estimation for LTMC increases significantly as well. Therefore reduction of the required computational complexity is one of the most challenging issues for LTMC. Also, if we locate motion search windows at fixed locations in a frame buffer for a given macro-block, it is highly possible that the oldest frames in a frame buffer do not contain matching blocks due to the reduced correlation between the current frame and the reference frames located further in the frame buffer. Therefore, if we can locate the motion search window at a good position adaptively in a frame buffer, we can enhance the gain performance of LTMS. In this paper, we propose a novel motion estimation algorithm for LTMC to reduce significantly the required computation complexity, and to enhance the performance of LTMC. For the proposed motion estimation algorithm, we introduce a directed search strategy. Also, we propose to employ hypothesis testing fast matching (HTFM) as a fast matching criterion. The goal of a directed search strategy is to let the location of the motion search windows change adaptively as the search proceeds to older frames in the frame buffer. The main benefit over standard, fixed window, approaches is that the algorithm can track larger motion and therefore, we can reduce the residual frame energy. In addition, because the directed search strategy keeps track of best matched blocks, we can reduce the computational complexity significantly by reducing the motion search window area in a frame buffer. Simulation results show that by employing the directed search with reduced motion search window, we can reduce the computational complexity approximately 30%-40%, and that by employing HTFM, we can reduce the computational complexity as additional 40%-50% as compared to the result of a full search algorithm which employs a non-directed search strategy for LTMC. The proposed algorithm provides particularly significant gains for video sequences which contain large motions. In the case of high motion sequences directed search also results in slight gains in PSNR over non-directed search.

In this paper, a large range motion estimation algorithm based on hierarchical motion vector estimation is proposed. Aimed at objects move in large area in image sequences, the proposed algorithm encounter with problems caused by large range search, such as huge computational complexity and relative bad performance. The computer simulation results are also given for various test sequences.

3D modeling of human face has been an active research topic in computer vision and computer graphics communities. In this paper we develop a face-modeling system from calibrated stereo video input. The stereo configuration is chosen according to the characteristics of the face structure. For data capturing the subject sits in front of the calibrated stereo video cameras and rotates his/her head from front view to side view. Then we can create his/her 3D face model with realistic texture mapped, which is ready for MPEG-4 compatible facial animation, by adapting a generic wireframe model to the input. In deciding 3D face geometry we utilize two kinds of 2D information. One is the correspondence between stereo pair; the other is self-occluded contour. After some 3D points in the specific face are obtained from both image clues in different frames, RBF (Radius Basis Function) data interpolation is performed to decide its shape. Afterwards texture image is generated from the face geometry and video. Experiments show that satisfactory results can be reached with our system.

Recent deployments of digital cameras and fast PCs have led real-time interactive systems to adopt vision-based interfaces. However, 2D vision-based interfaces, due to the lack of 3D information, have inherent weakness in tracking 3D objects in real world. In this paper, we propose a vision-based 3D interface for interactive systems that allows an object tracking on the fly in 3D by exploiting multi-view images. Due to the characteristics of interactive systems, requiring real time processing, the main challenge is a simple and robust estimation of 3D information from the images captured by asynchronous digital cameras. The proposed vision-based 3D tracking consists of three steps: (i) dynamic stereo calibration (ii) simple object segmentation (iii) robust 3D movement tracking. To show its effectiveness of the proposed framework, we applied the proposed 3D interface to an interactive 3D Gumdo simulation. Due to the simplicity and robustness of the proposed framework, it can be applied to various real-time interactive applications.

This paper presents a video watermarking scheme for copyright notification and protection. Each video frame is clustered into high motion and low motion regions. Dual watermarks are designed by exploiting video motion, perceptual characteristics, and some other informative cues. They are embedded into certain macroblocks in these two regions respectively. The watermark in the low motion region is resilient to frame-level attacks such as frame dropping, frame averaging and frame reshuffling. The watermark in the high motion region is robust to statistical attacks. The detection does not need the original frame or any a priori index information of frames in video sequence. Good invisibility as well as strong robustness can be obtained. The experimental results validate the effectiveness of the system.

In this paper we present a new method to extract both superimposed and embedded graphical texts in a freeze-frame of news video. The algorithm is summarized in the following three steps. For the first step, we convert a color image into a gray-level image and apply contrast stretching to enhance the contrast of the input image. Then, a modified local adaptive thresholding is applied to the contrast-stretched image. The second step is divided into three processes: eliminating text-like components by applying erosion, dilation, and (OpenClose + CloseOpen)/2 morphological operations, maintaining text components using (OpenClose + CloseOpen)/2 operation with a new Geo-correction method, and subtracting two result images for eliminating false-positive components further. In the third filtering step, the characteristics of each component such as the ratio of the number of pixels in each candidate component to the number of its boundary pixels and the ratio of the minor to the major axis of each bounding box are used. Acceptable results have been obtained using the proposed method on 300 news images with a recognition rate of 93.6%. Also, our method indicates a good performance on all the various kinds of images by adjusting the size of the structuring element.

In this paper, we present a novel technique for classification of face images that employs Curvature-based Multi-scale Morphology (CMM). Multi-scale Morphology is an image analysis technique that employs mathematical morphology with structuring elements whose spatial dimensions are scaled successively. This scale-space representation of images has proven to be an efficient technique for indexing a large database of images. A majority of the existing techniques for multi-scale morphology employ regular and symmetrical structuring elements like cylinders, hemispheres or circular poweroids. The shape of these structuring elements is controlled only by the scaling parameter. In this paper, we propose the use of a structuring element whose shape is a function of both the scaling factor and the principal curvatures of the intensity surface of the face image. A high-dimensional feature vector is obtained by applying the CMM technique to the face images. The dimensionality of the feature vector is reduced by using the PCA technique, and the low- dimensional feature vectors are analyzed using an Enhanced FLD Model (EFM) for superior classification performance. Experimental results have shown that the proposed CMM technique outperforms existing approaches based on multi- scale morphology.

We develop a method for automatic segmentation of natural video sequences. The method is based on low-level spatial and temporal analyses. It features three designs to help facilitate good region segmentation while keeping the computational complexity at a reasonable level. Firstly, a preliminary seed-area identification and a final re-segmentation process are performed on each video frame to help region tracking. Secondly, a simple way to measure homogeneity of texture in a region is devised and the segmentation tries to locate object boundaries at where the texture shows significant changes. And thirdly, a reduced-complexity motion estimation technique is used, so that dense motion fields can be computed at a reasonable complexity. The overall method is organized into four tasks, namely, seed-area identification (for each frame), initial segmentation (only for the first frame in the sequence), motion-based segmentation (for all later frames), and region tracking and updating (also for all later frames). Some examples are provided to illustrate the performance of this method.

Background registration technique is useful to solve still object problem and uncovered background problem for video segmentation. However, it is hard to automatically decide the threshold of the frame difference for background registration to make it more feasible for real-time applications. Many previous works made efforts on automatic threshold decision for change detection. In this paper, we propose a new method of automatic threshold decision algorithm in a totally different viewpoint. Not only change detection but also the quantization effect in discrete domain is concerned. A Gaussianity test is first applied to find the standard deviation of Gaussian noise from the camera. Then, the quantization effect in discrete domain is taken into consideration to derive the relation between the standard deviation and the optimal threshold value. A couple MPEG-4 sequences and experimental sequences are tested as examples. Simulation results show that the calculated threshold values are suitable for background registration to give good segmentation results.

In order to support the object-based coding in the MPEG-4 video coding standard, we need to represent each video frame in terms of video object planes (VOPs). Segmentation is an essential operation to generate such VOPs. In this paper, we propose a semi-automatic segmentation algorithm using a concept of the virtual blue screen (VBS) that is defined as an image where its background is colored by a particular value, such as blue. After we extract moving objects in the first frame using VBS, we predict locations of the moving objects in the following frames via VBS. Experiment results show that the proposed method reduces the computational complexity significantly, while providing reasonably good segmentation results under an assumption that the video sequence has a still background.

We present a new block-based true-motion estimator, based on the 3D Recursive Search (3DRS) algorithm. It uses a three-frame match measure that solves the occlusion problem, and allows for an aggressive search strategy. Using a new level-of-detail measure it can distinguish good matches from bad, allowing it to perform a structured search on badly matching blocks only.

Video segmentation plays an important role for efficient storage and transmission in visual communication. In this paper, we introduce a novel video segmentation system using point tracking and contour formation techniques. Inspired by the results from the study of the human visual system, we intend to solve the video segmentation problem into three separate phases: user-assisted feature points selection, feature points' automatic tracking, and contour formation. This splitting relieves the computer of ill-posed automatic segmentation problems, and allows a higher level of flexibility of the method. First, the precise feature points can be found using a combination of user assistance and an eigenvalue-based adjustment. Second, the feature points in the remaining frames are obtained using motion estimation and point refinement. At last, contour formation is used to extract the object, and plus a point insertion process to provide the feature points for next frame's tracking.

A new technique for dynamic object segmentation in outdoor environments is presented. It is based on an adaptive thresholding pixel-oriented strategy applied on a combination of static and dynamic differences that segments the motion presented in each image. A detection and tracking scheme helps reducing the processing areas to those in which the moving objects are present. Therefore, computational cost is highly reduced without any loss in the accuracy of the algorithm. Besides, shadows and occlusions (due to the perspective that is used) are detected and eliminated keeping similar computational savings. Results are shown on road traffic monitoring sequences.

JPEG-LS is the latest pixel based lossless to near lossless still image coding standard introduced by the Joint Photographic Experts Group (JPEG). In this standard simple localized edge detection techniques are used in order to determine the predictive value of each pixel. These edge detection techniques only detect horizontal and vertical edges and the corresponding predictors have only been optimized for the accurate prediction of pixels in the locality of diagonal edges. In this paper we propose a low complexity, low cost technique that accurately detects diagonal edges and predicts the value of pixels to be encoded based on the gradients available within the standard predictive template of JPEG-LS. We provide experimental results to show that the proposed technique outperforms JPEG-LS in terms of predicted mean squared error, by a margin of up to 8.51%.

We introduce techniques that improve the error resilience of JPEG2000 against packet losses. The presented methods operate on JPEG2000 codestreams and consider the properties of different codestream segments. Experiments indicate that error resilience of JPEG2000 codestreams against packet losses can be improved significantly using these techniques. The performance of the proposed methods compares favorably with existing algorithms.

In this paper, we have a close look at the runtime performance of the intra-component transform employed in the reference implementations of the JPEG2000 image coding standard. Typically, wavelet lifting is used to obtain a wavelet decomposition of the source image in a computationally efficient way. However, so far no attention has been paid to the impact of the CPU's memory cache on the overall performance. We propose two simple techniques that dramatically reduce the number of cache misses and cut column filtering runtime by a factor of 10. Theoretical estimates as well as experimental results on a number of hardware platforms show the effectivity of our approach.

The new ISO/ITU-T standard for still image coding, JPEG2000, has been shown to provide superior coding efficiency to the previous standard, JPEG. Because of the superb performance of JPEG2000, it is reasonable to argue that Motion-JPEG2000, the corresponding moving picture coding standard of JPEG2000, has equally outstanding performance. However, there has not been a sufficient performance evaluation of Motion-JPEG2000. To this end, we have studied the potential of Motion-JPEG2000 for video processing. Our experiments show that Motion-JPEG2000 provides high compression performance, strong error resilience, and good perceptual image quality. Together with a rich set of features inherited from JPEG2000, Motion-JPEG2000 has advantages as a coding standard for video processing in many applications.

In this paper we present a compression scheme for the Digital Cinema application. Specifically, we developed a rate allocation algorithm for applying JPEG-2000 to coding Digital Cinema movies to achieve near lossless and smooth visual picture quality over the decoded movies at the required coding rate. First, a rate-distortion model is established based on the estimated source characteristics of the pictures, and a rate allocation algorithm is derived to determine a target rate for each picture based on the model so that 1) the overall distortion is minimized and 2) each picture has the same distortion at the required average bit rate. Then JPEG-2000 is employed to encode the high resolution motion picture according to the target rate allocation to achieve efficient compression performance and smooth picture quality over the picture sequence. The test results on Digital Cinema movie clips have shown that our scheme has achieved near lossless coding performance and very smooth picture quality both visually and in PSNR.

In this paper, several VLSI architectures and implementations of Shape-Adaptive Discrete Wavelet Transform (SA-DWT) with odd symmetric biorthogonal filters are presented. The hardware implementation issues of SA-DWT algorithm are first addressed, and one lifting scheme together with some appropriate shape information processing units is introduced for the 1-D SA-DWT architectures. These architectures can efficiently perform 1-D length-adaptive DWT on continuous line segments and solve the corresponding boundary extension and subsampling problems without any overhead. In addition to 1-D architectures, two 2-D SA-DWT architectures are also proposed. One is the direct method, which uses the 1-D SA-DWT architectures as basis to extend to 2-D SA-DWT by direct row-column approach. The other is the line-based method, which is an efficient and feasible architecture that uses internal line buffers rather than external frame memory. These two 2-D SA-DWT architectures are both VLSI implemented by using cell-based chip design flow. The Daubechies (9,7) filter is used for the implementation by direct method, and the Daubechies (9,3) filter is used for the line-based implementation. The proposed architectures can be applied both in arbitrarily shaped visual objects coding such as MPEG-4 still texture coding and rectangular region image coding such as JPEG2000 still image coding.

This paper proposes a classification of the parallelisms in general-purpose processor based systems in three main categories. One category is the intra-processor parallelism that includes multimedia instructions and superscalar and VLIW architectures. The former takes advantage of data parallelism. The latter benefit from instruction level parallelism. Another category is the inter-processor parallelism. We consider the parallelism between processors inside shared memory symmetric multiprocessor systems and in distributed memory clusters of workstations. Finally, in the last category, main features of the system level parallelism are studied including the input/output operations, the memory hierarchy and the exploitation of external processing. The potential gain is studied for each type of parallelism available in general-purpose processor based systems from a theoretical point of view as well as for existing image and video applications. The results in this paper showed that the exploitation of the different levels of parallelism available in PC workstations can lead to considerable gains in speed when optimizing a multimedia application. Finally the results of this work can be used to influence the design of new multimedia systems and media processors.

This paper presents a system-level codesign methodology of hardware/software for HDTV source decoder. We give out the hardware/software codesign method, which can meet the requirements of HDTV source decoding application. According to the algorithm development and analysis, the MPEG-2 MPHL decoder process is partitioned into software and hardware at the system level. We construct the system model that can represent the system architecture and application program of interest. A platform-based design environment is developed that can provide modularity, scalability, and flexibility in cosimulation and development for system on chip. Our main contribution is to introduce platform-based codesign tools where the software and hardware are validated and developed concurrently at the cycle accurate level.

Recently, we reported on a recursive algorithm enabling real-time object-based motion estimation (OME) for standard definition video on a digital signal processor (DSP). The algorithm approximates the motion of objects in the image with parametric motion models and creates a segmentation mask by assigning the best matching model to image parts on a block-by-block basis. A parameter estimation module determines the parameters of the motion models on a small fraction of the pictorial data called feature points. In this paper, we propose a new, computationally very efficient, feature point selection method that improves the convergence of the motion parameter estimation process.

This paper proposes a new MPEG-4 rate control algorithm for single or multiple object video sequences. The algorithm aims to achieve an accurate bit rate with the maximum picture quality while efficiently handling buffer fullness and scene change. In addition to estimating the bit budget of a frame based on its global coding complexity, the algorithm dynamically distributes the target bits for each object within a frame according to its coding complexity. Even though the VM8 solution and other algorithms adopt a simple proportional buffer controller, their control ability is rather ineffective. The proposed algorithm exploits a novel Proportional Integrated Differential (PID) buffer controller to effectively minimize the buffer overflow or underflow. The PID based controller reduces the deviation between the current buffer fullness and the target buffer fullness, mitigates the overshoots, and improves the transient response. The combined effect is a more smooth and effective buffer control. Furthermore, the algorithm defines a new and effective coding complexity of an object and dynamically optimizes several parameters. Overall, the proposed algorithm successfully achieves accurate target bit rate, provides promising coding quality, decreases buffer overflow/underflow and lowers the impact of a scene change.

This paper presents a novel adaptive H.263+ rate control algorithm for interactive video communication under the networks supporting bandwidth renegotiation, which can communicate with end-users to accommodate their time-varying bandwidth requests during data transmission. They are especially suitable for the transmission of non-stationary video traffics. The proposed rate control algorithm communicates with the network to renegotiate the required bandwidth for the underlying video and choose their control strategies according to the renegotiation results. The proposed algorithm treats both the spatial and temporal qualities at the same time to enhance human visual perceptual quality. Experimental results are provided to demonstrate that the proposed rate control algorithms can achieve superior performance with a low computational.

In low bit rate coding applications, high quantization levels might be needed to achieve a target bit rate. However, such high levels of quantization are likely to decrease picture quality. A possible solution is to reduce temporal resolution by dropping, for instance, selected frames thereby lessening the requirement for high quantization levels and thus improving video quality. Similarly, the spatial resolution of the encoded video could also be manipulated to achieve the target bit rate. Therefore, it might be possible to maximize picture quality by adjusting dynamically these three parameters while still meeting bit rate constraints. To do so effectively, the relationship between these parameters, alone or in combination, and subjective picture quality must be known. In this paper, we investigated the effect on subjective quality of: quantization alone (Experiment 1); a reduction in spatial resolution either alone or combined to moderate levels of quantization (Experiment 2); and a reduction of temporal resolution either alone or combined with moderate levels of quantization (Experiment 3). The results suggest that at very low bit rates reductions in spatial or temporal resolution combined with moderate levels of quantization might be an effective means of reducing bit rate without further loss in video quality.

In this research, we have developed a novel and unified rate-distortion (R-D) analysis framework for all standard DCT-based video coding systems, including MPEG-2, H.263, and MPEG-4. Within this framework, we consider the coding bit rate as a function of the percentage of zeros among the quantized DCT coefficients. From our extensive simulations over a wide range of video data, we observe that this rate function appears linear in all standard video coding. Based on Shannon's source coding theorem, we provide a heuristic justification for this linear rate model, which helps us better understand the unique behavior of the video encoder. Within the proposed R-D analysis framework, we have also developed an accurate distortion model. It is expected that this simple yet unified R-D modeling shall have significant impact on various video coding related research and development works, including rate control, bit allocation, joint source and channel coding, and so on. In this work, we have developed a unified bit allocation and rate control algorithm for MPEG-2, H.263 and MPEG-4 video coding. Compared to other rate control algorithms reported in the literature, the proposed algorithm controls the coding bit rate more accurately and robustly. In addition, the coded video quality is significantly improved.

The joint temporal-spatial rate control problem was examined in to achieve proper temporal-spatial quality tradeoff. Theoretical analysis and the performance benchmark of several optimal/suboptimal algorithms were presented. However, the high computational complexity of these solutions prevents them from being used in practical applications. To address this problem, good rate-distortion (R-D) models used to approximate empirical R-D curves are developed in this work. The complexity caused by actual encoding is reduced so that the process of finding the optimal/suboptimal solutions is significantly expedited. Different approximating R-D models are proposed for INTRA, INTER and skipped frames, respectively, by considering both coding and interpolation dependencies among successive frames. Simulation results show that the approximation error of the proposed R-D models are very small, which results in small degradation of coded video quality.

This paper presents a simple technique for estimating the space location from which a certain image has been taken. The basic assumption is that the scene portrayed in the image is planar. The method is based on the acquisition of a new set of images, closely resembling the given image. The location is recovered from the parameters describing the camera's pose during the acquisition of that among the new images showing the highest degree of correlation with the original image. An example of application of this technique is discussed in the paper.

In this paper, we present a complete framework for automatic analysis of soccer video by using domain specific information. In the proposed framework, following shot boundary detection, soccer shots are classified into 3 classes using the ratio of grass-colored pixels in a frame, and the size and number of soccer objects detected in a shot. These classes are long shots, in-field medium shots, and others, such as out-of-field of close-up shots. The long shots and in-field medium shots are further processed to analyze their semantic content. We observe that different low-level processing algorithms may be required to process different shot classes. For example, we introduce different tracking algorithms for the long shots and in- field medium shots. Furthermore, frame registration onto a reference field model is not usually applicable to in-field medium shots, because the field lines may not be visible. The proposed framework enables development of more effective low-level processing algorithms for high-level scene understanding, which perform nearly in real time. The results show the increased accuracy and efficiency of the proposed methods.

Many imaging systems consist of a combination of distinct electro-optic sensors that are constrained to view the same scene through a common aperture or from a common platform. Oftentimes, a spatial registration of one sensor's image is required to conform to the slightly disparate imaging geometry of a different sensor on the same platform. This is generally achieved through a judicious selection of image tie-points and geometric transformation model. This paper outlines a procedure to improve any such registration technique by leveraging the temporal motion within a pair of video sequences and requiring an additional constraint of minimizing the disparity in optical flow between registered video sequences.

An algorithm is proposed for the detection of scene changes in video sequences. The algorithm is based on the comparison of several features which represents the characteristics of frames. More specifically, the feature extraction is confined to several blocks that contains strong edges instead of the overall image as in the conventional algorithms, in order to concentrate more on the important colors and objects than the backgrounds. Several non-overlapping blocks of predefined size are first found, which contain strong edges in the frame. Then, three different kinds of features are extracted by using the pixels in the blocks. One is the color histogram of pixels in the blocks, the second one is the sum of absolute difference (SAD) between the blocks of current and previous frame as in video coding, and the third is the number of active blocks, which have the edge strength larger than a given threshold. The dissolve and wipe are detected by comparing the histogram of the blocks, the cut is detected by the SAD, and fade-in/outs are detected by the number of active blocks. The comparison of several test sequences shows that the color histogram from the strong edge blocks is a promising feature for detecting wipes and dissolves. Also, cut detection performance by the SAD of strong edge blocks is shown to be comparable to the conventional feature based algorithm. The fade-in/outs are also easily detected with high precision, by counting the number of active blocks.

The Hausdorff distance can be used to measure the similarity of two point sets. In matching the two point sets, one of them is translated, rotated and scaled in order to obtain an optimal matching, which is a computationally intensive process. In this paper, a robust line-feature-based approach for model-based recognition is presented, which can achieve good matching, even in a noisy environment or with the existence of occlusion. We propose three different ways for forming the line segments for a point set. Two of the methods are based on a fixed reference point, which are the centroid and the center of the minimal disk of the point set. The third method is based on the longest possible line segments formed within the point set. The line features extracted based on the respective line segments are insensitive to noise and can be used to determine the rotation and scale of the image point set accurately and reliably, and more importantly, 2D-2D matching algorithm can be adopted. The first 2D-matching is to compute the relative scale and orientation between the two point sets. Having rotated and scaled the image point set, the M-Estimation Hausdorff distance is applied for the second 2D-matching to measure the relative translation between the query point set and the model point set. This 2D-2D matching can greatly reduce the required memory and computation when compared to a 4D-matching. Both the performance and the sensitivity to noise of our algorithms are evaluated using simulated data. Experiments show that our 2D-2D algorithms can yield a high level of performance when determining the scale, the orientation and the similarity of two point sets.

Fine-Granularity-Scalability (FGS) has recently been adopted as the MPEG-4 standard for the efficient and flexible distribution of multimedia over the Internet. The FGS framework allows spatio-temporal-SNR tradeoffs in real-time (not limited to encoding time) at transmission or (post-) processing time, depending upon the available bit-rate, sequence characteristics, priority, user preferences or receiver resources. This paper discusses the versatility of FGS and how it can be used to great advantage in a variety of application domains beyond Internet video streaming, such as robust wireless video transmission and flexible scalable storage. In particular, the focus is on investigating the use of FGS for enabling the Universal Media Access (UMA) paradigm for wireless networks. To cope with detrimental channel conditions, we use several joint source-channel coding schemes for wireless transmission, termed Robust FGS (RFGS). RFGS exploits the FGS property of compressing video data in a set of multi-priority streams, based on the channel characteristics and terminal capabilities, by adapting the channel coding. Our results show that RFGS can provide graceful adaptability to varying network conditions while also offering additional benefits such as flexible Quality-of Service (QoS) management and channel adaptation and allocation. Moreover, the use of FGS for scalable storage of video content in gateways and in-home centralized storage units is also investigated. The results show that the proposed scalable method allows storing ten times or more video data on the same storage device depending on the desired guaranteed minimum quality level.

This paper proposes an optimal rate allocation scheme for Fine-Granular Scalability (FGS) coded bitstreams that can achieve constant quality reconstruction of frames under a dynamic rate budget constraint. In doing so, we also aim to minimize the overall distortion at the same time. To achieve this, we propose a novel R-D labeling scheme to characterize the R-D relationship of the source coding process. Specifically, sets of R-D points are extracted during the encoding process and linear interpolation is used to estimate the actual R-D curve of the enhancement layer signal. The extracted R-D information is then used by an enhancement layer transcoder to determine the bits that should be allocated per frame. A sliding window based rate allocation method is proposed to realize constant quality among frames. This scheme is first considered for a single FGS coded source, then extended to operate on multiple sources. With the proposed scheme, the rate allocation can be performed in a single pass, hence the complexity is quite low. Experimental results confirm the effectiveness of the proposed scheme under static and dynamic bandwidth conditions.

In this paper we study two rate control strategies for fully fine-grained scalable (FFGS) video coders. Usually, in scalable coders the bitstream is divided into a base layer, which is decoded by all the decoders, and one or more enhancement layers which can improve the quality provided by the base layer. In Internet video streaming it is important that the bitstream be scalable in rate, which allows a server to adapt the bitstream to changes in the available bandwidth in the network. FFGS coders allow the maximum degree of rate scalability by using scalable encoding in both the base and enhancement layers. In this paper, we propose a rate control algorithm which is based on the rate distortion characteristics of the encoded bitstream and prevents large jumps in quality. We show that due to the embedding property of FFGS encoders, we can properly select the number of bits of every layer and frame by taking into account the quality of the video sequence. In addition, by allowing a controlled amount of prediction drift, we can set the rate control of the base layer much higher and gain in some cases several dB of PSNR performance at the highest rate. Experimental comparisons are made using SAMCoW, a FFGS video coder based on the wavelet transform and motion compensated prediction, and the MPEG-4/FGS coder using the TM-5 rate control algorithm.

To compensate for the unpredictability and variability in bandwidth between sender and receiver(s) over the Internet, a new scalable coding tool has recently been introduced in MPEG-4: Fine-Granularity-Scalability (FGS). The FGS framework is very flexible and can adapt in real-time to the Internet bandwidth variations by supporting both SNR and temporal scalability through a single pre-encoded fine-granular stream. However, while FGS is very flexible, it has a lower coding efficiency than non-scalable MPEG-4 coding. To reduce this visual quality penalty at low and medium bit-rates, a subjective quality improvement tool has been introduced in MPEG-4, allowing the prioritized transmission of low frequency DCT coefficients that contribute more to the visual quality. This tool is termed Frequency Weighting (FW). In this paper, we propose a novel scene-characteristic-dependent adaptive FW method that considerably improves the visual quality of FGS at low bit-rates. First, a thorough analysis of the FGS enhancement layer is performed at various bit-rates. Based on this analysis, we concluded that for improved visual quality, different FW methods should be used depending on the video sequence characteristics. Subsequently, we developed an automatic FW matrix adaptation mechanism that measures the brightness, motion and texture activity of each sequence and selects an appropriate FW method for that sequence from a set of a priori determined FW classes. This adaptive FW method has been subjectively evaluated and showed a clear improvement in visual quality compared with non-frequency weighted or non-adaptive frequency weighted sequences.

A flexible and effective macroblock-based framework for hybrid spatial and fine-grain SNR scalable video coding is proposed in this paper. In the proposed framework, the base layer is of low resolution and is generally encoded at low bit rates with traditional prediction based coding schemes. Two enhancement layers, i.e., the low-resolution enhancement layer and the high-resolution enhancement layer, are generated to improve the video quality of the low-resolution base layer and evolve smoothly from low resolution to high resolution video with increasingly better quality, respectively. Since bit plane coding and drifting control techniques are applied to the two enhancement layers, each enhancement bitstream is fine-grain scalable and can be arbitrarily truncated to fit in the available channel bandwidth. In order to improve the coding efficiency and reduce the drifting errors at the high-resolution enhancement layer, five macroblock coding modes with different forms of motion compensation and reconstruction, are proposed in this paper. Furthermore, a mode decision algorithm is developed to select the appropriate coding mode for each macroblock at the high-resolution enhancement layer. Compared with the traditional spatial scalable coding scheme, the proposed framework not only provides the spatial scalability but also provides the fine granularity quality scalability at the same resolution.

Arriving at improved picture quality often requires combining various video enhancement algorithms. These algorithms may be interdependent and global or local in nature. It is, therefore, far from trivial to tune the parameters that control an individual algorithm such that the optimal output quality is achieved. Moreover, due to the nature of various algorithms and their interdependencies consistency in both space and time is often sub-optimal. In this paper we introduce an enhancement model that guarantees spatio-temporal consistency while optimizing the parameters that control video enhancement.

In old movie film, most of sharp brightness transitions have been blurred, and film materials are often corrupted by several distortions such as blotches. To restore original edge sharpness without augmenting visibility of such distortions, first we characterize such distortion areas, repair them and then sharpen only blurred edges selectively. This paper presents a locally-adaptive sharpening method based on the coupled nonlinear reaction-diffusion time-evolution equipped with a second-order nonlinear smoothing term, a reaction term and an overshooting term. The overshooting term adds an overshoot only to the blurred edges. The coupled nonlinear reaction-diffusion method utilizes information about local image contents and the characterized film distortions, to control the degree of the second-order smoothing and the magnitude of the overshoot to be added. Our method sharpens blurred edges selectively much better than our previously presented adaptive peaking method. Our method, of course, is applicable to sharpness enhancement of general blurred images.

This paper outlines a generalized image reconstruction approach to improve the resolution of an Electro-Optic (EO) imaging sensor using multiple frames of an image sequence. This method only assumes the constituent video has some ambient motion between the sensor and stationary background, and the optical image is physically captured by a staring focal plane array.

In this paper, we present the implementation of the robust detail preserving filters with complex noise suppression for image processing applications. The designed filter is the consequential connection of two filters. The first filter uses the value of central pixel of the filtering window to provide the preservation of fine details and the redescending M-estimators combined with the median estimator to provide impulsive noise rejection. The second filter uses the output of the first filter as the pre-estimator for an adaptive calculation in the redescending M-estimator. We investigated various types of influence functions in the M-estimator those are similar to the ones used in the Sigma filter to provide multiplicative noise suppression. The optimal values of the parameters of designed filters in presence of different noise mixture are determined. Different simulation data are presented in the paper and shown the statistical efficiency of the filters.

Imagery gathered by a Focal Plane Array (FPA) based sensor often suffers from the intrinsic non-uniform response of the individual detectors of the FPA. A digital Non-Uniformity Correction (NUC) can compensate for this distortion by implementing a functional transformation to the numerical output of each digitized FPA pixel. Such a NUC is often measured by exposing the sensor to one or more sources of uniform flux, and computed so that the post-NUC image of such uniform scenery has minimal spatial variation. Alternative NUC implementations adopt a scene-adaptive approach , using only the data in the gathered video sequence for which one wants to NUC. Several implementations, such as temporal high-pass filtering, neural-network, steepest-descent, or adaptive LMS, fundamentally depend on scene-predicted image necessary compute the appropriate functional correction. Such predicted images are invariably a spatial transformation of a single frame of video; it is because of the limited accuracy of such a single-image prediction that mandates algorithm compromises between slow convergence and pathological collapses, such as image scene burn-in or image washout. Previously reported research in image resolution enhancement mandates the construction of a Temporal Accumulation of Registered Image Data (TARID) composite image as a pre-processing step. Such TARID composite images have significantly improved accuracy and robustness over any single-frame predicted image applied to scene adaptive NUC algorithms, resulting in markedly improved performance in both convergence and stability.

In this paper we focus on the detection of moire patterns, which may appear during film-to-video transfer using Telecine devices. A model of moire distortion is created and studied. A new suppression algorithm using the spectral model is presented and applied for real video sequences. The non-linear filter is based on analysis of the magnitude of the Fourier spectrum of the image. As an alternative, an algorithm using parametric identification of the moire patterns is investigated. It is found that spectral domain manipulation gives the best results.

In this paper, we propose a spatially adaptive wavelet thresholding method using a context model that has been inspired by our prior work on image coding. The proposed context model relies on an estimation of the weighted variance in a local window of scale and space. Appropriately chosen weights are used to model the predominant correlations for a reliable statistical estimation. By iterating the context-based thresholding operation, a more accurate reconstruction can be achieved. Experimental results show that our proposed method yields significantly improved visual quality as well as lower mean squared error compared to the best recently published results in the denoising literature.

In this paper, we propose a watermarking method that does not requires an original image to extract embedded data. In our proposal, watermark information data are embedded by using the differences between the neighboring wavelet coefficients of the lowest band in the wavelet domain. Though such a concept have been already proposed by H.S. Kim et al., the original image is needed for extracting embedded data in their method. We developed a new scheme which does not requires the original image to extract embedded data, by modifying the embedding algorithm. Simulation results show that our proposed scheme indicates good picture quality in the watermarked image and robustness to some types of image processing attacks including JPEG compression.

Current wavelet-based image coders obtain high performance thanks to the identification and the exploitation of the statistical properties of natural images in the transformed domain. Zerotree-based algorithms, as Embedded Zerotree Wavelets (EZW) and Set Partitioning In Hierarchical Trees (SPIHT), offer high Rate-Distortion (RD) coding performance and low computational complexity by exploiting statistical dependencies among insignificant coefficients on hierarchical subband structures. Another possible approach tries to predict the clusters of significant coefficients by means of some form of morphological dilation. An example of a morphology-based coder is the Significance-Linked Connected Component Analysis (SLCCA) that has shown performance which are comparable to the zerotree-based coders but is not embedded. A new embedded bit-plane coder is proposed here based on morphological dilation of significant coefficients and context based arithmetic coding. The algorithm is able to exploit both intra-band and inter-band statistical dependencies among wavelet significant coefficients. Moreover, the same approach is used both for two and three-dimensional wavelet-based image compression. Finally we the algorithms are tested on some 2D images and on a medical volume, by comparing the RD results to those obtained with the state-of-the-art wavelet-based coders.

This paper discusses a model of combining multiple error factors based on multicriteria optimization. Frequently this involves fitting calculated errors to subjective data, and using the resulting scalar weighting as the single-valued error measure to optimize. Instead of finding a way of optimizing a fixed combination of the different factors, we consider the multiple error measures as a vector-valued function, thus producing data for an optimization problem with multiple objective functions. Applying multiple criteria optimization techniques to the resulting problem can yield a range of potentially optimal weightings for each factor. By adding a degree of freedom by way of the utility function, which describes how strongly each objective function contributes to the optimal weighting, we can remove the dependence on fixed scalar combinations resulting from fixed viewing conditions. Using this model, an end user or compression method designer can adaptively set their own preferred weighting. This paper discusses the relevant multiple criteria optimization theory, and describes our experiments with applying these techniques to the PQS model of Miyahara, Kotani and Algazi, applied to greyscale still images. We also describe how such methods could be generalized to models in which each error measure is described using entire images rather than single factors. These include Osberger's Region-of-Interest map, and Daly's Probability Detection Map.

We present an MPEG--7 compliant description of generic video sequences aiming at their scalable transmission and reconstruction. The proposed method allows efficient and flexible video coding while keeping the advantages of textual descriptions in database applications. Visual objects are described in terms of their shape, color, texture and motion; these features can be extracted automatically and are sufficient in a wide range of applications. To permit partial sequence reconstruction, at least one simple qualitative as well as a quantitative descriptor is provided for each feature. In addition, we propose a structure for the organization of the descriptors into objects and scenes and some possible applications for our method. Experimental results obtained with news and video surveillance sequences validate our method and highlight its main features.

Image retrieval and image compression are both very active fields of research. Unfortunately, in the past they were pursued independently leading to image indexing methods being both efficient and effective but restricted to uncompressed images. In this paper we introduce an image retrieval technique that operates in the compressed domain of vector quantize images. Vector quantization (VQ) achieves compression by representing image blocks as indices into a codebook of prototype blocks. By realizing that, if images are coded with their own VQ codebook then much of the image information is contained in the codebook itself, we propose the comparison of the codebooks, based on a Modified Hausdorff distance, as a novel method for compressed domain image retrieval. Experiments, based on an image database comprising many colorful pictures show this technique to give excellent results, outperforming classical color indexing techniques.

An embedded space-time coding method is proposed for wireless broadcast applications. In the proposed system, a transmitter sends out multi-layer source signals by encoding different layers with different space-time codes. Then, the receiver can retrieve different amount of information depending on the number of antennas. The receiver with only one antenna can decode only the base layer information with a low complexity, while the receiver with more antennas can retrieve more layers of information. We derive an analytic bound on the error probability, and show both analytic and experimental results in this work.

The application of Human perceptual models in image and video coding is motivated by the fact that non-perceptual distortion metrics (mean square error) do not correlate well with the perceived quality at lower bit-rates despite their acceptable signal to noise ratio. In this paper, we propose a novel approach for indexing the visual content of images based on human perceptual thresholds employed for encoding. In other words, the thresholds that are employed in perceptual coding also serve as an index. These thresholds depend on the overall luminance, frequency/orientation, and the variety of patterns in an image and can serve as indexing features. These features therefore have the potential to retrieve perceptually similar images in response to a query image. Detailed simulations have been carried out using the proposed indexing concept in the DCT compressed domain. Here, the indices have been computed using the DCTune coding technique, which has been shown to provide a superior visual quality in encoding images. Simulation results demonstrate that superior retrieval performance can be achieved for specific classes of images while comparable performance is obtained for other image classes.

A new image compression approach is proposed in which variable block size technique is adopted, using quadtree decomposition, for coding images at low bit rates. In the proposed approach, low-activity regions, which usually occupy large areas in an image are coded with a larger block size and the block mean is used to represent each pixel in the block. A novel classification scheme, which operates based on the distribution of the block residuals is employed to determine whether the processed block is a low-detail or a high-detail block. To preserve edge integrity, a new edge-based coding technique is used to code high-activity regions. In this method, the orientation of edge pattern within a high-activity block will be computed as an aid to the classification. A novel edge-oriented classifier, operating based on the histogram analysis of the pixels' orientations, is also proposed to for edge classification. Each edge block is represented by a set of parameters associated with the pattern appearing inside the block. The use of these parameters at the receiver reduces the cost of reconstruction significantly and exploits the efficiency of the proposed technique. Experiments have been conducted to compare with variance-based quadtree technique, vector quantization-based variable size algorithms, as well as the standard JPEG. Results show higher PSNR at competitive reconstruction quality.

A novel scheduling mechanism and a new bandwidth allocation scheme are proposed in this work to support multimedia traffic over DOCSIS (Data Over Cable System Interface Specification)-compliant cable networks. The primary goal of our research is to improve the transmission of real-time variable bit rate (VBR) traffic in terms of throughput and delay under the current DOCSIS specifications. To support integrated services, we also consider the transmission of constant bit rate (CBR) traffic and non-real-time traffic in simulation. To demonstrate the performance, we compare the result of the proposed scheme with that of a simple multiple priority scheme. It is shown via simulation that the proposed method provides a significant amount of improvement over existing DOCSIS QoS scheduling services.

With the increase of the complex of VLSI such as the SoC (System on Chip) of MPEG-2 Video decoder with HDTV scalability especially, simulation and verification of the full design, even as high as the behavior level in HDL, often proves to be very slow, costly and it is difficult to perform full verification until late in the design process. Therefore, they become bottleneck of the procedure of HDTV video decoder design, and influence it's time-to-market mostly. In this paper, the architecture of Hardware/Software Interface of HDTV video decoder is studied, and a Hardware-Software Mixed Simulation (HSMS) platform is proposed to check and correct error in the early design stage, based on the algorithm of MPEG-2 video decoding. The application of HSMS to target system could be achieved by employing several introduced approaches. Those approaches speed up the simulation and verification task without decreasing performance.

We proposed an algorithm for the tracking of facial feature points based on the block matching algorithm (BMA) with a new shape of window considering the feature point characteristics and scale/angle changes of the face. The window used in the proposed algorithm is the set of pixels in the 8 radial lines of 0 degree(s),45 degree(s),... from the feature point, i.e. the window has the shape of cross plus 45 degree(s) rotated cross. This shape of window is shown to be more efficient than the conventional rectangular window in tracking the facial feature points, because the points and their neighbor are not usually the objects of rigid body. But since the feature points are usually on the edges of luminance or color changes, at least one of the radial line crosses the edge and it gives distinct measure for tracking the point. Also the radial line window requires less computational complexity than the rectangular window and more readily adjusted with respect to scale and angle changes. For the estimation of scale changes, the facial region is segmented at each frame using the normalized color, and the number of pixels in the facial region are compared.

This paper presents our approach to using a stereo camera to obtain 3-D image data to be used to improve existing lip boundary detection techniques. We show that depth information as provided by our approach can be used to significantly improve boundary detection systems. Our system detects the face and mouth area in the image by using color, geometric location, and additional depth information for the face. Initially, color and depth information can be used to localize the face. Then we can determine the lip region from the intensity information and the detected eye locations. The system has successfully been used to extract approximate lip regions using RGB color information of the mouth area. Merely using color information is not robust because the quality of the results may vary depending on light conditions, background, and the human race. To overcome this problem, we used a stereo camera to obtain 3-D facial images. 3-D data constructed from the depth information along with color information can provide more accurate lip boundary detection results as compared to color only based techniques.

The paper provides a novel algorithm for face rendering applications. Ensuring algorithms of low complexity to render virtual humans in VLBR networks is at the heart of our new facial rendering system. The system differs from others such as parametric animation models and interpolation solutions. The novelties include a dual segment growing algorithm and a heat diffusion rendering method. The extracting process takes into account information both in gradient domain and topographic feature. And segments are used to carry this information, which greatly reduces the transmitted packet size. Face rendering is based on this segment and is carried out like a heat diffusive process. Experimental results, as reported in following, prove that this proposed system. Furthermore this scheme can be extended to deal with more general video or image analysis and synthesis systems.

Adaptive Block Transforms (ABT) were recently proposed for inter coding in the emerging video coding standard H.26L. The basic idea of ABT is to connect the block size used for transform coding of the prediction error to the block size used for motion compensation. The maximum possible signal length is exploited for transform coding. In this paper, the concept variable block size transforms (VBST) is proposed for intra coding. The selection of the transform block sizes is based on a rate/distortion criterion. Several prediction modes including directional prediction are employed. In terms of PSNR the simulation results reveal better or at least comparable performance for the VBST coding scheme compared to the TML. A gain of about 0.5 dB can be observed for intra as well as for inter coding. The VBST scheme tends to preserve more detail information of the signal. Therefore, the subjective quality of the decoded video is generally increased.

The performance of superresolution video enhancement relies heavily on the robustness and accuracy of motion estimation techniques. In this paper, we propose a novel and efficient block matching motion estimation algorithm suitable for estimating general motion existing in low resolution video frames. We exploit the spatial correlations between motion vectors and apply a coarse-to-fine multi-stage scheme to get the dense motion fields. We incorporate our motion estimation technique into the Projection Onto Convex Sets (POCS) superresolution framework. Experimental results show that the resulting high resolution images yield visual sharper enhanced images with significant PSNR improvement.

The prediction error can be decreased by incorporating with high accuracy estimation and compensation, and the performance of compressed video can be improved. Two fast algorithms of fractional-pixel accuracy video motion estimation are proposed in this paper. After half-pixel accuracy motion estimation, a high accuracy motion estimation can be calculated with the intermediate results. The algorithm is based on the intermediate results in half pixel accuracy motion estimation, and traditional fast block matching algorithms can also be implemented in half pixel accuracy. The arbitrary fractional-pixel accuracy motion estimation can be achieved directly, at the cost of small computational overhead. The approach described in this paper eliminates the systematic limitations of conventional block matching. Experimental results using typical video sequences show that the proposed algorithm can achieve better PSNR and lower bit rates in higher fractional pixel accuracy than in the half pixel motion estimation and compensation. These fast motion estimation algorithms provide methods for studying higher pixel accuracy motion estimation in video compression coding. The proper fractional pixel accuracy motion estimation and compensation by truncating the precise results will be the best way to achieve more efficiently video compensation and higher image quality.

Motion Estimation (ME) for video coding has taken an entirely different direction with the emergence of zonal algorithms. These algorithms made it possible to significantly reduce, if not almost eliminate, the tremendous computational overhead of motion estimation that was incurred in a video encoding system, while having little if any loss in quality. In this paper we further improve on these algorithms with the introduction of the Enhanced Predictive Zonal Search Algorithm (EPZS). By considering an additional set of predictors, improving the thresholding process, and simplifying the search pattern employed by these algorithms, we not only manage in achieving better output quality, but also reduce complexity of the motion estimation process even further. Our algorithm was compared with the algorithms accepted in the Optimization Model 1.0 of MPEG-4, and our simulations prove its outright superiority versus the existing algorithms. Furthermore we present a 3-Dimensional implementation of EPZS, which can be easily applied in the case of multiple frame motion estimation (ITU H.263++ and H.26L).

How to develop a multi-resolution representation of video signals for both efficient and scalable coding? What are fundamental advantages and limitations of a resolution scalable video coder? How to make use of the scalability features of a video codec to compromise other requirements (e.g. delay)? These are the central issues we address in this paper. We first demonstrate the importance of resolving the phase uncertainty on the efficiency of motion-compensated prediction (MCP) in the wavelet domain. Improved understanding of the relationship between phase associated with any wavelet transform (WT) and motion accuracy of MCP motivates us to develop a novel multi-resolution representation for video signals. The salient feature of our new representation is that MCP can be performed both effectively and independently at different resolutions. We apply previous theoretical results on fractional-pel MCP to analyze the sacrifice of coding efficiency due to the resolution scalability constraint. We also investigate the issue of delay in video coding and propose a framework of trading delay with spatial resolution. A video decoder can display a low-resolution frame with low delay first and then gracefully enhance the frame resolution as the delay increases. The low-delay resolution-scalable MCP-WT coder built upon our new wavelet-based multi-resolution representation of video signals has achieved significant Rate-Distortion improvements over previously-reported scalable coders in the literature and even outperforms non-scalable MPEG-2 coder by 1-2dB at the bit rate of 2-9Mbps.

This paper proposes a novel low-complexity region-based video-coding algorithm that automatically identifies moving foreground objects, compresses them with higher quality than the background and efficiently encodes the video in an H.263+ compliant bitstream. Global motion estimation is first performed using the MSE algorithm. The original sequence is then segmented into foreground and background regions by using global and local motion information predicted from the previous frame. This enables the separation of moving objects with respect to a static background, even in the presence of camera motion. A modified TMN8 rate control algorithm is proposed to assign more bits to the foreground region, and the segmented video is then encoded into an H.263+ compliant bitstream. As block-matching motion estimation is used to obtain the local motion field and foreground/background identification is also block-based, the proposed algorithm has lower complexity than previously proposed pixel-based algorithms. Hence it is can be easily implemented in software or ASIC-based real-time applications. It is also particularly useful for mobile applications where bandwidth is highly constrained and low power requirements restrict processing complexity.

Wavelet transform is a popular tool for image and video coding. It has several advantages in multiresolution analysis and subband decomposition of images. Motion estimation and motion compensation methods are widely used to reduce temporal redundancy in video sequences. Although there have been several attempts to compensate for the motion in the wavelet domain, their performances are limited due to the shift-variant problem. In this paper, we propose a new motion compensation method in the wavelet domain to overcome the shift-variant problem. Experimental results show that the proposed method outperforms the previous motion compensation methods in the wavelet domain as well as the full search algorithm in the spatial domain.

This paper proposes an algorithm for extracting the boundary of an object. In order to take full advantage of global shape, our approach uses global shape parameters derived from Point Distribution Model (PDM). Unlike PDM, the proposed method models global shape using curvature as well as edge. The objective function for applying the shape model is formulated using Bayesian rule. This method can extract a boundary of an object by evaluating the solution maximizing the objective function iteratively. Experimental results show that the proposed method requires less computational cost than the PDM and it is robust to noise, pose variation, and some occlusion.

Video transcoding is one of the key technologies in implementing dynamic bit rate adaptation of a pre-encoded video stream to the available bandwidth over various networks. Though there have been many different transcoding schemes, since they do not concern the contents of video, the quality of transmitted video becomes proportionally worse as the rate of reduction increases. In this paper, we propose a content-based MPEG transcoding method using a discontinuity feature in the Discrete Cosine Transform (DCT) domain. The discontinuity heights in DCT blocks are evaluated by a method of model-based DCT alignment, and a DCT block is transcoded differently depending on the height of a block. In the experiment, we show the result that the video quality of content-based transcoding is better than that of a constant cut-off method and the processing time of the adaptive method is much faster compared with the pixel domain methods in the same bandwidth.

A novel concept for SNR scalability with motion compensation in the enhancement layer is introduced. The quantization of the prediction error at different quantization step sizes is performed in the same loop. This allows the application of bit plane coding if the configuration of the quantizers is appropriately chosen. Since a layered prediction is employed at the encoder a drift can occur at a base layer decoder. The concept is, therefore, extended by a drift limitation operation. In this context, two approaches are investigated. One is based on a modification of the prediction error. In the second approach the drift is controlled by dynamic clipping of the enhancement prediction. The proposed SNR scalability concept is applied to the lowpass band of a wavelet-based video coding scheme. The performance is compared with a conventional approach to SNR scalability with two and three quantization layers, respectively.

Former research on perceptual image coding was mainly developed in the traditional sequential coding framework, where the codestream is neither rate nor resolution scalable. In this paper, our earlier embedded subband/wavelet image coding algorithm EZBC is further developed for highly scalable image coding applications. Special attention is given to perceptual image coding under varying viewing/display conditions --- a common situation in typical scalable coding application environments. Unlike the conventional perceptual image coding approach, all the perceptually coded images (individually targeted at particular viewing conditions) are decoded from a single compressed bitstream file. The experimental results show the bitrate savings by the proposed algorithm are significant, particularly for coding of high-definition (HD) images.

As media processing gradually migrates from hardware to software programmable platforms, the number of media processing functions added on the media processor grow even faster than the ever-increasing media processor power can support. Computational complexity scalable algorithms become powerful vehicles for implementing many time-critical yet complexity-constrained applications, such as MPEG2 video decoding. In this paper, we present an adaptive resource-constrained complexity scalable MPEG2 video decoding scheme that makes a good trade-off between decoding complexity and output quality. Based on the available computational resources and the energy level of B-frame residuals, the scalable decoding algorithm selectively decodes B-residual blocks to significantly reduce system complexity. Furthermore, we describe an iterative procedure designed to dynamically adjust the complexity levels in order to achieve the best possible output quality under a given resource constraint. Experimental results show that up to 20% of total computational complexity reduction can be obtained with satisfactory output visual quality.

In this paper, we present a novel design of a wavelet-based video coding algorithm within a conventional hybrid framework of temporal motion-compensated prediction and transform coding. Our proposed algorithm involves the incorporation of multi-frame motion compensation as an effective means of improving the quality of the temporal prediction. In addition, we follow the rate-distortion optimizing strategy of using a Lagrangian cost function to discriminate between different decisions in the video encoding process. Finally, we demonstrate that context-based adaptive arithmetic coding is a key element for fast adaptation and high coding efficiency. The combination of overlapped block motion compensation and frame-based transform coding enables blocking-artifact free and hence subjectively more pleasing video. In comparison with a highly optimized MPEG-4 Advanced Simple Profile coder, our proposed scheme provides significant performance gains in objective quality of 2.0-3.5 dB PSNR.

In this paper, we present a novel wavelet image coding method based on the concept of directional dilation. We investigate the relationship among coefficients within and cross subbands and find that the concept of directional dilation is very helpful in expressing positions of significant coefficients. Furthermore, we try to combine two efficient ways: in-band directional dilation and quad-tree prediction together. According to this, we designed a new wavelet encoder, and experiment results show that the proposed method can outperform many famous algorithms in literature yet still possesses the embedded property.

Uniform scalar quantizers are widely used in image coding. They are known to be optimum entropy constrained scalar quantizers within the high resolution assumption. In this paper, we focus on the design of nearly uniform scalar quantizers for high performance coding of wavelet coefficients whatever the bitrate is. Some codecs use uniform scalar quantizers with a zero quantization bin size (deadzone) equal to two times the other quantization bin sizes (for example JPEG2000). We address the problem of deadzone size optimization using distortion rate considerations. The advantages of the proposed method are that the quantizer design is adapted to both the source statistics and the compression ratio. Our method is based on statistical information of the wavelet coefficients distribution. It provides experimental gains up to 0.19 dB.

In this paper, we propose a simple but efficient wavelet-based embedded image coder that employs a new inter-band magnitude relationship in the wavelet coefficients and block trees. The proposed scheme includes multi-level dyadic wavelet decomposition, raster scanning within each subband, formation of block trees, partitioning of block trees and adaptive arithmetic entropy coding. Although the proposed scheme is simple, it produces a bitstream with a rich set of features, including SNR scalability and the embedded nature. Experimental results demonstrate that the new scheme is quite competitive to and outperforms other good image coders in the literature.

Traditional algorithms of 3D surface reconstruction include iteration and linear interpolation and surface fitting technique etc. These methods are mainly suitable for regular shapes and smooth surfaces. For scenes with affluent texture, these methods can't preserve their statistical characteristics. A new method combining wavelet decomposition with fractal interpolation for 3D surface reconstruction is proposed in this paper. For 3D surface reconstruction purposes, the wavelet decomposition extracts strong image self-similar characteristics that can be utilized to three dimension surface reconstruction. With a fractal interpolation, we can generate the practical simulation of three dimension visual surfaces. Our results indicate a good approximation of realistic looking mountainous terrain.

The transmission of large data streams over error-prone channels as e.g. in multimedia applications is inherently linked to long transmission delays if automatic repeat request schemes are used. As this article will show, the delay can be reasonably traded against residual bit errors if a short transmission time has highest priority. The dependency of the delay on two important factors, packet length and channel bit error rate, is determined to be non-linear and strictly monotonously growing. Furthermore, transmission behavior and properties of a plain binary symmetric channel and one with additional repeat request technique are simulated and compared to previous research. The simulations finally lead to a redefinition of the formula for the estimation of the residual bit error rate of a non-transparent channel.

Video transmission on the Internet often requires a transcoding in order for the bitrate to meet the available bandwidth. Since the bitrate of videos is determined by an amount of bits per frame(BPF) and framerate(FPS), one of or either of these should be reduced in transcoding. In addition, because there is a tradeoff between the BPF and FPS under a bitrate constraint, the BPF and FPS should be optimally determined to get the best subjective video quality. This paper proposes a framerate optimization method, which estimates an optimal framerate directly from a target video and a specified bitrate. Actually, an optimal framerate is derived by using a characteristic surface of Mean Opinion Score(MOS), which is estimated by suitable features of a target video. Comparison of the optimal framerate obtained by the actual measurement with one by the proposed technique has proved that the accuracy of estimation by the proposed technique is satisfactory for several open data sequences.

In this paper we propose a novel approach for the bit allocation performed in an ADSL modulator. This new method is based on the observation that the transmission speed using ADSL strongly depends on the distance between the central office and the subscriber's side and does not permit real-time transmission for high bitrate video on long distances. The algorithm we develop takes into account the characteristics of a video sequence and distributes the channel error according to visual sensitivity. This method involves variable transmission Bit Error Rate.