Analysis of Mismatched First Order A Priori Information in Iterative Source-Channel Decoding

Due to complexity and delay constraints a usually significant amount of residual redundancy remains in the source samples after source coding. This residual redundancy can be exploited by iterative source-channel decoding for error concealment and quality improvements. One key design issue in joint source-channel (de-)coding is the index assignment. Besides conventional index assignments optimized index assignments have been developed, e.g., considering zeroth or first order a priori information of the source samples. However, in real-world scenarios it is unlikely that the amount of residual redundancy is constant over time and thus it may occur that the just deployed index assignment is suboptimal at times when the residual redundancy differs too much from the amount that it is optimized for. In this paper the performance of optimized index assignments is examined that consider first order a priori knowledge under such suboptimal conditions.


Introduction
Since the discovery of Turbo codes [1], which allow for channel coding close to the Shannon limit with moderate complexity, the Turbo principle of exchanging extrinsic information has been extended to various components of the receiver chain.Iterative source-channel decoding (ISCD) [2,3] is such an extension.Instead of a concatenation of two or more channel decoders, a channel decoder and a soft decision source decoder (SDSD) [4,5] are iteratively combined exchanging extrinsic information.Unlike Turbo channel decoding, which aims at minimizing the bit error rate, ISCD mainly aims at error concealment and signal restoration which is not necessarily connected to a lower bit error rate, but to a higher parameter SNR.ISCD exploits the a priori knowledge on the residual redundancy of the source codec parameters that remains after imperfect source coding.The a priori knowledge can be a nonuniform probability distribution, an autocorrelation or a cross correlation.The source codec parameters can be, e.g., scale factors or predictor coefficients for speech, audio and video signals.Delay or complexity constraints prevent a complete removal of the residual redundancy and therefore, in practice, a quite large amount of residual redundancy remains in the source codec parameters, which can be exploited by ISCD.
One major design issue in ISCD systems is the index assignment.Besides the traditional index assignments for noniterative systems, such as natural binary or Gray, optimized index assignments have been developed that take into account the possible feedback due to the Turbo loop between channel decoder and SDSD.In [6,7] index assignments have been introduced that are optimized considering a nonuniform probability distribution, i.e., the zeroth order a priori information, of source samples.Further enhanced index assignments have been presented in [8] and the corresponding optimization process even takes the first order a priori information, e.g., the autocorrelation, of the source samples into account.This paper focuses on the latter type of index assignments.
In general, an index assignment is chosen in advance and is not exchanged during a transmission, otherwise side information has to be transmitted to notify the receiver of the change.However, in this paper the index assignment is assumed to be constant during a transmission.If it is a priori known that the source codec parameters bear a specific overall autocorrelation, an appropriate index assignment can be applied exploiting this autocorrelation.But since most signals have a time-varying autocorrelation it has to be examined how much the performance degrades, if the signal correlation does not match the correlation that the index assignment is optimized for.
At first, the underlying ISCD transmission scheme is described in Section 2. Since in this paper the performance of index assignments that consider first order a priori knowledge will be under examination, more details on this topic will be given in Section 3. In Section 4 the performance (degradation) of the ISCD system is shown by means of simulation results utilizing the index assignments under both optimal and suboptimal conditions.In Section 5 the simulated scenarios are then analyzed and confirmed by so-called EXIT (extrinsic information transfer) charts [9].

Iterative Source-Channel Decoding
The baseband model of the utilized ISCD transmission scheme is depicted in .Source codec parameters are generated by a Gauss-Markov source, with an inherent autocorrelation M bits is assigned according to the utilized index assignment.The single bits of a bit pattern are indicated by with , and the frame of bit patterns is denoted as x .Three parameter SNR optimized index assignments considering first order a priori knowledge (SOAK1) are used and the natural binary (NB) index assignment serves as a reference.The SOAK1 index assignments are optimized for different source correlations, thus they are referred to as ) ( SOAK ρ .
The bit interleaver π scrambles the incoming bits x of the frame τ x of bit patterns to x ~ in a deterministic manner.To simplify the notation, we restrict the interleaving to a single time frame with index τ and omit the time frame index τ in the following, where appropriate.
For the channel encoding of a frame x of interleaved bits x we utilize LDPC codes, which were first proposed by Gallager [10] and rediscovered by MacKay [11].LDPC codes have a very high error correction capability with iterative decoding that is very close to the Shannon limit.Their performance is comparable or even superior to that of convolutional Turbo codes.In this paper we use a modification of short LDPC codes as presented in [12].Identical instances of a short LDPC code are combined to a long LDPC code, whose frame size is flexible in multiples of a subframe size, i.e., the frame size of the short LDPC code.By serially concatenating the subframes with a bit-interleaver and a second component that provides extrinsic information according to the Turbo principle (e.g., a soft decision source decoder (SDSD) as in this paper), extrinsic information can also be exchanged between subframes.Such concatenated LDPC codes approach very well the performance of long monolithic LDPC codes of the same frame size [12].The performance of the concatenated LDPC code strongly depends on the performance of the short code.Therefore, the short code has to be chosen carefully.As short LDPC code a (21,11) difference set cyclic (DSC) code [13] is used.DSC codes feature a high minimum Hamming distance, and especially at short block lengths they can outperform comparable pseudorandom LDPC codes [14].
The resulting codeword is denoted as with bits , which are mapped to bipolar bits . We choose the simple BPSK modulation scheme, since modulation is no design issue in this paper.
On the channel, the signal is superposed with additive white Gaussian noise (AWGN) with the known power spectral density , i.e., The received symbols z are evaluated in a Turbo process, in which extrinsic reliabilities between the LDPC decoder and the SDSD are exchanged.Utilizing LDPC codes results in an additional iterative loop in the LDPC decoder, in which extrinsic information is exchanged between the variable nodes and the check nodes.These iterations are denoted as LDPC-iterations.
Details about the ISCD receiver can be found in [2,3,15].The LDPC decoder uses the belief propagation algorithm [16,11] to generate extrinsic information.The SDSD determines the extrinsic information mainly from the natural residual source redundancy, which generally remains in the bit patterns after source encoding.Such residual redundancy appears on the parameter level, e.g., as a nonuniform distribution , in terms of a correlation, or as other possible time-dependencies.The latter terms of residual redundancy are generally approximated by a first order Markov chain, i.e., by exploiting the conditional probabilities .These conditional probabilities heavily depend on the source correlation.For specific source correlations, e.g., , they can be calculated once in advance.The technique for combining this a priori information on the parameter level with the soft input values on the bit level is also well known in the literature.The algorithm for the computation of the extrinsic has been detailed, e.g., in [2,3,15].As a quality measure we consider the parameter signal-to-noise ratio (SNR)

Index Assignments
The index assignment is a major design factor influencing the performance of ISCD transmission systems.Several conventional index assignments already exist, such as natural binary (NB) or Gray [17].They are well-suited for noniterative systems [15,7,6], but they exhibit only a suboptimal performance in ISCD., to bit patterns with In this paper we consider only bit patterns consisting of that are assigned to quantizer levels.The utilized index assignments are listed in Table 3, but first, the optimization algorithm will be briefly explained.This was introduced in [8] and can be found in more detail there.According to the notation in [15], the index assignment  Parameter SNR optimized index assignments that take into account first order a priori knowledge are generated by minimizing the overall noise energy function [8] ∑∑ ∑ , which yields a global optimum, or by the binary switching algorithm [18,6,15], which may lead to a local optimum only.

Simulation Results
In Figure 2 the parameter SNR performance of ISCD utilizing the optimized index assignments is compared to the one using the conventional natural binary index assignment.The parameter SNR performance is shown for various combinations of the source parameter correlation at the transmitter T ρ and the assumed correlation at the receiver R ρ .For exploiting the residual redundancy at the receiver it is necessary that the source parameter correlation is known at the receiver.To that end, the source correlation either has to be transmitted as side information or it has to be estimated at the receiver.The latter approach has turned out to be very precise and easy to implement.However, in this paper, the mismatch between T ρ and R ρ is preset.
A reference parameter SNR of is assumed at which the parameter SNR performances will be compared.The parameters emitted by the Gauss-Markov source exhibiting a source correlation of dB 13 [ref] = P ) 1 ( 2 = σ T ρ , are grouped to frames of size and are quantized by an 8-level Lloyd-Max quantizer (LMQ) resulting in 990 uncoded bits per frame.After channel encoding this yields 1890 coded bits per frame.On the receiver side, iterations are performed, which means that during each of the six iterations between LDPC-decoder and SDSD three LDPC-iterations between the variable nodes and the check nodes of the LDPC-decoder are carried out.
The set of curves in Figure 2 that is labeled by shows the scenario in which no correlation is exploited at the receiver, independently of the actual source correlation.This is also the current state of today's transmission systems, where the available correlation is not utilized at the receiver in order to enhance the signal quality.In this case the three curves for NB,   In systems with a high and slowly varying source correlation a reliable estimation of the source correlation is possible, and thereby it is reasonable and feasible to exploit the performance gain of an index assignment optimized for high source correlations.In systems with fast and high source correlation fluctuations a more conservative choice of the index assignment is recommended.

EXIT Chart Analysis
In this section the system is analyzed by EXIT (extrinsic information transfer) charts [9], which are a powerful tool for analyzing and optimizing the convergence behavior of iterative systems utilizing the Turbo principle, i.e., systems exchanging and refining extrinsic information.The capabilities of the components, in our case the LDPC decoder and the soft decision source decoder (SDSD), are analyzed separately.The extrinsic mutual information [ext]  I obtained by each component is determined for a certain a priori mutual information [apri]  I .Both, [ext]  I and z , are calculated on the basis of the actual data and the available extrinsic or a priori information of the data.As a basis for this calculation usually histograms of the respective L-values, e.g., for , are used.Lvalues are the log-likelihood ratios of the corresponding probabilities I [19].For the EXIT characteristics, the a priori L-values are simulated as uncorrelated Gaussian distributed random variables with variance and mean .In most cases, e.g., for a classical Turbo Code [1], this is a good assumption [9].The applicability of EXIT charts to ISCD is demonstrated, e.g., in [15].z .The decoding trajectory (step curve) shows the mutual information I for each iteration in a simulation of the complete system.In the optimization process of the system design the components are chosen such that the (first) intersection of their EXIT characteristics moves towards the point .In the design of the system, i.e., the design of the characteristics, we have always

Conclusion
In this paper the performance of iterative sourcechannel decoding utilizing optimized index assignments has been analyzed.The studied index assignments are optimized with respect to the parameter SNR and by considering first order a priori knowledge.The simulation results show that high gains are achievable if the source parameter correlation is high and if the correlation at the receiver matches.These findings have been confirmed by an EXIT chart analysis.With optimized index assignments additional gains can be achieved compared to conventional index assignments.In case the assumed correlation at the receiver is higher than the source parameter correlation, high deteriorations can occur depending on how mismatched the correlations are.Thus, depending on the dynamics and the amount of correlation of the source codec parameters, either the optimized or the conventional index assignments are better suited.The index assignments optimized for high correlations perform better in systems with a high and slowly varying source correlation, while the conventional index assignments have an advantage in systems with fast and high source correlation fluctuations.

uρ
in order to obtain comparable and reproducible results.At time instant τ , K source codec in the frame.In this paper the autocorrelation ρ is constant in order to simulate a fixed mismatch between the correlation T ρ of the transmitted source parameters and the assumed correlation R ρ at the receiver.The autocorrelation takes on values from a finite set, e.g., matching.Such high values for T ρ are not unusual for several source codec parameters in current communication systems like GSM or UMTS.The parameter SNR optimized index assignment ) gain of compared to the reference index assignment NB.The gain of and the performances of almost the same.However, this shows that if the source parameters exhibit a certain amount of correlation it is definitely expedient to exploit it.The rightmost set of curves labeled by ) of the scenario in which the source parameters are uncorrelated, but the receiver assumes a high correlation can be observed.However, this scenario is very unlikely, since the correlation can be estimated very accurately at the receiver, so that this high mismatch is temporally limited to the instances of abrupt changes of the source parameter correlation.

Figure 2 :
Figure 2: Parameter SNR performance of source correlation optimized index assignments in combination with an autocorrelation mismatch.) ( SOAK1 8 3 ρ Figure 3: EXIT charts for SOAK1 ) 0 .0 ( and SOAK1 ) 9 .0 ( . a) and Figure 3(b) into account, we can observe that by means of EXIT charts we can accurately analyze, explain and confirm the parameter SNR results.

Table 1 :
Index assignments