Speech signal modeling using multivariate distributions

Abstract

Using a proper distribution function for speech signal or for its representations is of crucial importance in statistical-based speech processing algorithms. Although the most commonly used probability density function (pdf) for speech signals is Gaussian, recent studies have shown the superiority of super-Gaussian pdfs. A large research effort has focused on the investigation of a univariate case of speech signal distribution; however, in this paper, we study the multivariate distributions of speech signal and its representations using the conventional distribution functions, e.g., multivariate Gaussian and multivariate Laplace, and the copula-based multivariate distributions as candidates. The copula-based technique is a powerful method in modeling non-Gaussian multivariate distributions with non-linear inter-dimensional dependency. The level of similarity between the candidate pdfs and the real speech pdf in different domains is evaluated using the energy goodness-of-fit test.

In our evaluations, the best-fitted distributions for speech signal vectors with different lengths in various domains are determined. A similar experiment is performed for different classes of English phonemes (fricatives, nasals, stops, vowels, and semivowel/glides). The evaluation results demonstrate that the multivariate distribution of speech signals in different domains is mostly super-Gaussian, except for Mel-frequency cepstral coefficient. Also, the results confirm that the distribution of the different phoneme classes is better statistically modeled by a mixture of Gaussian and Laplace pdfs. The copula-based distributions provide better statistical modeling of vectors representing discrete Fourier transform (DFT) amplitude of speech vectors with a length shorter than 500 ms.

1 Introduction

Statistical-based speech processing algorithms have attracted wide interests during the last three decades in numerous applications, e.g., speech coding [1], speech recognition [2, 3], speech synthesis [4], and speech enhancement [5]. In all statistical-based speech processing algorithms, a probability density function (pdf) is assumed for the signal or its representation. Therefore, it is not surprising that proper selection of the pdf has been one of the challenges persistently addressed in this area [6–8].

Most of the statistical-based speech processing algorithms assume Gaussian probability distribution density for speech signals [2, 9–13]. The simplicity of the formulations and the semi-support of the central limit theorem are the main motivations for using Gaussian pdf [14]. Recently, a number of researchers have studied the distribution of speech signal more precisely using goodness-of-fit (GOF) test [15, 16] in time domain and transformed domains, e.g., discrete cosine transform (DCT) and discrete Fourier transform (DFT). In this regard, Gazor et al. [14], Martin [6], Shin et al. [7], Chen and Loizou [17], and Erkelens et al. [18] have shown that speech signals in various domains are modeled more accurately by super-Gaussian pdfs than by Gaussian pdf. Their evaluation results have demonstrated that the pdf of speech signals for time and DCT features are closer to Laplace [14], for jointly amplitude and phase of DFT features are closer to complex Laplace [17], for amplitude of DFT features are closer to Rayleigh [18], and for real or imaginary parts of DFT features are individually closer to either Laplace or Gamma [6]. In addition, Shin et al. [7] have reported that generalized Gamma pdf models the distribution of real parts of DFT features more accurately compared to the Gaussian pdf. Table 1 summarizes the results of the published evaluations in this field.

Table 1 Proposed super-Gaussian univariate distribution of speech signal in different domains

All aforementioned publications have aimed to address the issue of modeling univariate pdf of speech signals for algorithms using univariate pdf. However, there are many statistical-based algorithms that take advantage of multivariate distribution of speech signals, and therefore, studying the multivariate distribution of speech to exploit a more proper pdf is a key issue for those speech processing algorithms too. There are typically several challenges in the studying and modeling of speech signals in the multivariate distribution case, e.g., the non-linear or linear inter-dimensional dependency, and the sparsity and complexity of the multidimensional space. These issues may have caused to mostly focus on the investigation of univariate distribution during the last two decades, and a small progress has been made in the multivariate distribution study of speech signal. The earlier studies on multivariate distribution of speech signal, performed by Brehm et al. [19] and LeBlancin et al. [20], suggested the multivariate Gaussian pdf for speech frames with a length of 5 ms. As the frame length and the process domain may vary the distribution [14], the multivariate Gaussian pdf may not be an appropriate choice for the algorithms using frame length other than 5 ms, e.g., 10 to 35 ms or exploiting process domain other than the time domain, e.g., DFT or DCT. In recent studies, Gazor et al. [14] and Jensen et al. [21] have used the moment test and have shown that Laplace multivariate distribution models speech signal or its representations are better than the Gaussian multivariate distribution. However, in these studies, the moment test as a GOF test was applied to each dimension individually and the possible contribution of inter-dimensional dependency to the multivariate distribution was not considered.

In this paper, we investigate multivariate distribution of speech signal in the time and transformed domains. We consider new plausible distribution candidates to tackle the multivariate distribution modeling challenges. Among the candidates, copula-based distributions are also proposed which are able to model the high-dimensional non-Gaussian distribution with non-linear inter-dimensional dependency [22]. The copula-based distributions have been popular over the last decade in the statistical fields, e.g., climate research, econometrics, risk management [22, 23], and finance [24, 25]. The other possible pdf candidates of speech including multivariate Gaussian, multivariate Laplace, the mixture of Gaussian, and the mixture of Laplace distributions are investigated in this paper too. We employ the goodness-of-fit test [15, 16, 26] to evaluate the degree of similarity between the candidate distribution and the real speech signal distribution. The GOF test is a tractable three-step approach to investigate distribution of data. In the first step, a number of candidates are assumed as the pdf of the real data. Next, an estimator, e.g., maximum likelihood (ML) is exploited to fit the candidates to the real data, and finally, the GOF test is performed to quantify the level of similarity between the fitted candidates and the real data. It is noted that although a wide number of GOF tests have been proposed, the most appropriate GOF test is the one that can highly cover underlying problem conditions, e.g., in our case study is high dimensionality of spaces. We briefly present a number of GOF tests, a summary of their strengths and deficiencies, and finally choose the one that has been reported as the most appropriate for high-dimensional space.

In general, speech processing algorithms using multivariate distribution exploit different feature types to process speech signals. For instance, traditional hidden Markov model (HMM)-based speech recognition and synthesis algorithms [3, 27] exploit Mel-frequency cepstral coefficients (MFCC); HMM-based speaker recognition [13] systems exploit either linear predictive coding (LPC) or MFCC; HMM-based speech enhancement algorithms use LPC, time, DCT, MFCC, or DFT [7, 9, 10]; and codebook-driven-based speech enhancement algorithms [28] employ LPC. However, all these algorithms assume the multivariate Gaussian pdf for extracted features of speech signals. As the feature type may influence the distribution [14], the multivariate distribution of the different feature types including DFT, DCT, time, LPC, and MFCC is studied in this paper. It is noted that a number of speech processing algorithms, e.g., proposed by Martin [6], Shin et al. [7], model the real and imaginary parts of DFT separately. Thus, we study the real and imaginary parts of DFT features separately. The whole study of multivariate distribution in this paper is concentrated on clean speech signals.

The remainder of this paper is structured as follows. In Section 2, the copula-based distributions are presented including their formulations and parameter estimation. In Section 3, the GOF tests are briefly reviewed and among them, the energy test is selected as the most appropriate one for the multivariate distribution study of high-dimensional space. Section 4 elaborates candidates’ formulations, their parameter estimation, and an algorithm for exploring the best-fitted candidate. Section 5 presents the evaluation setup and experimental results. Finally, Section 6 concludes the work.

2 Copula-based distribution

A copula is defined as a multivariate probability distribution where the marginal probability distribution of each variable is uniform and is used to describe the dependency between random variables [22, 29–33]. As all the multivariate joint distributions can be written in terms of a copula and univariate marginal pdfs [29], copulas are used as a popular statistical tool for modeling multivariate distributions. In this regard, copulas allow to easily model the distribution of multivariate random variables by estimating only marginal pdfs and copulas. A copula-based distribution can capture important characteristics of a vector, e.g., the appropriate pdf for margins and the appropriate correlation structure with a possibly simple form.

The purpose of this section is to briefly review the basic definition of the copula and a number of the most commonly used estimation methods for fitting the copula to the real data.

2.1 Copula model

The mathematical basis of the copula was proposed by Sklar [29] and Hoeffding [33]. To define a copula model, let x be a d-dimensional random vector x = {x ₁, …, x _j , …, x _d }. Sklar [29] showed that the joint cumulative distribution function (CDF) of x, F _X (x), can be expressed as a function of marginal CDFs ${u}_j={F}_{{\boldsymbol{X}}_j}\left({x}_j\right),\kern0.24em j=1:d$, as shown in Eq. (1), where C _X  : [0, 1]^d → [0, 1] denotes copula function. Based on Sklar’s theorem [29], any arbitrary multivariate pdf f _X (x) can be expressed as the product of two terms: the marginal pdf of dimensions ${f}_{{\mathtt{X}}_j}\left({x}_j\right),\kern0.24em j=1:d$ and the copula density function c _X (.) as shown in Eq. (2). The copula density function c _X (.) can be derived from the copula function as given in Eq. (3). As the copula density function characterizes the inter-dimensional dependencies, it is also called correlation structure in literature [22].

$$ {F}_X\left(\boldsymbol{x}\right)={C}_X\left({u}_1,\dots, {u}_j,\dots, {u}_d\right) $$

(1)

$$ {f}_{\boldsymbol{X}}\left(\mathtt{x}\right)=\left({\displaystyle \prod_{j=1}^d{f}_{X_j}\left({x}_{\mathrm{j}}\right)}\right).\;{c}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_j,\dots, {u}_d\right) $$

(2)

$$ {c}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_j,\dots, {u}_d\right)=\left({\displaystyle \prod_{j=1}^d\frac{\partial }{\partial {u}_j}}\right).\kern0.24em {C}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_j,\dots, {u}_d\right) $$

(3)

The two most used parametric forms for c _X (.) are elliptical and Archimedean [22, 30]. The Archimedean-based copulas are mostly used in the bivariate form and they are not practically usable for high-dimensional spaces due to its high-computational cost [22, 34]. In contrast to the Archimedean, the elliptical-based copulas, including Gaussian and Student-t copula, can be used for spaces with any number of dimensions [22]. We therefore briefly review the Gaussian and Student-t copulas in the following sections.

2.2 Gaussian copulas

Let us assume the marginal CDFs, u _j , are known. Each u _j can be transformed to a standard distributed random variable y _j , by using the inverse of univariate Gaussian CDF ${F}_{N\left(0,1\right)}^{-1}(.)$ as shown in the following equation,

$$ {y}_j={F}_{N\left(0,1\right)}^{-1}\left({u}_j\right)={F}_{N\left(0,1\right)}^{-1}\left({F}_{X_j}\left({x}_j\right)\right)\sim N\left(0,1\right). $$

(4)

As a consequence, the set y = {y ₁, …, y _j, …, y _d} has a multivariate Gaussian distribution N(0, Σ _Copula), where Σ _Copula denotes a symmetric, diagonal, positive definite matrix with unit diagonal elements. The joint CDF of y can therefore be expressed by Eq. (5), which can be interpreted as the copula function of x using terminology of Eq. (1). By using Eqs. (3) and (5), the copula density can be derived as shown in Eq. (6), where I denotes an identity matrix and Tr denotes transpose operator. For further details on this topic, see [22] and [31].

$$ {F}_{\boldsymbol{X}}\left(\boldsymbol{x}\right)={C}_{\boldsymbol{X}}\left({u}_1,\dots, .{u}_j,..,{u}_d\right)={F}_{\boldsymbol{N}\left(0,{\varSigma}_{\mathrm{Copula}}\right)}\left({y}_1,\dots, {y}_j,\dots, {y}_d\right) $$

(5)

$$ {c}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_d\right)=\frac{1}{{\left|{\varSigma}_{\mathrm{Copula}}\right|}^{0.5}} \exp \left(-\frac{1}{2}\kern0.24em y\left({\varSigma}_{\mathrm{Copula}}^{-1}-I\right)\kern0.24em {y}^{Tr}\right) $$

(6)

2.3 Student-t copulas

The Student-t copula is defined analogously to the Gaussian copula; however, the transformed variables y _j are obtained using univariate Student-t CDF inverse ${F}_{t(v)}^{-1}\left({u}_j\right)$, where t(v) is a univariate Student-t distribution with v degrees of freedom. Consequently, y = {y ₁, …, y _j , …, y _d } follows a multivariate Student-t distribution t(v, Σ _Copula) where Σ _Copula denotes a positive definite matrix with unit diagonal elements. Similar to the Gaussian copula function, the Student-t copula function, i.e., joint CDF of y, is shown by Eq. (7). By using Eqs. (3) and (7), the Student-t copula density then is computed as shown in Eq. (8). For further details, see [22] and [31].

$$ {C}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_d\right)={F}_{\boldsymbol{t}\left(v,{\varSigma}_{\mathrm{Copula}}\right)}\left({F}_{t(v)}^{-1}\left({u}_1\right),\dots,\;{F}_{t(v)}^{-1}\left({u}_d\right)\right) $$

(7)

$$ {c}_{\boldsymbol{X}}\left({u}_1,\dots, {u}_d\right)\kern0.36em ={\left|{\varSigma}_{\mathrm{Copula}}\right|}^{-0.5}\kern0.24em \frac{\varGamma \left(\frac{v+d}{2}\right){\left[\varGamma \left(\frac{v}{2}\right)\right]}^d}{{\left[\varGamma \left(\frac{v+1}{2}\right)\right]}^d\varGamma \left(\frac{v}{2}\right)}.\;\frac{{\left(1+\frac{1}{v}y\;{\varSigma}_{\mathrm{Copula}}^{-1}\;{y}^t\right)}^{-\frac{v+d}{2}}}{{\displaystyle \prod_{j=1}^d{\left(1+\frac{{\mathtt{y}}^2}{v}\right)}^{-\frac{v+1}{2}}}} $$

(8)

2.4 Fit a copula model

There are several estimation methods proposed for copula parameters [22, 35–37]. Among them, maximum likelihood (ML), inference functions for margins (IFM), and canonical maximum likelihood (CML) techniques are used more frequently than others [22]. Generally, these estimators maximize the log-likelihood function of Eq. (2) with respect to $\varTheta =\left\{\theta, \alpha \right\}$ in different ways, where α denotes a set of parameters concerning the copula density function, e.g., Σ _Copula, and θ = {θ ₁, …, θ _j , …, θ _d } denotes a set of parameters concerning the marginal pdfs, ${f}_{{\mathtt{X}}_j}\left({x}_j\right)$. The log-likelihood function is derived as shown in Eq. (9) [31] where ${\boldsymbol{x}}_{n=1}^N=\left\{{\boldsymbol{x}}^1, \dots,\;{\boldsymbol{x}}^n, \dots,\;{\boldsymbol{x}}^N\right\}$ denotes a set of N observation vectors of real data and ${x}_j^n$ represents jth component of vector x ⁿ.

$$\begin{aligned}\text{l}(\Theta) & =\sum_{n=1}^N \sum_{j=1}^d\text{ln}\,f_{\;\mathrm{X}_{j}}\left({x^{n}_{j}};\theta_{j}\right) & + \sum_{n=1}^N\text{ln}\,c\left[F_{\mathrm{X}_{1}}\left({x^{n}_{1}};\theta_{1}\right),\ldots,F_{\mathrm{X}_{j}}\left({x^{n}_{j}};\theta_{j}\right),\ldots,F_{\mathrm{X}_{d}}\left({x^{n}_{d}};\theta_{d}\right);\boldsymbol{\alpha}\right] \end{aligned}$$

(9)

The ML approach estimates the parameters of marginal pdfs and copula density function jointly using numerical optimization [22]. This is the only way to estimate all the parameters consistently [22].

The IFM method is the most used method. It estimates $\mathtt{\varTheta}$ by maximizing Eq. (9) in two steps. First, the parameters of the margins θ are individually estimated as shown in Eq. (10). It is noted that the type of marginal distributions, e.g., Laplace or Gamma, are assumed to be determined in advance. The copula parameters α are then derived as shown in Eq. (11) by using the estimated $\widehat{\boldsymbol{\theta}}$.

$$ {\widehat{\theta}}_j= \arg \kern0.24em \max\;{\displaystyle \sum_{n=1}^N \ln \kern0.24em {f}_{X_j}\left({x}_j^n;{\theta}_j\right)} $$

(10)

$$ \widehat{\boldsymbol{\alpha}}= \arg\;\max\;\left[{\displaystyle \sum_{n=1}^N \ln \kern0.24em c}\left({F}_{{\mathtt{X}}_1}\left({x}_1^n;{\widehat{\theta}}_1\right),\dots \right.\right.\left.,{F}_{{\mathtt{X}}_j}\left({x}_j^n;{\widehat{\theta}}_j\right),\left.\dots, {F}_{{\mathtt{X}}_d}\left({x}_d^n;{\widehat{\theta}}_d\right);\boldsymbol{\alpha} \right)\right] $$

(11)

The CML method first empirically estimates ${f}_{{\mathtt{X}}_j}\left({x}_j^n\right)$ and ${F}_{{\mathtt{X}}_j}\left({x}_j^n\right)$, denoted as ${\widehat{f}}_{{\mathtt{X}}_j}\left({x}_j^n\right)$ and ${\widehat{F}}_{{\mathtt{X}}_j}\left({x}_j^n\right)$, using non-parametric methods, e.g., kernel smoothing density estimator [38]. Thus, it is not needed to determine the type of marginal distributions in advance, in contrast to the IFM method. The parameters of copula are then estimated using Eq. (12).

$$ \widehat{\boldsymbol{\alpha}}= \arg\;\max\;\left[{\displaystyle \sum_{n=1}^N \ln \kern0.24em c}\left({\widehat{F}}_{{\mathtt{X}}_1}\left({x}_1^n\right),\dots \right.\right.\left.,{\widehat{F}}_{{\mathtt{X}}_j}\left({x}_j^n\right),\left.\dots, {\widehat{F}}_{{\mathtt{X}}_d}\left({x}_d^n\right);\boldsymbol{\alpha} \right)\right] $$

(12)

In order to estimate the parameters of the copula-based distribution using one of the discussed methods, the following issues should be considered:

• The ML method is used for spaces with a small number of dimensions due to the numerical complexity issue.

• The IFM method ends up a sub-optimal solution for parameter estimation since the log-likelihood function is maximized in two individual steps [31].

• The IFM and CML methods result in closed-form formulas only for Gaussian copula case [31].

• When one of the values of off-diagonal components of the covariance matrix ΣCopula of either the Student-t or Gaussian copulas takes 1 or −1, the estimation procedure of the copula parameters using CML method may fail [39]. It is due to Cholesky decomposition performed in CML methods.

In this paper, the parameters of copula-based distributions are estimated using IFM method and Gaussian copula density function that result in closed-form formulas and avoid failing in the estimation procedure. To estimate α = Σ _Copula of Gaussian given a vector sequence ${\boldsymbol{y}}_{n=1}^N=\left\{{\boldsymbol{y}}^1, \dots,\;{\boldsymbol{y}}^n, \dots,\;{\boldsymbol{y}}^N\right\}$, the Eq. (6) is plugged into Eq. (11) resulting in Eq. (13). Regarding the estimation of θ, as the type of marginal pdfs of copula density function should be given in advance, it will be discussed in Section 4.

$$ {\widehat{\varSigma}}_{\mathrm{Copula}}=\frac{1}{N}{\displaystyle \sum_{n=1}^N\;{\left({\boldsymbol{y}}^n\right)}^{Tr}{\boldsymbol{y}}^n} $$

(13)

3 Goodness-of-fit test

A wide number of goodness-of-fit (GOF) tests depending on underlying conditions of the case study have been proposed. In our study, high dimensionality and possible non-linear inter-dimensional dependency are the most crucial issues. Although various GOF tests are proposed for one-dimensional space, only some of them are extendable for high-dimensional space. As the number of space dimensions increases, the tests become inefficient [26]. For instance, the extension of χ ² test [15] to higher dimensions suffers from the curse of dimensionality [40] caused by the space sparsity unless the sample sizes are large enough.

There are several GOF tests particularly proposed for the multivariate case, e.g., the nearest neighbor test which exploits the nearest neighbors [41], Mardia test which exploits the skewness and kurtosis [42] and Freidman–Rafsky test which exploits the minimum spanning tree [43]. In this regard, the energy test has also recently been proposed by Zach and Aslan [44]. The performance superiority of the energy test has been demonstrated compared to Mardia, nearest neighbor, χ ², and Friedman–Rafsky tests. Accordingly, the energy test is selected as a more appropriate GOF for underlying conditions of the study in this paper to evaluate candidates. In the following, the energy test is discussed.

3.1 Energy test

Given a candidate pdf ${f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ and a sequence of observation vectors ${\boldsymbol{x}}_{n=1}^N=\left\{{\boldsymbol{x}}^1, \dots,\;{\boldsymbol{x}}^i, \dots,\;{\boldsymbol{x}}^N\right\}$ which follow an unknown pdf f _X (x), the energy statistic for the hypothesis ${H}_0:\kern0.24em {f}_{\boldsymbol{X}}\left(\boldsymbol{x}\right)={f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ against ${H}_1:\kern0.36em {f}_{\boldsymbol{X}}\left(\boldsymbol{x}\right)\ne {f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ is computed by

$$ {\phi}_{NM}=\frac{1}{N\left(N-1\right)}{\displaystyle \sum_{t>i}R\left(\left|{\boldsymbol{x}}^i-{\boldsymbol{x}}^t\right|\right)}+\frac{1}{M\left(M-1\right)}{\displaystyle \sum_{j>n}R\left(\left|{q}^n-{q}^j\right|\right)}-\frac{1}{NM}{\displaystyle \sum_{i=1}^N{\displaystyle \sum_{j=1}^MR\left(\left|{\boldsymbol{x}}^i-{\boldsymbol{q}}^j\right|\right)}} $$

(14)

where ${\boldsymbol{q}}_{j=1}^M=\left\{{\boldsymbol{q}}^1, \dots,\;{\boldsymbol{q}}^j, \dots,\;{\boldsymbol{q}}^M\right\}$ denotes a sequence of M observation vectors following ${f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ and generated by Monte–Carlo simulation [45] and R denotes a continuous, monotonically decreasing function, i.e., R(r) = − ln(r). In the limit N → ∞ and M → ∞, the statistic ϕ _NM approaches minimized value, near zero, if ${\boldsymbol{x}}_{i=1}^N$ and ${\boldsymbol{q}}_{j=1}^M$ are from the same distribution [44]. For further details, see the Appendix.

The required steps for performing the energy test are summarized:

• 1. The real dataset is segmented, resulting in N vectors each of length d, ${\boldsymbol{x}}_{i=1}^N=\left\{{\boldsymbol{x}}^1, \dots,\;{\boldsymbol{x}}^i, \dots,\;{\boldsymbol{x}}^N\right\}$. Depending on the process domain, x ⁱ represents the segmented real data in that process domain, e.g., time, DFT, and DCT.

• 2. A possible candidate pdf ${f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$, e.g., either copula-based or conventional distributions, is hypothesized and fitted to the real data vectors ${\boldsymbol{x}}_{i=1}^N$.

• 3. The number M of simulated data vectors following fitted pdf ${f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ is generated using Monte–Carlo.

• 4. The energy test statistic is computed using Eq. (14) to determine the level of similarity between the distributions of real data vectors ${\boldsymbol{x}}_{i=1}^N$ and simulated data vectors ${\boldsymbol{q}}_{j=1}^M$.

4 Multivariate distribution candidates

To study the multivariate distribution of speech features, two classes of pdfs are considered as the candidates in this paper: (1) copula-based distributions and (2) conventional distributions. The first pdf class includes five distributions:

• 1. Copula-based Laplace distribution (CLD)

• 2. Copula-based Laplace distribution with mutually independent dimensions (CLID), i.e., c _X (.) = 1

• 3. Copula-based generalized extreme value distribution (CGevD)

• 4. Copula-based Rayleigh distribution (CRD)

• 5. Copula-based Gamma distribution (CGD).

As formerly mentioned, the IFM method is used to fit the copula-based distributions to real data. In this regard, marginal distributions of each candidate should be first determined. The following univariate pdfs are used as the marginal distributions for each candidate: univariate Laplace pdf as shown in Eq. (15) [14] for CLD and CLID, univariate generalized extreme value pdf as shown in Eq. (16) [46] for CGevD, univariate Rayleigh pdf as shown in Eq. (17) for CRD, and univariate Gamma pdf as shown in Eq. (18) [21] for CGD. In these equations, μ _j and α _j denote the location parameters, b _j , ξ _j , and σ _j denote the scale parameters, and k _j and γ _j represent the shape parameters. As a consequence, CLD is a copula-based distribution with marginal distributions of Laplace and its parameters Θ _CLD = {θ _Laplace, α _Copula} are estimated using the IFM method given an observation set of vectors ${\boldsymbol{x}}_{n=1}^N=\left\{{\boldsymbol{x}}^1, \dots,\;{\boldsymbol{x}}^i, \dots,\;{\boldsymbol{x}}^N\right\}$, where θ _Laplace denotes a parameter set {b _j , μ _j } concerning marginal distribution ${f}_{X_j,\mathrm{Laplace}}\left({x}_j\right),\kern0.24em j=1:d$ and is estimated using Eqs. (19) and (20). The parameter of Gaussian copula density α _Copula is estimated using Eq. (13). Similarly, θ _Rayleigh = {ν _j } is estimated using Eq. (21) and θ _Gamma = {γ _j , ξ _j } is estimated using Eqs. (22) and (23) [47], where $s= \ln \left(\frac{1}{N}{\displaystyle \sum_{n=1}^N{x}_j^n}\right)-\frac{1}{N}.{\displaystyle \sum_{n=1}^N \ln \left({x}_j^n\right)}$. It is noted that CGD and CRD both model one-side distributions and are therefore proposed as candidates for modeling one-side distributed random vector, e.g., the amplitude of DFT feature. For estimation of CGevD parameters θ _GEV = {σ _j , α _j , k _j } using the IFM, as it results in no closed-form solution [48], the MATLAB function “fminsearch”, which employs numerical method of Lagarias [49], is employed.

$$ {f}_{X_j,\mathrm{Laplace}}\left({x}_j\right)=\frac{1}{2{b}_j}. exp\left(-\frac{\left|{x}_j-{\mu}_j\right|}{b_j}\right) $$

(15)

$$ {f}_{X_j,\mathrm{G}\mathrm{E}\mathrm{V}}\left({x}_j\right)=\left(\frac{1}{\sigma_j}\right). exp\left(-{\left(1+{k}_j\frac{x_j-{\alpha}_j}{\sigma_j}\right)}^{-\frac{1}{k_j}}\right).{\left(1+{k}_j\frac{\left({x}_j-{\mu}_j\right)}{\sigma_j}\right)}^{-1-\frac{1}{k_j}} $$

(16)

$$ {f}_{X_j,\mathrm{Rayleigh}}\left({x}_j\right)=\frac{x}{\nu_j^2}. exp\left(-\frac{x^2}{2{\nu}_j^2}\right) $$

(17)

$$ {f}_{X_j,\mathrm{Gamma}}\left({x}_j\right)=\frac{{\left({x}_j\right)}^{\gamma_j-1}}{{\left({\xi}_j\right)}^{\gamma_j}\varGamma \left({\gamma}_j\right)}. exp\left(\frac{-{x}_j}{\xi_j}\right) $$

(18)

$$ {\widehat{\mu}}_j=\mathrm{median}\left({x}_j^n\right) $$

(19)

$$ {\widehat{b}}_j=\frac{1}{N}.{\displaystyle \sum_{n=1}^N\left|{x}_j^n-{\mu}_j\right|} $$

(20)

$$ {\widehat{\nu}}_j\approx \sqrt{\frac{1}{2N}.{\displaystyle \sum_{n=1}^N{\left({x}_j^n\right)}^2}} $$

(21)

$$ {\widehat{\gamma}}_j\approx \frac{3-s+\sqrt{{\left(s-3\right)}^2+24s}}{12s}\sqrt{\frac{1}{2N}.{\displaystyle \sum_{n=1}^N{\left({x}_j^n\right)}^2}} $$

(22)

$$ {\widehat{\xi}}_j=\frac{1}{\gamma_jN}.{\displaystyle \sum_{n=1}^N{x}_j^n} $$

(23)

The following steps summarize the estimation procedure of Gaussian copula.

• 1. A marginal distribution, e.g., one of Eqs. (15)–(18), is accounted for ${f}_{X_j}\left({x}_j\right),\kern0.24em j=1:d$ and parameter set is accordingly estimated.

• 2. The corresponding CDF of ${f}_{{\mathtt{X}}_j}\left({x}_j\right)$ is derived to compute ${u}_{\mathrm{j}}={F}_{{\boldsymbol{X}}_j}\left({x}_j\right),\kern0.24em j=1:d$ and ${y}_j={F}_{N\left(0,1\right)}^{-1}\left({u}_j\right)$ as shown in Eq. (4).

• 3. The parameter of Gaussian copula density α _Copula = Σ _Copula is estimated using Eq. (13).

The second class of candidate distributions, i.e., conventional distributions, includes five conventional pdfs: multivariate Laplace (MLD), multivariate Gaussian (MGD), and three multivariate Gaussian–Laplace mixtures (MGLD). The MGD is considered as shown in Eq. (24) where the full covariance matrix Σ and mean vector μ are estimated using the maximum likelihood method [16]. Regarding MLD, as the ultimate purpose is to compute the energy test statistic using simulated vectors following MLD, the required vectors ${\boldsymbol{s}}_{{}^{\mathrm{Laplace}}}^i$ are generated using Eq. (25) [50] where ${\boldsymbol{x}}_{{}_{\mathrm{Gaussian}}}^i$ and w _i denote simulated vector following MGD of Eq. (24) and simulated sample following univariate exponential distribution of Eq. (26), respectively. For MGLD observations, the simulated vectors are generated using Eq. (27) where ${\boldsymbol{s}}_{{}^{\mathrm{Laplace}}}^i$ and ${\boldsymbol{x}}_{{}_{\mathrm{Gaussian}}}^i$ denote the simulated vectors following distributions of Eqs. (25) and (24), respectively. In this equation, variable p shows the amount of contribution of Laplace distribution compared with Gaussian distribution in generating z ⁱ. Three multivariate Gaussian–Laplace mixtures are considered with corresponding values of p ∈ {0.25, 0.50, 0.75}, denoted as MGLD with p = 0.25, MGLD with p = 0.50, and MGLD with p = 0.75, for the experimental evaluations.

$$ {f}_{\boldsymbol{X},\;\mathrm{Gaussian}}\left(\boldsymbol{x}\right)=\frac{1}{\sqrt{\left|\varSigma \right|{\left(2\pi \right)}^d}}. exp\left(-\frac{1}{2}\left(\boldsymbol{x}-\mu \right){\varSigma}^{-1}{\left(\boldsymbol{x}-\mu \right)}^t\right) $$

(24)

$$ {\boldsymbol{s}}_{{}^{\mathrm{Laplace}}}^i=\sqrt{w_i}{\boldsymbol{x}}_{{}_{\mathrm{Gaussian}}}^i $$

(25)

$$ {f}_{\mathrm{Exponential}}\left({w}_i\right)=\frac{1}{\lambda } \exp \left(\frac{-{w}_i}{\lambda}\right) $$

(26)

$$ {\boldsymbol{z}}^i=p{\boldsymbol{s}}_{{}^{\mathrm{Laplace}}}^i+\left(1-p\right){\boldsymbol{x}}_{{}_{\mathrm{Gaussian}}}^i $$

(27)

Consequently, eight candidates are considered for multivariate distribution study of speech features, except for amplitude of DFT feature. For the amplitude of DFT feature, two additional candidates CGD and CRD resulting in total ten candidates are considered. Table 2 summarizes the candidates.

Table 2 Multivariate distribution candidates considered for experimental setup

5 Evaluation results

In this section, the experimental evaluation results of the multivariate speech distribution study are presented. To perform the evaluations, 100 sentences, uttered by 11 male and female native English speakers (with New York City dialect region), with sampling rate of 16 kHz from TIMIT database [51] were randomly selected (see Additional file 1). Two experimental setups were considered for evaluations. For the first experimental setup, all speech information of 100 sentences was exploited for computing the statistic of the energy test. For the second experimental setup, phoneme-based evaluations were performed, i.e., five classes of English phonemes were used (fricatives, nasals, stops, vowels, and semivowel/glides). For each phoneme class, the relevant information was first extracted from the 100 sentences used in the first experimental setup and then concatenated to produce one file. As a result, five output files for five phoneme classes were produced. Table 3 shows the content and the number of extracted phonemes of each of the five files. The evaluation results of the second experimental setup benefit the statistical-based speech recognition and synthesis algorithms statistically modeling phoneme classes. As non-speech information (silence interval) may influence the distribution [14], it was removed for both experimental setups. The total duration of the first setup data after excluding silence intervals was 337.84 s. For the second setup, the duration of each file is represented in Table 3. Prior to performing evaluation, the datasets of both setups were segmented in frames with lengths of 20, 30, 100, and 500 ms. For the first setup, the number of frames, N, corresponding to the segment lengths of 20, 30, 100, and 500 ms were 13936, 9291, 2788, and 558, respectively. For the second setup, the number of frames, N, corresponding to the segment lengths for each phoneme class is shown in Table 3.

Table 3 The data set of the second setup of evaluations

The experimental evaluations were performed for time features (T), amplitude of DFT (ADFT), real parts of DFT (RDFT), imaginary parts of DFT (IDFT), DCT, LPC, and MFCC features. Regarding the LPC and MFCC, 10 and 12 coefficients were extracted from frames, respectively. The MFCC vectors were extracted from 23 Mel-frequency filter banks. To set up the energy test, the value of M was taken equal to N. All the reported energy test values were computed at a significant level of 0.01 using a bootstrap method [44, 52].

Figure 1 represents the experimental results of the energy test for ADFT features, concerning the first setup. Fig. 2 illustrates the experimental results of the energy test for other features including LPC, T, MFCC, RDFT, IDFT, and DCT features. Regarding Fig. 2, as the energy test values of some cases were much lower in comparison with the others, they were scaled up ten times to be illustrated better and punctuated by * on the right side of frame length of horizontal axis, e.g., 500 ms *. Moreover, as the energy test values of CGevD for all features were far greater than the others, they were schematically removed from Fig. 2 to have a more comparative demonstration for the small energy test values.

Fig. 1

Values of the energy test for different multivariate distributions (candidates) resulted from ADFT features with frame lengths of 20, 30, 100, and 500 ms

Fig. 2

Values of the energy test for different multivariate distributions (candidates) resulted from a LPC, b T, c MFCC, d RDFT, e IDFT, and f DCT features with frame lengths of 20, 30, 100, and 500 ms

Table 4 summarizes the best-fitted candidate for different speech features and frame lengths according to Figs. 1 and 2 evaluation results. According to Figs. 1 and 2 and Table 4, the following conclusions are conducted:

• The best-fitted candidate in the sense of the energy test for the T, RDFT, IDFT, and DCT features with frame length of 20, 30, and 100 ms is MLD, despite the often used assumption of multivariate Gaussian distribution in the speech enhancement algorithms [8–10], but consistent with the univariate Laplace distribution proposed by Martin [6] and Gazor et al. [14].

• The univariate Rayleigh distribution has been proposed for ADFT feature with a short frame length. Maybe as a consequence, it was expected that multivariate Rayleigh distribution (CRD) would be superior in modeling the multivariate distribution of ADFT; however, the energy test evaluation results proposed the CLD as the best-fitted candidate for frame lengths shorter than 500 ms.

• Regarding statistical modeling of ADFT features with short frame length, although CLD and CLID are both Laplace-based distribution, CLD was proposed as the best-fitted candidate. As the copula density function c _X (.), which models inter-dimensional dependency, is non-unit for CLD and unit for CLID, the superiority of CLD over CLID shows how the modeling of inter-dimensional dependency contributes to the proper multivariate statistical modeling.

• Increasing frame length to 500 ms caused the best-fitted candidate corresponding to ADFT, RDFT, IDFT, and DCT features to be shifted from either CLD or MLD toward MGLD. This finding suggests that the Gaussian distribution contributed to the actual multivariate distribution of those domains when the frame length sufficiently increased, which is also supported by the central limit theorem. Similarly, varying the best-fitted distribution for LPC features from MGLD (with p = 0.25) to MGD verifies this contribution, too.

• The best-fitted candidate for the MFCC with different frame lengths is MGD, consistent with the assumption of multivariate Gaussian distribution used in most speech recognition algorithms [2, 3].

Table 4 Best-fitted multivariate distribution in the sense of energy test for different features and frame lengths of speech signals

According to the conclusions, representative frames of speech signals containing T, RDFT, IDFT, or DCT features that are often used in statistical model-based speech enhancement algorithms [9, 10, 28] can be better statistically modeled by the MLD than MGD distribution. Furthermore, if the statistical-based algorithm exploits MFCC and ADFT, the energy test proposes MGD and ADFT, respectively.

Tables 5, 6, 7, 8, 9, 10, and 11 present the evaluation results of the energy test for the second experimental setup. In each table, there are 15 blocks surrounded by bold lines belonging to each phoneme class with a determined frame length. Each block in these tables contains four cells, as shown by Fig. 3. The A and B cells show the first and the second best-fitted candidates, respectively, and C and D cells indicate the energy test value corresponding to the first and the second best-fitted candidates, respectively.

According to Tables 5, 6, 7, 8, 9, 10, and 11, the following conclusions are conducted:

Table 5 Best-fitted multivariate distribution based on the energy test for ADFT coefficients of five phoneme classes in different frame lengths

Table 6 Best-fitted multivariate distribution based on the energy test for LPC coefficients of five phoneme classes in different frame lengths

Table 7 Best-fitted multivariate distribution based on the energy test for time coefficients of five phoneme classes in different frame lengths

Table 8 Best-fitted multivariate distribution based on the energy test for MFCC coefficients of five phoneme classes in different frame lengths

Table 9 Best-fitted multivariate distribution based on the energy test for RDFT coefficients of five phoneme classes in different frame lengths

Table 10 Best-fitted multivariate distribution based on the energy test for IDFT coefficients of five phoneme classes in different frame lengths

Table 11 Best-fitted multivariate distribution based on the energy test for DCT coefficients of five phoneme classes in different frame lengths

Fig. 3

Values of cells in each block of Tables 5, 6, 7, 8, 9, 10, and 11

• The univariate Rayleigh distribution has been proposed for statistical univariate modeling of ADFT feature. Maybe as a consequence, it was expected that multivariate Rayleigh distribution (CRD) would be also superior in modeling multivariate distribution of ADFT; however, the evaluation results proposed MGLD, CGD, or CGevD as the best-fitted candidates.

• The best-fitted candidate in the sense of the energy test for all phoneme classes in T, RDFT, IDFT, and DCT features with different frame lengths was either MLD or MGLD (with p ∈ {0.25, 0.50, 0.75}). In particular for frame lengths of 20 and 30 ms, which are mostly exploited in speech processing, either MLD or MGLD with p = 0.75 dominated. As a consequence, the Laplace distribution contributes more compared to the Gaussian distribution in the statistical multivariate modeling of T, RDFT, IDFT, and DCT features with short frame lengths.

• The best-fitted candidates for different phoneme classes with LPC feature was mostly MGD or MGLD with p = 0.25. As a consequence, the Gaussian distribution contributed more in the statistical multivariate modeling of LPC feature compared to the Laplace distribution.

• As the first or second best-fitted candidates for different process domains of a phoneme class with a fixed frame length mostly varied between MLD and MGLD (with p ∈ {0.25, 0.50, 0.75}), the statistical modeling of phonemes with a mixture of Gaussian and Laplace distributions is proposed.

• The best-fitted candidate for most phoneme classes with MFCC features extracted from frames of length less than 500 ms is MGD, consistent with the assumption of multivariate Gaussian distribution used in most speech recognition algorithms [2, 3].

• The only copula-based distribution proposed by the energy test evaluation results was CGD for statistical modeling of the stop phoneme class in ADFT domain with frame lengths of 20 and 30 ms, and CGevD for semivowel/glide with frame length of 500 ms.

• Based on the evaluation results, in the sense of the energy test, the copula-based distributions using IFM method were mostly overcome by conventional distributions in the second experimental setup. As only one of parameter estimation methods of copula-based distribution, IFM method, was taken into account in the experimental evaluation, and the IFM method ends up a sub-optimal solution for parameter estimation, it is difficult to have a generic conclusion on copula-based distribution’s benefit in statistical modeling of speech frame. One of future work perspective might therefore be to study the power of statistical modeling of copula-based distribution of speech frame using optimal parameter estimation methods.

• In some cases, where the energy test values of the first and second best-fitted candidates are almost the same, there is almost no superiority in the sense of energy test between the first or second best-fitted distributions, e.g., the case of fricative phoneme in time domain with different frame lengths in Table 7.

6 Conclusions

In this paper, the multivariate distribution of speech features in various domains, e.g., time, DFT, DCT, MFCC, and LPC, was studied and a framework was proposed for exploring the best-fitted distribution among different candidates. Ten plausible candidates including five conventional distributions, e.g., the multivariate Gaussian, multivariate Laplace, and the mixture of Gaussian–Laplace distributions (in three forms), and five copula-based distributions with marginal Laplace, independent marginal Laplace, Rayleigh, Gamma, and generalized extreme value (GEV) distributions were considered to explore the effect of feature type, phoneme class (for English language), and frame length on the distribution.

The evaluation results of the test energy showed that the multivariate Laplace distribution statistically better models time and DFT features of speech signals compared to the multivariate Gaussian distribution. For the amplitude of DFT features, the copula-based distribution with marginal Laplace distribution was proposed as the best-fitted candidate. For the MFCC features, the best-fitted candidate was MGD, consistent with the assumption of multivariate Gaussian distribution used in most speech recognition algorithms. For multivariate statistical modeling of different phoneme classes, the first or second best-fitted candidates for different domains (and also for different frame sizes) mostly varied between MLD and MGLD (with p ∈ {0.25, 0.50, 0.75}), i.e., a mixture of Gaussian and Laplace distributions. The future work of this study can lead toward the development of statistical speech processing algorithms exploiting Laplace, mixture of Laplace and Gaussian, or copula-based multivariate distribution, depending on the feature type, phoneme class, and frame length.

Although the copula-based distribution was proposed as the best-fitted distribution for the modeling of amplitude of DFT, it is not the case for other features. It means that the copula-based approach requires more investigation in numbers of ways. First, the practical issues, e.g., the computational cost and the lack of sufficient amount of data for parameter estimation of some phoneme classes, e.g., stops, are needed to be considered. Second, as the IFM method used for parameter estimation of copula-based distribution ends up in a sub-optimal estimate, developing an optimal parameter estimation method for large vector dimensions is needed to have a fair evaluation of the copula-based distribution power in the statistical modeling of speech signals, e.g., compared to the optimal parameter estimation method used for MLD and MGD.

Appendix

The quantity ϕ, the energy, is defined as the difference between two pdfs ${f}_{{\boldsymbol{X}}_0}\left(\boldsymbol{x}\right)$ and f _X (x) by

$$ \begin{array}{c}\phi =\frac{\mathtt{1}}{\mathtt{2}}{\displaystyle \int {\displaystyle \int \left[f(x)-{f}_{{}_0}(x)\right]\left[f\left({x}^{\prime}\right)-{f}_{{}_0}\left({x}^{\prime}\right)\right]}}R\left(x,{x}^{\prime}\right)dxd{x}^{\prime}\\ {}=\frac{\mathtt{1}}{\mathtt{2}}{\displaystyle \int {\displaystyle \int \left[f(x)f\left({x}^{\prime}\right)+{f}_{{}_0}(x){f}_{{}_0}\left({x}^{\prime}\right)-2f(x){f}_{{}_0}\left({x}^{\prime}\right)R\left(x,{x}^{\prime}\right)\right]}}dxd{x}^{\prime}\end{array} $$

(28)

where the weight function R is a monotonically decreasing function of Euclidian distance and the integrals extend over the full variable space [44]. As the product of same distribution occurs in the first and second terms, it is not necessary to draw two different samples of the same pdf, and thereby, the first two terms can be neglected. The remaining third term has the form of expectation value of R and can be computed from the mean of all combinations ${\boldsymbol{x}}_{i=1}^N=\left\{{\boldsymbol{x}}^1, \dots,\;{\boldsymbol{x}}^i, \dots,\;{\boldsymbol{x}}^N\right\}$ following an unknown pdf f(x) and simulated Monte–Carlo samples ${\boldsymbol{q}}_{j=1}^N=\left\{{\boldsymbol{q}}^1, \dots,\;{\boldsymbol{q}}^j, \dots,\;{\boldsymbol{q}}^M\right\}$ following f ₀(x), thus the energy statistic can be given by Eq. (29).

$$ {\phi}_{NM}=\frac{1}{N\left(N-1\right)}{\displaystyle \sum_{t>i}R\left(\left|{\boldsymbol{x}}^i-{\boldsymbol{x}}^t\right|\right)}+\frac{1}{M\left(M-1\right)}{\displaystyle \sum_{j>n}R\left(\left|{q}^n-{q}^j\right|\right)}-\frac{1}{NM}{\displaystyle \sum_{i=1}^N{\displaystyle \sum_{j=1}^M}R\left(\left|{\boldsymbol{x}}^i-{\boldsymbol{q}}^j\right|\right)} $$

(29)

It is noted that since the evaluation of ϕ requires a summation over integrals, which is typically difficult, f ₀ is preferred to be represented by a set of samples generated through a Monte–Carlo simulation.