Model Order Selection Based on Information Theoretic Criteria: Design of the Penalty Article Swipe
YOU?
·
· 2015
· Open Access
·
· DOI: https://doi.org/10.1109/tsp.2015.2414900
· OA: W2056601790
Information theoretic criteria (ITC) have been widely adopted in engineering\nand statistics for selecting, among an ordered set of candidate models, the one\nthat better fits the observed sample data. The selected model minimizes a\npenalized likelihood metric, where the penalty is determined by the criterion\nadopted. While rules for choosing a penalty that guarantees a consistent\nestimate of the model order are known, theoretical tools for its design with\nfinite samples have never been provided in a general setting. In this paper, we\nstudy model order selection for finite samples under a design perspective,\nfocusing on the generalized information criterion (GIC), which embraces the\nmost common ITC. The theory is general, and as case studies we consider: a) the\nproblem of estimating the number of signals embedded in additive white Gaussian\nnoise (AWGN) by using multiple sensors; b) model selection for the general\nlinear model (GLM), which includes e.g. the problem of estimating the number of\nsinusoids in AWGN. The analysis reveals a trade-off between the probabilities\nof overestimating and underestimating the order of the model. We then propose\nto design the GIC penalty to minimize underestimation while keeping the\noverestimation probability below a specified level. For the considered\nproblems, this method leads to analytical derivation of the optimal penalty for\na given sample size. A performance comparison between the penalty optimized GIC\nand common AIC and BIC is provided, demonstrating the effectiveness of the\nproposed design strategy.\n
