Corrections to Published Values of Frequency Sampling Filter Transition Coefficients

TL;DR

This paper corrects and updates the published optimal transition coefficients and peak sidelobe levels for frequency sampling filters, providing more accurate values than previously reported in key DSP references.

Contribution

It provides corrected and optimal transition coefficients and PSL values for frequency sampling filters, addressing inaccuracies in prior published data.

Findings

Corrected transition coefficient values for various filter configurations.

Updated peak sidelobe level (PSL) values based on new calculations.

Identified discrepancies in previously published filter parameters.

Abstract

Tables of optimal transition coefficients and peak sidelobe level (PSL, in dB) associated with frequency sampling filter (FSF) design were published by Rabiner et al. (Jun 1970), and reproduced, for example, in the book Digital Signal Processing by Proakis and Manolakis (4/e, 2007). A set of values are also given in Appendix H of Understanding Digital Signal Processing} by Lyons (3/e, 2011), but there are significant differences between these two sets. For example, for $N=16$ and BW$=4$, two different transition coefficient values have been reported, viz., $0.38916626$ (Rabiner, et al.) and $0.34918551$ (Lyons). Neither is optimal, for we find the optimum value to be $0.40474097$. The published values of the corresponding PSLs were also found to be incorrect. In this paper we give the optimal values of the transition coefficients and PSL values as estimated by our program for the lowpass and bandpass filters listed in Rabiner et al. and Lyons.