In noise robust speech recognition, for PMC (parallel model combination) method, the performance of the combined model can approach that of the model matching the noisy environment theoretically, so it is an important noise robust speech recognition research field. In this paper, a novel feature MFCC_FWD_BWD, which is based on forward-backward difference dynamic parameters, is presented to satisfy the requirement that the feature construction matrix is invertible for PMC. On this condition, a novel structure model named parallel sub-state hidden Markov model (PSSHMM) is presented for PMC and each state of this model has parallel sub-states with transitions. In experiment, PSSHMM achieves good results under each kinds of noise and each levels of SNR, especially for non-stationary noise, its robust performance is also excellent.