Abstract—Estimating emotional states in music listening basedon electroencephalogram (EEG) has been capturing the attentionof researchers in the past decade. Although deep belief network(DBN) has witnessed the success in various domains includingearly works in emotion recognition based on EEG, it remainsunclear whether DBN could improve emotion classification inmusic domains, especially in dynamic strategy that considerstime-varying characteristics of emotion. This paper presents anearly study of applying DBNs to improve emotion recognitionin music listening where emotions were annotated continuouslyin time by subjects. Our subject-dependent results usingstratified 10-fold cross-validation strategy suggested that DBNscould improve performance in valence classification with fractaldimension (FD), power spectral density (PSD), and discretewavelet transform (DWT) features and improve performance inarousal classification with FD and DWT features. Furthermore,we found that the size of sliding window affected classificationaccuracies when using features in time (FD) and time-frequency(DWT) domains, while smaller window (1–4 seconds) couldachieve higher performance compared with a larger window (5–8seconds).