machine learning - Increasing training data doesn't reduce overfitting in CNN -

i working on cnn classification problem:
using cnn classify audio emotions 6 classes (anger, disgust, fear, happy, sad, neutral). using emodb dataset, , input features fourier transforms [256*36]. , cnn network has 3-4 convolutional layers max-pooling each conv, plus 1 connected layer. learning curve shows large gap between training , validating loss, indicating severe overfitting. , best validating accuracy can between 75% 80%.

learning curve here 1 of learning curve got. black , blue ones training accuracy , loss, respectively. other 2 validation accuracy , loss. validation result don't improve anymore when training loss goes 0.

i have tried augment data set, add 50% dropout fc layer, add l2 regularization fc layers, , use learning rate decay policy (as 'inv' in caffe). gap still remains.

is because dataset small?
have around 500 audios in total, , extend around 6,000 samples. when increased data 15,000, gap still large. 15,000 still small dataset cnn?

would because data augmentation process introduced error?
original dataset consists of around 500 audios of different lengths, ranging 1s 6s. randomly extracted samples of 1.2s. longer durations is, more samples get. , can have more 15,000 samples training now. thinking long audios, sample of 1.2s lose information, , may not able represent feature of corresponding emotions. best method can come with, because cannot use rnn or hmm deal data reason.

would possible feature computation went wrong? (even though have check several times) have tried mfsc features [120*40], 2 feature set have similar overfitting problem..

or because network not enough? thought more complicated networks introduce more overfitting.. simple ones didn't show performance.

even though list many reasons overfitting, cannot figure out ones real factors influenced performance. there way know part went wrong? or advice reduce overfitting?

thank you!