javascript - How do you normalize weights q-learning with linear function approximation -

i developing simple game program show q-learning linear function approximation. screen shot

in game, there uncountable state. have consider many factors player's position, speed, , enemy's position (there 12 ~ 15 enemy objects). ended changing algorithm using table use linear function approximation.

i decided around 20 ~ 22 features.(constant, player position, player speed, of enemies position). , there

after implementing algorithm, got stuck in problem.

weight value overflowed in few second after running program. found didn't normalize features , weight.

it easy normalize feature value because each feature has bound . however, wasn't enough normalize feature value. still end overflow.

my problem how normalize weights.

below code implement normalize features.

//f feature       f[0] = 1;     f[1] = this.getnormminmax(this.player.x,0,cc.winsize.width);     f[2] = this.getnormminmax(this.player.vel,-80,80);      for(var i=0; i<poolist.length;++i)     {         f[3 + 2*i] = this.getnormminmax(poolist[i].x,0,cc.winsize.width);         f[3 + 2*i+1] = this.getnormminmax(poolist[i].y,0,cc.winsize.height*3);     } 

and below code updating weight without normalization.

for(var i=0; i<this.featuresize; ++i)         {             var w = this.weightarray[this.doaction][i];             this.weightarray[this.doaction][i] =                 w + this.learningrate*(this.reward + this.discountfactor*maxaction - this.updateqsa) * f[i];         } 

it seems you're using linear regression without regularization, , there collinear features. try adding l1 or l2 regularization (use ridge, lasso or elastic net models).