When i opened my 360 to replace the thermal paste on the gpu and cpu, i noticed that the thermal paste on the gpu was on the 2 big square thingys and there was also some thermal paste on the tiny rectangle ones. So i've cleaned all of it off and was wondering if im suppose to put the thermal paste on the tiny rectangle ones too.

No, that's just the manufacturers being lazy, and using way too much of that bullshit thermal paste to begin with.