Heatsinks at neo motors are there for two reasons. They look cool and they keep temperatures for Neodymium under control as it is much more temperature sensitive than ferrite material is. AlNiCo or Samarium magnets would have the best stability over the temperature changes but they have their own drawbacks which is why they are not really used in Pro Audio drives.
Purpose of those holes in the back plate and the big one in pole piece is to allow venting through the back plate and maximize the air circulation around the voice coil. They also serve on purpose to have equal pressure for cone movement in both directions. If that hole in the pole piece wouldn't exist, it would cause lot of pressure for behind the dust cap to build up in the driver and cause distortion and non linear cone movement. Air gap where voice coil is, isn't large enough to allow that much air to flow through it without restricting it.
Of course the plate set and magnet gets warm over the time as the voice coil heats them up, but as such, they are not (especially the magnet itself) a significant method of cooling the voice coil. Only the bit of top plate and pole piece are, which create the air gap. + any shorting rings close to voice coil. But majority of the cooling is done with air and that is why size of the magnet is irrelevant for cooling.