recently there was a funny debate on the fb group where somebody states that you always had to read disk at the same speed they were burned.
it turned out to be a rare case of an old drive not beeing able to read a disc which was burned at 50x.
you have to assume that the faster an optical media has been burned, the more inaccurate the "printings" are; imagine the holes get increasingly more rounded corners the faster you burnt it, that should be the right picture.
if you want to take care of that or not is everyones personal decision. if the discs are for yourself and you have a desktop mac, it is probably easier to get a better drive so that you can read everything. if you are giving media to others, who have a cube or pismo, it might be wise to burn slower.
the same is true for audio CDs, btw. thats why you never gave audio masters as media but always as disc image file to your pressing plant.