Why should we believe that ALL of the time-turners in the world
were kept in one little cabinet?
That's even less realistic.

And no, I don't think you could do too much damage unless you spend some time thinking about it.
If you went back and changed things too much, you wouldn't know that you had done it, so you couldn't do it. It has to be a near-miss, and you have to think of going back to change something that never happened.
It's not a sport for the shallow-minded.

The Buckbeak story worked because they believed he died, when he never did.
Dumbledore was there and knew that Buckbeak survived, but he didn't tell.