I have only heard it used in golf when a golf club hits the ground and pulls a chunk of the ground up leaving a small hole.
Google image search "Divot" for pictures.
It's a missing chunk from a surface, as DrKno pointed out. Typically used to describe a missing chunk of lawn following my swing with a driver or a chunk of missing roadway that isn't as big as a pothole.