That's exactly it. How else do you think you delete files? The hard drive is not write-once storage - why even bother then? ("Write-once" is like CD-R or DVD-recordable - you can only save to it once, and then the data is there forever.)
What the hard drive does is set a pointer, and that pointer is where your computer starts when you find a file. It goes from one area of data to another to another, until the file is amassed and can be opened. That is the point of defragmenting - it puts the files back together, so the drive isn't going from here to there to here to there.
When you delete a file, the pointer is deleted - so the balance of the file is around somewhere, but, when you save a new file over that pointer, the old file is not accessible.
Now, that is the essence of data recovery - finding files where the pointer is gone (both for drive failures and for law enforcement). Likewise, that is how digital "shredders" or "erasers" go - they put junk on ostensibly unused areas (which are actually data that no longer are linked to).
Clear as mud? Delete away, and watch the space open up! Just make sure it's stuff you REALLY don't need - but, if that's class stuff, movies, music, and so on, it's not critical.
