Indeed, files stay on your hard drive. There is no manner in which you can completely destroy a file simply by deleting it; overwriting it, however, is another matter. It is true that it is possible to completely eliminate something from your hard drive, but that does not mean what it was was any illegal. No one is going to sue you for downloading ROMs, but that isn't the point.