The ISS routinely eats micrometeorites or other tiny debris and also maneuvers to avoid larger trackable junk on occasion. If it was travelling at half the speed of light, it would be a bit more problematic.
We don't know how much of this stuff exists in the interstellar space. It's possible that a probability to hit something there is much less than ISS hitting something big enough to destroy it, even with all radars and maneuvers. So I wouldn't consider it as an argument against interstellar travel, not yet.
I'd say it's one of many potential problems. We've already observed a couple interstellar comets, right? So we actually already know that there are objects out there. Certainly it wouldn't hurt to send some laser accelerated probes out there and see what we find, but human travel seems like pure sci fi.
