It IS possible that it's your router, and in most cases, more likely that it's your router or a faulty Ethernet cable. What you can do to trouble shoot is, when you're having problems, connect your computer to your modem for a short amount of time. I say a short amount of time because this essentially removes your main source of protection from Internet bad boys like smt.
Anyway, if the problem persists even when taking it out of the equation by connecting directly to your modem, it leaves 3 possibilities. A faulty cable, a faulty modem, a crappy ISP. Get to troubleshooting!