Here's a problem that's more difficult to solve than it first seems:
You have a traditional land line that serves all the phones in the house. You switch to VOIP. Your phone that's connected to the router/adapter works fine. So how do you get all the other phones to connect to the same line? The not-so-easy way is to rewire the phone lines in your home. There are plenty of instructions available on the Internet for doing this, but unless you have technical savvy, you could end up making more of a mess. Consider hiring someone who knows how to wire to come in and help.
Or you could do it an easier way: For about $100, you can buy a wireless home phone with extensions you can put around the house. Connect the main phone to the adapter and the wireless extensions will connect themselves to the main phone. It's cheaper than hiring someone to do the wiring and you can get everything up and running with one quick trip to the mall.