It's slightly more complicated. The main idea of special relativity theory is that you can't compare "time" at two separate points of "space". E.g. it makes absolutely no sense to compare age of a twin left on Earth with the age of a twin on a moving space ship as long as they are not at the same point in space. So if you really want to compare age you will have to turn your space ship around and return to Earth. Now, that part is covered not by special relativity, but general relativity. Here we need to consider effects of a gravitational field (any acceleration is equal to gravitation). The most interesting question in the twin paradox is what's the difference between space ship and Earth. Why it's the ship which is accelerating, but not the Earth. Why Earth is considered an inertial system (according to the Newton's first law), but a space ship making U-turn is not.