Thoughts on Time Travel (unrelated to script)

Recently I have seen several shows and stories that deal with time travel, and being the over-analytical person I am I have thought of a potential flaw shared by just about every theory ever depicted in literature, movies, shows, etc. That is a person's relative position while time traveling. In a lot of shows and movies when you see time travel you see a person's world changing around them while they are staying static. This implies that when in a time machine or using a time-traveling device, you move freely back and fourth throughout the fourth dimension while not affected by stuff going on outside of your little bubble. If that is so, how does the time traveler end up in the same geographical spot as when they began. If they are moving freely throughout the 4th dimension from a singular location then everything should continue moving around them, including the earth rotating, not to mention around the sun at thousands of miles per second and that around the center of the galaxy and so on and so in. Even just one second in time travel should potentially displace a person miles away from where they started. If you jumped for a continued amount of time you you be displaced off the earth and into empty space. That is unless somehow you still have inertia while time traveling (in which case you would still be displaced because your movement would be in a strait line if you were no longer acted on by outside forces). So the only way you could theoretically survive time travel is if while traveling freely in the fourth dimension you were still acted upon by the physical forces that drive the universe. Otherwise when you re-appear it would be in empty space and you would instantly die. So, yeah, that's it for my time travel musings.