Most of the tracks we see are spurs, built to service a factory, mine, produce area, stockyard, or some place that needed freight service. To go from point A to point B, the car you get on at A travels the spur back to the main line. Then you or the car has to switch to a main line train, then down the line to the spur going to B, switch again, finally up the spur to B. That's a lot of time and equipment to get you some place you could easily drive to in far less time
They'll only run passenger service if there's a huge demand and they don't get in the way of the freights. You don't see mixed trains like the old days, with a mix of freight and people cars, except Alaska. The only place passenger trains really work is the main lines between big cities and commuters in and out of cities, where they can count on a certain level of ridership most every day.
I always though that trans Canada would be cool, but like the states, a big hunk is flat and not much to look at, then the mountains send you into sensory overload trying to look in nine directions at once.
