I can see a little muscle soreness for people that have never really worked or stretched their muscles in that way, but all they need to do is adjust the width more gradually. It is pretty much low impact otherwise since it is smooth motion similar to a elliptical, vs. a jarring motion where your body weight is slamming into the joints.
I'm talking about straight-up ligament soreness. The regular elliptical machine does not put valgus torque on the knee, not unless the person is really throwing their hips side-to-side.
You are right that muscle conditioning is also a factor for some. Since muscle conditioning directly affects kneecap tracking, I've seen the pain be a lot more serious that 'a little' though, requiring months of patience and pure-strength non-cardio workouts to get past what bike riders used to call "Easter knees".
http://www.livestrong.com/article/528877-bicycling-cold-weather-and-knee-injuries/Notice that neither problem has to do with 'jarring' or impact.
I am not going to say anything about others workout. I am sure I get my share of eye rolls as a 46 year old desk jockey who spends way too many hours working and on the road , enjoys good food and drinks, trying to hold onto some level fitness.
I'm right there with ya. I look at it like Jane Austen did:
"For what do we live, but to make sport for our neighbours and laugh at them in our turn?"
EDIT: Got an hour in on the lateral on Sunday, averaging mid-80s cadence with some bursts of 120+.