Endurance sports activities provide a very different physical experience from adrenalin-high sporting activities. The rewards of each are likewise different, I believe. I don't cross-country ski, but I was a competitive cross-country distance runner in high school and college. That involved a lot of individual, solitary training runs, at least once a day. I enjoyed the solitude, the concentrated quiet of the mind that such training requires. Where I live (Anchorage, Alaska), cross-country skiing is a major winter activity enjoyed by many hundreds of people. There is a big woodland park with groomed ski trails directly across the street from my house, and it is used heavily by skiers all winter. I personally know a lot of people who do this.
And the fitness aspect is huge. People who run, ski, row, swim or do any other kind of endurance activity generally derive great physical satisfaction simply out of staying fit.
If you're an adrenalin-junkie and get your kicks out of things that provide that jolt, you probably won't enjoy cross-country skiing. Which is okay.