I know this thread is titled "White privilege" specifically, but I think that's muddying the waters more than it should be, and I also think you're making a genuine effort to understand better.
The core principle of privilege (as I understand it) that addresses what seems to be your main concern, is the phrase "All other things being equal"...
All other things being equal, being white (in a Western nation) will give you an advantage over those who aren't.
All other things being equal, being male will give you an advantage.
All other things being equal, being heterosexual will give you an advantage.
All other things being equal, being born into a middle-class family will give you an advantage over those who were born into poor families.
I'm a straight white male, as were all my peer group growing up. But they were largely middle class and above, with both parents around, whereas I grew up fairly poor with a single mother (for most of my childhood). All other things being equal, their economic privilege gave them a measurable leg-up on me. I could have done better than I did with hard work, but I also didn't have the role-models to show me how and why.
It doesn't mean you need to feel bad about your successes, or that you didn't earn them. It's just a matter of acknowledging that others didn't always get the same opportunities you may have, based on something they had no control over.
Comparing Tiger Woods to an average white Joe in Appalachia is pointless, because of course he enjoys some privileges they don't. If you compared him to a white golfer of similar fame levels though, you might see some divergence. That's all privilege really is.