Oh, no, Freckles is not the only one. Caspian usually highly dislikes bareback. He has a very weak back, which, although it's getting better, drops down right behind his withers, right where I'd sit. I think the extra pressure of my seat just makes it too painful. Unless we're having a really good day, he has a fit if I try to ride bareback.
It helps ENORMOUSLY to have a very thick bareback pad, so I know the weight distribution is key for him. As far as securing the pad, I have a bareback pad (thick fleece) that I double pad with my memory foam pad, then secure with a western girth. It's not super-secure, but it works fine for me.
I really suspect he'll get much better once he actually builds muscle, but at the rate he's building it, he'll be 40 years old and too senile to care anymore when I get on bareback...
Did Freckles get worried when you got on, when you moved around, or some time else? Because that could give you a clue as to what he finds worrisome. Caspian hates the mounting, and when I scoot my seat into a better position, telling me that it's painful. Once we're moving, he's fine, so I don't think it's the way I'm sitting that's blocking him. Is it the same with Freckles?