Bless you and hugs for you for taking this pony away from the horror that he's likely experienced.
Very early in my career, after a lot of ranch work with pretty rank horses, and then with racehorses gone sour (particularly stallion work), I learned somethings about these fighters.
A general statement about them to establish where one might come from initially with them is in order here: they are often more full of fear than other horses, or ponies, they lack confidence and much like human bullies make up for it with threats, aggression, and if they can, to violence.
My own work rehabing such horses, and watching "hands," breaking horses on my uncle's ranch back in the 50's (yep, I'm that old), gave me a lot to consider over time.
I've come to the conclusion that horses are not very confident creatures by nature, very cautious, and they tend to use what first works to make things they fear leave them alone.
My speculation is that this pony received some abuse, but that is not necessary for this violent pattern of behavior to emerge. With no cause other than feeling anxious and reacting to it with a bit of aggression and successfully driving off the object that provoked the anxiety the pattern can begin.
A few more positive rewards (which it is when a horse is aggressive and people retreat from that aggression) and behaviorally the horse or pony is now "trained," to be aggressive.
There is another field to look to for some answers.
Mustang taming for the auctions.
People don't just come to the roundups and pick one out, have it roped down and drug into a truck. These horses are gentled, halter-broken, ground handling familiar and tolerant. And the number one rule (as far as I can see, as it was for me as the broken bones mounted up over the years) is safety for the human handler first.
So they begin most often in a pen, with the mustang inside, and the human outside, and a bamboo touch pole with a rag or feather on the end. Regardless of the mustang's response the end of the touch pole is introduced and reintroduced, gently of course, until it accepts the touch.
You may well know some of these things yourself. You don't write like a novice horse person, but bear with me. Something new and useful may turn up in my narrative.
After than first touch, that comes from approach and retreat methods, the area that has been tolerant of touch is expanded, again with approach retreat. You go only so far as you get the reaction you wish. That is that the horse, or pony, stay positive with the pole and rag. Always stopping before you get a negative reaction. Then restarting at the last "tamed," spot and working out from it.
Why treat this pony in this way, like a mustang? Because the reactiosn you are seeing are the same ones that mustangs have naturally, flight or fight.
The wonderful thing about mustangs is that they usually learn so very quickly and get over their fear, as they have not been trained by human presence to be afraid and to fight.
Your pony, sadly for the little guy, has probably been trained. "humans bad, bite humans, humans go away."
I'm sure you'll study him, and note what triggers his biting or striking and work with that.
We have, we human horse handlers, for thousands of years more often than not assumed the worst of the horse and any unwanted behavior. The change has come for many in recent years, say the last three decades or so, that the horse may not be hostile at all, but simply expressing his fear, the primary feeling, with the secondary behavior of fight and resist.
So if we think in those terms, and presume the horse would rather, over all, live a peaceful gentle life and frame our tactics around that goal, and approach with gentle nurturing and kindness in our hearts and minds, and hold to that, even when the little pony is rective and still wants to fight us, eventually he will come to be able to read our true intent.
A few little practical things one can do. Secrets of the so called "Horse Whisperers," most of them:
The rag you touch him with? Breath heavily into it, thinking good thoughts, how beautiful his is, how brave he is, how much you wish to nurture and protect him like his mother would have when he was little, etc. Yes, I'm quite serious. I've exerimented a quite about with this.
Horses, as predator food, have learned to read the thoughts and emotions from other animals breaths...think about how horses greet each other. The are even clearing in reading other creatures than we are, and mostly through breath
And yet another thing. Do not be silent approaching, with, or leaving the pony. Predators make a point of approaching with great stealth. Do not approach slowly, or fast, just a medium pace, and the same with movements.
And another; think 'mother mare.' That is to use your body like a mare with her foal. If something startles your pony, move between him and the object of fear. Go, if you can, to the object and touch it with your hand (your hand, and arm, to a horse, is more your head and neck for inspection and other purposes ... we don't use our nose and mouth for this, as the horse can see, so they think our hand and arm are our touching sensing "nose and mouth.")
Do not be afraid to use your hand and arm to signal "no" to the pony. That is, should he strike or bite in your direction, make the mother mare motion with your hand and arm, swing it high over above shoulder level, your own shoulder, of course. ONCE. Don't keep swinging it for a single aggression move. Just once. MORE means you want to fight, in pony and horse language.
It does mean too you want to play, but context is everything. One "cue," at a time for one "offense" at a time.
You need no more than that. Forget punishment. Even a water bottle. The arm wave in horse language is very clear as a neck and head gesture of "don't DO that."
Your pony has probably had plenty of "punishment," that is negative consequences heaped on him and he's probably a master at steering his own response from suppression of a behavior to outright aggression again. He'll get around that water bottle trust me. And likely change his tactics. Instead of a bite or strike, a kick or a charge and shoulder hit.
Leave the world of negative consequences behind, and of course the world of punishment too.
If you don't click and treat (that is use positive behavioral conditioning, operant conditioning) then now would be a wonderful time to learn it.
Learn how to charge the clicker. That is to associate the sound with a toothsome yummy reward. You can pitch him the reward to stay safe.
As soon as he alerts on the clicker, and comes to the treat reliably - usually no more than 15 or 20 minutes of slow work, you can look at his behaviors, pick the ones you want (such as standing quietly when you lift the touch pole to shoulder level) and click and treat.
The pony, I guess very like us all these days, needs to find peace, and once found will want it for life, as I do.
Give him peace every change you can.
Find out how to give him pleasure in ways besides the treats too. Locate some scritching spots. They will often reveal themselves during the first stages of taming him to touch. You'll spot that he not only holds still for a particular spot to be touched but moves it into the touch, and later, will "present," that spot to the touch pole and rag. Bingo, you have another "reward," to give when you click.
I do not agree with the "herd buddy," taming method any more. I once did. Nor with the herd leader method either.
It's the behavior of the mother mare that we should study. Even at very high levels of horse training for performance. The mare traits work.
That is who to study to learn how to tame and socialize horses to humans, even badly abused horses and ponies. Study the mother mare. Take them back to their mother.
When I worked with emotionally disturbed youth I got my first inkling of this concept. I learned to mother 15 year old kids. So called delinquents. And it worked very well indeed.
It does for horses and ponies. In fact even those not abused and traumatized respond to mare mothering.