I'll jump in this one, as the topic of the dopamine reward system / implications of operant conditioning has been something I have worked on in my "day job", though related to humans. I first brought this up in my first experience on AND, but have learned much since then.
Key points: operant conditioning (especially +r with variable reward schedules) is VERY powerful but has a potential side effect in SOME contexts.
This is how I describe it to people dealing with people:
If using operant conditioning (like c/t) for behaviors that are NOT rewarding on their own, it is almost all benefit with no downside. The potential side-effect happens when the behavior you are rewarding IS an intrinsically-rewarding (meaning pleasurable all by itself) one. AND is focused ON trying to help encourage more of the intrinsically rewarding behaviors: things horses do for fun, to show off, play, etc. so here we fall into the tricky zone for operant conditioning.
More than forty years of research on humans, mostly, have formed the leading theory of motivation today under the label of Self Determination Theory. In the US and the UK, at least, (don't know about other countries) it is the main theory of human motivation with respect to extrinsic and intrinsic motivation, and provides a continuum ranging from NO motivation ate rough four levels of extrinsic motivation and finally to intrinsic motivation on the end. ( The leading researcher for this was Edward Deci).
Intrinsically motivating behaviors are the most robust -- the most likely to be repeated.
Extrinsically motivated behaviors -- those that are NOT on their own rewarding/pleasurable -- are NOT robust and will often disappear when no longer reinforced.
But the REALLY tricky one (and where I have gone so wrong with my horses) is THIS: when you EXTRINSICALLY reinforce an already INTRINSICALLY motivating behavior, you can replace/crowd out the INTRINSIC pleasure and exchange it for EXTRINSIC reinforcement. In other words, extrinsic rewards can -- in some circumstances -- DEMOTIVATE the previously intrinsically pleasurable behaviors. This has been demonstrated (in hundreds of studies) with humans, but also the first experiment that started most of this work was on monkeys: they enjoyed solving wooden puzzle toys, but when given a special treat for solving the puzzle, they slowly lost interest in the puzzles.
Possible ways to avoid this bad side-effect: (very little research on this with animals, so I am guessing now)
* Use c/t (or any other reward system) with these intrinsically motivating behaviors ONLY for the teaching/marking effect, and up the criteria much more rapidly than you would for a NON rewarding activity. In other words, use the rewards JUST enough to say, "yes, that is in fact what I would love for you to do" and then, hopefully, the rewarding effect of the behavior will kick in and the horse will want to do it just because he loves doing it. We have of course all discovered those of these with our horses, since AND is all about finding and encouraging the things they already love doing (or could love doing) just being a horse.
In this way, the reward functions more as feedback/information, and there seems to be evidence that when rewards are primarily information (rather than reward), the DEmotivating side-effect is minimized.
* Use c/t (or any other reward system) in a way that acts as a RECOGNITION rather than a reward. This is also tricky... Since it is all about how the reward receiver feels about it. For example, when someone studies martial arts, earning a black belt is NOT seen as a reward but rather a recognition of really hard work and "awesomeness"
. No demotivation. But it WOULD be (according to the research) DEmotivating if a child was given a reward each time they performed a simple movement they were learning. It is the DO THIS AND EARN THAT form that -- when used on behaviors that are rewarding on their own -- causes the demotivation.
Also, this demotivation in humans is subconscious. It is as not a decision the person makes to say "because I am being rewarded for doing this, it must not be rewarding on its own". It is, the theory for now assumes, somehow a process of the brain that -- without our awareness -- says, "I am doing this for the reward, not because I love doing it or it is worth doing."
I was once skilled at c/t and carefully tuned and timed reward schedules and believed I was successful at shaping and building incremental behaviors. BUT I had a horse that responded in the most extreme way when I crossed over into rewarding him for the behaviors that SHOULD have been intrinsically rewarding: like moving with energy. ALL use of c/t or any form of operant conditioning with +r produces an initial SPIKE in motivation as the dopamine system kicks in, REGARDLESS of whether it is an intrinsically rewarding behavior or... Something that is not on its own pleasurable. If the behavior is not in itself pleasurable on its own, then the reward system is a big help, and continues to be a big help.
So you do not see the DEmotivation potential at first. When it happens, it can look like a plateau (or in my case worse, a total backwards slide in the behavior). It can begin to feel as if the horse is simply "phoning it in", going through the motions but NOT demonstrating a love of the behavior itself. This can be difficult to see because the animal is simultaneously STILL INTERESTED in the rewards, and possibly still enjoying the "game" of getting rewarded, BUT he is not finding the behavior/activity being rewarded especially stimulating and motivating.
We often see this as just a difference between horses rather than a bad side-effect of operant conditioning when used for motivating behaviors.
I ended up with Draumur in such a demotivated state that he was diagnosed with EPM. Obviously I eventually turned this around, as you could see in his "fancy trot" video. But it was such a dark time for us that it prompted me to spend a lot more time figuring out what might have gone wrong. At first I had no idea there was such a strong body of research on the DOWNSIDE of extrinsic rewards, and how subtle it can be... After all, it is almost impossible for us to imagine ANY scenario where rewards will not HELP.
What I try to do now that is different from what I did before:
* if the behavior is NOT likely to be FUN TO DO on its own for a horse (fun as in AND behaviors that are those demonstrated in play or flirting, etc.), then I may still use c/t for its MARKING/TEACHING behavior, but the reinforcement schedule is MUCH less reinforcing than for other behaviors. If the behavior is just hard work but not necessarily fun on its own (a lot of lateral work can fall into the Safe-to-Reward category because for a horse it may feel like weight-training in the gym... Good to HAVE DONE, not really good to do while doing it.)
* If the behavior is NOT necessarily fun to do, then reward like crazy (following operant conditioning principles) as it is the dopamine reward system that makes it worth doing. Again, this is like us going to the gym vs. skiing -- two different difficult physical activities, but one is (for most people) a necessary but not fun while doing experience while the other is pleasurable by itself, even though both are strenuous.
* If the behavior is REALLY physically challenging AND intrinsically pleasurable, like really high energy high speed or high prancey things like passage, THEN it is possible (I am guessing now) that the reward takes on a DIFFERENT role. -- but only if given for the REALLY big efforts -- and functions as a recognition rather than reward. In other words, it may be like the horse saying, "DAMN RIGHT I DESERVE THIS FOR MY AWESOMENESS. You are a perceptive, smart, respectful person for having noticed this..." There is a related bit of research which may be working here, which is that animals given two different but equally-favored rewards will begin to prefer the one given during the physically strenuous work vs. the one given during less strenuous work, even though initially both were preferred equally. They tested this via brain scans to show that the animal actually began perceiving the treat given for harder work as MORE TASTY.
* So, the attitude I try to take now is that I am rewarding a spectacular athlete for spectacular work the way you would give a gold medal rather than "oh what a good boy". I am looking not for the higher *quality* but the higher big show-off effort to reward.
* if the behavior is not going to be a big awesome look-at-me behavior, then all normal operant conditioning rules apply. I use c/t for these things as most do.
* NOW the controversial part... I have re-introduced pressure into my training for these big intense things, but only AFTER I developed the relationship with my horse where instead of "owner" or even "trainer" I took on the role of "Olympic Athlete Coach". When doing things I never wanted to do that someone else wanted me to do, the last thing I wanted was pressure, and rewards at least made it tolerable. BUT... As a college (and later) competitive athlete, having a coach that was not abusive but still REALLY pushed and pressured you to to more than you thought you could -- but made you grateful they pushed you -- was suddenly a BENEFIT and PLEASURABLE. They made you better than you were before, in ways you found gratifying, and yet they did it through pushing you.
I do not know if this is an actual useful idea or if it simply worked well with MY two. However, I have spent the last year with one of the most spectacular trainers I have ever seen, and he consistently gets horses to no just step up to his expectations but appear to really really love it... They all end up acting far more proud of themselves, despite his use of pressure. Or... Risky as this may be... In part BECAUSE of the pressure.
However, he does NOT use pressure at all with horses that are already proud and sensitive and extremely athletic. He simply helps them. It is with the horses that are NOT showing a lot of pride that he does use pressure, but in a way that says, "You are a fabulous athlete. You are the best horse in the herd. Let's show them what you can DO!" and sure enough, even the least motivated horse starts acting like, "Damn right, about time SOMEONE noticed!" and then they start trying really hard and acting really into it. Ears change, interest changes, etc.
About tail swishing: both of my trainers do not necessarily see all tail swishing as a negative thing, but rather the horse expressing his opinion which is a good thing. A horse that is being forced into a frame and consistently pushed to do work that is uncomfortable is not expressing a good opinion, but a horse that is working hard and then maybe you ask for a little more than they are ready for and they swish their tail, may NOT be a really bad thing but just part of the conversation between coach and athlete... Part of a partnership. I am sure I DO NOT know where the line should be drawn, but I know my two trainers do.
I would never advocate the use of pressure to increase a horse's true intrinsic motivation, because I have no idea exactly how to figure out the boundaries, and because too much pressure seems more harmful than not. However, I absolutely do NOT see pressure as AlWAYS a negative thing for the horse, and all I have to do is imagine my former -- and favorite coaches -- as a model for how it CAN work wonders in the right situation.
The one other area where pressure may be a net gain is when the horse has a lot of physical problems and does not KNOW he really can feel better, but he needs to push through pain that even the best tuned operant conditioning reward schedule cannot break through. HERE, the model is a physical therapist doing rehab. If you have ever been through physical therapy, you may understand this feeling -- you might sometimes thing your therapist is a mean, heartless person and yet part of you recognizes that they are helping you and when you DO make the breakthroughs, this person becomes your hero and the person you are grateful to for life. Even while you were sometimes swearing during the process. Of course this assumes the trainer knows absolutely that this pressure in this way WILL lead to a breakthrough the horse will recognize as a good thing. BOTH of my trainers can do this, but I cannot. Or at least I am not confident that I would be pressuring them at the right time and in the right way...
OK, long long post just to say where I am right now about all this. Grateful to have a place where people actually CARE about the horse's intrinsic motivation for doing intrinsically rewarding behaviors. This is rare today, though of course Xenophon a couple thousand year's ago was all ABOUT inspiring a proud, show-off horse...