Using Novelty in Reinforcers

Last week on my Bookends Farm blog, I wrote about one of my horses sneaking into the barn and finding enjoyment exploring the area. He found this somewhat novel experience preferable to being outdoors with his friends eating hay. That was an example of him creating his own novelty in enrichment. I promised a blog post on novelty in training with dogs so here it is.
 
IMG_6851
ruts are for roads, not training

While we tend to think that our animals need routine, routines can become ruts. Ruts aren’t fun. Animals understand patterns and when they predict something enjoyable, they can become disappointed and frustrated if it doesn’t happen. On the other hand, if we surprise them with something new, it helps to build some resilience for those times when unexpected or unavoidable situations disrupt the routine. We can help our dogs learn that different can be a good thing.

 
The routine in our house is that mealtime for dogs is when we come in from barn chores, both morning and evening. Eloise, my Jack Russell Terrier, runs up the stairs in anticipation as soon as we come in. When I get upstairs, she is sitting on the little rug where she waits for meals. If I am in a hurry to do something when I get in and don’t immediately feed her, she continues to sit there. She learned that sitting on that rug is what causes meals to appear, and my occasional lateness taught her that she needs to keep sitting there. That behavior has become resilient to the passage of time. How have I made that a good thing? Oftentimes what delays her breakfast is that I am sharing mine with her. If I feed her immediately, it’s because I have prepped her dog food quickly. If I am making scrambled eggs or oatmeal for breakfast, I will make extra so she can have some for her breakfast. More often than not, a late breakfast equals a special breakfast!fullsizeoutput_194
 
One of the first handouts I give to dog clients is about treats. I explain that treats should be small and soft but that each dog will have preferences in taste. I ask people to try different foods to see what their dog likes best and then vary those treats according to the difficulty of the training. I suggest commercial treats and “people food” such as cheeses, meats and peanut butter. People find something the dog likes, but then they stick with it. The dog gets accustomed to that treat, and it stops being quite so reinforcing. The dog doesn’t focus as well. When I visit, I bring something new, such as ham, or chicken and the dog can’t wait to work for me! The people see how well it works and then use that as a reinforcer…and stick with it. They neglect to take advantage of novelty. I tend to have something different each week so that the dogs never know what to expect from me. It’s not the specific food I have in my treat pouch, it’s the knowledge that there will be something new and exciting.

IMG_9022
Bandit is happy to see me arrive
 
The reaction from the dogs shows that it’s not what I have; it’s the anticipation of what it will be this week. They don’t ignore me until they see what I have. They know I’m good for something novel and so they focus on me as soon as I step in the door. That’s what makes positive reinforcement and novelty both so powerful and so different from luring, even though both involve food and both can use novelty. Positive reinforcement includes the element of surprise.
 
In addition to varying foods as reinforcers, we can also vary activities. I like to surprise Eloise after a recall. Sometimes she gets a treat. Sometimes I pick up a stick and toss it for her. Other times I pat my chest which is her cue to leap into my arms. Then she gets her wiry hair scritched before being put back down to run off again. In the fall, our road is littered with wild apples and they make wonderful things to throw and fetch. They bounce and taste good too. My old Jack Russell, Beetle loved snowballs tossed for him to leap up into the air and catch. Nature provides novelty if we take advantage of the opportunities.
 
When training a new behavior, novelty may cause problems by interrupting the flow of training. While the dog is still trying to figure out what earns the click, a high rate of reinforcement (the number of click/treats per minute) gets many repetitions of the correct response. Offering the dog something novel could slow things down. The dog might sniff first, then savor the morsel and finally look around on the floor to see if any was dropped.
 
I prefer to use novelty to maintain behaviors that might otherwise lose their fluency over time. The problems that distractions cause can be counterbalanced by novelty. It becomes “can I catch that squirrel?” against “what goodie might she have”? I want to be sure that my reinforcer is worthy of the decision my dog makes!
Advertisements

The First Week with a Clicker Puppy

This past week I have considered making up bumper stickers which say “Puppies are a pain in the butt: Rescue!”.  I realized that of the many, many dogs we’ve had over the years, very few have been 8 weeks of age when we got them. Some were adults, some were adolescents but the mayhem of an 8 week old puppy had faded into the deep recesses of my mind. The last 8 week old we had was my dear Jack Russell, Beetle, who is now 15 years old. When he was a pup, I had two kids at home to help exercise, entertain and supervise him. Thankfully, we have several mature dogs who are taking the place of the kids in this department.

I have become a huge supporter of rescuing dogs.  There are just so many great dogs  in need of homes and so much support for choosing, training and rehabilitating if necessary. So why did we get a puppy? We need a working dog. We rely on a Border Collie to manage over 100 head of livestock (NOT horses). In the past, I have tried a couple older rescue Border Collies and I just did not do a good job of transitioning them from pets to working dogs, though both were well bred and both had the instinct.  I will take all the blame.  I think I could do a better job now but my confidence was shaken and the stakes are too high. Our current working dog, Nell, is 7 and she needs an assistant!  I usually plan on the young dog being two years old before he can be a reliable assistant so by then poor Nellie will be nine years old and her body will be ready to hand over the more strenuous tasks.

Enter Case:IMG_5095

Don’t be fooled.  He looks like all sweet cuddly softness but I only have pictures of him like this from above when he has finally crashed.  If I move, he’ll be up and a blur in no time.  And as soon as you reach for that soft puppy fur, you feel sharp puppy teeth. My husband and I both have the wounds to prove it.

So how to deal with this monster?  Just like I tell all my clients: management and training. I have to manage every minute of his day until I can use positive reinforcement to train the behaviors we want.

I do not want him to “practice” peeing in the house.  Nor do I want to punish him if he does. If I say “no!” or grab him as he pees, he will learn that it’s not safe to pee in front of me and in rooms that I am in most often.  If he sneaks off to the bedroom where no one is looking and pees there, he will learn that running off to a back room is a safe place to pee. What I WANT is for him to learn to pee (and poop) outside. I know that puppies don’t like to mess their sleeping area.  By confining him when we cannot watch him, and reinforcing him strongly for peeing outside, we have used both management and positive reinforcement. At the end of the first week, we have great success.  It’s only noon but we haven’t had any accidents in the house yet today 🙂  We have had several successful trips outdoors.

Last year I attended a talk by Ian Dunbar and my favorite part was his protocol for puppies. He suggests breaking the day up into cycles.  The younger the puppy, the shorter the cycle.

Crate->outside for a potty break which is rewarded with 3 treats->walk, fun time, training time->back in the crate with a food stuffed toy->repeat

When we get up in the morning, we let Case out of his crate and take him straight outdoors.  After a long night (only once this week have we heard and responded to a request to go out in the night), he immediately pees.  For this, he gets three tiny treats, handed to him one at a time in a flat palm. Then he poops and again, gets three tiny treats. Ian recommends that this takes place where you want the puppy to learn to use as his toilet spot for his life.  That’s a bit of a challenge for us because we still have so much snow that there are limited places to go.

After he has taken care of his potty needs, then we go for a walk. Puppies love walks, or as Jane Killion calls them, “sniff and strolls”. IMG_5089 Young puppies should be allowed to go at their own pace, stopping to examine their world as they wish. This activity functions as a reward for Case’s successful potty break.  If instead, you take the puppy right back inside after he pees, that can be viewed as punishment and he’ll learn to hold out as long as possible.

If the weather is not conducive to a sniff and stroll, then we come inside for an interactive play time or a mini training session. We play tug, or fetch and allow him to investigate new toys while we watch him knowing the bladder is empty.  He learned how to play with Eloise and they will IMG_5080wrestle, tug with some encouragement and play keep away. Beetle lies on his large bed in the corner curling a lip in warning if Case gets too close. It only took one day for Beetle to teach Case that he was not going to tolerate puppy nonsense in HIS corner. Doing his best Clint Eastwood “stay off my lawn”  imitation, he growled, snarled and snapped if Case got too close. At first this was sufficient, but being a bold and brash puppy, Case decided to push it and torment the old man by sneaking closer and closer, ignoring the warnings.  That’s when he found out the old man’s gun was loaded. Beetle leapt off his bed, landed on Case and gave him a good thumping.  Case came running to us and hid behind our feet. Since then, if Case gets too close in a moment of forgetfulness, all is takes is a rumble from Beetle to send the puppy running to us for cover.

IMG_5078
Don’t be fooled, in 24 hours he learned how to climb out of this 3 foot ex-pen, howling and shrieking indignantly.

When playtime winds down and we need to get on with our day, then it’s time for Case to go back in his crate.  Every morning I measure out his day’s food and put it in a safe place near his crate. When I need to put him in his crate, I put a little kibble in his dish, moisten it with hot water and then scoop it into a kong.  This is what he gets when he goes in his crate.  After all that play, he’s ready to eat and having it in a kong or other stuffed toy is the other brilliant part of Ian Dunbar’s protocol.  Not only does it make the crate a pleasurable place to be (he gets fed in there!), but it teaches him, by practice, an acceptable chew object.  Kongs are rubbery and with food down inside it, Case works at it with paws and tongue until he’s sleepy and drifts off for a nap. This builds a good habit, as opposed to building a taste for furniture legs or shoes.

note: Dunbar recommends doing this each hour especially for young puppies, but Case seems to have a longer cycle.  He will stay awake to play inside or out for a couple hours, and then sleep for a couple hours during the day.  It is working for us to go by his schedule, waiting until we hear him stir to take him out of his crate, but you can also set a schedule as long as it’s age appropriate.

That isn’t to say that he hasn’t grabbed a shoe or gnawed on a table leg. But because we are actively watching him when he’s loose, we see it immediately and we can redirect him with an acceptable toy.  Toys are much more fun when there is a person on the other end of them.  They move and wiggle and bounce! The bulk of his “practice” of toy play and chewing is with items we have chosen so that as he grows, he will have a habit of playing with dog toys, not household items.

I have a training journal for Case in which I wrote goals for the week and I log important things like successful potty trips outside, when and where he has accidents inside, what new things he is exposed to and how, people he meets, etc.

For the first week, my goals were “name association, housetraining, fear free habituation to house, car and yard. All eliminations outside to get verbal praise and treats”. We have been careful with his name. Only half joking, I have said what we are calling him most of the time is Hellboy. He has picked up several other nicknames and we use those most often. I reserve using “Case” for when I have high value treats to follow it up immediately. Initially, this was specific training sessions.  With ten treats in my hand, I’d say “Case” and hand him a treat, immediately followed by nine more repetitions of that. It only took a time or two before he knew that “Case”=treats! (it did take a day for him to learn about treats themselves, never having had them before). Once we had an immediate reaction to the name, then we began using it when we were 100% sure he would respond. If he was wrestling with one of the big dogs or exploring a wonderful smell outside was not one of these times. There was a good chance he would choose to stay with his current activity rather than leaving it. But if he was quietly hanging out almost looking for entertainment, we could say “Case” and he’d look and we’d hand out a treat followed by some play. In this way, we now have a great response to his name. We still don’t use his name unless we are 100% sure he will respond and we are prepared to follow up with great reinforcement.

His first car ride was a long four hours home from Massachusetts. So this week he has gotten in the car just for treats or for short rides (a mile down the road and back). As a result, he will now voluntarily put his front feet up into the car if we open the door.

Finally, fear free habituation. We took Case from his birthplace, his first environment, his dam, his litter mates and his first people. We brought him to a new environment, new dogs, new smells, new people, new sounds. We wanted him to settle in with as little stress as possible. As I’ve already mentioned, he’s a pretty bold little monster so there were very few things which frightened him but we were also very careful to watch for any signs of concern: body posture, eyes, tail. If any of those signs told us he had any concern, we removed him immediately to a safer spot or used food treats and body comfort.

The one thing which we did notice was the noise of motors: vacuum, blender, airplanes. Outside, if an airplane went overhead, I’d see his body posture go into a little cower as he looked up and around so I’d scoop him up and feed treats (not being able to get away from the noise).  Indoors, we were careful to have him on a different floor with Ed feeding treats and playing or comforting if I ran the vacuum (didn’t happen often but we did break a glass one day that needed to be vacuumed up). When I ran the blender, I’d turn it on and then go across the room to feed until it shut off (luckily my blender has an automatic shut off after 60 seconds).  At this point, he is no longer reacting to any of these sounds, but we’re still counter conditioning to be sure.

IMG_2017
In case anybody sees the wound on Ed’s face, here’s proof it wasn’t me.

All in all, it’s been a successful week. An exhausting week, but a successful one.

 

Pay It Forward: Reinforcement Is Not Bribery

IMG_2946Many times when I do an evaluation on a new dog, the owner will tell me “she’ll come if she knows I have treats”, or “if I shake the box of dog biscuits, he’ll come running”.  But, they say, the dog will not come when called, unless they know the goodies are waiting. This causes several concerns with dog owners regarding using positive reinforcement to train.  They don’t see how positive reinforcement is any different than what they are already doing and they think that if they don’t have the treats, the dog will ignore them.

This is a hard concept to break through when we do use food as a reinforcer. When I talk about using high value reinforcers depending on the difficulty of the behavior, they think that they need to show the dog that they have hot dog bits in their pouch before they step outside so the dog will pay attention.

The opposite is actually true. We do not want the dog to know ahead of time what, if any, reinforcer is available.  Shaking the treats, showing the cheese, making a show of putting on the treat pouch: these are all forms of bribery.  Effective use of positive reinforcement is not bribery.  The proof of that is a dog who has been trained with the correct use of treats as reinforcers.  These dogs come running every time, from a cue, whether it’s verbal, visual or other.  They don’t look to see if the person has worthwhile treats- they come immediately, with great speed, and directly to the handler. 

The reason positive reinforcement works so well is in the definition of the word reinforcement itself.  In behavioral terms, reinforcement is something which makes it more likely the behavior will happen in the future. So this past week I found myself pointing out to clients that the treat they deliver has no effect on what the dog just did- the behavior is over!  But it does have an effect on the next time you ask the dog for that behavior.  If you call your dog, without any bribery, and then are able to reach into your pocket and pull out a piece of cheese to give him, that may make it more likely he will come when you call in the future. He did not even know you had cheese in your pocket, but you did!  You want to be full of happy surprises like this.  Every single time your dog does something you like, you want to be able to surprise her with something she likes.  This is called building a reinforcement history.

When you have a strong reinforcement history, meaning you have a reputation with your dog of “paying well” for behaviors, your dog will begin to respond to cues more rapidly and with much greater reliability.  We refer to reinforcement history as “putting money in the bank”.  You want to have plenty in the account so that when you need to make a withdrawal, the reinforcement account is flush.  When your dog responds to your cue, for instance when he comes when you call, you are making a withdrawal from that “trust account”.  The level of difficulty of the behavior will determine how much is withdrawn.

In the house, getting ready to go for walk and you call your dog, well that might even be free because the dog wants to go out with you!  The opportunity to go outside is payment enough.  In your yard with no distractions and you call your dog, that might cost you a little bit.  Dogs love to sniff and smell things so while it may look to you as if it’s an easy thing, really he had to give something up- the pleasant occupation of sniffing around- to come to you.  So you just made a small withdrawal from your account.  But you can replenish that easily by giving him a treat when he returns.  The balance has been restored. There is enough in the account so that next time you call, he will respond.

The challenge, of course, is when the UPS man pulls in the driveway.  Or at least that’s the biggest challenge at my house.  That UPS truck is the source of so many distractions: movement, intruder on the property, friendly guy in the truck, who gives dog biscuits, and oh the smells on his tires, his clothes, the packages.  I don’t know about you but my dogs act as if the property is under attack when he pulls in and then switch to “here’s Santa Claus!” mode when he stops.  They are in that truck checking everything out.

For me to be able to recall them in that situation, I have to have an account that is full to bursting.  The only way to get it that full is to put lots and lots of little deposits in over time.  When we are outside, whether it’s to go to the barn, or to get the mail, or to go for a walk, I make sure I have low and high value treats with me.  For my dogs, examples are kibble, cheese and hot dogs.  I cut them up in tiny bits (smelly hot dogs go in a tiny plastic container with a lid), I hide them away in a pocket before we leave so the dogs never know what’s there. And then we practice.  I call them when nothing is distracting them and pay with kibble.  I call when there is a mild distraction (sniffing in the woods) and pay with string cheese.  I call when a car is coming and I pay with hot dogs.  My dogs don’t know what I have, but they know that it will be good.  Each time I give a treat, I am putting money in that bank.  Every day I put more and more in- if I’m cheap and only put in kibble each time, that might keep the account level…but will it put enough in so that when the UPS man comes, I have what I need?

I never know when the UPS man is coming, so I can’t slap a steak in my pocket for that particular occurrence.  I can, however, pay it forward.

Keep treats on hand and on you all the time.  You want to pay for the easy stuff, so that the account can cover the cost.

Meet Eloise!

DSCF0433

This is Eloise.  She is the face of The Dog Chapter at Bookends Farm.  Eloise came to live at Bookends Farm when she was a year and a half old.  She had a different name then and her people decided she wasn’t happy living in an apartment and spending a lot of time in a crate.  They were right.  They contacted a woman named Pauline who worked with Russell Refuge (Pauline now has her own rescue called Jacks Galore!).  I was looking for a snugly little girl and so Beetle and I went down to meet her.

29511_400506320924_6992394_n

She did not try to take Beetle’s tennis ball away from her so he said it was ok if she came home with us.  We named her Eloise because everyone at Bookends Farm and The Dog Chapter has a book themed name- she is named for Eloise in the children’s book…a wild haired little girl.  That Eloise lives in the city at the top of the Plaza Hotel.  This Eloise came to live on a farm.

The other thing about everyone at The Dog Chapter and Bookends Farm is that we use Clicker Training and Positive Reinforcement to do all our training.  And that is what this blog will be about.  I’m Jane, by the way.  I will be sharing stories of how Clicker Training and Positive Reinforcement work to be the best training we can use for our friends.