The difficulty of testing cognition

With all the fuss lately about high-stakes testing and the seeming inability to make judgments of any kind about whether schools are worth the price of admission, I found this study of age-related cognition decline interesting.  The blog is entitled "The Importance of Being Wrong," apparently in honor of the author's interest in the process of learning not only by association but by elimination.  (In other words, why a really effective dog trainer--not me--not only rewards the right action but, perhaps even more important, never fails to discourage the wrong one, even when tired or bored.)

The linked post focuses on tests designed to discover whether the ability to recall words alters with age.  The author makes an interesting case that there is a stronger association of poor performance with the size of the sample than with age.  Among the possible explanations are the greater difficulty of excluding specific cognitive diseases from a large sample, or the greater difficulty of ensuring consistent evaluation techniques with a larger research staff.  It's also possible that older research subjects are more anxious being tested by youngsters, or in a university setting, than are younger subjects.

It's a tricky process, determining whether cognitive declines exist and, if so, what causes.  Nevertheless, though testing is fraught with challenges, I remain skeptical that it should be all that hard to get a rough idea whether a group of youngsters is noticeably less ignorant and untrained after a year of publicly funded instruction than before, though I suppose it can be quite tricky to compare two such sets of youngsters, taught by different people or methods, and obtain any certainty about which techniques were more effective.

Another post by the same author explores the difficulty in teaching children to recognize color apart from the objects conventionally associated with it.  Very young children may know that bananas are yellow and apples are red without gaining much skill in assigning colors to neutral objects in the laboratory.  I thought that was interesting in light of our discussion a few weeks ago about the oddly indeterminate use of color in Homer, and the differences in color language across cultures.  Anyone who's ever tried to paint in color knows how surprisingly hard it is to choose a color from a palette that will genuinely recreate the impression of a colored object in the real world.  It's not necessarily a natural skill to tell what combinations of frequencies correspond to what we carelessly call "red" or "blue" in various objects.

4 comments:

Grim said...

...never fails to discourage the wrong one, even when tired or bored.

Not necessarily! There's an alternative theory of dog training (which works even better with horses) based around what operant conditioning calls "extinction." Extinction in this context means that a trained behavior can disappear or decrease in frequency if you stop rewarding it. But it also means that a bad behavior can become extinct if it stops drawing any reaction at all.

Many dogs and some horses are keenly attuned to you, and want to please you. With these animals, reward/punishment works well. Many horses and some dogs, though, are ornery. These animals are often best trained by rewarding the desired behavior, but showing absolutely no reaction to them doing what they know damn well is wrong.

If they can't get your goat, they'll often lose interest.

Joseph W. said...

This fellow suggests that cats don't respond to negative reinforcement at all well, though they do to positive.

But it also means that a bad behavior can become extinct if it stops drawing any reaction at all.

One of my relatives is a part-time dog trainer, and tells me one technique she's learned is to look away from a dog that misbehaves...the idea being that if you're looking away, the dog starts to feel ostracized from the "pack," and will be eager to get back into your good graces. If that's right, and if I'm not garbling what I heard, ignoring the animal at the right time can be effective even if it's attuned to you.

Texan99 said...

One of our dogs has a bad habit of jumping up on people. Someone suggested dropping the arms and looking stiffly up and away. It does seem to work.

I said "discourage" rather than "punish" because I meant to include any action by the trainer that signals displeasure. It might be nothing more than completely failing to interact, which is a way of withdrawing reward. But my impression is that good trainers get good results by being extremely consistent in that kind of withdrawal. Assuming a reaction is clear enough for the dog to be reasonably able to notice it, consistency seems to trump intensity of message.

Ymar Sakar said...

One dog had the habit of resting the fore limbs on a chair, when it was time for the humans to eat. Either it was going to get some scraps from its owner, or it would get attention.

When it tried to do it to my chair, I first ignored him for a few seconds with a verbal command to get off. Then I looked directly at the target and pushed it out with index and middle finger. Dog tried again. I repeated the process. Dog then gave up and put its head down at the dining table's floor. I looked at dog, to see if it becomes hyperactive again, verifying that it has calmed down and is in a neutral behavior state, I gave it some meat off the table.

While the dog got hyper again, it didn't try to jump on my chair any more.

But the moment I get off the chair and someone else is there, like the owner, dog goes back to its neurotic habits once again.

Most of these methods are also used to train slaves or servants. So long as people don't know how they are used, explicitly, nor study or accept how the Left uses them, many things in life are mysteriously incomplete.