Thursday 7 March 2013

Real understanding versus procedural pseudo-understanding: a collage of sentences

*

The typical modern educated person - by educated I mean someone with advanced educational certification - has zero understanding of complex concepts; including the specific concepts which his educational certification purports to validate.

Modern educational certification is based on evaluations (one can hardly call them examinations) which are so procedural, and so difficult to fail except on procedural grounds, that they are incapable of evaluating understanding.

*

Only another human being, in sufficiently dense and sustained human contact, is capable of evaluating understanding.

What we have instead is the evaluation of collages of sentences - evaluated one sentence at a time, in terms of the accuracy of reproduction.

*

In multiple choice examinations, students are required to match up sentences in a very explicit way - but in extended writings such as essays and dissertations and theses, the principle is the same: these extended writings (or indeed conversations) are collages of sentences - individual factoids learned and assembled according to prescribed procedure.

*

It is not that such evaluations are easy - many people cannot do them; simply that they are grossly misleading in terms of over-estimating understanding.

These evaluations primarily test adherence to procedure; and to adhere to a procedure requires approximately one standard deviation of intelligence less intelligence than to understand that procedure.

*

But if our educational evaluations were to become genuine tests of understanding, then not only would nearly all students fail nearly all examinations - but so would their 'teachers'.

All this being a consequence of the decline of general intelligence combined with the inheritance of a highly complex society and culture bequeathed by earlier (and more intelligent and much more creative) generations.

http://iqpersonalitygenius.blogspot.co.uk/2012/11/the-over-promoted-society.html 

*

11 comments:

stats79 said...

You've been making this claim for a while now and yet you have never given a concrete example of what you mean. An illustration of this general lack of understanding by moderns would be helpful. Is this a problem in the hard sciences as well as liberal arts, religion, theology and philosophy?

PhilR said...

This is precisely correct. A level exams are so wedded to the mark scheme that anything other than a rigorous adherence to the 'procedure' meets with bureaucratic rebuff from the examiner. My best students almost never get the best grades. It's the dutiful processors and regurgitators that get the A*.

FHL said...

@stats79

I am unsure if I am thinking of this in the same way that Dr. Charlton is thinking of it, but I can give you a few examples.

Example #1: In almost all science courses students are given formulas to use but the formulas themselves are almost never explained in any detail. Students know how to use the formula, but where it came from or what it even means is never explained.

I majored in philosophy but I was a biology student until I switched at the last moment. In a genetics class, we were taught how to use "Chi squares" to determine whether or not a certain trait had a genetic link.

You ask: "what is a Chi square?"

Hell if I know.

I know the formula, and if you have the data I can plug in the numbers and tell you if there is a link or not, but as to how or why the formula works, I couldn't tell you. I suppose I could figure it out, but that would be on my own time; it is not taught at the university and the knowledge of how the Chi square actually functions is not necessary to pass any exams. You just have to know where to plug in the numbers and that's all.

Example #2: I built my own computer. I bought the parts at a store and assembled them at home. People are usually impressed when they hear this, but really, I was just reading instructions and sticking things into slots.

I do know what the parts do. I know, for example, that a hard drive works by writing and reading information onto a disk (or several disks) through a "head," a needle like thing that hovers above the disks as they spin. It doesn't touch it, but it is similar to how a record player delivers music through the record.

I also know that if you defragment your drive in alphabetical order, many of your programs will run faster, since many programs have scripts and files named in alphabetical order corresponding to the order in which they are used. So if your files are physically arranged in alphabetical order on the the disk, the head doesn't have to waste any time looking or moving to the location of the next file, it just moves sequentially from one to the next.

Most people don't know stuff like that. For most people, a hard drive might as well work by magic. So people usually find it impressive that I know how it physically works.

But wait- what do I actually know about hard drives? There is a spinning disk and a needle-looking thingy? And then it "writes"? It "reads"? What the hell does that even mean? How does it do this? How does the information transfer over and become produced into a program, or music, or a video game or whatever? What is the information even encoded in the first place? Different charges transferred to the disk from the head and vice-versa, like forces corresponding to a code of ones and zeros or something?

I couldn't tell you. I don't have the slightest clue.

I can, to a certain extent, say I "know" how a hard drive works, but I could never build one on my own, even if I had the money to set up any necessary equipment. What could I say? "I need to build a disk and a platform with a rotating center that will spin the disk. And then I will need a needle-looking-thingamabob connected to an arm that will move it around all over the disk as it spins. After that, it will then read and write. Whatever the hell that means."

Of course not. I sound educated when I tell people "how it works," but in reality, I don't have a bloody clue of how it really works.

Like the Chi square, I could probably look it up. I never took a university course on computer hardware, but I can somewhat safely guess that even if I did, I still probably wouldn't know how the hard drive actually reads or writes anything.

deep_junior said...

Your post reminded me of a short article by Eliezer_Yudkowsky, Guessing the Teachers Password. Here is a quote:

"Suppose the teacher presents you with a confusing problem involving a metal plate next to a radiator; the far side feels warmer than the side next to the radiator. The teacher asks "Why?" If you say "I don't know", you have no chance of getting a gold star—it won't even count as class participation. But, during the current semester, this teacher has used the phrases "because of heat convection", "because of heat conduction", and "because of radiant heat". One of these is probably what the teacher wants. You say, "Eh, maybe because of heat conduction?"

This is not a hypothesis about the metal plate. This is not even a proper belief. It is an attempt to guess the teacher's password."

George Goerlich said...

I found this in my masters level courses. I had to avoid the most interesting lines of inquiry in my papers and stick to a simple structure following repetition to obtain the best grade. My role was nothing more than a regurgitation, but it seems that is the actual intent now.

My favorite class was actually undergrad and taught by a Master, not a Doctorate, who worked at the University only part time. More engaging and creative.

Bruce Charlton said...

@FHL - I agree that the educational systems have given up on understanding of difficult concepts, by and large - but that is because neither the teachers nor those they teach are capable of understanding.

I have spent most of my academic life trying to understand concepts that everybody uses all the time - what is 'depression' and an 'antidepressant', what does the 'randomization' do in a randomized trial - for example.

When I have worked out what these mean (or could mean) then it becomes clear that people - and at the very highest level - are using these terms without understanding.

But, even worse, most people (and at the highest level) cannot understand even when it is explained to them.

At a certain point it becomes embarrassing to persist in trying to explain basic concepts to people who are 'international experts' in 'advanced concepts'.

I should emphasize that this is not a matter of people refusing to accept that I am correct! This is that people cannot understand the subject under discussion.

Now, I would be in just the same situation were I to try and discuss something like negative numbers or logarithms with someone who really understood these. But I think I would know that I did not know, and could not know.

Why has this situation arisen?

Because the educational system is not structured to discover whether people understand - but is instead structured to make understanding irrelevant - so that the whole system is long since poisoned and corrupted.

It is rather as if the classical music training programme that leads via apprenticeship through Grades to a Diploma and teaching by acknowledged performers was to give up on performing pieces of music, and to become entirely a matter of learning and being tested on scales and arpeggios. So that a 'great pianist' was one who could play scales and arpeggios quickly and without mistakes.

The successful pupils would not need to be musicians, and pretty soon the teachers would stop being musicians - and the whole system would be incapable of evaluating musicianship. Except they would still call it music.

Bruce Charlton said...

@FHL - I agree that the educational systems have given up on understanding of difficult concepts, by and large - but that is because neither the teachers nor those they teach are capable of understanding.

I have spent most of my academic life trying to understand concepts that everybody uses all the time - what is 'depression' and an 'antidepressant', what does the 'randomization' do in a randomized trial - for example.

When I have worked out what these mean (or could mean) then it becomes clear that people - and at the very highest level - are using these terms without understanding.

But, even worse, most people (and at the highest level) cannot understand even when it is explained to them.

At a certain point it becomes embarrassing to persist in trying to explain basic concepts to people who are 'international experts' in 'advanced concepts'.

I should emphasize that this is not a matter of people refusing to accept that I am correct! This is that people cannot understand the subject under discussion.

Now, I would be in just the same situation were I to try and discuss something like negative numbers or logarithms with someone who really understood these. But I think I would know that I did not know, and could not know.

Why has this situation arisen?

Because the educational system is not structured to discover whether people understand - but is instead structured to make understanding irrelevant - so that the whole system is long since poisoned and corrupted.

It is rather as if the classical music training programme that leads via apprenticeship through Grades to a Diploma and teaching by acknowledged performers was to give up on performing pieces of music, and to become entirely a matter of learning and being tested on scales and arpeggios. So that a 'great pianist' was one who could play scales and arpeggios quickly and without mistakes.

The successful pupils would not need to be musicians, and pretty soon the teachers would stop being musicians - and the whole system would be incapable of evaluating musicianship. Except they would still call it music.

stats79 said...

Okay, "randomization" for example. Seems fairly obvious to me. What am I and the experts missing?

Bruce Charlton said...

@stats - You have illustrated one reason why I don't provide examples usually. You ask for an example, I give a couple. Now you want to debate the examples. And so on.

stats79 said...

I don't want to debate the examples, I just don't understand why they are illustrative. I don't know enough about the subject to debate it with you. I would just like to understand your argument more clearly.

Bruce Charlton said...

@stats - OK - you might read:

http://charltonteaching.blogspot.co.uk/2010/10/scope-and-nature-of-epidemiology.html

and

http://www.trialsjournal.com/content/2/1/2