A Guide to Arguing Against Man-Made Climate Change

If you must, then at least do it properly…

The debate about climate change ranges from people arguing that it isn’t happening at all, to those who argue that it is happening, but is entirely natural. The debate can become quite nasty, and part of the reason for this is not that people disagree, but that they disagree without following the rules of scientific discourse. I’m guessing in many cases this is accidental- a kind of cultural unawareness. It’s like making an otherwise innocuous hand gesture while on vacation in a foreign country, only to learn later that it was the rudest possible thing you could have done.

I’ve been annoyed by poor-quality discourse on this topic for some time, and written a few draft blog posts about it, but I’ll defer to the INTJ Teacher for a summary of the key issue (and the main reason I no longer read comment sections after news stories about climate change).

critical thinking2

So now that you know the problem in general terms, let’s talk specifics.

Dismissing the data

First of all, if you’re going to make claims that the data about climate change are problematic in some way, then you should know that there is no one data set. There are thousands of data sets worked on by thousands of people.

Some people seem to think that the whole matter rests on the “hockey stick” diagram of Michael Mann, Raymond Bradley, and Malcolm Hughes published in 1999. (You can download the paper as a pdf here.)


Annotated hockey-stick diagram

Briefly, this was an exercise in solving two kinds of problems: (1) taking temperature information from a variety of sources (e.g., tree rings) and turning it into something that could reasonably be plotted on the same diagram, and (2) figuring out how to take temperature measurements from all over the world and combine them into something representative of climate as a whole. The main reason it became controversial was that it showed a clear increase in temperature since 1850, and that result was not optimal for a certain subset of individuals with a disproportionate amount of political clout. There is a nice description of the debate about the diagram here, including arguments and counter-arguments, along with the relevant citations.

Those arguments are moot at this point, because the PAGES 2k consortium has compiled an enormous amount of data and done the whole project over again, getting essentially the same result (the green line in the figure above).  I can’t help but think that this was an in-your-face moment for Mann et al. (“In your face, Senator Inhofe!  In your face, Rep. Barton!  How d’ya like them proxies?!”)

Despite these results, if you still want to argue that the data are bad, you will need to do the following:

  • Specify which data set you are referring to. Usually this takes the form of a citation to the journal article where the data were first published.
  • Specify what is wrong with it. Was the equipment malfunctioning? Was the wrong thing being measured? Was there something in particular wrong with the analysis?
  • Assuming you are correct about that particular data set, explain why problems with that one data set can be used to dismiss conclusions from all of the other data sets. This will mean familiarizing yourself with the other data and the relevant arguments (although if you are arguing against them you would presumably have done this already).

Things that are not acceptable:

  • Attacks against the researchers. It is irrelevant whether the researchers are jerks, or whether you think they’ve been paid off. What matters are the data. If you can’t supply the necessary information, you have only conjecture.
  • Backing up your argument with someone else’s expert opinion (usually in the form of a url) if that opinion does not cover the points in the first list. It is discourteous to expect the person you are arguing with to hunt down the data backing someone else’s opinion in order to piece together your argument.
  • Arguing from the assumption that man-made climate change isn’t happening. If that’s your starting point, your arguments will tend to involve dismissing data not because there are concrete reasons to do so, but because based on your assumption, they can’t be true. This may be personally satisfying, and ring true to you, but it lacks intellectual integrity. If your argument is any good, that assumption won’t be necessary.

Climate models and uncertainty

It is a common misconception that uncertainty in the context of climate models means “we just don’t know.” Uncertainty is an actual number or envelope of values that everyone is expected to report. It describes the range of possibilities around a particular most likely outcome, and it can be very large or very small.

If you plan to dismiss model results on the basis of uncertainty, you will need to demonstrate that the uncertainty is too large to make the model useful. In cases where the envelope of uncertainty is greater than short-term variations, it may still be the case that long-term changes are much larger than the uncertainty.

Another misconception is that climate models are designed to show climate change in the same way that a baking soda and vinegar volcano is designed to demonstrate what a volcano is. Climate models take what we know of the physics and chemistry of the atmosphere, and add in information like how the winds blow and how the sun heats the Earth. Then we dump in a bunch of CO2 (mathematically speaking) and see what happens. In other words, models specify mechanisms not outcomes. They are actually the reverse of the baking soda and vinegar volcano.

The mathematical equations in a model must often be solved by approximation techniques (which are not at all ad hoc, despite how that sounds), and simplified in some ways so computers can actually complete the calculations in a reasonable timeframe. However, I would argue that they are the most transparent way possible to discuss how the climate might change. They involve putting all our cards on the table and showing our best possible understanding of what’s going on, because it’s got to be in writing (i.e., computer code).

The models aren’t top secret. If you really want to know what’s in them, someone will be able to point you to the code. If the someone is very accommodating (and they often are if you’re not being belligerent or simply trying to waste their time) they might explain some of it to you. But whether or not they do that effectively is irrelevant, because if you’re going to make claims about the models, it’s your obligation to make sure you know what you’re talking about.

If climate changes naturally, then none of the present change is man-made

This is a false dichotomy. No-one is arguing that nature isn’t involved in the usual ways. What they are saying is that the usual ways don’t do all of what we’re seeing now.

A simple way to think about it is as a shape-matching exercise. We would expect that if some trigger in nature is causing the climate to change, then a graph of the temperature change should resemble that of the triggering mechanism. The IPCC has done a nice job of making this comparison easy. In the image below I’ve marked up one of their figures from the Fifth Assessment Report in the way I usually do when I’m researching something. Panel a shows the temperature record (in black), and the panels below it show the changes in temperature attributable to different causes. In the upper right I’ve taken panels b through e and squashed them until they are on the same scale as panel a.

IPCC comparison

A common argument against man-made climate change is to say the sunspot cycles are to blame. You can see the temperature variations that result from these cycles in panel b, and again at the top right. While there are small scale fluctuations in a, it is quite evident that the shape of the effects of sunspot cycles cannot account for the shape of the temperature record, either in terms of having an upward trend, or in terms of the timescale of the temperature change in a. Even if you added in volcanoes (panel c), and the El Niño/ La Niña cycles (panel d), you couldn’t make the trend that appears in a.

The only graph with a similar shape is the one that shows the temperature variations we would expect from adding CO2 and aerosols at the rate humans have been doing it (panel e). The red line in panel a is what you get if you add together b through e. It doesn’t have as much variation as the black line, meaning there are still other things at play, but it does capture the over-all trends.

You needn’t rely on someone else’s complex mathematical analysis to do this. This is something you can do with your own eyeballs and commonsense-o-meter. You may still be inclined to argue that all of these graphs are made up out of thin air, but if you have a look at the many different studies involved (you can do this by reading the chapter in the IPCC report and looking at the citations), you should realize that it’s a pretty lame argument to dismiss all of them out of hand.

But if you are undeterred by said lameness, at that point anyone interested in a serious conversation is going to decide that it isn’t worth their time debating with you, because you’ve already decided that any evidence contrary to your point of view must be wrong. Nothing they can tell you or show you will make a difference, ergo the conversation is pointless. You will appear to be impervious to reason which, incidentally, will be assumed to be the case for your opinions on other matters as well, whether that impression is deserved or not. (“It’s not worth arguing with Jim… if he has an idea in his head, he won’t change his mind no matter what you tell him. He would stand under a blue sky and tell you it’s pink.”)

Scientists are paid off to say climate change is man-made

This argument is quite irrelevant given that the data are what matter, but I think part of this argument might be related to another misconception, so I’m going to address it anyway. It is true that there are millions of dollars spent on climate research grants, but this isn’t pocket money for scientists. To get a grant researchers must justify the amount of money they are asking for in terms of things like lab expenses, necessary travel, and the like. Often their salaries don’t even come into the picture because they are paid by employers, not grants. It is more likely they will be paying grad students and post docs than themselves. When they do apply for funding that will cover their own salaries, that salary must be justifiable in the context of what others in similar positions get paid. In many cases this is a matter of public record, so you can go look up the numbers for yourself.

Most research being done on climate change is funded by government grants. A very few scientists have funding from private donors (though there isn’t nearly as much money as for petroleum-related research), but there is a big check on what influence those donors can have. Research must still go through review to be published. Even if biased research did make it through review, scientists on grants are highly incentivized to pick it apart because that can be an argument for additional grants to further their own research. Getting a grant is a matter of professional survival, so competition for research grants is intense.

In conclusion

There is only one way to make arguments against man-made climate change, and that is to address data and conclusions honestly and appropriately. It may feel good to add your two cents, but if your comments amount to ad hominem attacks or generalizations so broad as to be silly, you shouldn’t expect a good response. You’ve just made the equivalent of a very rude hand gesture to people who value thoughtful and well-informed discourse.

This all seems obvious to me, and I’ve struggled to understand people who argue in a way that I can only describe as dishonest.  But maybe psychology is a factor.  The climate-change deniers need only suggest that scientists are making things up. People don’t want to feel that they’ve been fooled, and most don’t have the background to easily check such claims, so it feels much safer to settle into uninformed skepticism.

Categories: Learning strategies, Science and such, Teaching strategies | Tags: , , , | 3 Comments

INTJ Teacher: An Alternative Approach to the Teaching Philosophy Statement

Some weeks ago I attended a workshop on designing a reflective teaching portfolio offered by the GMCTE at the U of S, and facilitated by two of their excellent instructors, Kim West and Wenona Partridge. The workshop was about making a written record of one’s ideas about teaching, of the methods for accomplishing teaching goals, and of evidence of how that was going. The starting point was a statement of teaching philosophy, which is exactly what it sounds like- a written description of how one views teaching, and how those views inform one’s approach to teaching. (You can find some examples here.)

Kim and Wenona had us do a number of activities to help us articulate a teaching philosophy. I found those activities useful, but what really helped me was something that appeared in their slides, but which we didn’t really discuss in the workshop: the Myers-Briggs survey. Myers-Briggs is a set of questions that helps to classify people into one of 16 personality types described by four letters: I or E for introvert or extrovert; S or N for sensing or intuition; T or F for thinking or feeling, and J or P for judging or perceiving. The figure below is a nice summary.


Chart of Myers-Briggs personality types

Chart of Myers-Briggs personality types. (Source: Jake Beech, CC-BY-SA 3.0)

I don’t know what motivated me to do the test, and from the figure, it doesn’t seem like the classifications should lead to any earth-shattering revelations. But I did do the test (I used the one at 16 Personalities, because it seemed to have the best interface and descriptions of personality types), and the outcome was something I would never have imagined: relief! It felt like a huge burden was lifted!

I learned that many of the things I though were dysfunctional about me, and which I’ve worked hard to overcome so that I could interact with “normal” people are actually themselves perfectly normal characteristics… albeit for a very small segment of the population. If you haven’t guessed yet, my type was INTJ, and I’ve read that approximately 0.8% of North American females fall into that category. That certainly explains my experience of high school (shudder).

On the one hand, nothing has changed, but on the other hand everything has. I suddenly feel like I’m allowed to be myself, which I would describe as someone with mildly misanthropic tendencies. The kinds of things that frustrate me are quite predictable based on the type, and this also explains why very few people seem to feel as cranky as I do about a number of things which I find blatantly objectionable (some of which I’ve discussed at Petragogy).

After a sense of relief, the next thing that occurred to me is that INTJs would, on the surface, not seem to be particularly suited to teaching roles. The phrases “does not play well with others,” and “does not suffer fools kindly” come to mind. But teaching is something I feel very strongly about, and something I put a lot of effort into getting right.

So naturally, I’ve decided to do a webcomic about being an INTJ teacher. I see it as serving a number of purposes:  I’d like it to catch the attention of others like me so I can hear about their experiences.  I think it’s a nice way to give advanced warning to those who might cross my path. It’s also a remarkably effective way to communicate ideas that took several paragraphs to articulate in my teaching philosophy statement. One would think a cartoon would be limiting as a means of communication, but somehow it’s just the opposite. I can get more across with a few lines of text and some pictures than I could in several pages of writing.

I’m calling my comic INTJ Teacher (as far as I can tell no-one on the internet has claimed that yet), and using the tag line, “For those who are, and those who should know what they’re dealing with,” because that pretty much sums it up. My first installment (below) is an introduction, and I will put it and subsequent installments on a new blog. With the small number of INTJs out there, and the fraction of those who are educators, I’m not expecting a huge readership. That’s fine, though. I just like the idea of being able to point someone to a url and say, “You’ve been warned.”

INTJ Teacher webcomic

Introducing the INTJ teacher

Categories: Teaching philosophy | Tags: , , , , | 2 Comments

The Levitating Wiener Standard of Formative Assessment

Formative assessment, or informative assessment, as I like to call it, is the kind of evaluation you use when it’s more important to provide someone with information on how to improve than it is to put a number next to a name. Formative assessment might or might not include a grade, but it will include thoughtful and actionable feedback. Formative assessment of teachers is no less important than formative assessment of learners- both are needed for the magic to happen.

I struggle with how to get truly useful formative feedback from my students. There are different instruments for evaluating teaching, including SEEQ (the Students’ Evaluation of Educational Quality), but the problem with the instruments I’ve used is that they don’t provide specific enough information. Sure, there is a place where students can write comments to supplement the boxes they’ve checked off elsewhere on the form, but those spaces are often left blank, and when they’re not blank, they don’t necessarily say anything actionable.

I’ve concluded that I need to design my own questionnaires. But when I get down to the business of writing questions, it feels like an impossible task to design a survey that will get at exactly what I want to know. I do have a pretty high standard, however: the levitating wiener.

The mentalist and magician Jose Ahonen performs a magic trick where he presents a levitating wiener to dogs. You can watch the videos How Dogs React to Levitating Wiener (parts 1 and 2) below. These are fascinating videos… have a look.

The dogs in the videos have one of three reactions:

  1. It’s a wiener! Gimme that wiener! These dogs react as one might expect, focusing on the existence of the wiener rather than on the fact that it is levitating.
  1. How the heck are you doing that? These dogs ignore the wiener and focus on the palms of Jose’s hands instead. It’s as though they’ve decided that it doesn’t make sense for a wiener to be levitating, and he must be doing it by holding strings. In other words, these dogs are trying to figure out how he’s doing the trick, and they all seem to have the same hypothesis. (Incidentally, it’s probably the first hypothesis most humans would come up with.)
  1. This is wrong… it’s just so wrong. These dogs watch for a moment and then get the heck out of there. Like the dogs in group 2 they also don’t think wieners should levitate, but they are too appalled by the violation of normality to formulate a hypothesis and investigate.

To my mind, most of the teaching assessment instruments are more like having the dogs fill out the questionnaire below than watching them interact with a levitating wiener.

Formative assessment for levitating wieners (loosely based on the SEEQ questionnarie)

Formative assessment for levitators of wieners

If the participants checked “agree” or “strongly agree” for “Weiners should not levitate,” it could mean something different for each dog. A dog from group 1 might object to having to snatch the wiener out of the air as opposed to having it handed to him. A dog from group 2 might think the question is asking about whether wieners are subject to gravity. A dog from group 3 might be expressing a grave concern about witchcraft. If the dogs wrote comments (we’re assuming literate doggies here), their comments might clarify the reasons behind their responses. Or they might just say there should be more wieners next time.

Now contrast the questionnaire with the experiment shown in the videos. Because of the experimental design, I learned things that I wouldn’t even have thought to ask about- I just assumed all dogs would react like group 1. I learned things the dogs themselves might never have written in their questionnaires. A dog from group 2 might not have noted his interest in the engineering problems surrounding hovering hot dogs in the “Additional comments” section. It might not have occurred to a dog from group 3 to mention that he was frightened by floating frankfurters. Maybe neither dog knew these things about himself until he encountered a levitating wiener for the first time.

A formative assessment tool that is up to the levitating wiener standard would tell me things I didn’t even consider asking about. It would tell me things that students might not even realize about their experience until they were asked.  Aside from hiring a magician, any suggestions?

Categories: Assessment | Tags: , , | Leave a comment

The open textbook arrives- For real this time!

Textbook coverApparently I’m not the only person who has seen a need for an open textbook for introductory physical geology. BCcampus has just released one, and I was lucky enough to get in at the tail end of the process and add a chapter on the origin of Earth and the solar system.

In case you haven’t heard of open before, these are resources which are free to use, and are available with non-restrictive or minimally restrictive licenses, or released to the public domain.  This particular book has a CC-BY license. That means anyone can use the book in any way they please, so long as they give the creator appropriate credit, and note whether any changes have been made.  Quite simply, it’s a free textbook.

A lot of work went into this book, and that is evident when you look through it. The author, Steven Earle, not only explained a wide range of topics in an enthusiastic and conversational style, but he also drew or adapted a great many figures. From my own experience, I can tell you that figure drawing takes up a lot more time than the actual writing. (But then again, I’m the sort to vacillate between shadow settings or tweak a font for 5 to 10 minutes before I’m happy with it.)

This textbook has a special focus on the geology of British Columbia.  That’s a Canadian focus that many commercially available physical geology textbooks have lacked until recently, judging by the regular shipment of freebies I get from the textbook companies.

The textbook is not only accessible as an online resource, but it can also be downloaded in a variety of electronic formats- you could put it on an e-reader, for example.

The online version of the book is built in Pressbooks, and I was immediately enamored of the aesthetics and ease of navigation… so much so I got my own Pressbooks account for experimentation. I’m finding it a bit slow, however my internet signal fails on really windy days (one of the perils of a rural home office), so maybe it’s not their fault.

The one feature of this textbook which has had the greatest impact on me is the fact that I can edit it- all by my lonesome, no permission required. In its current form, the book presents some of the topics differently than I do in my course, so I inquired about making the changes. The result? Amanda, a lovely lady from BCcampus, emailed me the latest version of the XML file, and I imported it into my Pressbooks account. It was that easy. (The XML file is one of the download options as well.)

I was pleasantly surprised at how straightforward it was to get something I could edit, but what surprised me most of all was the immense sense of relief I felt when I discovered that there were no barriers to making the textbook fit my needs. I’ve always been a bit of a control freak when it comes to my course materials- on a bad day that translates to downright resentment at not being able to fix things that need fixing- but I had no idea that I was so stressed out about not being able to control my textbooks.

Categories: Learning technologies, Textbooks | Tags: , , , | 1 Comment

Help for students, part 4: Exam panic

Exam panic is a tricky problem, because once you experience it, it can make you worry about panicking in the future.  Once you are anxious about panicking, that makes it all the more likely. Fortunately there is a way to fix this. The solution is, go ahead and panic… sort of.

Brain sees exam monster

The problem: you see an exam but your brain sees certain death.

Your brain is an amazing bit of biology that has evolved over millions of years to serve the needs of our ancestors. Unfortunately, somewhere during that evolutionary process it became a toddler-like entity which, regardless of your good intentions, is willful, easily bored, and prone to inconvenient emotional outbursts. It learned a few good tricks that were suitable for helping our ancestors to escape from predators and each other, but since then it has stubbornly refused to acknowledge that those same tricks can be counterproductive when dealing with anxiety over situations that are not likely to kill you.

Brains in a panic

Brains do not react well to certain death.

When you see an exam and feel anxious, your brain sees something else entirely. As far as it’s concerned, that exam is actually a large carnivore about to eat you for lunch. Your brain will try its best to persuade you that you are about to die, and that you should run for your life. Your brain is wrong, but it is also convincing.

Expect some exam anxiety or even outright panic, but realize that you don’t have to accept what your brain is telling you about the situation. Sit back and let it have a fit, like you’re waiting out a child’s temper tantrum. Without your complicity, your brain will not maintain its high panic state, and will settle down again in a few minutes. If you happen to imagine it as an obnoxious pinkish-grey wrinkly thing running back and forth, waving its arms in the air, and screaming at the top of its lungs, that might speed things along.

Brains exhaused after their panic

Sometimes you just have to wait them out.

Exam panic is only a disaster if you think it is. If you begin to panic, and mistakenly believe that the panic is the result of an accurate assessment of your situation, then more panic follows. Even worse, when you panic, your cognitive functioning can diminish- amongst other things, you can forget what you’ve studied. So now you’re suddenly unable to remember anything you studied, and becoming convinced that you are facing catastrophe. This leads to the all too common experience of blanking on an exam only to suddenly remember all of the answers 30 minutes later, once you’ve begun to relax.

Brain not committed to behaving itself in future

Unfortunately, you can’t leave it at home.

Fortunately, this can be managed by expecting that your brain will do stupid things in response to stress, realizing that you might have to let it freak out for a while, and then just waiting until it has regained its composure.

Categories: Challenges, For students, Learning strategies | Tags: , , , , | Leave a comment

Help for students, part 3: Reasons for miscalculating course expectations

Bert and Sally are two students who ran into difficulty on their final exams, and complained that there was an unreasonable amount of material to memorize for the course. So why did Bert and Sally not try to understand the course material rather than just memorize it? Maybe they thought memorizing would be easier and faster, or maybe they weren’t far enough along to transition toward understanding. But I suspect there is another reason. I suspect that they underestimated how much they would be expected to know, and how well they would have to know it. As a result, they prepared too superficially.

1. Approach to the assignments

In the course that Sally and Bert took, students do assignments which are often accomplished by paraphrasing the textbook in a way that is only slightly better than copying it outright. Because of this approach, they can get the right answers (and therefore good grades) without actually understanding key parts of what they’ve written. There seems to be a chain of reasoning that runs: “I didn’t really understand the assignments, but I still did well on them. Therefore, I will be able to do well on the final exam with a similar level of understanding.”

In fact, it is never safe to take one’s performance on assignments at face value unless one can be confident that the conditions of the exam will match the conditions of the assignment. For example, if you refer to your textbook while solving physics problems, this is not the same as having to solve a physics problem on an exam without your textbook and while under pressure. A good grade on that physics assignment would tell you very little about how you will do on the exam. In the case of the course that Sally and Bert took, a look at students’ performances over many offerings of the course shows that there is effectively no correlation between assignment grades and exam results.

Another problem with paraphrasing the textbook very closely is that while I suspect that students who do so are not clear on what they are writing, I have no way of knowing for sure what they do and don’t understand. That means I may give a student full points on a question even though that student might have misunderstood the text that he or she paraphrased. In that case, getting full points might convince a student that his or her understanding is correct, when in fact it isn’t. That student has eliminated any chance for me to find the error… until I grade the exam, that is. Then I hear from Berts and Sallys.

2. Reasonability assumption

When filtering out what is and isn’t necessary to study, a starting point might be the assumption that an instructor will not be unreasonable and will avoid demanding complex details, or asking questions about extremely difficult topics. One problem with this assumption, however, is that someone who is new to a field of study will not have the same perception of what is difficult or complex as someone who has worked in the field for a while. An idea that might seem complicated to the uninitiated could be a very basic principle in that field. A second problem is that sometimes a complicated or difficult topic can be very important for a particular area of study, and therefore necessary to learn even though learning it might seem nearly impossible at the time.

You may think that another problem with the reasonability assumption is that some instructors are unreasonable and use exams to punish students. I can’t say that’s never the case, but I will point out that a “reasonable” exam is not an exam that any student can pass- it is an exam that a student can pass if he or she has done a reasonable job of covering the stated course objectives.

In the end, if you’re not sure about whether something is important or not, and you can’t determine that from the learning objectives or course objectives, just ask your instructor.

3. Perceived importance of the course

I sensed that Sally was unhappy about taking the course. It was the last one she needed to get her degree, and she was anxious to move on with her life. She seemed to feel that the course was a pointless hoop to jump through, and just wanted to get it over with. Understanding the course material was not a priority for her, and maybe her feelings about how much work she should have to do for the course coloured her perspective on how much work would actually be required.

Sometimes students in Sally’s position assume that the instructor understands that the course is not important to them. They assume that the instructor knows better than to make the course too demanding and get in the way of a student graduating. However, even if a student’s reasons for taking the course colour his or her expectations about what the exam will be like or should be like, it does not affect the reality. The requirements will be the same regardless of why a student is taking the course, and students should expect that there could be the same kinds of demands as in courses that they view as more serious, or more important for achieving their goals.  Put another way, no-one should expect to get credit for a course without fulfilling its requirements.  I would also recommend against telling your instructor that he or she should pass you because the course doesn’t matter.

4. Learning is what someone else does to your head

Every now and again I run into students who prefer to be passive participants in their own learning. These are students who think that I should put more effort into helping them than they are willing to put into helping themselves. Frank was a classic case. Before assignments came due, he would email to ask what pages the answers were on in the textbook. An email exchange with Frank would look like this:

Frank: “I can’t find the answers to questions 1b, 1c, 2a, 2b, 3, 5, and 7. Can you tell me where I should read in the textbook?”

Me: “For question 1b asking what igneous rocks are, you can find the answer in the section titled “What Igneous Rocks Are” starting on page 53.”

Sometimes it is hard to find a specific topic amid other details, so I explained to Frank how he could look up the page numbers in the index of the textbook. Frank disregarded my explanation, and continued to ask similar questions.

I want to be careful to distinguish between students who take a passive approach, and those who ask a lot of questions about different topics, those who ask for help with the same topic repeatedly, or those who need assistance deciphering their textbooks. By definition, these students are not taking a passive approach because their questions have arisen out of an effort to understand the course material. In fact, I would prefer that more students contacted me with those kinds of questions. But this is very different from asking me to look up pages for you in the index.

Students who are passive about their learning will inevitably underestimate the amount of understanding that is required because they believe on some level that learning and understanding are things that they are given. That’s just not the way learning works.

Categories: For students, Learning strategies | Tags: , , , | 1 Comment

Help for students, part 2: Memorizing vs. understanding

Sally and Bert are two students who fell prey to unknown unknowns on their final exam. They both sent me emails complaining that the exam was unreasonably difficult- that they were required to supply more information than a student could reasonably be expected to memorize.

Sally said:

“I found the exam very difficult obviously and would like to see the format change as the amount of content that needs to be memorized is something I feel uncapable [sic] of. If it were a multiple choice exam, I believe the outcome would have been different.”

The exam did require more knowledge than a student might reasonably be expected to memorize, and that was because the students were expected to understand the course material rather than just memorize it. Memorization is a very inefficient way to attempt to store information. Understanding is much better. It’s the difference between learning the lyrics to your favourite song by trying to remember the words in random order, or learning them as they fit into lines and verses and tell a story.  One is next to impossible, and the other you can do after listening to the song a few times.

That’s not to say there is no memorizing involved, but ideally the situation would look something like Plan A in the diagram below. The diagram is a sketch of what part of your knowledge would depend on memorizing, and what part would depend on understanding as you progress through the course. Initially, there is quite a lot of memorizing required as you encounter new terms for the first time, but at the same time your understanding is increasing. Eventually, you are able to add new knowledge by building on your understanding, and less memorizing is required.

Memorizing versus understanding

If a substantial amount of learning gets left to the last minute, then to be as prepared as in Plan A, learning must happen a lot faster. In that case, we’re looking at Plan B where memorizing and understanding are condensed into a small amount of time. Unfortunately, your brain can only learn so much before it needs a break, so what actually happens is C. There is insufficient time to prepare, and that time is taken up mostly by memorizing because you don’t yet have enough of the individual puzzle pieces to start to build the big picture.

Categories: For students, Learning strategies | Tags: , , , | 1 Comment

Help for students, part 1: Breaking the curse of the unknown unknowns

Students often ask whether I can offer any tips on preparing for and writing exams. Sometimes they are new students who haven’t developed study strategies yet, and sometimes they have just become frustrated with strategies that don’t seem to be working for them. Sometimes they are panicked and desperate, and end their emails with “HELP” followed by several exclamation points. (Never a good sign.) So I thought it might be time to jot these things down in one place, rather than writing them over and over again in emails to unhappy students who waited to ask for help until it was too late.

If there is one thing that causes more problems for students preparing for exams than any other, it would be the unknown unknowns:

“…as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know.”  Donald Rumsfeld, US Secretary of Defense, 12 Feb 2002

When studying, known knowns are the topics you are confident about, and which you are right to be confident about. Known unknowns are the deficits in your knowledge that you are aware of, and which you therefore have a chance to fix. Where you get in trouble, however, are the unknown unknowns- the deficits in your knowledge that you don’t realize exist. You can’t fix those because you don’t know they’re there. At least, you don’t know they’re there until you hit an exam question you didn’t realize you were unprepared for. Then they become known unknowns, but it’s too late to do anything about them.

Here are two examples of what a run-in with unknown unknowns can sound like. Unfortunately, I receive emails like this on a regular basis:


“I realize I am not going to pass this course even with the 20+ hours I studied over the last week. I have trouble putting the definitions on paper, I remember reading them and seeing them but can’t find the definition…”


“I felt as though I at least I completed the test and did not leave it blank, and felt confident that half my responses where right, but must have gotten confused…”

Note: “Bert” and “Sally” are not the real names of these students, and may or may not reflect their gender(s).

Sally’s unknown unknowns turned into known unknowns during the exam. In contrast, Bert emailed me because he was shocked that his exam grade was so low- Bert’s unknown unknowns were so sneaky that he got right through the exam without even noticing them.

Both Sally and Bert blamed the exam format for their problems. Their exam was short answer, and they felt that if they had clues in the form of multiple choice questions, then things would have gone better. As Bert put it,

“… there is no way someone first year can be capable to do this, let alone without instruction, or scientific key terms without getting terms mixed up, since there is no multiple questions [for] deductive logical reasoning…”

I think that part of Sally’s and Bert’s problem was that they underestimated how much understanding they would need to be successful on the exam. Ultimately, though, exam format should not be an issue.  If you know the answers, it shouldn’t matter whether the question format is short answer, multiple choice, essay, or interpretive dance. If you know it, you know it, and if you don’t, it makes just as much sense to blame your pencil.

The main problem that Bert and Sally had is that brains can be deceiving. In Sally’s case, after more than 20 hours of studying, everything looked familiar to her brain, and she believed it when it told her that she was ready for the exam. Unfortunately for Sally, the appearance of the page was what was familiar, not the information on it.

For both Sally and Bert it would have been a simple matter to set a trap for the unknown unknowns: if Sally and Bert had put their notes away every few minutes and tried to explain verbally or in writing what they had just read, they would have found very quickly that they couldn’t do it. Then they could have fixed the problem. Unfortunately, this is very hard work and should not be done for more than 45 minutes or so without taking a break. In Sally’s case, after many sustained hours of studying, her brain would have been too tired to manage it. She probably continued reading and not absorbing partly because she was too tired to do anything else.

Some of the sneakiest unknown unknowns hide so well that you might need someone else’s help to find them. Those are the kind where you remember information, but don’t realize that you have some part of it incorrect. The best way to trap these is to work with someone who might be able to pick up errors in your understanding as you explain the course material to them. This could be someone else in the class, or just a friend who asks you questions by referring to the textbook. Here are a few strategies that I’ve found helpful for turning unknown unknowns into known unknowns:

  • Scare them out into the open: Imagine that your instructor were to call you out of the blue to ask you questions about the course. What would you not want him or her to ask you about? Along the same lines, what would you not want to be asked about on the exam?
  • Treat learning objectives as questions and attempt to answer them without looking at your notes.
  • Reorganize information into diagrams and tables. For example, if you made a table to compare and contrast Neanderthals and Cro-Magnons, you might find that you can say something about Neanderthal body size, but you don’t remember how that compares to Cro-Magnon body size.   Diagrams and tables have the added benefit of being much easier to remember than lists of facts.
  • Study by explaining topics out loud to yourself or a friend. There is a difference between reading facts and trying to mentally organize them so you can say them out loud, and that difference can be enough to throw you off balance and expose unknown unknowns.
Categories: Assessment, Challenges, For students, Learning strategies | Tags: , , , | 3 Comments

Why I don’t give extra credit assignments

I view extra credit assignments as problematic because they can be unfair to other students in the course, they don’t necessarily solve the problem of missed learning outcomes, and they’re a hassle for me.

Let’s say I’m teaching a carpentry class called Potting Sheds 101. Students sign up to learn how to build potting sheds. Their final exam is building a potting shed. They may or may not go into the potting-shed building industry after graduation. On the last day of class the final projects are evaluated. Bob’s potting shed is out of square, and collapses when the door is opened. Bob fails. Later I receive the following email from Bob:

“Hey, how are u? I’m Bob in Potting Shed 101. I failed my final project. It’s been a really hard month for me. I was sick for the last two weeks, plus I didn’t have money to buy the textbook or a hammer. I found the final project did not suit my learning style, and was shocked at how difficult it was. Talk about being expected to run before even learning to walk! I will definitely be commenting about this in the course evaluation. Plus I was delayed getting started because I had to borrow a hammer from the library, and it was recalled and still hasn’t been returned yet. Potting Shed 101 is the last class I need for my degree, and I don’t plan to build potting sheds for a living, but I really need to pass the class to graduate. Is there some extra credit work I could do to pass the course with a high enough grade to get my degree? I feel I already learned a lot, and I would need at least 65 to graduate.”

 So what should I do with Bob? Here are some considerations:

  • Bob knew he would need a hammer to build potting sheds. Other students made sure they had the supplies necessary before signing up. It is unfortunate that Bob doesn’t have a hammer, but does this justify extra credit work?
  • Bob says he was sick, but I can’t verify that independently. Previously, Bob didn’t say anything about being sick, but if he had I would probably have granted him an extension to complete his potting shed.
  • Bob should have expected that building a potting shed would be part of Potting Sheds 101, so I don’t accept his argument that the final evaluation was unreasonable.
  • Bob is suggesting that the class doesn’t mean anything to him, but is just a course that his program required for some reason, and that he won’t use the skills (although he still claims to have learned something).
  • Bob expects that whatever he will do for extra credit will get him at least 65% in the course, and can be done in time so that he will graduate as expected.
  • If I give Bob the opportunity for extra credit, are the other students any less deserving? Should they not be allowed extra credit projects too?

What if I cave in to Bob’s request? Bob suggests that he make ten bird houses for extra credit. Bird houses are not potting sheds, so he would be getting credit for doing a task that is much easier than the original task. Bob assumes that demonstrating a willingness to work hard is equivalent to demonstrating competency in potting-shed building. While a good work ethic is admirable, it is not the same as being able to build a potting shed. If Bob changes his mind about working in the potting shed industry, he will use the grade I gave him to convince an employer that he can build potting sheds. If Bob shows his grade in potting-shed building to prospective employers who don’t deal in potting sheds, they may take it as a sign that he is somewhat handy, has reasonable hand-eye coordination, and can handle complex tasks that require precision and attention to detail.

Let’s go one step further and assume I let Bob hand in his 10 bird houses. They are consistent with his skill at potting-shed building. Am I required to give him extra credit even though his work is substandard? If I don’t, must I allow him to do extra extra credit work?

What if the day after Bob hands in his 10 bird houses, Carrie sends me an email:

 “I heard you let Bob build bird houses for extra credit. Can I build bird houses for extra credit, too? I’d really like to improve my grade because I want to get into the Advanced Potting Sheds program.”  

This is a very competitive program, and if I let Carrie do the extra credit work, her grade would not reflect her skill at potting-shed building, but it would give her an advantage compared to other students who apply to the program.  Is that fair?

Then I hear from Marty:

“I heard you let Bob hand in bird houses for extra credit. I made some when I was in grade four. Can I hand those in for extra credit?”

If Marty has demonstrated the skill, does that not count? If he had brought a completed potting shed to class on the first day, should he have received credit for the course? Some would say yes.


“I heard you were taking bird houses for extra credit. My neighbours have some. Can I get credit for those?”

I would have to explain to Beatrice that she must make the bird houses herself. She would then request step-by-step instructions on how to build a bird house, and ask if she could come to my office hours to get help.

On a box delivered to my front door, containing 20 bird houses with the “Made in China” stickers still attached:

“Here are my bird houses for extra credit. Thx. Pete”

In an email from the department head:

“WHY are you letting students build bird houses for credit in Potting Sheds 101? They’re supposed to be building POTTING SHEDS!”

You see, it’s just way too complicated.

Categories: Assessment, Challenges | Tags: , , , | Leave a comment

Crowd-sourcing distance education (or, Why Athabasca University’s problems are just getting started)

mortar quoteLast week there was another missive from Athabasca University interim president, Peter MacKinnon. The post appears to be damage control after a Metro article by Jeremey Nolais, Fears arise that Alberta’s Athabasca University will be lost as tough budget looms.  The post says that while Athabasca is facing “financial challenges that are acute,” and “a decline in the rate of enrollment growth,” the rumors in the media that it will be merged with another school or shut down are untrue.

What I found interesting about the article were the comments. There are only 6 of them at this point, and three of the comments were complaints about insufficient interaction with tutors. They weren’t complaints about the call centre model, where students contact the call center and a call center employee determines whether an Academic Expert should be informed that the student needs assistance (contact ensuing within 2 business days). Instead they were complaints by students who had tutors but felt that they weren’t hearing from the tutors enough. As one student says, “… I did not pay to be completely ignored and paid to be TAUGHT.” [original emphasis]. There were complaints about the quality of education, and the blame for that laid at the feet of tutors as well.

Given the controversy surrounding the call center, and the seemingly obvious thing to comment about- that those who were unhappy with insufficient contact from their tutors could rest assured that they would soon have no contact with a tutor whatsoever- it is surprising that no comments of that nature were posted. After some experimentation, I determined that those points of view were being moderated out. The upshot is that readers will come away with the conclusion that what is wrong with Athabasca is its tutors, which is very convenient for the administration at present. However, there is also a very real risk of discouraging students who might otherwise register for courses that are still running under the tutor model. If someone at AU thought that was a risk worth taking… well, make of that what you will.

I’m not saying that all tutors do the job that students want them to- some tutors may not do the best job they can for any number of reasons, and some students may have bought into the misconception that they have a full-time teacher assigned to them.

But Athabasca’s problem isn’t tutors not doing what students want them to. Its problem is a structure designed in the days when distance education required sending students boxes of paper in the mail. This is a problem because the system that can most efficiently manage hard-copy course materials is one in which teachers cannot have the autonomy to alter their course materials at will to respond to students’ needs. Think of the nightmare that keeping track of document versions would be! There would be no control on the workflow (and therefore on costs) if instructors could alter materials whenever they found a better way to teach. In order for the school to function, teaching has to take a back seat to operations.

kids these days

Kids these days

On the surface, it would seem that Athabasca has moved past this, with an online learning management system, online exams, and digital textbooks. At its core, however, Athabasca is still structured so that it is necessary to inhibit its teachers in order for it to operate as efficiently as possible. The reason I think that Athabasca’s problems are just beginning is that the costly and harmful structure they are fighting to protect is rapidly becoming unnecessary for distance education.

Think of it this way- these days most universities run courses through online learning management systems. Using these systems, instructors can post documents, set up quizzes and exams, post video links, post videos and podcasts of lectures, host class discussions, and more. It is entirely doable with a very minimal outlay for me to broadcast my in-class lectures live online, and have students answer questions in real time through Poll Everywhere while watching that lecture. We could discuss their answers as a group, and I could adjust my lecture on the fly to address issues that they were having. Students could even submit questions through Twitter during the lecture.

With courses set up this way, no-one sends students a box of paper course-materials. Students download and print what they want, access the rest online, and purchase textbooks directly from vendors in the format they want. Students can take a course and engage fully with a community of learners and an instructor without being in the classroom, whether that course is designated as distance education or not.

In contrast, Athabasca is structurally incapable of empowering its front-line teaching staff to act in their students’ best interest. It has people to grade papers and answer questions, but it doesn’t afford those people the mantle of teachers, the salary so they can be committed to students full-time, or the autonomy to fix issues with courses as they arise. Consider this: I don’t have access to the course materials that I wrote.  If I want to fix a typo, there is a separate group of people who handle that sort of thing, and I have to make a request to get it done. I had to hunt around to find out who those people were. If I had the same control over my Athabasca courses as I do with some other courses I teach, I would just take the three seconds to fix the typo myself and not tie up IT people, and who knows who else. I would likely be updating the course regularly to improve it, which means that a separate expenditure on a Subject Matter Expert (who is also me) to revise the course every so many years would be unnecessary.

From a business perspective, it might have been safe at one time to compromise on teaching if you were the only game in town that could mail out those boxes of paper. But what happens when mailing out boxes of paper becomes irrelevant to serving students at a distance? What happens when the competition is no longer other distance education schools or programs- when it becomes hundreds or even thousands of individual creative, energetic, and innovative instructors at traditional brick-and-mortar schools who choose to build and manage their own online courses? What happens when the additional cost of running those courses is trivial, because the resources are already there as part of how on-site students are served? Well, what happens is that the competition is essentially crowd-sourced, and can do a better job with lower costs and happier teachers.

I don’t know what will become of Athabasca. As long as it offers programs that no-one else does, there will be a demand for its product, and perhaps it will begin to focus on that segment of the market instead of a broad swath of undergraduate courses. But if it does offer programs that no-one else does, that will have more to do with no-one else choosing to offer those programs, rather than being unable to do so in a cost-effective manner. Athabasca will not change the way it does business because it is firmly committed to the notion that as long as the school is run as a business, the rest will take care of itself.  The call centre model- where by design, the first person students talk to will never be the person teaching them- is evidence of that. There is an entrenched culture which holds front-line teachers in such low regard as to view answer databases and non-teaching call-center employees as a better alternative.  This exists because at some level, Athabasca views itself as an organization for delivering courses rather than for teaching students.

Categories: Distance education and e-learning, Learning technologies, The business of education | Tags: , , , , , , , , | Leave a comment

Blog at WordPress.com.