Arnold Kling, economist and teacher, talks with EconTalk host Russ Roberts about recent technological innovations in education and Kling's forecast for their impact on learning and how they might affect traditional education. Examples include the recent explosion of online lessons and classes, new teaching styles that exploit those offerings, and the nature of learning in various kinds of classrooms and student-teacher interactions.
About this week's guest:
About ideas and people mentioned in this podcast:Articles:
Web Pages:
Videos, Podcasts, and Blogs:
| Time | Podcast Episode Highlights |
|---|---|
| 0:33 | Intro. [Recording date: September 28, 2012.] Russ: Topic is education and in particular, online education, a field that is just starting to take off. We are going to talk about what it is, what its impact might be on traditional methods of teaching folks. And as a jumping off point we are going to use a recent essay that you wrote for The American. And if we run out of things to say about education--which I think is unlikely--I may pick your brain on other topics of the day. But our plan is to talk about education. You start off with a very thoughtful observation about the stagnant nature of educational technology. What was your point there? Guest: Well, that if you ask a, if you took a farmer out of the 18th century and put him on a farm today, they would not be able to function. If you took a factory worker out of the 18th century and put him in a factory today, they wouldn't be able to function. But if you took a professor out of the 18th century and put him in a classroom, they might be a little surprised to see a whiteboard instead of a black chalkboard, but they would be able to function pretty much just fine. Russ: Of course their knowledge might not be up to date. It depends on the field, though. Teaching Shakespeare, they should be just great. It's a shame those lectures are lost, actually, of the past. Now, that was until recently, at least--that education hasn't changed much. We did have the opaque projector. I mean, that was a breakthrough. But you are right. The technology is: You stand in front of a room and you tell folks some things where you talk at them or listen to them, answer some questions, and there's a text sometimes that helps you organize it. But that hasn't changed very much until recently. So, what you do in this article is you take a number of recent innovations that use technology and you talk about whether you think they are winners, losers, or a magic bullet. And by "magic bullet," I assume you mean really great. So, winners, losers, and really big winners, potentially, in terms of their effect on education. So, the first one you look at is what's sometimes called massive open online courses. This would be Khan Academy, Udacity; there's a number of other-- Guest: Coursera. Russ: Udemy. The idea here is that anyone all over the world, not just at any time, not just when the course is being taught, can get online and access the material and learn. And to my surprise--I'm a big fan of this, and may defend it--but you call it a loser. Guest: Yeah. I think it's overhyped, at the very least. I don't think it's a killer application or a magic bullet to take the course that you maybe lectured 100 people and blow it out to 100,000 people. I think that's kind of the wrong direction, trying to take what's a relatively ineffective education process and sort of make it bigger. I think education has to be more customized, has to be more toward the individual. And my impression about what happens with these massive online courses is that you get two extremes that benefit. The people who are extremely close--let's say, use a Stanford massing online course as an example. So the students on campus actually benefit, because the course was sort of geared to them to begin with. And they have the option of attending or not attending class. And they often find that not attending works just fine. Russ: And they watch the material being covered on the web. Guest: Right. Russ: As opposed to skipping it. Which also happens in universities. When you say "they don't show up and that's fine," they don't show up and they watch the material online. Guest: Right. And then when they encounter quizzes or projects, if they want to ask somebody down the hall for help, that person down the hall is available just as they would be if they were taking the course in the class. So, it's not that different an experience. I tend to assume that the lecture is one of the least important components of education, and so if that gets moved to a different environment it doesn't make that big a difference. So, that group of students, the Stanford students, who have colleagues who can help them and for whom the course is geared, they do fine. Another group that benefits is the group that's farthest away from Stanford. People in, let's say, the former Soviet Union, or in Africa, who happen to have the ability and enough background to handle the course but who never would have had the money to come to Stanford and to take it. And that's very few people, because there are not very many people in those places that have that ability. But they benefit. So, you have those two extreme groups. And in the middle, you have tens of thousands of people who don't benefit at all. My guess is that the drop-out rate from these courses is probably 90, 95%. So that you get a huge proportion that have no benefit at all from this. |
| 6:28 | Russ: Well, I think that's overly pessimistic. Let me push back and you can respond. So, Sebastian Thrun taught a course Artificial Intelligence, I think at Stanford, as the first example of this that you are using as an example. Where I think something like 100,000 people took it around the world. A few hundred took it at Stanford, live. As you say, a bunch of those watched it online rather than physically come to class when the class was scheduled. My memory is that actually a substantial number did finish the class. We can debate about what that means. But I think it was something like 20,000 did all the homeworks. So, the dropout rate--not 95%. More like, maybe oh, whatever it is, 70, 60%, something like that. 75%. Maybe 80%. But still a pretty good attendance. A lot of people had an incredible experience; they couldn't have done it before. We can ask why they dropped out. Maybe it wasn't meant for them. It could be that the material wasn't presented in a way that was tailored for them, as you are suggesting. Which I think is part of it. But I think where the potential for this technique lies is in different kinds of material. I think the potential is different for different kinds of material. For example, in one of their courses--this is Udacity, now--Udacity offers a course on how to program in Python. That's something that you can learn, get better at. You could master it. You can't master Keats, Shakespeare. Probably not even microeconomics, by following a set of prescribed steps and being quizzed and responding. It's a lot harder. Different set of ideas. But for things where there is potential for technique and mastery such as computer programming, possibly statistics, maybe some kinds of economics--don't you think the ability to give access to that kind of material in a way that is visually interesting and that is potentially more interactive than a traditional classroom with 100 students--don't you think that has some real potential? Guest: I think it does. However, I don't think that taking a traditional course and migrating it to the online environment is the best way to do it. The best way to learn computer programming is sort of on an as-needed basis. I don't think a computer course as traditionally structured is usually the best way to learn it. It's certainly easy to learn programming online. I mean, I did some programming this summer and it was just a much different experience than it was 15 years ago. Fifteen years ago you'd have had two or three books spread out and you are like flipping pages through books. And now you just ask Professor Google, and Professor Google links you to a little short video that demonstrates exactly what you need. But it's Professor Google. It's not a structured course taught by one person. It's just little snippets from different people that you pick up in order to learn computer programming. Russ: It's more crowd-sourced. Essentially. Guest: Yeah. Russ: Let me ask you one more question about something you just said: You said lectures aren't very important. I certainly agree with you that if your idea of teaching is to read out of the textbook or to reproduce what's in the textbook, the textbook has already honed the best way to transfer the material, that would be true. But of course that's not true. And it really makes your point--that if you think you are going to port your classroom lecture onto the web as a way of making your web-based class on the web, you are not going to gain much. And similarly if you just read the textbook in front of your class, they are not going to get much, either. Clearly what happens in a great classroom is something intangible that's hard to describe, where it's partly what we're right now--it's partly dialog; it's partly a back-and-forth, question and answer. But a great lecturer isn't just the equivalent of a good book on the topic. You get into that person's head and you get a whole new way of looking at the world. Guest: Perhaps. Or perhaps you just get a little bit of entertainment. Maybe you've convinced yourself that you are learning something. It's really hard to say. But one of the things that you'll notice if you play around, search around in some of these, even some of the well-known, massive online courses--like, I went to a statistics course--I don't know if it's part of Coursera or one of these large popular consortiums. And it's taught by somebody at Princeton; and it's obvious that he just said: Well, I can just film the lectures I've been giving all along. So the course begins with a 25-minute lecture on histograms. And I submit that there are very few people who will put up with a 25-minute lecture on histograms if they are not going to Princeton-- Russ: Feel like they are getting their money's worth. Guest: You just take one look at that and you go--I'm sure it's a great lecture on histograms and that his idea of using that as a way of introducing statistics is valid. But that just will not cut it long term. My prediction is that that course will not survive as a popular course. Russ: And was he being filmed standing in front of something or was it just his voice with histogram stuff being filmed? Guest: Well, there's a little bit of him. I'm sure that they also show some drawing. It's more of him than you would think is reasonable, in my view. Russ: I think the big breakthrough of Kahn, the Kahn Academy, Udacity, and some of the other ones I've seen, is that they realize that filming someone teaching is not visually compelling. Unless that person is remarkably attractive, telegenic, and it's lit and produced in a high end way. And that would be a Technology, Entertainment, Design (TED) talk. So, a TED talk to me looks good; it's fun to watch. Whether you learn anything is a different issue; there's been a little bit of a backlash against TED. But I think the great insight of the Khan approach or the Udacity approach is that you listen to the voice, you don't watch the professor; the material you see on the screen is what engages you visually, not the person's face and mouth, etc. Guest: Right. But then I think people have gone beyond that. First of all, they realize that actual writing is too slow. So that the best-produced videos speed up the writing process to get the writing going quickly. The visual images move more quickly than they would if you had an actual professor writing on the board. And the next step is to realize that passively watching the video is not sufficient. So, the best ones are now adding questions that happen during the video that the student is supposed to answer. The good news is that's very engaging. The bad news is that you begin to see the challenges--that a lot of students are just not going to be able to answer those questions. And the responses that they get back are going to frustrate them. They are not going to explain why they got it wrong. |
| 14:20 | Guest: And so what that to me gets back to is the goal really should be to personalize learning. It shouldn't be to blow out your lecture to 100,000 students. But ask yourself: What would I do if there were just one student in the room? You wouldn't lecture to them. You would probably give them lots of things to think about, problems to work. And as they worked the problems you would coach them directly and say: Well, no, think about it this way. Or: here's what's wrong with your current way of thinking about it. Russ: A hint. Guest: Yeah: here's a hint. Here's another question that will kind of help you, an intermediate question that'll maybe help you get there. And there would probably be some emotional coaching as well--if the person is just completely tuning out, you would say: No, you can do this. Here, just take a moment and think about this. Or, maybe you are trying to go too quickly. Slow down. Again, I think that the best way to think about teaching is to think about what would you do if you just had one student in the room rather than think about: Wow, I could get 100,000 students to watch this lecture. Russ: Yeah. You and I have talked about these topics before, and that's one of the two deepest things I've heard you say about this. I think it's an incredibly provocative and important idea. When I think about it, I think about how I interact with my economics students. Traditionally, I teach a very Socratic class, where I am probing and pushing and asking who understands this? When they can't get it, if not enough hands go up, I'll give them a hint; not too much of a hint. There's a big sweet spot there, just like in the quizzes you are talking about. If the quiz is too easy, people just start skipping it because it's a waste of time. If it's too hard they skip it because it's a waste of time. You have to find--it's like a crossword puzzle. You have to find the right level of difficulty so people get the satisfaction of doing it well but not so hard that they can't get through it and they get frustrated. When I think about my children, the one-on-one model, when I teach my kids economics at the dinner table, it's something like what I do in class; but of course ideally it's better because it's totally tailored to their level, they are close but not quite, so I can give them the right kind of hint. Plus, I know them pretty well, so I know what kind of hint to give them so they don't get frustrated but they can make progress. Guest: And you also would not give them a lecture. Russ: Well, sometimes I give them a lecture. No, you are right. I don't sit there and expound. There are things that come up they are confused about; I might explain something. But in general, you are right. What we enjoy talking about at the dinner table are problems, things that are interesting to puzzle out and explore. And that's where the learning takes place. I don't want to miss your second deep idea. I'm sure you have more than two, Arnold. But the two that caused me to think a lot about these issues is your insight about feedback. Guest: Yeah. My line is that teaching equals feedback. It's really the back and forth between the student and the teacher that generates the learning. It's like, so the student is finding out where they are wrong or where they are right, where they need to go next. The problem with just videos or textbooks is that it's very hard for the student to get that feedback. And I think everyone who has tried the Khan Academy flip the classroom model-- Russ: Explain what that is. Guest: So, that's a model where instead of lecturing in class and sending the student home to do problems, you send the student home to watch video lectures and then have the students come to class and work problems and sort of walk around the classroom and help them work problems. And I've done that in my high school teaching. I'm very happy with it because it gives me the opportunity to be giving more feedback. As I'm walking around, I'm seeing where students are stuck, where they are doing okay. And they are constantly asking me for help and hints on problems. I think that that is the essence of teaching. That if the student is just passively watching a video or passively listening to a lecture, they are not sort of coming up against the sandpaper of working problems, of trying to see whether they've really grasped the material. You create this sense of uncertainty: Did the student really get something out of it? Does the teacher know that the student got something out of it? Whereas when the student does something themselves, then the teacher has an idea. A very common problem--you'll see this in books, advice for teachers--is a teacher will give a lesson thinking that the objective is x, and the student gets something completely different out of it. They either pay attention to an anecdote that isn't central, or they actually come away with the wrong point of view. Until there is some sort of feedback, nobody knows that that kind of disconnect has happened. Russ: I used to give what I called a one-minute feedback forum--this is for me to get feedback--at the end of every class. A few questions. One of those questions was: What was the most important thing you learned today? And often it was not the most important thing I wanted them to learn. That's a big wakeup call for a teacher. The other thought I had, thinking about what you just said, is that I think what makes a person a successful learner, a successful student, is the ability to self-monitor. So, I tell my students: When you read a passage, when you try to do a problem, do you say: Oh, yeah, I know how to do that; let me move on. When you kind of really don't: I kind of get it. And you really have to get to a point where you master an idea or a concept. And students of course can't do that at first. They don't know whether they've mastered it. I think one aspect of your point is you have to give them a way both to know whether they've mastered it and then to assess, to test themselves. And that's what I think grown-up learning is about. I don't mean just as an adult--at any point in your life: I know this now; I really understand this. And that is a real art. And it's something I often think as teachers we often fail to teach them how to do that. And to the extent we do teach that and the extent that students can learn how to do that, they become better learners. Guest: It's hard work to teach that way. If you really want to teach that way, what you do is you say, every time you assign something, you ask the student to write something about it. And then you comment on what they wrote. And then say: This missed the point; this was on the point; this was interesting, I hadn't thought of it before. But that constant feedback is what you would need to do. Russ: A standard technique of learning is to teach people what you have learned to see if they've learned it. And that's one of the things teachers do when they teach. You think you understand something, but when you teach it you realize you didn't understand it as well as you thought you did. I always tell my students: When you understand something, tell someone else. See if you can explain it to somebody else. And that is a measure of self-assessment. Guest: Yeah. And that's a good teaching practice. Put something in your own words. Something good teachers are always asking students to do. But again, that's hard work. It's a lot harder work than tossing off a lecture and saying: I assume people know what I'm talking about. Russ: Or giving them some multiple choice questions to see if they can understand what the key points were. Which is not the same as using the material. I mean, my main complaint about most textbooks isn't the passive aspect of it. It's related to that. But it's really the idea that, economics, say, is a set of facts. The ratio of the prices equals the marginal rate of substitution, and maximizations in an optimal situation. That's good for spitback but it doesn't teach you to think like an economist. There is no really good textbook on how to do that; and there's no online aspect of that. And I like to think part of what we are doing here on EconTalk is a little of that; but even here, we are doing a bunch of other things at the same time. I think there is an opportunity, if someone can figure out how to do that on the web to make a real impact. Guest: Yeah, well that is certainly a challenge. The easiest sort of feedback to automate is the multiple choice type question. The hardest feedback to automate is the open-ended question, the critical thinking question. I think we are so far--the distance we need to do that is so far that it'll be a long time before we are capable of doing that. Russ: Right. In the example that you gave--I used to tell my students this all the time--which would be: If you think you understood it, write it up. Put it in your own words. When they'd say: How do I study for your exam? I'd say: Why don't you go home and try to write up some of the ideas we talked about in your own words? You'll find out if you understand it and you'll get better at thinking about it as you write it up. But ideally you'd do that every night and you'd grade it. You'd give personal feedback on the quality of their write-ups. And some people do that. You can have a blog that you grade where students write things up. But I think that's a superb teaching method that's incredibly time intensive. Guest: And we don't know how to automate it. You do need a teacher to do that. |
| 24:32 | Russ: Let's talk about some of the other innovations that you discuss in your article. You're a big fan, I think for reasons you've made clear in this last few minutes, of adaptive textbooks. What is an adaptive textbook, and why are you a fan? Guest: It may be more of a dream than a reality, but it would be a textbook that responds to students' questions, that give students questions, and adapts to their level of ability. So, if a student shows that they have mastered some material, it would give them harder questions. If a student is struggling, it will keep repeating material until they learn it. The other sort of fantasy for an adaptive textbook would be one that sort of got a sense of the student's mood. So, if it sees the student is just rushing too quickly, it does something to soothe them, slow them down. If the student is angry, it tries to soothe them. If the student is impatient, it tries to deal with that. If the student loses confidence somehow, an adaptive textbook would kind of encourage them to move on. So that would be the fantasy version. The closest thing to reality might be, there's a biology textbook that was announced a couple of months ago, an online, or actually I guess it's an iPad version, an app. It's called Project Inquire, where they used a knowledge map to make it possible for a student to ask a question in the student's own words and then get answers back from the textbook. So, if the student just has enough curiosity and a little bit, just enough curiosity to ask the right questions, they can kind of work their way through the knowledge base by asking questions and getting answers. They claim that they've tried that and shown that it works. Russ: They claim. I don't know. I think there's an enormous, as you say, a gap here between the idea and the reality, of the application of artificial intelligence to this whole process. Your description of the adaptive textbook is really just a personal tutor. It's Mr. Google as personal tutor. It's the ability to access the "knowledge" of the web. It's kind of an absurd idea. The question is: What would the reality, how close can we get to that? Maybe not today but soon, that would allow a person to explore at that level and get that kind of feedback. I don't know if that's ever going to be possible. It's kind of a-- Guest: Yeah. Well, one scenario is that the world just belongs to self-learners. I've sometimes heard Tyler Cowen make this claim: that if you have somebody who has the self-motivation and the know-how to sift through what's available online, they will have tremendous advantages. And everybody who is dependent on a teacher of any sort will be at a disadvantage. That is one scenario that you can lay out. Russ: I don't know. Hard to know. When I think back on my formal education, I taught myself a ton. It's a huge part of learning. And I learned a lot from my classmates. That was a huge part of my graduate school learning, for example. But there were teachers who--I mean, they obviously taught me more than the material. They taught me things about integrity, passion, curiosity that inspired me. Guest: And also just watching someone's thought process and seeing how they tried to--in economics I think the most interesting ones are the ones who try to see multiple points of view. And see things from different points of view rather than present a single point of view. Russ: I remember being in Robert Lucas's class when he was working on business cycle models--this was 1976, 1977, somewhere around there. And he hadn't gotten very far yet. But we got to see him trying to figure it out. It was exhilarating, very frustrating as a student at times. But the main thing I got out of that class wasn't an understanding of business cycles. It was a respect for a great scholar who was trying to struggle with reality. It was phenomenal. I think there are a lot of--again, I think most of great teaching is a mix of that inspiration. And also some window-opening, door opening, or maybe a better word is window washing. There's something that's opaque to you and that teacher makes it clear in a way that you can't do on your own, you can't learn it yourself, it's a lot harder. I remember taking a class on William Faulkner in college--I didn't have much respect for Faulkner at the time; I was 19 years old and snotty. And the class was on Faulkner and Joseph Conrad. And I loved Conrad. By the end of the class I thought Faulkner was the greatest writer of all time and I thought Conrad was, eh. And I don't think I could have done that on my own, without Dr. Patterson [?]--I think that was his name. Anyway, I thank him for that. |
| 30:03 | Russ: I want to talk about some of these other examples. We talked about the flipped classroom. What's independent certification and why do you think that's important? Guest: One of the phenomena that's important in the United States today is that if you go back to the end of WWII, I think something like 5% of jobs required a formal education to get a license. Now it's something like, I read, about a third of jobs require that. Russ: It's because we have too many lawyers. Swelling the numerator there. Guest: Well, there's a lot of rent-seeking going on in the creation of these requirements. People have mocked them; people have mocked the requirement that in some states you need a license to give somebody a manicure. That's the extreme. So, what that says is that a lot of the value of education is just the credential, just the certification that you've learned it. So, let's say you had a self-learner now who could sift through the web and actually learn as much as a Harvard student about English and economics and physics and whatever. How could that student demonstrate that they've achieved that level of knowledge? Another way to put this is that if you are really going to scale up college education--and we can talk about whether that would be a good thing or not--but if you are going to sort of, you say: My goal is to have everyone go to college. If you are going to do that, then you can't have this tight credentials cartel that only certain places are able to certify that someone has a college degree. You are going to have to come up with alternative certification. So, if we are going to broaden access to college education, a necessary condition will be some way of certifying. I think an additional benefit is I think there's actually an agency problem in having the same professor teach the course and grade the course. Russ: A conflict of interest. Guest: Yeah. That you can--and it shows up in lots of ways. It generally shows up in grade inflation. My guess is that a lot of teacher evaluations, the rate-my-professor type stuff, really boils down to how generously they grade. Russ: Well known correlation between easy grading and positive evaluations. Guest: And that creates a real problem in education. But if separated out the grading from the teaching, that would completely reverse itself. The more rigorous teachers who were maybe tougher on you in class but who helped you earn a higher grade from an independent certifier, they would suddenly be popular instead of being unpopular. So, I think there's a lot to be said for separating the functions of teaching and certifying. And I suspect the main reason that that's not done is because control over the certification is considered by most colleges to be their main source of value for them. And they are very threatened by the notion of an independent certification. Russ: We'll come back to the issue of whether education is actually valuable or whether it's just about credentials. But I think: how could we do an independent certification? How might that work? Guest: Well, we have examples of it now. Things like the Advanced Placement tests for high school courses are a good example of that. It tells you independently whether you've learned it. Now, the complaint about something like that is it's teaching to the test. Suppose I'm a creative teacher and I'd like to teach something else. When I was an undergraduate I went to Swarthmore College, and they have this program that about half the students are in called the Honor Seminar Program, where the curriculum is made up entirely by a Swarthmore professor, but the exam is given entirely by an outside examiner, typically a professor from another school. So, the Swarthmore professor will send the syllabus to the outside examiner; the outside examiner will create an exam and then will grade the exam. It's really taking a risk from the Swarthmore professor's point of view. Russ: Oy. It's a frightening thought. Guest: Yeah. But it's really making a statement that we can teach. And you can go ahead and test us on what we say the syllabus is. I think that's very powerful. I don't know whether that kind of external evaluation can be done cost effectively for many institutions, but if you can come up with a way to do that, I think that would be a very powerful tool. Russ: I think it also, just thinking about it forces you to confront the fact that there's a very idiosyncratic aspect to so many classes. Right? What I expect my students to learn is not what you expect your students to learn; and my students wouldn't do very well on your test and your students might not do well on my test. Which means either one of us is a very bad teacher, or we are both teaching two things that are very different; or neither of is doing a good job. I'm not sure. Guest: Or that you really need to be clear about what your objectives are so that another person actually could write a test: even though they might want to teach the course differently, they understand your objectives clearly enough, they can write a test that's appropriate for yours. So, my line is for an independent certifier: instead of making somebody teach to the test, you have to be able to test to what you teach. Russ: Interesting idea. One last example and then I want to move on to more general issues. You are negative about clickers, in-class feedback devices, which would seem to be a good thing. It's feedback, it tells the teacher who is learning. Explain what they are first. Guest: Okay. I have to say, I haven't used them. I know some people who are using them. I'll be curious to see how they like them. I gather it's a device where a teacher can ask a question, maybe even a multiple choice question, and they have the student answer it. So you get an immediate idea while you are talking of what proportion of students are getting a particular problem. There are other uses for these things, but lets go with that one. There are other things--there are Smart Boards, all these things. My concern with all these tools is they are designed to reinforce a one-to-many type of teaching. And what I want again to think about is: What would you do if you had one student in the classroom? And if I had one student in the classroom I wouldn't give him a clicker. I'd just be talking to them back and forth. So that's why I didn't give that one a good vote. Russ: Yeah. I think I've referenced this story before; maybe you'll help me, maybe you know the story. I think the story is that Arthur Pigou was a student of Alfred Marshall. I think the timing for that is correct, right? Marshall's older than Pigou. And my memory is that he was the only student in the class. But that didn't stop Marshall from coming in, standing at the front of the class at the lectern with his notes and lecturing. But he had a different model. Guest: I guess Pigou survived that. Russ: Could have been had he had a different interaction with Marshall. |
| 38:26 | Russ: Before we move on, I want to give you one example of the clicker application, I've used in class. It was a physics professor, who would give a multiple choice question, like you said, and asked the students to click and vote what the right answer was. And most of the people knew the right answer, or most of the people got the wrong answer, if it was the right answer, he'd move on. If it was the wrong answer, he'd lecture some more and explain some more stuff. But if they were pretty evenly divided, he would then have the class, he would have the class break into groups; he would say: Find someone next to you with a different answer from yours and see if you can convince him that you're right and they're wrong. And then he'd re-vote. And this gets at the idea I mentioned earlier, that when you can teach it yourself you learn something. So this takes this classroom setting and makes it much more interactive. And then they revote. And he found that it actually--of course, sometimes someone with the wrong answer convinces other people that they are right. But most of the time the students who can't defend the answer because it's wrong learn that they've got the wrong answer. So, to me that's an incredibly valuable idea. You've got the students into teaching, not just listening. You've got them to engage with the material in a different way. You've got them reinforcing the learning. And when I tried that in my classrooms I was shocked at how the room exploded into conversation. I was afraid they'd sit there, but immediately they jumped into action. So, it's an interesting technique. Guest: And that gets into another topic, which is social learning. Students learning from one another. That's one of the-- Russ: Highly touted. Guest: Very hyped. There are a lot of online companies, startups, venture things, that say: We facilitate social learning. And I'm skeptical about that because first of all, I'm not sure that we know what works, what fosters social learning in a live classroom, much less how to reproduce that in an online world. Now, you gave an example that sounds like a great example of social learning. But I've seen things in real life where the student interrelationships are dysfunctional. They discourage each other. Russ: Sure. Guest: The morale of the class is actually lower than the morale you think you could get if you talked to them individually. So, I don't think we know what the magic is in social learning. I don't think we're ready to bottle it and say we can make that work online. |
| 41:12 | Russ: Let's move on to some general questions about education that these issue inevitably trigger your thinking about. The current state of high school and college, just as the technology hasn't changed much in the last 150 years, they haven't changed much either. The dorms have gotten a lot nicer and the food's better probably--no, not probably, certainly. And a lot more people are going. Do you think that lack of dynamism is a government problem, a lack of competition? Government is very involved in high school education. It's sort of very involved in college education, but there's a lot of private universities that are obviously affected by the public universities' competition. What are your thoughts on that? How do they look different? Guest: Well, there's one line of thinking that's very popular among academic economists, which is to say, let's look at cross country comparisons. America has great colleges and universities and we're mediocre to maybe even worse than that in terms of K-12. So, we've got probably the highest proportion of public education in K-12 of any country; and maybe the highest proportion of private universities and colleges of any country--so look at that correlation. We're great when we've got a lot of private competition and we're not so great when we have K-12. And so as an economist that makes you feel good; like you are saying, competition makes a difference. That's what makes us better in colleges and universities. Russ: It makes some difference. Guest: As an academic it makes you feel good because you are at a college or university and you say: We're great; it's the people in high school, K-12 who are bad. I'm skeptical just because in general the ability of different educational methods to make a difference is not well demonstrated. Russ: Yeah. Guest: So, you typically only find anecdotal evidence and statistical evidence tends to cut the other way. There's the famous study, I guess it's Krueger and somebody else comparing students who were admitted to Ivy League schools and state schools. You can think of three groups of students: students admitted to Ivy League and go, students who only get admitted to state schools and therefore have to go there, and students who are given the choice, who can go either way. And that third group of students who have the choice either way and then let's say choose to go to the state school, what do they look like? Do they look like the students who go to the Ivy League schools? Do they look like the students who had no choice but to go to the state schools? And it turns out that they look like the students who go to the Ivy League schools. It makes you suspicious that what's going on is that colleges and universities are selecting the better students. So, yeah, it's easy to say--another way of putting this is if you believe that there's some educational method that's superior to another method, I dare you to do the following, which is: swap a set of students who are subject to one method for the method that you think is better. So, if you think a suburban high school is really a better school than an inner city school, I dare you to swap the students in those schools and show me that you still observe the same direction of relative results that you had before. Russ: Yeah. Obviously there are some of what we call fixed effects, unmeasured variables that are explaining what's going on. Guest: And that raises the question of whether to think about education as a transforming system or a sorting system. So, a metaphor for a transforming system is that the students come in as clay; you shape them as clay and you turn them into pottery. Something. They are all the same raw material but you can make something beautiful out of them. The sorting mechanism would be, if you go to a bank nowadays you can take a jar of coins that you've dumped into a jar, pour it into this machine, and it will sort out which are the quarters, the dimes, the nickels, and so on. But it hasn't changed the coins. They are still quarters, dimes, and nickels. It's just sorted them. And when you look at education, when people talk about education they always have this idealistic view that they think of it using the clay model. But the reality is much more often the sorting model. It is very rare to see provable, durable, scalable examples of actual transformation, where you can see students being transformed rather than just sorted. Russ: I'm sympathetic to that view. But I think it clearly misses part of what's going on in some educational settings. I think if we armchair theorize--not theorize--armchair, I don't know, armchair history: you think about on your own education, which of course is a flawed exercise because your memory is very selective; it's you. But I think you can look back and say: I can identify classes where nothing went in that lasted. Robert Frank was on this program ages ago and we talked about the fact that students who take a standardized test in economics, 18 months after the class is over don't do any better than people who have never taken economics. I'm not sure it's quite that dramatic, but there is certainly some truth to that. There are classes that all you did was absorb some facts or some information and then after a while it got replaced by other information and facts, it didn't stay around. And it was just sorting: you got an A, which put you into the quarter pile, and somebody else got a C, which put him into the nickel pile. And that's what happened, that's the experience. At the same time, there are these teachers and classes that are transformational, that change the way you look at the world, that give you insights into things you couldn't have learned on your own or had a chance to learn on your own. So, it seems to me it's a mix. I think the question to me is: Were those classes transformational because of the space I was in, where I was ready to absorb those lessons? Or was it simply a great teacher and if we just had enough great teachers that could happen? Or is it the process? We just need to create a classroom where I can teach myself and I don't need a great teacher. It's the McDonald's model: you don't have to be a great cook to run McDonald's. They've done all that for you. And you get this standardized, pretty good meal. Is that what we should be striving for, or should we be striving to attract these extraordinary transformational people who can change people's lives? |
| 49:05 | Guest: We don't know the answer to that. It would be great to be able to do experiments. I know you've had Rick Hanushek, who claims that teacher quality makes a difference and in particular bad quality makes a difference. And maybe that's true. I recently saw an article, I think by Martin West or something, sort of, things that other countries do differently. Because other countries seem to have better K-12 outcomes. Again, that could be a sorting issue; it could be a teaching issue. But he cited three things. One is that they have more competition at the K-12 level from private schools. Now, can I remember all three on the spot? The second one--I'll be like Rick Perry--is that they have high stakes testing, not just a low bar, you pass this test and you get to graduate. And the third is that they draw teachers from a higher level within, a higher academic level; we get teachers from teachers colleges, where there is often the lowest of the college-educated group. So, that's an interesting claim. What I noted about it is those are all factors where any change would be resisted by the current teachers' unions. But it would be interesting. Because it's often claimed that one of the things that happened in the United States was that as women got access to occupations is that we used to have some of the brightest women in K-12 teaching but we moved them out of that into other occupations. And so that might have lowered the quality of our K-12 education. It would be interesting to sort of see a controlled experiment world where higher level of academic accomplishment people are put into teaching and you compare a similar group of students being taught by lower level accomplished people, and see: do you have durable differences? A lot of times when people say something works in education, the evidence is anecdotal. You can't prove that it was transformation versus sorting. A lot of times the difference disappears after a few years. So, your example of the economics not making a difference after 18 months--there are lots of examples where cognitive ability tests stronger after 1 year but after 4 years that difference goes to zero. And a lot of these things don't replicate. So, people will say: Oh, the Perry preschool project did this, but then you try something similar somewhere else and you see no impact. It's very hard to find--I dare somebody to find something that is durable, based on controlled experiment, and can be widely replicated. Very hard. Russ: I agree. Again, you think about yourself, what works for you. And what works for you in one setting doesn't always work in another. And what worked for you when you were younger doesn't work when you are older. I'm not sure that what you remember is the best measure of what you get out of a class, though. I think sometimes you get out of it something you can't put your finger on, not a set of facts. A way of thinking, or just some inspiration. Tough issues. |
| 53:22 | Russ: Do you think we have too many kids going to college in America or too few? Most people would say too few. Because everybody should go to college and college graduates make more money than non-college grads. Guest: So, let's think in economic terms: What does the supply and demand situation look like? There's some data that suggest there's sort of an excess demand for college students, and that is first of all the salary differentials on average are higher; and they are high enough that they more than make up for the cost and even the opportunity cost of going to college. Russ: Foregone time you have to give up before you can do what you have to do. Guest: And the other data are that they are more likely to be employed. So that in addition to the salary differential, you have the higher probability of being employed. So, you put those together and it seems like a pretty powerful case. The evidence that goes the other way, however, includes a couple of things. One is that a large and increasing number of college graduates are now doing jobs that you don't need a college graduate to do: waiting tables, or answering phones, what have you. So, if there's truly an excess demand for college students, society wouldn't waste those resources that way. They wouldn't be putting college graduates into jobs that you don't need a college degree for, if there's this unmet need for college graduates. So, there's something a little wrong with that picture. And the other alarming statistic is if you get outside of the top 100 or so colleges, the completion rate falls to 50% or less. And so that says that if anything we are sending too many students to college. I gave a talk recently a community college where a professor said: What exactly are we supposed to be doing with students who read at a 4th grade level? Am I supposed to be teaching an anthropology course to that student? What he's saying is a need for what we used to call basic adult education or remedial education, and if we are increasingly asking colleges or community colleges to meet that need, what's going on? So, overall I would make the case that we are trying to send too many students to college. That there are too many who cannot read and write a college level, and we need to--if we need more college graduates the way to get them will be either to import them from other countries or come up with some magic solution that makes people come out of 12th grade more capable of operating at a college level. Russ: But you use an example of somebody reading at a 4th grade level. You could also make the case that somebody who is very gifted isn't well served by college, either. There's a lot to be said for imagining that instead of hanging out with other 18-year olds, 21-year olds, drinking and going to class a few hours a week, you ought to be doing something real. Maybe taking some Udacity classes at night. But college is very expensive. It's four years, and there is the out-of-pocket, which has become quite expensive, whether it's paid for by the state or by the student. The opportunity cost is enormous. It's fascinating to me how little competition there is in alternatives that are life-transforming. Other than college. Guest: Yeah. That raises the issue of why does this endure as a ritual that people go through. We've had a lot of discussions, as you know--Bryan Caplan, who blogs on EconLog has suggested that it's all about signaling. So, imagine I'm thinking about hiring you and I interview. And in the interview you come off as having a lot of cognitive ability, so I'm not really worried about that. But you haven't gone to college; and that makes me wonder: Well, maybe you aren't really conscientious. Maybe the reason you didn't go to college is you clearly couldn't turn in papers on time. And as an employer, I need people to turn in their projects on time. Or maybe you demonstrate somehow in the discussion that you've turned in your papers on time, but now I get the impression that you could have gone to college but you just chose not to because you just want to be different. And what Bryan would argue is that you are signaling at this point that if you are bright enough and conscientious enough to go to college but you don't, you are signaling that you are willing to be different. That you don't accept the norms of society. And so again as an employer I've got to worry that sometimes you have to do something that doesn't make sense to you but we do it because we do it in the organization and you just are supposed to do it. Russ: It needs to get done. Guest: It needs to get done because the organization demands it; and if you won't do it, that's going to cause problems. And you may be exactly that type of person if in today's environment where the normal thing to do is go to college, you chose not to. You may be this rebel that it's going to be hard for me to manage. So, his argument is that right now not going to college sends a negative signal. Russ: It doesn't make you stupid; it's not that you lost a chance to get smarter. It's just that you don't have this credential that you need to prove that you can be a good peg to put in a hole somewhere. It's a depressing thought. Guest: Yeah, that you have demonstrated clearly that you are nonconformist. And in a world with many organizations you need some level of conformity. Russ: And the flip side of course is that by going and jumping through all the right hoops and getting the good grades, I've demonstrated stick-to-it-tiveness, conformity, reliability. Guest: And you are willing to do something just because somebody told you to do it. Russ: What do you think of that? Guest: I think it raises the question of there's a lot of $20 bills left on the sidewalk. That if somebody could come up with an alternative way of demonstrating that, that's less expensive than college, then presumably the world would beat a path to their door. If I could come up with a way of certifying that you are bright, conscientious, and conformist--the conformist challenge becomes almost by definition a problem--but if I could solve that, then if Bryan is right that that's the main thing then you could envision the economy jumping very quickly to a different equilibrium, where this alternative certification method dominates and where people leave college in droves. Because once it becomes conformist to accept the alternative certification, then the current college has got nothing, in some sense. Russ: And since they are so expensive, you can make an enormous amount of money charging something in between the tuition rate and zero, half of tuition. And people could be done in six months or two years. You could still have the four year thing, but it would be at a fraction of the cost. You wouldn't need the bricks and mortar. You'd need something, presumably, the alternative. But it suggests there is, as you say, a $20 bill lying on the ground, meaning that there's some unexploited profit opportunity. There's a lot of people talking about it, too, so it's not a secret that this profit opportunity is allegedly out there. Guest: Yeah. And a lot of people are trying it. And my guess is that some people will succeed at it. Again, go back to the technologies I think will help. I think some of them are farther away than people think. It's not just a matter of slapping up a few videos and a few multiple choice tests and saying we can replace college. But I think at some point people will start to pick up some of those $20 bills. Russ: To give you my perspective: I think college is three things. It's a sorting mechanism, it's a credential thing, some of it. It is a transformational experience, if you take the right classes and it's fit for you. But the third thing is it's a place, a time to explore who you are, what you are interested in, what you want to do with your life in a fairly safe setting. And that's an incredible luxury that hasn't been available to humanity until the last half century or so. And it's a statement about how rich we are, to think it's okay to take four years off to dabble in a few different subjects, see what you like, try to figure out what you're about, stay up late night and argue with your new friends. It's a consumption good. Guest: There's an episode of Dawson's Creek, which used to be one of my daughter's favorite shows when she was in high school; one of the characters says something to the effect that college is the world's nicest holding pen. The view that maybe it doesn't do much for you but it's certainly a nice place to hold onto these people that society doesn't know what to do with because they haven't grown up but they are not children. Russ: I can't end on that. I was going to end there. It's too depressing. You want to say something cheerful about where our education might be going down the road with these new innovations? Guest: Well, I think the really--and I'm sort of channeling Tyler Cowen on this, but there are two kind of exciting trends. One is this trend for the self-learner, somebody who is able to teach themselves. Even if they can't get a college degree, at some point in their lives if they develop that ability and have that ability, already they can become great computer programmers by asking Professor Google. And soon they'll be able to learn things from their tablet computers. Apple wants to talk about Apple University; and Google has Google Course Filter. So, there are tremendous opportunities for the self-learner. And the other thing is that there are tremendous opportunities for people in other countries who had no access to education. All sorts of stories. India has a horrible primary education system. People don't realize this. People see people from India in the United States and say what a wonderfully educated people they are. These Indian immigrants. Russ: Not a representative sample. Guest: So they think wow, that must be a great system. But in fact it's a horrible primary education system. A lot of places have little or no access to college. The numbers of people who can take advantage of education from these other countries, that's going to be really exciting over the next 20 or 30 years. It's going to be, as I think the last 20 years have been for the United States a challenge for the United States, in that a layer of people will find themselves that being an America citizen, the marginal value of that is no longer as high as it was before because they are going to be facing competition from Africans and Indians and Chinese that they wouldn't have faced before. But for the people in those countries it's going to be a tremendous opportunity. |
READER COMMENTS
Nathan
Oct 15 2012 at 7:28am
Here in the UK all essays and final exams are ‘internally examined’ by somone at the University who did not teach the course and all coursework done in a single department is ‘externally examined’ by someone from another university who is paid a small fee. The system is a disaster. Essentially the second and external examiner are playing complicated bureaucratic and political games of the type ‘how do I mark this exam in the minimum time in a way that looks like it is independent of the initial marker, but not in a way that would cause any hostility with my colleagues’. The whole thing is expensive and inefficient and adds no value.
Perhaps for someone who thinks the idea of independent examination is a good thing it would all work if the teacher were not an assesor at all. But in any case, if you want this proposal to be serious then it must consider international practice.
James
Oct 15 2012 at 2:19pm
Great topic! I am actually doing my thesis on a topic related to this, so it was quite a treat.
I agree with most of what Kling said. The main point I would disagree with is the argument that school is almost entirely a sorting mechanism. While schooling certainly does play this role, and Kling is right that differences in large-scale quality of schooling (e.g. private vs. public college) often have little demonstrable effect, he is ignoring the mountain of evidence on the quantity of schooling, as well as more “micro” research on comparisons of different instructional methods.
The place to start is Ceci’s 1991 review:
http://psycnet.apa.org/journals/dev/27/5/703.html
Ceci reviews 8 different sources of evidence on the effects of schooling on intelligence and finds plenty of evidence that quantity of schooling produces large effects. One can find nearby towns that received different levels of schooling at different times, for example. Quasi-experiments on this issue go on all the time because children start school based on a purely arbitrary number, their birthday. Controlling for age allows the estimation of grade effects (Cahan, 1987, 1989, 2001, 2008). The effect of 1 year of school is (very roughly) twice the effect of 1 year of age, so that schooling accounts for about 2/3rd of growth on most tests.
Within courses and classrooms, there is ample evidence that individualized instruction of exactly the type Kling is recommending is superior. Indeed, why would he be recommending it if it had no effect? Tutoring, for example, has long been known to have a massive effect. Here is a recent meta-analysis:
http://www.fau.edu/CLASS/CRLA/Level_Three/Educational_Outcomes_of_Tutoring_A_Meta-analysis_of_Findings.pdf
The average effect size of tutoring on achievement was .40 (52 studies). Multiple studies also showed positive effects on student attitudes toward the subject matter. Interestingly, benefits were even stronger for the tutors themselves than for the “tutees” (goes to your point about teaching, Russ).
One can find similar effect sizes in the literature on other individualized curricula like the Perry method, computer tutors, and blending learning. Obviously much depends on what is being learned and what stage of learning students are in.
I find it strange that Kling left this entire literature out of his discussion, as it buttresses his arguments against lecture and one-size-fits-all. The main problem with traditional instruction is that a huge fraction of instructional time is wasted on material that is either too difficult or too easy. It must be admitted that this probably effects the speed of learning more than the final outcome given infinite time, but in reality instructional time is never infinite.
Evidence of large-scale quality differences is harder to find, but there are some examples. Carpe Diem schools in Arizona has implementing “blended learning” to great effect. Blended learning is basically the “flip the class” model Kling was recommending. Rote information and practice is fed to students via computer, while project-based discussion, application, and teamwork take place in a live classroom. The military has also adopted blended learning in many training programs and has found that it can cut training time by 1/3rd (Army learning concept 2015: http://www.tradoc.army.mil/tpubs/pams/tp525-8-2.pdf).
I suspect that Kling is aware of all this, but still it is misleading to talk as if there is no evidence of schooling effects. School clearly effects speed of learning if nothing else.
Brad Hutchings
Oct 15 2012 at 6:49pm
I hate for my comment to appear right after James’ comment, but I suspect there is a lot of scientism parading as education research, and I suspect that’s why Dr. Kling doesn’t acknowledge it. When you look at the effects of No Child Left Behind (NCLB), which has likely been the source of the plethora of “results” that James has to plow through in his own research, you see lots of strategies and approaches wrapped up with a “seal of science”.
Why? Because it’s now a requirement for selling those strategies and approaches to schools. You have to have a study that validates what you’re selling. So little companies with great ideas have to check that box before getting a chance to experiment. And incumbent players throw research dollars out to keep the fences around their product market share strong.
Quite often, what a crappy school needs more than anything is an organizing principle or activity paired with a cheerleader to jumpstart the process and get everyone on board. That can get you all the things we all seem to agree are important: engaged students, parental involvement, a sense of mission, measurable achievements, etc. How do you think that gets put into a scientific paper?
Before NCLB really set in (2005ish), there was a pretty robust market for this kind of product in crappy schools. You could parlay one small success into several opportunities. You could create your own measurables, which tended to be less bland than test scores, and expect that test scores would follow as well. Now, the focus is primarily and nearly exclusively on effect on test scores.
As a cynical aside, I kinda wonder what the mobile gaming market might have looked like if instead of choosing winners by actual sales, it chose them with “game science”. Mobile game enthusiasts have seen all kinds of innovation in the space over the past few years, as developers play and experiment and fail and sometimes succeed. For example, Rovio had no idea that Angry Birds would be even a minor “hit”, and yet it it has become a franchise that could have Disney-esque staying power. I think education could benefit from that kind of innovative dynamic a lot more than from “science-driven” innovation. But that’s just my heretical opinion :-).
Ken P
Oct 15 2012 at 11:01pm
This was a good podcast, but I get the feeling that Arnold hasn’t ‘been there’. Or maybe he has just taken the wrong MOOC courses. There is no way I could have learned the amount of material that I learned last fall in Stanford’s Artificial Intelligence Class and Machine Learning Class in any other way. I also found the forum to be as good or better than any student interaction I’ve had in any course.
I agree with Bryan (although I’ve been slow to accept) that there is a sorting/signaling thing going on with education and that it may be the biggest aspect. However companies think they are getting ‘educated’ students and I believe the sorting function is a faulty aspect of the current system. Someone who turns in papers on time, thinks only deep enough to pass tests and easy falls into group think is not my idea of a great employee. They do lots of busy work, require lots of feedback from managers (who are often similarly unprepared) and there is often much less real progress than there would be with truly educated students. I also have the belief that a significan number of intelligent students are weeded out of the ‘success track’ through boredom in grade school.
What I really like about the current state of MOOCs is that there are many of them with different approaches and the technology is in it’s infancy. I also think the advances in Natural Language Processing will make big contributions to these companies in the future. I wouldn’t be surprised to see some of Arnold’s concerns about interaction improved greatly as a result.
There are clearly problems with assessing competency in non-science/math type subjects and these problems are magnified with class size.
It may take some time for employers to accept this approach to education, but would-be entrepreneurs looking to gain the education needed for a tech startup can begin now.
Other major benefactors include intelligent middle and high school students. Should they be able to CLEP out of a Math, Physics or Computer Science degree??
Joe W
Oct 16 2012 at 3:08pm
One small critique with regards the short conversation on learning computer programming. The key thing you missed was that simply learning the skill of computer programming does not make you a programmer or for that matter a computer engineer or computer scientist. Your analysis of learning the skill of programming seemed a bit naive and suggested something along the lines of learning to multiply makes you a mathematician, obviously this isn’t the case. “Mastery” of any skill requires experience not available in books, videos or from professors.
Ryan R
Oct 16 2012 at 4:43pm
Kling is making the perfect the enemy of the good. MOOCs don’t need to achieve perfection or even the level of a university course to be useful. What matters is whether the reduced quality is still worth the reduced cost. I’ve completed one course and dabbled in a couple of others, and I’ve found them to be a worthwhile alternative to trying to take the equivalent course at the university. I could learn more and get a credential from the university, but the added flexibility of what lectures I watch, when I watch them, what speed I watch them, and how many times I watch them is often superior for skills and knowledge that I wish to learn.
His ideal of giving students personalized feedback from highly skilled experts is a fantasy, at least until we develop highly sophisticated artificial intelligence. He is proposing taking a sector that is suffering from increased costs partially due to its productivity not keeping up with the economy as a whole and suggesting that we make it even more labor-intensive.
John Berg
Oct 16 2012 at 7:49pm
As a gedanken experiment, I suggest the teaching of Chess via computer teaching. The results are objective, the goals are measureable, and the testing is clearly do-able against current masters and current pupils beginning their path to mastery.
A scosh harder would be the teaching of GO and monitoring the path to mastery. In the mid-70s, the Japanese, under their nation-wide PIP program had a project to teach the computer how to play GO and I wonder what was learned that could be useful.
in the mid-80s, Artificial Learning made a perceived giant-step forward which received recognition in the national press. (I remember the Fortune article that touted the progress.) This progress turned out to be due mainly to the big leap forward in cheap memory beginning with terabyte memory.
John Berg
Charlie
Oct 16 2012 at 9:47pm
Robin Hanson argues that the Dale and Krueger study is mis-cited
http://www.overcomingbias.com/2009/03/college-prestige-matters.html#more
“In fact his original 1998 working-paper abstract said:
We find that students who attended colleges with higher average SAT scores do not earn more than other students who were accepted and rejected by comparable schools but attended a college with a lower average SAT score. However the Barron’s rating of school selectivity and the tuition charged by the school are significantly related to the students’ subsequent earnings.”
Mike G
Oct 16 2012 at 11:23pm
Part of the fun of this topic is indulging in fantasy. Imagine if there were a market for explanations – potential suppliers could compete on the quality of their video explanations for some concept to be used in an actual lesson. Multiple explanations of the same concept might be purchases so that if a student wasn’t satisfied with the 1st explanation, he/she could see the same concept explained by someone else.
Since knowledge is dispersed, I bet there are plenty of well-informed folks who, if given the right incentive, could produce excellent explanations for the things they know most about.
Lauren
Oct 17 2012 at 6:20am
A major effect of the Internet on education that wasn’t highlighted in this podcast is the importance of variety.
Even within a single subject, the existence of the Internet makes available a wide range of opportunities that now can be customized by a student to suit a person’s individual learning style. There is no reason to think that only one teaching method will survive the shakeout. Some students may prefer–meaning learn best for their purposes–asking Professor Google. Others may prefer a flipped classroom model. Still others may prefer the traditional lecture model, either by taking a course in person or by listening to it online. Some may even actually prefer just reading a well-written book. Moreover, different subjects may call for a different range of available learning options.
I do not think I am a typical learner; but I do represent a learning style that may characterize enough students to support at least some traditional books/teachers. Specifically, I hated being called on in class. I enjoyed and felt most relaxed–and learned the most–by listening to the teacher expound, taking notes, and then mulling it all over later. I would not have liked a teacher hovering over me or offering me advice while I was trying to work out a problem. I got pleasure from working things out for myself. I enjoyed reading books; and I learned calculus and computer programming from reading books. For many purposes, I prefer to read a book where someone has thought out, organized, edited and reworked an idea to hunting and pecking around Google. I note in passing that I am also someone who always got pleasure from delayed rewards, which may be a factor in learning styles.
The reason I liked to attend classes in person–traditional lectures–was because it was a kind of calibration of what I knew with where the teacher/professor was coming from, all in a somewhat anonymous framework, without my being under the teacher’s (or parent’s!) one-on-one eye while I absorbed it. It takes me a while to process new stuff, and I just don’t do it best when I am under someone’s eye. But: If I went home after a lecture, read through my notes or tried to do homework or read the book, if something didn’t make sense, I knew exactly what to ask the teacher during office hours or after the next class. “In class you said x but that’s exactly where I got confused. What does this mean?” or “I tried to work out this problem but I got stuck exactly here.” Or “In class you mentioned that James Joyce uses imagery like smoke, rope, and colors to allude to religious and political symbolism. I don’t get it. Should I just hold out for the next lecture? I know sometimes it takes more than an hour-long snippet.” Anticipation can be one of the most exciting incentives for learning. Education is not all about instant gratification.
Because of a lecture, the teacher and I now speak the same language and can communicate. That makes possible the best kind of feedback, on both sides.
Jason Haines
Oct 17 2012 at 2:54pm
I agree with ‘Ken P’s comment on MOOC. Kling does seem to have overlooked not only the value that MOOCs are currently delivering but also their future potential.
Kling gives several good arguments as to why MOOCs are not a magic bullet. I would agree that coursera-style courses probably do have a high drop-out rate: less talented/motivated students won’t get the same sort of support and encouragement from a MOOC setting that the could get from an on-site course.
However, the benefits of coursera are huge – a talented educator like Andrew Ng reaches many (ten of) thousands of online students with his coursera Machine Learning course. For his students (including myself), this is a valuable opportunity that would otherwise be impossible to receive.
Some of Kling’s criticisms of MOOCs seem premature and resolvable. He says that he watched a boring 20 minute statistics lecture on histograms. No doubt there are bad MOOCs. However, it’s not hard to imagine a better statistics teacher seeing that video and deciding to make a better one. Once it exists, it can exist forever. A huge advantage of MOOCs over on-site courses is that MOOCs can offer everyone access to the very best teachers of a subject, rather than just the one available on-site.
Kling also criticises the lack of custom feedback. While this may be true of current courses, this is an area that will see improvement. Daphne Koller’s TED talk introducing coursera.org even mentions custom feedback. By gathering statistics on frequent incorrect answers, an explanation could be given to explain common misapprehensions. I could also imagine community based approaches where students support each other. coursera.org has excellent discussion forums (and meetup groups) where students ask and answer each other’s questions. It’s not hard to imagine a wiki page accompanying a video where students collectively build lecture notes to clarify the concepts
While underestimating the benefits, Kling doesn’t talk about the other side of the MOOC equation: cost. Here is where MOOCs are outstanding. The marginal cost for each additional student is essentially zero and the fixed costs (website hosting) are small and shrinking. Once created, the course can also be re-run for close to free.
Are MOOCs are magic bullet? Certainly not on their own. But ‘loser’? That seems to miss the point.
Nick Zbinden
Oct 17 2012 at 10:31pm
Prof. Kling is much much to pesimistic and doesn’t see everything that goes on.
First I have to add that I am a Computer Programmer that has followed this online education revolution since the beginning and has done or looked into many diffrent courses.
Computer Programming:
You both have somewhat wrong views on programming. You can not master programming any more then economics. Lecturs do not cover the absolut basics (what is a loop) are very importend. Explaining the Ideas behind algorithms and how to apply them is very useful.
Prof. Kling suggestion that you can just type your problem in to the computer only cuts it for very simple stuff that just does something with the computer (get a html page from the web for example).
Acctually programming is something very diffrent. A big part of programming is taking algorithems and apply them to you problem domain. This is the real meat of programming.
It is true that learning by doing is very powerful in learning to be a programmer but there are somethings that are very hard to pick up along the way.
I agree that the lectures are not the most importend but the are importend. You can not just tell somebody “google on the internet how to apply constraint satisfaction algorithms to X” The lectures are importend but most of these courses have 2h of lecture and then you have to do 3-10 hourse of programming.
The good online courses model this quite well.
[Comment edited: Guest’s name corrected from Klein to Kling.–Econlib Ed.]
Joe Mangum
Oct 18 2012 at 11:58am
Russ,
You commented that college is a place to find yourself, and that we are rich enough as a society for people to take 4 years off and explore themselves. That may be the college experience for the middle class and above, but as a member of the lower income class that was not my experience nor of most people I know. The equation changes significantly when college is something you do in addition to working a full time job. At that level the usefulness of a mechanism to signal my willingness to conform, for me, drops to about nil. My decision to stop dumping hard earned cash into a stupid piece of paper that was going to take me a decade to receive and that guaranteed nothing was probably one of the better I have made. Most of my college graduate friends are neck deep in debt and not working in a career related to their field, assuming they are working a job that qualifies as a “career” in the first place. I give you the benefit of the doubt and assume you are well aware of these circumstances, but I often get the sense from academia folk that they just don’t get that poor people don’t take four years off to educate themselves. For them college is a second job.
TravisZ
Oct 18 2012 at 4:44pm
I strongly disagree with what Arnold Kling had to say. Yes, it is true that one on one private class with lots of interactions is probably better than mass online class. But, it is very expensive and unrealistic. It is like saying we should not give Plumpy’Nut to malnourished children in Africa, because the fresh full course dinner prepared by world-class chef is better.
7 years ago, I was a college student studying engineering in China. I found videotaped courses from Berkeley online and they were extremely helpful in my studies. I can imagine there are a lot of people around the world who are like me and who could benefit from these mass online courses tremendously.
We need to democratize the education like we did with technologies. Lowering the cost is the key to achieve that.
Mark Crankshaw
Oct 18 2012 at 6:38pm
@ Joe Magnum
Here! Here! As a self-financing student of very modest means, I had to work full time (working nights and the grave yard shift on weekends) and attend school during the day. No time taken off, no exploring, and no fun.
Had Cornell University been willing to exchange the diploma directly for cash, I would have gladly forgone the gruelling “college experience” that I had to endure, that’s for sure.
I very much subscribe to the signaling model of education as espoused by Bryan Caplan. That’s all the University experience was for me, from the first day to the last. I took difficult courses and got good grades, sure, as a signal to future employers, but that would have gotten me any future remuneration without attaining the diploma. And I was there solely for the future remuneration.
I choose to complete my graduate studies in the UK (London School of Economics) because the face time with professors and other students was at a bare minimum. Another reason why I feel the signalling model fits better with my own personal experience: not only did I not encounter any “inspirational” professors in my university career, but my experience was that the overwhelming majority of what I did learn was self-taught with professors and other students contributing little, if anything. The only function of the professor or the classroom was to give me a reference point to determine what I had to teach myself to successfully clear the next hoop, so I could successfully signal to future employers, and so successfully aquire that monetary remuneration. For me personally, education was simply a means to that end. The quicker I could have gotten to that end, the better…
Russell
Oct 18 2012 at 9:02pm
Russ,
It struck me you may be a Keynesian with respect to education (parallel between emphasis on the role of the teacher in learning and the role of government in the economy).
The power of Khan (Khansian?) (sorry) The power of the Khan Academy is that it is so inexpensive to access the knowledge.
I find the freedom to learn independently to be liberating.
Great program! Love your show!
Thanks!
twv
Oct 18 2012 at 10:13pm
The legal profession has something like external accreditation now: “passing the bar.” And in some jurisdictions one may practice law without going to college: All one is required to do is “read law” and “pass the bar” examination.
I’m told not many do this. Why?
Giovanni Battistini
Oct 20 2012 at 1:39pm
Fascinating conversation. The real. Issue is, in my opinion how we teach or learn how to learn. The lecture one can get online is just a trigger to our curiosity and there is so much more out there on the Internet that we can learn from, but we lack the appropriate tools to pursue all this information and transform into knowledge. We keep thinking in terms of self-contained classes or books, but ignore the important fact that the Web has dramatically changed this paradigm, or should have. Russel makes the important point that self-monitoring is a crucial aspect of our education, but how do we learn it or teach it? At Zonebee, a spinoff from UofA, we tried to make some progress in this direction, although today this tools would be an iPad app integrated with something like coursera or Khan Academy. Some ideas to check out though, still a little ahead of its times: http://www.youtube.com/playlist?list=PLB163129F321D78A5
Trent Whitney
Oct 24 2012 at 11:55am
Russ,
Thoroughly enjoyed this podcast, and as you were discussing teaching styles, classroom engagement, etc., I couldn’t help but think back to your MBA Microeconomics class at Olin/Wash U.
What I remember most, apart from group-singing “The Flintstones” theme, was the lively discussion between you and the class that you incorporated into your lecture. You also employed the “turn to a person with a different answer & try to convince him/her that you’re right” method…and I think it worked well. I’ve admittedly used that myself – both in business (training/meetings/etc.) and briefly as a teacher.
As I recall, this thoroughly engaged the class to the point that there always seemed to be a mad rush to the podium at the end of class by several students to challenge you on the material. And, of course, you’d stay there 5, 10, even 15 minutes after class to answer questions, promote more thinking, etc. Meanwhile, students who had left the classroom were still talking about that day’s class afterwards.
Without naming the professor, the same group of students also had to take the MBA Macroeconomics class. This “teacher” made us purchase photocopies of his slides, which we had to put into a 3-ring binder, and then he would display the slides on the screen & talk to them one at a time…..basically read you his notes that we had sitting in front of us. No Q&A…very little interaction…minimal class engagement. No talk about the subject matter after class…merely a recap of how many slides he flipped through during his lecture (should add that some classmates created a numbers game to help pass the time). Needless to say, I don’t recall much at all about the content from this class.
Bottom line, you’re definitely on to something in your discussion about classroom atmosphere and the importance of all that goes on outside the standard lecture. But I agree that it’s very difficult to quantify things like classroom engagement. It may well be one of those factors such that you know it when you see it & you know it when it’s not present.
Farhad
Oct 25 2012 at 7:04am
Unfortunately this was one of the most wishy-washy and biased talks that I have ever heard on Encon Talk and on the subject of online education. It is a clear example of people discrediting a subject when they feel their jobs are threatened by it. It also shows that economist should not necessarily talk about anything and everything. I haven’t seen people in other fields act this way but it seems economists see themselves as entitled. Here are some more specific comments about the talk:
1. Kling claims there are only two groups of people who benefit from online courses and one group is the type of Stanford students who can interact with each other on the subject on campus. I assume he is not aware of or ignores the fact that there are already plenty of active online and offline forums where people who take courses at Udacity, etc. can gather and discuss the course material. Wasn’t this worth mentioning?
2. He also mentions a large drop rate in these courses as a sign that they are not useful. Wrong again! He ignores the fact that there is almost no barrier for entry in these courses so it is natural that a lot of people sign up and many later drop out. But the fact remains that even if a mere 10% finish the course (which by the way added up to 20000 people in the Artificial Intelligence course taught by Sebastian Thrun and Peter Norvig), we have achieved an unimaginable task: teaching a course by the best professors in the world to a large audience. If we assume each AI class at Stanford has 50 students, how long does it take to train so many people in AI?
3. Kling outrageously claims that in this day and time, we don’t need courses on programming languages like Python. That’s an amazing claim. Obviously god-like economists can make any claim and we have to accept but I suggest he raises this issue within his own university and asks them to stop offering courses on programming because people can use Google. Roberts also plays an interesting game along with Kling and declares macroeconomics is an exception and we still need to teach that subject in classrooms. Why macro but not python? Nobody knows. Again a clear example of the fact that people talk aligned with their interests.
4. I agree on one thing with Kling though and that’s his comment about Stat One course taught by Andrew Conway on Coursera. I listened to most of the lectures and did the assignments and I agree that the course was not well planned and the materials were not well explained. However, I have a PhD and throughout my years of education, I have sat on many college courses in which the profs were much more disorganized and disconnected that at any given time I prefer to take an online mediocre course at my own pace and place instead of taking those college courses.
Although solvable, I think the main problems with online courses are as follows:
1. Labs and experimental works
2. Brand (a big benefit of universities comes with their brand)
3. Training (how do you create an environment to train best instructors?)
Brit Cruise
Oct 26 2012 at 12:33pm
Just wanted to say I’m a long time listener and faculty member at Khan Academy. I loved this episode, I thought it was a very productive discussion.
Michael
Oct 30 2012 at 11:52pm
Kling struck a nerve. Online learning, especially MOOCs, is about educating a large number of people. Remote learning, perhaps even isolated, is inherent to online learning. Both Kling and some online students want both the luxury of learning at their own pace, on their own time, yet expect frequent and immediate feedback from the instructor. The cake cannot be had and eaten, too.
Having recently attended a workshop on Online Learning, the student feedback panel opined about the lack of contact with the instructor. Duh! At our current level of technology, a distance between the instructor and the student is not a probability, it’s a surety. Contrary to Kling’s comment about not wanting to sit through bland video lectures, all of the students on the panel stated they wanted more video lectures, more classroom capture videos. “We love Eluminate,” was the chorus. My university recently bought a license of Tegrity to provide an additional lecture capture tool.
Many seem to apply the same yardstick to all courses. A 100-level World Geography course does not and should not require the same level of attention as a 300-level Econometrics course.
A rational approach to online learning based on the complexity of the course and course material is needed.
And, I thought I heard Mr. Kling advocated “teaching to the test,” i.e independent testing and certification and instructor popularity for helping students do better on independent examiner. It’s hard enough to make an instructor give up some sovereignty for exams in one institution, let alone push them outside the university for assessment.
I think his “one student in the classroom” notion is way too simplistic. Sounds like he wants everyone home-schooled.
Comments are closed.