Why Today’s Training Methods are Failing

Speaker 1 (00:00): I was like, I found it. I found it. This is what I’ve been looking for. I can honestly say it has genuinely changed the way I run my business. It’s changed the results that I’m seeing. It’s changed my engagement with clients. It’s changed my engagement with the team. I couldn’t be happier. Honestly. It’s the best investment I ever

John Jantsch (00:16): Made. What you just heard was a testimonial from a recent graduate of the Duct Tape Marketing certification intensive program for fractional CMOs marketing agencies and consultants just like them. You could choose our system to move from vendor to trusted advisor, attract only ideal clients, and confidently present your strategies to build monthly recurring revenue. Visit DTM world slash scale to book your free advisory call and learn more. It’s time to transform your approach. Book your call today, DTM World slash scale.

(01:03): Hello and welcome to another episode of the Duct Tape Marketing Podcast. This is John Jantsch. My guest today is Matt Beane. He does field research on work involving robots and AI to uncover systematic positive exceptions that we use across the broader world of work. He’s an assistant professor in the technology management department at the University of California Santa Barbara and a digital fellow with Stanford’s Digital Economy Lab and MIT’s Institute for Digital Economy. He received his PhD from the MIT Sloan School of Management. We’re going to talk about his book today, the Skill Code, how to Save Human Ability in an Age of Intelligent Machines. So Matt, welcome to the show.

Matt Beane (01:49): Delighted to be here. Really appreciated the invite and excited to talk.

John Jantsch (01:53): We’re going to do our best to save human ability today. I think that sounds like a noble goal. So the premise, I’ll be very brief and let you talk more about it, but the premise of the book is this idea that we’re losing a lot of skills that the master taught the apprentice over centuries of time. So kind of led you to studying that particular gap or space,

Matt Beane (02:17): Right? The gap I wasn’t sure was going to be there. I’ve always been interested in training, learning and skill development my whole career one way or another. And since about 2000 and the confluence of that in AI and robotics got my fierce and firm attention. When it came time to pick a dissertation focus at MIT, I knew robotic surgery was a very interesting place to go because it boy, oh boy, doing robotic surgery is radically different than doing it the old fashioned way or even the modern way before that, which was with straight sticks, basically laparoscopic surgery as we knew about it in the nineties. And I just looked at the control apparatus for that robot, and it’s a giant Xbox controller basically with a 3D vision goggles that you wear and foot control. So you’re using your feet and your hands and I’m like, this bears almost no resemblance to the old fashioned way. How did you learn

John Jantsch (03:13): How to do? Yeah, the parts are looking at are called the same thing. But other than that, right?

Matt Beane (03:16): Yeah, no kidding. Although by the way, looking at it through this robot, you’re seeing it at 10 x magnification and an inch movement on the outside with your hands and the controller translates to a millimeter on the inside. So it’s a real different game. I didn’t know all that to begin with, but I figured the training for this has got to be real different for the training the old school way. How does it work? What are the upsides? What are the downsides? That’s the sort of going in question after no more than three months in the field in the operating room, watching real procedures with this thing, talking to residents, talking to surgeons, it was very obvious that everyone was assuming the way to learn how to use this thing is the same way that you should learn how to do the old procedure. And that was a fatal assumption, turned out to be false because, and by the way, the old school way is something we have encoded almost in our DNA.

(04:09): It’s literally 160,000 years old in surgery. They call this C one, do one, teach one. You basically show up, help a little bit, watch, get a little bit more involved as the expert decides you’re ready and sooner or later you’ve got somebody looking over your shoulder. Look at any profession I defy you. The book is full of examples of this. We just take that for granted as the way that you learn how to do stuff and build skill, which is the ability to do the thing under pressure basically reliably. That’s different than knowing conceptually. Like book smart. This is like, can you do it? So anyway, it turns out that mechanism of learning, this taken for granted thing, it was getting busted in robotic surgery because the console and the controller allowed the senior surgeon to do the whole thing themselves. So the resident becomes an optional participant at that point.

(05:00): I have to make mental effort to involve that person who wants to become me someday and there’s a patient on the table and therefore I’m never going to do that because they’re going to be slower and make more mistakes than me. So instead of the old school way, a four and a half hour procedure, that resident is busy for actually about six hours. They are working from before the patient gets, there’s any incision all the way till well afterwards they’re sweating the whole time and they’re helping out in consequential work. Now, they might be swimming in the shallow end of the pool, but they’re swimming now. They show up, help get that robot doc to the patient, sit in a separate control console and they watch a movie. That’s all they get to do. And so that was no one. Everyone recognized that was unsatisfactory on some level, but boy oh boy, they were just kind of tolerating it and surgeons would come out of programs after six years of big air quotes training. I have, well, I quote a chief of urology for one of the top hospitals in the states and he said, top surgeons suck. Now. That’s what he said to me, to my face. They’ve watched a lot of surgery, they haven’t done it. We have to retrain them when they arrive. So it was about halfway through that study that I realized most of what I just told you, because

John Jantsch (06:19): Really when it comes down to it, learning is reps, right?

Matt Beane (06:24): Yeah. And not reps on a putting green with no wind and a perfect sunny day, you got to have somebody throwing rocks at you. So it’s sure some reps in practice conditions, it’s helpful. But you’re right, even in surgery, they have a name for this. In their research on surgery, they call it dwell time, which means how much time are you in the or? That used to be the old proxy for you’re in there, that’s the best place for you to learn. Now it’s the worst. Anyway, I also made it a point in that study to try to find surgeons who were learning anyway, in spite of that barrier, I was very lucky to get some guidance to do that early. That turned up some really interesting and new ways to build skill that defied the new barrier of novice optional world that we’ve got now.

John Jantsch (07:12): Is the skill required to become a great surgeon different now? I mean motor skills maybe are different needs, or is it really just learning the tools?

Matt Beane (07:22): Sure, there are different skills involved, and that’s much less important than do you know a healthy way to build skill period in this context because the skills that you need to do your job are always changing a little bit, constantly. New version of this thing for my iPhone, gosh darn it, I got to learn a little something every day, right?

John Jantsch (07:40): Why did they move that button?

Matt Beane (07:42): Right? And big things come along every once in a while and we get a really big thing with chat T these days. But for most folks, that doesn’t mean you have a whole new job or a whole new set of skills. It just means say 30%. But if you have a healthy path to learning new skills, humans love to learn, then we’re going to do that. That’s no problem. If the way you learn is getting busted, then we have a serious issue.

John Jantsch (08:10): So you have talked primarily about medical industry, but you actually studied a number of industries, didn’t you? That this equally applied to?

Matt Beane (08:17): Yeah, over 30 now. So that was the immediate question. I published that study in 2019. I gave a TED talk in 2018, but actually from about 2016 on those findings were done. And I wanted to know where else is this happening? Is this novice optional thing happening elsewhere? It should be. That’s all I knew because I had the hypothesis that the more intelligent technologies, robots, ai, they allow a single expert to get more done with less help. If that’s really true, then let me go to some radically different places in the economy to study things like policing, investment banking, bomb disposal. I have a list of more than 30 now to see different technologies, by the way, different skills, different cultures. If this same problem is showing up there, then it’s time for all hands on deck. We’ve got a serious problem. And that resulted in that Harvard Business Review and TED talk piece, which came later.

(09:16): It looked to the world. I had just trumpeted my surgical findings, but in fact those were three years in the can. And I had spent the next three years being like, hang on. So that was the sobering news in 2019. This is everywhere I can look except for one place. I found one exception. It was accidental where tech was getting deployed in a way that improved skill development, which gave me more hope. And I think it was worth studying. And I keep trying to find and study exceptions like that. We need them. But in general, yeah, no, once I hit north of 20, 22 professions, occupations, industries, and so on, I just kind of stopped trying to find it.

John Jantsch (09:55): It’s my pleasure to welcome a new sponsor to the podcast. Our friends at ActiveCampaign. ActiveCampaign helps small teams power big businesses with the must have platform for intelligent marketing automation. We’ve been using ActiveCampaign for years here at Duct Tape Marketing to power our subscription forms, email newsletters and sales funnel drip campaigns. ActiveCampaign is that rare platform that’s affordable, easy to use, and capable of handling even the most complex marketing automation needs. And they make it easy to switch. They provide every new customer with one-on-one personal training and free migrations from your current marketing automation or email marketing provider. You can try ActiveCampaign for free for 14 days and there’s no credit card required. Just visit active campaign.com/duct tape. That’s right. Duct Tape Marketing podcast. Listeners who sign up via that link will also receive 15% off an annual plan. That’s activecampaign.com/duct tape. Now, this offer is limited to new active campaign customers only. So what are you waiting for? Fuel your growth, boost revenue and save precious time by upgrading to active campaign today. So I want to get obviously a little more into the findings too, but just even your process is fascinating. Did you have any industries that kind of question your goals? What are you up to? What are you really studying? What are you trying to, how is this going to change my job?

Matt Beane (11:22): No, because already on the ground. So number one, I should be real clear all those 30 plus industries I’m talking about and occupations and so on, I didn’t do all those studies myself because each one of these studies takes about a year and a half. I got primary access to data collected by other scientists who had done similar studies in their context. So investment banking, for instance, I’ve collaborated with a woman named Callan Anthony at NYU. She gave me her data set, I gave me her mine. We compared notes. We wrote another paper off of that. But in general, surgery’s a perfect example of this. Everywhere I’ve gone, everyone already knows things are not well when it comes to skill development, the next generation is struggling, but the productivity boost coming from these technologies is such an enticing, saccharine hit upfront that there’s no one party whose job it is to pay attention to. That unintended negative side effect of getting this productivity hit. The expert is happy. The organization, the person who’s investing in that capital is happy. And yeah, it’s kind of a pain to learn now. Can’t find a mentor to save my life, says the 24-year-old, nobody’s problem really

John Jantsch (12:34): Learning. Well, lemme ask this, will that catch up over time? Will that 24-year-old when he’s 48 and that’s all he is known be the mentor?

Matt Beane (12:41): Exactly right. This is why I’m trying to sound the alarm bell as loud as I can. This is why I wrote the book. It’s 3, 5, 7 years later, depending on the cycle time for your talent that the organization, the profession, the economy is going to start to realize that the next generation of talent just isn’t prepared for duty. And surgery’s noticing this now. So I announced my findings in about 20 17, 20 18, and I’ll just ground this out for you. They went kind of early. So I think other professions should get ready for this. The next generation of robotic surgical talent that’s floating in the hospitals have to spend an extra, oh, between 80 and 180 grand on what’s known as a proctor, which is another senior surgeon, either inside their own hospital from the outside to come in and retu this person on how to do the job. They just went through five years of training for it. That’s supposed to be good enough and everyone knows on the street it’s not. And so it adds to the cost base, the time to ramp to skill. That’s a bandaid solution. That’s just, there’s no way that’s going to work.

John Jantsch (13:45): Are there other factors besides the technology that are really kind of leading to this, but maybe even culture inside an organization? Demographics of the next generation coming up? I mean,

Matt Beane (13:56): Yes. Yeah. And so I think the fair thing to say is the problem I’m describing has been around since the advent of language, so about 160,000 years old. Let’s go back to ancient Greece, which I do in the book. There were big disputes about the shift from hand pinched pots to using a potter’s wheel and what was that going to do to skill around here? And an expert doesn’t need mentee or an apprentice quite as much. If they’ve got a powders wheel, they can do a whole bunch more themselves. So it’s just the intensity and the pace of what happens when chat GPT is free for everyone to use. They’re dramatically more self-serve on a much broader span of their work. Their dependence on a novice in different ways is dramatically the span of management control. These days. I’ve been doing studies in warehousing, for instance, the last few years. You have one manager to 50 to 80 people. And part of the reason for is technology. You can manage them through their iPhone, partially their schedule, their timing, discipline, pay, all that stuff. So it’s a tale as old as time. It’s just so much more intensified now that lots of things are digital, that it’s kind of a difference in quantity, amounting to a difference in quality, I think.

John Jantsch (15:09): Is there a way that organizations can kind of protect that idea of the learned skill and the knowledge base inside the organization? Or are you really actually saying those are obsolete in some ways? No,

Matt Beane (15:23): I hear you. And the back third of this book is what do we do now? And so definitely I have evidence to prove from studies that folks are fighting for their skill out there. They’re not doing it in an organized way, they’re not quite aware of what they’re doing. I’m doing my best to report on things that are working. So individuals can do things every day to protect their skill as they engage with these technologies. For sure. I have a post on my substack called Don’t let AI Dumb You Down, for instance. There’s some things that you as a user of that technology can do to avoid this subtle slide towards B plus territory. If you’re not careful, in fact nudge yourself up in towards a plus. You can in fact not just tread water. You can actually use this tech to enhance your skill by the end of every session.

(16:08): It’s just we’re not doing it. So that’s one level. But tactically managers in many ways, and people who run businesses have all the tools they need around handling this new tech. Anything. If you went to a top flight MBA program or more likely learn some great lessons outside of academia about how to mountain run a business and lead people, your tactics for dealing with new surprising events in your environment are probably just about as relevant now. It’s just everyone’s kind of got their hair on fire running around saying, when you know perfectly well how to run a healthy experiment with something that could be good or bad for your business right now. And what does that take to make sure that folks aren’t just trying it all on their own right now, which is not a terrible thing, but it’s just like yelling, fire in a crowded theater. It’s better to get folks organized. And what is an organized, healthy experiment in a business look like, by the way? That’s aggressive. You’ve got to be aggressive, fine but focused. So sure there’s stuff in the book and also on my substack about let’s just everyone remind ourselves. We know a lot. We know a lot about how to handle change and technological change. Let’s just put it to work.

John Jantsch (17:18): You know what I’ve been telling people for the last five, well, probably 30 years, but it’s really ramped up the last five years, is instead of saying, how can we use this technology to do something? We’re already doing faster or better or whatever, how can we actually ask better questions? You got it. That becomes our job, right?

Matt Beane (17:36): Yes. And in fact, part of the way you do that is the temptation immediately is just to use it to, I know how to do X, I’m just going to do X about two times faster, which is boy is entrancing.

John Jantsch (17:50): It’s

Matt Beane (17:50): Amazing. You can get it to write a memo in five seconds when it would take you 30 minutes. Geez. And that’s where folks stop. That’s the difference between top performers and average performers. The top performer goes and says, what could I do with this that I would’ve never even dreamed possible before? For instance, I made my master’s students who were most of them fearful of coding, had no coding expertise. And I said, you have three weeks on your own no help, but from chat GT to become a data analytics person, analyze this dataset and do plots in Python, use coding a solution in Python and then post it on GitHub, which is a shared software manipulation platform. And they looked at me like, are you joking me right now? What kind of class is this? This isn’t computer science. And I said, go. And they all did it.

(18:43): And by the end, I post this on my substack too. In the beginning I have their ratings on how afraid they were, what they thought with that nudge, mandatory, you must go do the thing that you thought was impossible before. And they all did it at the end. That was a shock to all of them, like hot diggity. I had no idea. I thought this was just kind of fancy, auto complete, like fake my own kind of thing instead of change the world. I wasn’t thinking broadly enough. I wasn’t asking the right questions. What could I do with this that I couldn’t do before? And if you can make yourself ask that question and then go for it. Try something really crazy. You’re going to fail. It’s going to be a waste in three nights or maybe 72 hours of your life, but you’ll learn a lot about this new world we’re entering that most folks will not have under their belt.

John Jantsch (19:31): And I think to some degree, what you’re describing is not about the end result, it’s about the journey

Matt Beane (19:37): And really in your bones skill, what you have at the end of that is not just, I know unquote a little bit about ai. It’s like, no, I’ve actually, I tried to decode sperm whale songs. I know talking to somebody who just took my challenge and tried to do that and they got some meaningful interpretation out of the data. It’s not as good as this paper that just got published. Actually doing that work by scientists who, by the way, four years ago got the memo and used AI to decode sperm whale songs. Another person wanted to sub Bruce Springsteen in for Luke Combs in that Grammy performance with Tracy Chapman. They got pretty far and this person didn’t code before. So yeah, that you will know in the deep sense. You’ll have skill that will inform your decision making and leadership in ways that most folks won’t.

John Jantsch (20:26): Well, Matt, I appreciate you taking some moments to stop by the Duct Tape Marketing Podcast and talk about the future, I suppose is what we were talking about, right? To some degree and strangely the past.

Matt Beane (20:38): Yeah, exactly. Thousands of years ago, and I’ll just toss in for your listeners only. I posted the first chapter of the book online if they want it, ducttapemarketing.mattbeane.com. It’s sitting right there. They can just go and grab it. Hopefully the book is helpful to them, but the future is now. But I agree. It’s an old dynamic too.

John Jantsch (21:00): Yeah. Awesome. Again, appreciate you dropping by for a few moments, and hopefully we’ll run into you one of these days out there on the road.