Dr. Jimmy Parker & Dr. Jonathan Reams - We Have a Measurement Problem

Dr. Jimmy Parker advises executives leading unprecedented transformations in large organizations. He has led over 100 transformation initiatives in multiple Fortune 50 companies and several industries (technology, healthcare, energy, retail, and the public sector). In his 30 years of transformation work, his innovative techniques have minimized the time, cost, and effort required to create true transformation that sticks. A lifelong student of the craft, he has studied over 5,000 titles on transforming leaders, teams, and entire organizations. Jimmy is a graduate of the U.S. Naval Academy, served 11 years as a Marine Corps pilot, and led agile software teams before earning a masters in organizational development and Ph.D. in developmental psychology applied to org transformation.

Quotes From Jimmy's Chapter

• "We’re ultimately trying to change leadership behavior. And if you want behavior change that sticks, you often have to change the mindsets that drive that behavior: whether those mindsets are outdated leadership paradigms, unrealistic expectations about yourself or others, or just plain false assumptions."
• "The weekly Do-Discuss-Document cycle: Do a bite-sized developmental activity on the job, discuss it in your peer group that week, and document key insights and lessons learned."
• This “knowing-doing gap,” or the difficulty us humans seem to have in actually doing what we know we should, both frustrated and fascinated me. 

More About Series Co-Host, Dr. Jonathan Reams

• Jonathan's Website
• Book: Maturing Leadership: How Adult Development Impacts Leadership
• Article: A Brief Overview of Developmental Theory

About  Scott J. Allen

• Website

My Approach to Hosting

• The views of my guests do not constitute "truth." Nor do they reflect my personal views in some instances. However, they are important views to be aware of. Nothing can replace your own research and exploration.

About The International Leadership Association (ILA)

• The ILA was created in 1999 to bring together professionals interested in the study, practice, and teaching of leadership. 

Transcript

Note: Voice-to-text transcriptions are about 90% accurate, and conversations-to-text do not always translate perfectly. I include it to provide you with the spirit of the conversation.

Scott Allen  0:00  
Okay, everybody, welcome to the Phronesis Podcast. Thank you so much for checking in wherever you are in the world. We are continuing the series of conversations from this book that was edited by Jonathan Reams. He again is my co-pilot today, and today we have Dr. Jimmy Parker. He is a developmental psychologist and executive coach that helps leaders accelerate large-scale organizational and cultural transformations. He's currently helping Kaiser Permanente's technology transformation mature into an org-wide transformation to create more healthy years for more people. Prior to joining KP, Jimmy established Home Depot's Executive Development programs and refocus their internal consulting practice on a multi-year transformation of culture and agile ways of working across the enterprise. His background includes a variety of roles, such as agile software product manager, enterprise learning, strategist, helicopter instructor pilot, and military officer. He's a graduate of the United States Naval Academy, has a master's in organizational development, and is nearly complete with a Ph.D. in human development. He lives in Atlanta with his wife and two teenage boys is a huge fan of The Princess Bride movie and enjoys roasting small batches of coffee every weekend. Oh, wow. Jimmy, there's a lot that I could jump into here! But Princess Bride -  just "as you wish!" right!?

Jimmy Parker  1:26  
Oh yeah, this is gonna be a fun interview!

Scott Allen  1:31  
Such a great film with so many wonderful lines. And one of the wins, I think as parents that my wife and I have achieved is that at random points our children will quote from either the Holy Grail or The Princess Bride. And it's always just we look at each other from across the kitchen and say, "Okay, we're winning." Well, Jimmy, thank you so much for being with us today. We're very, very appreciative of your time. And Jonathan, how are you, sir?

Jonathan Reams  2:02  
I'm good. And I'm looking forward to this too. So I think I got to know you, Jimmy - three/four years ago through a mutual friend, John Oliver, and have kept up with your dissertation work, and I've been learning a lot through tagging along on the journey. But for today, what I want to do because there's so much that you have to offer, and we're gonna give the listeners just some snippets of these different chunks. In the book chapter, you talk a little bit about how you got into leadership in the military context. Could you say a little bit about that?

Jimmy Parker  2:45  
Yeah, sir. Oh, my goodness, I was halfway through the military academy. And I had the opportunity to be a kind of like the equivalent of a drill instructor, but for officers, you know, sort of new freshmen coming into that school, there was a six-week very intensive period, and I got to be a squad leader. And that was really my first taste of leadership - ever. I don't know if there's just something in me that lit up. And it wasn't like the ego boost, you know, that gave me a lift, it was just, I was now I had some responsibility for really the development of other people. And I just took it seriously, and I slept better at night, and I was more happy, even though it was a very intense environment. So I was like, what is that? And I realized later it was; it was like my first real taste of, like, a life purpose kind of thing.

Jonathan Reams  3:36  
Cool. And how did that translate into your corporate life?

Jimmy Parker  3:40  
That grew from those very early days in the military, and I was in the Marines flying helicopters for about 11 years total. During that time, my interest in leadership grew and grew. It started off with me wanting to be the best leader that I could, obviously, and that led me naturally into the self-help aisles of the bookstores, and just doing the best I could to be a good leader, but development in all domains, the best parent, I could be the best partner, spouse, you know, that I could be and spiritual growth, like all kinds of, you know, everything sort of development and growth and learning, have sort of all kinds just really grabbed me. And then, at some point, toward the end of that 11-year period in the military, I realized I wanted to share what I was learning. So what started with me wanting to be the best leader that I could morph into wanting to build the best leaders that I could. And that took me on a different track. And I realized that if I were to stay in the military, my opportunities would be fairly limited in that new sort of career trajectory that I was calling me. And so, once I fulfilled my service commitment, I opted for a job with more flexibility that allowed me to pursue the track I'm on now.

Scott Allen  4:50  
As you were speaking, I thought of my conversation with Doug Lindsay. Doug is at the United States Air Force Academy and is the editor of the Journal of Character and Leadership Development. And he had a quote from Bob Hogan that said, "who you are is how you lead." I love that quote from Bob Hogan, "who you are is how you lead" what I love about that quote as you could probably, at the end of it, put in a parent, or coach, right? I mean, it's just this becomes much broader than just a leadership role. It's how am I as a father? how am I as a partner? And I love that framing; how do I be the best person I can be? And then that will translate to different domains.

Jonathan Reams  5:31  
Well, and I think that's a good segue, too, because I gather that through all this voracious reading and adopting of things, you somehow came across adult development in some form.

Jimmy Parker  5:44  
Yeah, well, I wouldn't say the form that, you know, we have these discreetly defined stages and, you know, specific techniques for driving, what do you call Vertical Development through those stages? I didn't discover that until I had gone back to grad school, fairly late in my career, that sort of urgent desire, I had to want to build leaders and get better and better at my craft. I had jobs that allowed me to do that, and it grew from there, and I got better, and I got better feedback. And then I realized I've only got a Bachelor's degree in Computer Science right now. And I've been on this, this development track now, for a couple of decades. And I was like, "well, what if going back to school would make me even better at my craft?" And that was what pulled me back, actually, to grad school and, ultimately, the Ph.D. And it was like, literally, in the first couple of weeks of the Ph.D. that I got formally introduced to these stages - And I'm like, "you mean there's a map I can start to follow? To try and figure out development? The kind that I've been interested in for so long? There's a map, my gosh!" And that just sucked me in deeper.

Scott Allen  6:46  
And the book was probably called In Over Our Heads, right?

Jimmy Parker  6:51  
That was definitely one.

Jonathan Reams  6:55  
Now what I find interesting, you said a moment ago, when I think one of the things that Scott and I have encountered in this series is that there are many more rich and diverse ways of talking about development than these simplistic vertical stage models. And we're realizing that there's much more to it than that; when you were in Home Depot, and you had the chance to try and develop leaders develop the best leaders, you could, you had a fairly creative way of trying to enable a kind of system for people to do that. Can you say a little bit about that?

Jimmy Parker  7:36  
Well, the system that you saw was not version 1.0; I made a lot of fairly common mistakes that a lot of people make in this field when they're new to it, particularly if they're enamored with the stages as I was - perhaps excessively so. And I would leave with that; I would teach people and participants in programs about these stages. And that comes with it's a pretty loaded conversation, really, it comes with a lot of baggage that can actually get in the way of development, surprisingly. And so one of the pivots there was to tie everything to developmental stages, but complexity - Snowden's Cynefin Framework, you know, simple, complex, complicated, and how there are different levels and stages of complexity inherent in life, nobody really pushes back on that. It's not an ethical question, usually, you know, whereas with development, you have all these sort of laced with ethics, oftentimes, and people, you know, right versus wrong, people better or worse, you know, later stages and better, you know, that stuff really kind of it gets put to the wayside, you know when you start to anchor everything, and the fact that some things are more complex than others, some jobs are more complex than others. And then certain people are better suited to that versus not; the essence of the program is, let's see, micro-learning is pretty popular these days. It's not microlearning. But that's a good sort of mental frame, I would say; it's not quite micro-learning because learning usually, for most people, means knowledge transfer of some kind. I take knowledge, I chop it up into small bits, and then I give that to you in sort of little bite-sized things on the assumption that if you knew better, you would do better. That's the assumption that I don't think is accurate. And I've been trying to sort of help people, you know, cross that knowing/doing gap, we humans seem to have even to build our own habits that we want to we struggle quite a lot. So instead of micro-learning, I would call it micro-experiences, maybe? Where there's instead of just the learning or the acquire acquisition of new knowledge, it's a micro-task; it's a micro action, it's a micro-experience that we designed for them to do on the job; we have very specific design criteria, there's a there's an action that they must do on the job, usually can't complete it by looking at your screen, you got to go do something, talk to someone you know, have a certain kind of meeting or conversation or something. And then, there's a reflection question that you answer after completing that task. Both together, that pairing of action and sort of a reflection or a question that they have to answer is designed to take 60 minutes or less, no more. So that's the micro piece, that sort of bite-sized piece, and then we build one of those pairings per week. And then we built a program around that to sort of get people doing and reflecting and many, many reps over time, instead of the firehose effect you can get in the classroom when we call it more like the drip method.

Scott Allen  10:13  
Jimmy, I just love that. That is, I have not heard of someone implementing anything like this, did you have a model that you look to? Or was this your creative process of letting up and saying, "Look, we need to get people out experiencing these things and engaging in that work?" That's just I'm so interested in knowing what you found from that experience.

Jimmy Parker  10:36  
Yeah, in some ways, you could say it was mine; in many ways, you can say that it wasn't really because the reading that I had done over time naturally led me away from learning knowledge acquisition and more around habit formation techniques and experiential learning. And, you know, John, Dewey, and all the sort of original folks that helped us understand how humans do fundamentally change in sort of meaningful, deep ways, you know, for good - sticky kind of change. And it was through studying that stuff that I naturally started bracketing in toward this approach. And one of the things that were the big shift was when I realized this type of growth often takes place in its most sort of concentrated and popular forms, and an off-site of some kind, you go leave the workplace, and you go into the forest or something, you know, for a while. And you've got expert facilitators, and you can have some amazing transformative experiences - life-changing, people never forget that they come back. And they're, they're different for life. I wanted to figure out how to crack that nut without having to leave the workplace - by leveraging the workplace or this more the spirit of, you know, Everyone Culture, another Bob Kegan book of how we can leverage that environment itself as a developmental stimulant. Because most adults spend most of their lives in an organization of some kind, why leave that place to drive development? 

Scott Allen  11:59  
Yes, I mean, exactly. Because what, how do we...this is a similar hunt that I have been on for more than a decade now? How do we design interventions that are in the flow of work? Right? So I had Joe railing on a few months back, and we had a conversation about some of his work, and it's, you know, they action learning and that whole kind of stream of literature. But how do we align it with the flow? And how do we keep it so very, very close to the work? So that it's, it's a with versus in other?

Jimmy Parker  12:34  
Yeah, exactly. building these kinds of programs and attempting to do them at scale had just tons and tons of lessons learned. There's a whole chapter or series of chapters that could probably write just about the design requirements, what a strong task on the job tasks, you know, kind of looks like you got to tick certain boxes for it to really have the most amount of developmental juice. And then, you know, there's a whole environment ecosystem that goes around that around small group peer learning; how big should the groups be? The use of coaches facilitators, you know, what makes a good coach facilitator? How do you train those coaches? And, even then, people who, what I found designing these types of tasks is not normal, like nobody learns how to do this. Instructional Designers are equipped, trained, and certified to design instruction. That's the learning by knowledge transfer. Yes. You know, and this is knowledge, by experience, this is creating an environment for people to sort of stumble upon a deep insight, you know, on their own and not be told that, you know, learning by discovery, and designing experiences that help people do that, in a powerful way, is a unique kind of design skill that's not even very abundant in the marketplace. So that was, how can I find enough people to build these kinds of programs that still have a lot of developmental impacts. And that was much more difficult than I expected.

Scott Allen  13:55  
Wow.

Jonathan Reams  13:57  
And I think what you're describing is much like what we saw in ILA from Mike Moscolo, who is deep into skill theory and this model of Do and Reflect. And it's very similar to Theo Dawson, who you also interviewed Scott, that model of learning skills, and that you were able to kind of also direct, I think you had a simple metric for people that you trained as coaches to get away from the being the hub, and everybody responds to them. And how much people respond, just simple things that allowed them to major their performance. And you also had something that was surprising about the changes that were long after the course.

Jimmy Parker  14:44  
There are more metrics, and you can shake a stick out of the course like this. And the book lays them all out in terms of the ones that we use the most - you just referenced Jonathan. The first one was it was sort of a quick and easy, simple feedback loop for coaches to know whether they were, we're doing a good job. You know, we tell them in coach training on day one, which is very lightweight, by the way, two to four hours. And we say, Hey, you're not the fountain of wisdom here. Even though these coaches were like one job level higher than the participants that they were kind of coaching, which is quite countercultural for the organization I was in then, at the end of each weekly session that they would have, you know, helped facilitate, there are two questions, and both of them are measured on a one to five scale. One is the hub and spoke model. The other one is more of a network, you know, sort of topology. And just, we had some anchors and some language in there around you as the coach throughout a discussion question. And then Scott answers you, and Jonathan answers you, and then Jimmy answers you; you're the hub in the middle, and all the spokes are just talking to the coach; they're not talking to each other. That's one out of five; you didn't do that? Great. We're going for a five here, where you might toss out one question at the beginning of the meeting. And there's enough psychological safety; there's enough interest and camaraderie and relationship building, where they are trying to learn from one another, and just this amazing conversation breaks out. The coach may not have to say anything else the entire meeting; that's a five. That's an idea where they're tuning into one another and learning from others. And there's a cocktail of, you know, sort of things that goes into creating that kind of magic environment. And that's also the coach's job to create that environment. And we equip them on how to do that. That was the one thing I could get to the second metric here on impact. That was more like, how do we know this program had an impact? And so we had a question; it was pretty simple. Again, there are much more sophisticated, quote "accurate" ways of doing that. But again, this is a very busy environment; some of the participants in our program ran multibillion dollars (with a B) PNLs. And they're super busy. You know, we had to, we had to figure out ways of producing good data in that environment. That was good enough. And one simple question we took the participants in the program, and we said all of them report to a manager or boss of some kind, even higher up in the org chart. He said, Hey, have you noticed any differences in your participant in these very specific ways which aligned with the objectives of our program? And they were behavioral? In other words, observable? Have you observed changes in these areas? And in the very beginning of the program, only about six to seven weeks in, we got about half of the bosses saying, Yeah, I've already seen a change in, like, one to two months. And by the end of the six-month program, over 90% of bosses said, I agree or strongly agree that I've seen some substantial changes. This is also surprising because the people who went into this program, in the first couple of years, were our highest potential leaders; these were our leaders who had already been trapped, tagged as being some of the best that we have. And we made them even better, you know, through this kind of a program. Well, we took that same metric system, that same simple question. The program ended, you know, 90% of bosses said to agree, strongly agree. And then we waited nine months after the program was over because usually, you get this drop-off, you know, as soon as the program ends, you know, the behavior drops back to, you know, old habits and all that stuff. We had, I don't remember the exact numbers. Again, it's in the book; we had a, I don't remember. I don't want to misquote myself. But essentially, when we did that same question again nine months later, the numbers were even stronger, which suggested that these participants continued to self-develop.

Jonathan Reams  18:24  
So it says, the manager ratings and participant behavior change was 3.93 out of five at the end of the program, and 4.5, nine months later,

Scott Allen  18:38  
That's durability right there.

Jimmy Parker  18:39  
Yeah, and again, going back to, like, before I discovered adult development, and all of these, you know, techniques, that was the thing I was trying to solve for was, how can you get people to grow and develop themselves, you know, in this sort of everyday kind of thing. And that was, that's my favorite stat from the program because it's almost like no one knew I was trying to design for that. And then when these numbers came out, nine months after the first program ended, I was like, yes. Oh, good.

Scott Allen  19:05  
Okay, you said a word a few moments ago; that was an important word. It was a cocktail. And I think, I think I oftentimes when I'm designing a course, or designing a learning experience, and some of our conversations, Jonathan have just been wonderful in this series because there are some cool clues about some of those ingredients. I view it as sometimes I use the phrasing recipe. The recipe is that there's the burger feels like it's from, you know, White Castle, no offense to any White Castle fans, or In and Out Burger for those not in the States. That's a highly coveted burger with long lines to get in and out. So which one are you serving up, right? And there's a recipe in place. And I think learning experiences or recipes as well, that there are some main ingredients that are just fundamental, so you use the phrasing cocktail. Can you highlight a couple of those super important ingredients in the car? a cocktail that you think helped yield these results?

Jimmy Parker  20:03  
I could talk for 40 or 50 minutes, maybe just answering that one question. So I'll stick it to a very high level, this concept that its folk it's rooted in doing not in learning to do, it's learning from doing so it's action forward action first, and then you know, reflection on the action, that's absolutely critical, the degree of difficulty of the action has to be pretty high, also, because you can think about if we're asking them to do a one hour task on the job every week, and then also meet up with a small learning group that did all the same thing that week, that's two hours a week, that's a lot of investment really, for most busy leaders in an organization. And if you're going to ask them to spend two hours a week doing this one thing, they don't spend two hours a week doing one thing hardly at all, you know, a very small number of things, it better be a good use of their time. And so that's why you got to ratchet up. Because if you don't, then the program that you built is running the risk of wasting people's time, you'll get thrown out, and you won't get the traction or the budgeting or the funding or whatever that you need. So it's got to be action forward, it's got to be difficult, really good, valuable use of their time and their investments. The learning environment, the peer learning environment is supercritical; we're looking at groups that may be as low as four, usually five, maybe as high as six, sometimes seven. That's the sort of the range there because any more than that's a party, people can hide any less than that, then the dynamics change a lot when someone doesn't show; we have a weekly cadence, not bi-weekly because again, if somebody misses a meeting, if it's bi-weekly, then they go a whole month. You'll lose too much momentum weekly; it's good to keep this sort of effort high. And I think the coach's role is quite important as well; one of the biggest things that I found was most leaders, even executives, have never experienced a program like this. They don't know what they don't know; you can't hold them accountable. So like, it's just like an HR thing in the beginning, like a check in the box. And what happens is, you need to get them over an initial hump where they have the lightbulb moment, and they go, oh, yeah, whoa, this is valuable to me. Yes, that doesn't happen on day one. That usually happens around week three, sometimes week four. But only when you have the right ingredients in your cocktail. And the coach's role there, we essentially leverage formal authority in the organization to get them doing what they're supposed to be doing, even though they don't, maybe some of them may not want to help them taste the sweetness that's available to them here. And we found that after the program is over about half, about half of these learning groups continue meeting on their own because they want to.

Jonathan Reams  22:38  
You're able to shift a lot of senior individual leaders' behavior. But I can imagine that at some point, you're trying to understand if this has a collective impact on the organizational culture. Is there a collective development going on? And I know that as you moved into doing your Ph.D., this became a central question for you. So let's head in that direction a little because I know there's a lot of juicy learning there.

Jimmy Parker  23:07  
Here, right? There's a whole lot different there. So essentially, the world of adult development is right next door to the world of what I'm calling these days vertical OD, which is a collective development, using a similar stage theory saying, Yeah, we know that people individuals can develop through these stages in somewhat predictable ways and scientifically valid and all that good stuff. What about teams? What about entire department divisions? What about the entire organization? What about all of humanity, there seem to be similar patterns that people have noticed that a group may behave at a particular stage of development that we know exists at the individual level, but we're not sure really if it exists; we believe it exists at the collective level or group level. And so I wanted to lean into that pretty heavily with these programs and design for that kind of vertical collective development.

Jonathan Reams  24:04  
And I want to interject a little bit because as you describe this, what I can imagine, you know, you come into a new team or an organization you join, and there are norms, and those norms can inhibit your performance, or they can enable it. And so there is something about the collective that supports the individual.

Jimmy Parker  24:25  
Yes, absolutely. You know, there's lots of theory about how the individual how individual development is related to enabled by or inhibited by collective development. And when I looked at that more carefully, I realized that our ability to measure individual development is quite strong. These days, we got 50 years of testing and learning and the numbers and the data and the scientific study that statistical validity is there. It's solid, and it's good—our ability to measure collective development using the same yardstick. The same sort of stages of development is at the opposite end of the spec. From it's very immature, we can't do it today. And that was what I did my dissertation research on. How are all the ways that it's been done? What are the advantages, disadvantages, and ways I sort of catalog all of that stuff that was my literature review? And then, I invented a new approach to do it in a way that no one had done in hopes of maximizing the advantages of everything that had been done before and minimizing the disadvantages; I had a pretty surprising result; I realized I was trying to isolate the individual, like, if I'm writing my team, if I'm writing my organization, it turns out that my stage of development affects how I rate another team or my team, or another organization or my organization, I was trying to isolate that effect, to where my stage of development wherever it was, I was trying to prevent my stage of development from affecting how I rated a collective group of people. And I leaned into that about as heavily as anybody has ever leaned into that. And I discovered that I ended up measuring individual development more than collective development, it was I got the exact opposite result that I was expecting to get, which just goes to say how hard this is and how we haven't figured out how to do it today.

Jonathan Reams  26:18  
I know Scott's got a question here. But I want to say it was so interesting to read through that, as I read through your dissertation, to see that all this work you put in to try and sort out, you know, make distinctions and filter out things that might be noise in what you're trying to major and isolate the things ended up showing you that it's much more difficult than we imagined.

Scott Allen  26:43  
Yeah, well, I know, I know, Keegan at least has mentioned in everyone's culture, I think they start talking about working to measure some of this in a collective group. I think they're calling it the LPOP; is that accurate? And Have either of you explored that as a tool, the LPOP?

Jimmy Parker  26:59  
I'm not familiar with the LPOP. I'm interested to see what kind of assessment it is. Because, again, that was part of my work was to canvass the entire field of development and organize all of the ways people have tried to assess this and sort of a different sort of framework that I had ended up creating. And my guess is it probably fits into one of those. 

Jonathan Reams  27:21  
Well, could you say something a little bit then about because I know you talk, for instance, about models that ask people to kind of match descriptions or other models that do other kinds of things, and you ended up looking at polarities. So could you say a little bit about high-level descriptions of typical ways people try to do that?

Jimmy Parker  27:44  
So broadly speaking, without getting too nerdy, and putting everyone to sleep here, the ways that vertical complexity or vertical development has been assessed you can put into two broad buckets. The first one that came on my first and is pretty popular is what I prefer; I like to call content matching. It's saying, hey, through our research, we've discovered that people at this stage of development, they do these things, they say these things, they behave these ways, they believe these things, they have these value systems, that's everything in defining a discrete stage. But at the next stage, things change. And they believe different things and may say different things. And they do different things or can do different things. Most stage-based models of vertical development have rich, thick descriptions at each stage of development. Okay. And so if I'm an assessor, and I want to figure out what stage you're at, or your team's added, or something like that, I observe you for a while, map it back to my definitions, and then render my opinion. And so I'm comparing what I see in you to what's in my manual, my scoring manual. And so that's what I call that. It's a comparison-based assessment, content matching, matching, you know, what I'm saying to content that's in a stage-based manual, other theorists had identified that as a person grows through these stages, we also see some meta properties that sort of change or alternate and flip back and forth, you know, throughout the stages. One of them is an individual orientation all about me. And then a collective orientation at the next stage is all about us. And then, at the next stage, they continued development, and suddenly it flips backward, again, to all about me, but in a slightly different way. But again, the focus is back on me, and you continue to develop four stages beyond what is started. It's, again, back to we, in a different way, a more profound way, a bigger way, you know, and so that one little like rocking chair back and forth throughout the developmental stages of individual orientation and collective orientation, back and forth as one develops has become one of the multiple parameters that people have also used as the basis of a staged assessment. So you can look at what the primary orientation is, individual or collective - it's individual for this person or this team, and then you add in a couple of other parameters, and boom, "there at stage three."

Jonathan Reams  30:02  
And you know, we referenced earlier Terry O'Fallon stages model. And when we talked with Jeff and Abigail went into that a bit, and that's one of the three parameters, and there's another parameter then around, is it something concrete? Or is it something subtle? Or is it something really kind of meta, and then another one is active and passive, I believe? So there's kind of three criteria that you can look at and they kind of move in different constellations. So that's the second model. How did you try and then design your study, trying to mitigate the challenges of measuring collective development? Compared to having given the review you did and knowing that content matching wasn't probably what you wanted to do? And other things weren't? So how did you end up where you did?

Jimmy Parker  30:54  
Essentially, I found that, well, if you're going to do the collective developmental measurement, it's going to try to measure a stage of collective you got to canvass a lot of people. And you can't interview a lot of people to any reasonable amount of you can't do it cheaply or quickly. So you have to use surveys; that's like everybody's using surveys; most people use surveys in the space. And everyone is using a content-matching approach. In other words, there's a multiple choice question that my team has behaved like this, and you got options A, B, C, D, and each option correlates directly to a particular stage of development, right? So you take the content out of the scoring manual, and you drop multiple-choice questions. And then people say, oh, yeah, that's like me, or that's like my organization or whatever. And then, you know, you create a list of 10, or 20, or 30 of those, and then you do the math; what I found was, and this is pretty well known, some studies have proven that most people have a natural intuition about which of those options are developmentally earlier later, they don't have to know adult development theory, they don't have to know anything about the stages. They just sort of intuitively know that. And if you ask them to arrange them in order, from least developmental to most developmental, most of them will get it right 90% of the time. And think about it, if you know how those answers are arranged, like that introduces all forms, all kinds of bias, Halo horns effect like if I'm reading you, Jonathan. I like you; I might rate you at a later stage of development than I really should, then you are just because I like you. And if I don't like you, that's called the horns effect, you know, I might rate you less favorably, even if it has little to do with your actual stages of development. And so we found that when people think about organizations, there's a halo horns effect. There's also the well-known, well-documented item response theory problem of people writing themselves more highly than they should. It's a very common, very common effect. And so what I tried to do was I tried to come up with a multiple choice. However, fixed choice questions, where when somebody read the question, you still had a fixed number of choices to choose from, but I tried to design so that people would not know which options are developmentally earlier or later than others. And it turns out that the content matching approach naturally feeds into a multiple-choice question where you have all of those problems. And so that was why I switched not from a content-matching approach to assessment but to a pattern-matching approach using these parameters that are more difficult to explain, frankly. Still, the advantage of them is most people don't know how individual versus collective orientation maps back to a developmental stage. It's completely lost on them. And so they're just sort of free to answer, you know, more honestly, I guess without with fewer biases.

Jonathan Reams  33:31  
So what I recall, if you were using a polarities approach, these things look like they could be equally applicable. And they would pick one or the other.

Jimmy Parker  33:40  
And instead of a multiple choice question, with slider questions, where you have a slider, there's a ball in the middle of it, and the participant, the person answering that question, will move that ball left or right to represent the relative weighting of that two anchors at the end. The way to do this properly is you craft the anchors at the end of each slider so that they're either equally positive and desirable, equally negative or undesirable, or equally indifferent. They didn't know which side was more developmentally earlier or later. The answer to any one question did not arrive at a stage theory. There's some math and some magic under the hood with the calculations that allowed me to combine questions to arrive at a developmental assessment using, you know, terrier filling stages, stages model under the hood to do that, after people that answered all the questions. Most people didn't know whether they were answering in a way that was developmental. It couldn't tell. And so when we did the analysis, it correlated quite strongly with measures of vertical complexity of the inherent work that people are in, which is sort of a proxy for individual development. That's what told us that, wow, people were answering in a way that more reflected their developmental stage than the stage of the organization. They were supposedly evaluating.

Jonathan Reams  35:01  
So given that finding, did you find anything interesting, though, about how you understand collective development or organizational culture, the complexity of work in relation to how people look at it? What could you take away in understanding organizational culture and its development in relation to leadership? 

Jimmy Parker  35:26  
The reason that people are focused on this topic is that it seems it's very interesting; it seems plausible that there is these sort of stages of the way people act with one another norms and cultural norms; they seem to be different. Nobody discredits that; it seems that you can probably, at some point, lay them on a spectrum of more complexity or less complexity; we just haven't figured out how to measure that just yet. So we don't know for sure. And so I would say, Jonathan, if anything, I'm approaching that particular question today with much more modesty than I ever have. You know, many consultants in this space say, this organization that I've worked in for three years is at Orange stage of development, or 3.5, or, you know, you name it. And I'm just like, and I'm not sure we even have a way to assert that right now confidently.

Jonathan Reams  36:14  
The finding that I would say is you gained a healthy degree of perspectival. Humility.

Jimmy Parker  36:22  
Yes. Yes.

Scott Allen  36:25  
I've never heard anybody. I've never heard that phrasing. perspectival. Is that a word? Jonathan?

Jonathan Reams  36:30  
Perspectival humility? It is awareness. And I think this is a good example that, for myself, understanding development has moved from simple stage descriptions to where you realize those were just conveniences that are not really at the heart of what we're trying to study or what we've experienced, or how we're experiencing it. And so you realize that the inculturation, or models you absorb, actually get in the way of understanding the phenomenon more deeply. And when you dig into it, as you did with your Ph.D. That's the result. In essence,

Jimmy Parker  37:11  
I'd say the, probably the even bigger surprise, and oh my gosh, how can that be true, is that even world of od the whole world of OD, not just this narrow slice of vertical od vertical development, but all of OD, you know, there's the letter D, you know, D means development. And I thought that appealed to a broader developmental spectrum where you can say this organization is further along; there's nothing like that anywhere in OD. And so that's why most people in OD, you'll often see the acronym now would be ODC, OD, and change organizational development and change because clearly, we can change an organization from some current state to some future state. Still, to assert that that's more development, that they are developed more than they were before, you have to make all kinds of assumptions, and it's become quite a slippery slope; you can say that they've changed, but to say that they've developed, that's a different story. And that's kind of, again, what my literature review discovered, where we have a measurement problem that we don't know how to measure organizations, collectives, teams, and cultures.

Scott Allen  38:13  
Okay, as we wind down, I'm going to switch gears on you. And this will be a little bit of a hard left turn. Okay, I am interested in this question. Okay. Jimmy, are you ready? What did you learn? What do we know from instructing helicopter pilots that we need to employ in developing leaders and helping individuals be more successful when serving in these challenging roles? What does instructing helicopter pilots do better than we do in leader education, development, etc.?

Jimmy Parker  38:48  
First, I love the question because I have a bit of a multidisciplinary background. And I, I've seen sort of the benefits of trying to solve similar problems, but completely different domains, you know, different people, different language even. And so I love that you're bringing together like my first love by act one of my life, you know, with this sort of act two in the corporate world, and even act three now as an academic. A great article by King and Nesbitt looked at the leadership development industry. And they looked at the factors that sort of drive that industry, like in the form of buyers and sellers, right? You've got providers of leadership development solutions, and you've got people who buy those solutions, right? The name of the article is "Collusion With the Nile: Leadership Development and Its Evaluation" meaning, meaning we don't pay enough attention to whether or not it's working. Still, we spend a lot of money to make sure you know that we're doing it for all kinds of different reasons. So it's, it's meeting someone's needs. It's meeting some people's needs in some way. And in many cases, it might just be to offer some perks to our high-potential leaders or because they're in this prestigious program, and that's the end, but is it making a darn bit of difference? Isn't it helping to change people's health? Is it helping the organization achieve its mission or not? Those connections are usually not looked at. And usually when you've got somebody leans into that pretty heavily to say, here's, here's what the numbers say, it's, it's usually pretty disappointing. And so you've got this, this whole industry here where you could say buyers and sellers are sort of colluding, you know, but denying the fact that it doesn't work. You don't get any of that in the military to go back to, like training pilots and stuff like that, like results matter. We pay attention, we measure, and we make sure that our investments are producing the return that we want; if not, we pivot and make sure that the mission is first and foremost. And that's true for the military. It's true for many organizations, including nonprofits. You know, you've got a mission, and you want to do good in the world. And how does development tie into that? That does your organizational strategy, is it supported by a developmental strategy for people, for teams for organizations, and that's been another area that I've really sort of worked hard to try to make those threads Connect, you know, as explicitly as possible so that this becomes an enabler of the business strategy of the organizational strategy of our overarching mission, that we're all here to do.

Scott Allen  41:17  
Love it. Thank you, sir. You know, at least for me, as I reflect on our conversation, we went to many different places in the quote, unquote, pool. And I love...I think we said this before we jumped on to the actual recording. Still, I love it when we're speaking with someone who is that, that scholar-practitioner, that individual who has these really big, interesting, cool, fun questions rooted in that literature, and then has a space in organizational life where we're working to operationalize some of this and the experimentation and innovation and the creativity. It's just beautiful. It's wonderful. So thank you for the work that you do. We appreciate it.

Jimmy Parker  42:01  
This has been fun. And I have to say that I deny the label scholar-practitioner and do not identify as a scholar-practitioner. If anything, it's If anything; it's practitioner-scholar. Okay. Okay. I guess your first

Jonathan Reams  42:12  
Well, I have to say that your dissertation and the defense meeting you had were one of the most riveting, engaging, and robust defenses I've ever been to.

Scott Allen  42:29  
That sounds like a lot. Well, Jimmy, I hope you will come back. Thank you so much for your time today, sir; we appreciate it.

Jimmy Parker  42:38  
It's been fun. Thanks, Scott. Thank you,