Phronesis: Practical Wisdom for Leaders with Scott Allen

Dr. Marcy Levy Shankman & Dr. Ralph Gigliotti - Leadership Assessments: The Good, The Bad, & The Ugly

• Scott J. Allen • Season 1 • Episode 86

Share episode

Send us a text

About Dr. Marcy Levy Shankman

After five years as the Leadership Coach and Strategist for the Cleveland Metropolitan School District, Marcy is now serving as the inaugural Chief Organizational Learning Officer. In this brand new role, Marcy continues to hold primary responsibility for providing leadership development support to the CEO, the District’s executive leadership team (“the Chiefs”), and 65-person Senior Leadership Team (SLT). Additionally, Marcy has been charged with designing and implementing a system-wide learning agenda for all employees of the District. Marcy lives in Shaker Heights with her husband Brett and loves being a mom for their two college-aged children, Rebecca and Joshua.

About Dr. Ralph Gigliotti

Ralph is Assistant Vice President for Strategic Programs in the Office of University Strategy and Director of the Center for Organizational Leadership at Rutgers University, where he provides executive leadership for a portfolio of signature academic leadership programs, consultation services, and research initiatives. He engages directly with academic and administrative leaders in strengthening the Center’s role as a hub for leadership development research and initiatives. He also serves as a part-time faculty member in the Department of Communication, PhD Program in Higher Education, and Department of Family Medicine and Community Health at Robert Wood Johnson Medical School.

Marcy and Ralph just edited the book:  Using Inventories and Assessments to Enhance Leadership Development

Quote From This Episode

  • "What we've learned is that assessments add tremendous value if used with intentionality, if linked with some broader theoretical frame, if facilitated appropriately, and effectively by individuals who are qualified to help with the assessment and the debriefing."

Resources Mentioned in This Episode

About The International Leadership Association (ILA)

  • The ILA was created in 1999 to bring together professionals with a keen interest in the study, practice, and teaching of leadership. 

Connect with Scott Allen




Note: Voice-to-text transcriptions are about 90% accurate. 

Scott Allen  0:02  
We have two veterans on the podcast today. They are for nice us veterans. But it's a small but mighty band. I think maybe Ron Riggio might be the only other repeat visitors. So today we have Ralph Gigliotti. And we have Marcy Shankman, they have been on a writing adventure. For probably think, Marcy, maybe you told me about this project about two years ago.

Marcy Shankman  0:30  
That would be corrected in a couple of years. It's a journey. Okay.

Scott Allen  0:35  
So I'm looking forward to this conversation. It's all-around leadership and assessment. And you all have been immersed in that topic for a couple of years now. So for everybody, we will put this in the show notes. But Ralph just was promoted at Rutgers. So we might hear a little bit about that. Marcy is with the Cleveland Metropolitan School District. And we had a really fun conversation. I think that episode published maybe last November, Marcy, that was a great conversation. And Ralph, we had a conversation around that time, as well. So maybe bring listeners up to speed with each one of you. And then let's jump into your writing project.

Ralph Gigliotti  1:12  
Sounds great. Yes. So good to be with you both. I was recently promoted to Assistant Vice President for strategic programs in the office of university strategy. So I'm really looking forward to that new, exciting adventure, in addition to the ongoing work with our center as the director of the Center for organizational leadership at Rutgers, and both sort of aligns nicely with my own personal professional goals, and really, really looking forward to it.

Scott Allen  1:40  
You know, you're now part of the strategery right? Is that strategery, is it?

Ralph Gigliotti  1:47  
I mean, we're using strategy or Rutgers. But other places by use strategery,

Scott Allen  1:52  
you might want to look into that, yes. And Marcy?

Marcy Shankman  1:57  
So I'm so excited, because strategy is in my title, as well. And it's yet another cool connection that Ralph and I have that has emerged over the years of us knowing each other. Phil, at the Cleveland Metropolitan School District heavily engaged in phenomenal strategic work as we look to transform what urban public education looks like? And how do you redesign culture and an organization to support that, given the post-pandemic realities that we're facing? It has brought new challenges and opportunities to the leadership team. So serving as the lead coach and strategist at the district means that I am busier than ever, and we are charting new terrain like I never would have imagined. So it is super exciting. And fascinating, endlessly fascinating, still.

Scott Allen  2:51  
Good, good. Well, both of you are immersed in this world of scholar-practitioner, steeped in the literature, familiar with it, but also doing the work. And so that's why I always love these conversations because I think it's a very, very special place to be at that nexus of the theory and the scholarship. And the work. And today is really kind of about this writing project that the two of you have been working on. Marcy, I heard about this first from you, maybe bring listeners in and it hasn't been published. Is it out in the world?

Marcy Shankman  3:28  
We are so close Scott, for this release to come. Oh, good. I would say that to the listeners, depending on when you post this podcast, it will either be hot off the press or nearly off the press. It's interesting because there's actually no press involved. We are looking at an E-book. Yeah. And that's a new endeavor for us both to not have this come forward in any hard copy format. We're close, we're super close. So we have a phenomenal team of authors who we pulled together to help us educate, inform, provoke and challenge leadership educators and professionals on this topic of leadership assessments. It was a true passion project that I will confess, I wasn't sure I was actually as passionate about it as I turned out to be good. But that was an exciting learning experience for me. I wrangled Ralph, to be the CO editor with me, and thank goodness, he said, Yes, we've had a great time discovering more about one another, professionally, through the lens of these topics of how do we think about leadership assessments broadly, what is the ethical and effective use of them, and that's actually the title of this upcoming manuscript turned book. That's in the new year. directions for student leadership series. And maybe that I'll pause there and see what else you want to know. And let Ralph fill in some of the gaps. No, I

Scott Allen  5:09  
think that's I think that's great. I mean, I think what I would like to call this episode is leadership assessments, the good, the bad, and the ugly. What did you find? I mean, I'm really interested because you hear kind of all across the board. I don't know if the two of you have watched this documentary on HBO Max, about The MBTI. Have you seen this now? So really interesting about how, and I love the fact that you're talking about the ethical use of these assessments? And how should they be framed? But maybe we start there, the good, the bad, and the ugly? What do you think, Ralph, maybe we go to you first? Sure. How about the good, what do you find?

Ralph Gigliotti  5:47  
So it has been a great project to work on. And with Marcy, and our co-authors, thinking about the structure of the book, we start out first in the first half by laying out sort of principles, some guiding themes for how these assessments might be used effectively and ethically. And then a series of chapters in the second half of the book focused on applications. Those are the practice chapters. And I think what we've learned, especially from the good is that assessments add tremendous value if used with intentionality, if linked with some broader theoretical frame, if this facilitates it appropriately, and effectively by individuals who are qualified to help with the assessment and the debriefing. And also can be tremendously effective if they meet students, or faculty, or staff, or clients, or whomever the end-user might be, in ways that meet them where they are, and ways that are inclusive. And for all of those reasons to not take this process lightly.

Scott Allen  6:55  
For listeners, I wish you could have seen this, because if you listen closely, Ralph underscored and bolded some keywords, and he kind of leaned in, as he was saying those keywords in his response that I love how you frame that, Ralph? Marcy, what else on the good, what do you think

Marcy Shankman  7:12  
one of the commitments that we held to from the beginning through to the end was that these tools, these inventories are phenomenal assets when used well. And we've got tremendous examples in a variety of settings, and fields. So we push the boundaries of this series, which is traditionally just focused on higher education, and the college campus to be inclusive of multiple sectors because leadership assessments are used all over. Yeah. And so we brought authors from those diverse sectors so that our higher education professionals would have some insights from the private sector would have some insights from the nonprofit sector, from the public sector, because those are the sectors that our students are going into. And we want them to be fully prepared in understanding their leadership capacity. And that's really what these assessments are about. That's one thing I would add. The other thing I would add, and Ralph highlighted this beautifully, which is so much of how valuable an assessment can be. So the good of an assessment is intentionally linked with why you're using it, who is using it, and to what end. So the role of learning outcomes and curriculum that is designed for the inclusion of an assessment, as opposed to here take a test. I'll tell you if you're a leader or not. Yeah, we tried to confront that head-on.

Scott Allen  8:47  
Yeah, I mean, I remember probably the first one I ever took was, you know, disk. And yeah, and at least what I walked away from this, and I'm probably you all 22. At that point, I just walked away, like, "Oh, I'm I'm a D. Okay, I'm a D." And, you know, there's an awareness-building piece there. But if not, as you said, facilitated appropriately. I think it can be limiting for sure. Any one of these right? Any one of these? What were some other insights on the good? What do you think, how are these things being used? How are these resources being used? And what are some best practices that you came across?

Ralph Gigliotti  9:26  
I think the one piece for me that I was really struck by and a number of our co-authors reflected beautifully on this was how some of these tools are triangulated in a really purposeful way. So we'll have sort of a menu of selected instruments or tools, some validated and perhaps some that are not validated to encourage students to sort of take different dimensions away from each of the assessments and helping to sort of pull together a profile that's most reasonable, most meaningful for them. And I think this sort of integration of a menu of different assessments was a really it's a nice model for leadership educators and those engaged in leadership development to consider recognizing the limitations associated with anyone.

Scott Allen  10:18  
Well, Marcy, in, in our past, we've done a little bit of that work is that what Ralph is talking about here where we would, you know, maybe provide the participant with Strengthsfinder, with MBTI, with Gk AI, with something measuring that the emotionally intelligent leadership inventory, but then they would also maybe talk with people they know, and find out what their perceptions around the individual strengths and weaknesses were. And then we kind of bubble up and we start looking for some themes. Is that kind of how you're thinking about that Marcy, as well.

Marcy Shankman  10:53  
So I think there's a yes and with what you just shared. So I think that's a great example.

Scott Allen  10:58  
Now that you know that you included the and I feel much better, because if there would have been a but I would have just been shut down now as the host.

Marcy Shankman  11:06  
And I have to be inclusive and how we talk about this work, Scott. being emotionally intelligent as we are, you've got to do that. Yes. And the process I will share as an editor, and working with these authors was a great dimension to doing this work. What I learned from a number of our authors was the ways in which you situate an assessment to your point, you do an assessment, and what else. So in the practice chapters that really comprise the second half of the book that Ralph outlined, we have our authors intentionally outlining how they use the assessments that they're describing. And the variety of ways in which assessments can be used, far exceed some of the basic expectations. So when I was reading these chapters from our authors, I was learning just as much as I was reviewing, and their insights to how do you use an assessment in a formal classroom? How do you use an assessment outside of the classroom? How do you use the assessment with graduate students? How do you use the assessments as part of the Human Resource Development pipeline, in a company, and we felt like all of those different ways in which assessments can be used was important for our reader to understand because even if you are in a traditional student affairs role, using an assessment with your students in a co-curricular setting, having the broader vantage of, here are some different ways to take the learnings, even outside of this activity that you're engaged in, and project into career development project into identity development, a lot of our authors emphasize dimensions of identity. And this gets to some of the bad and the ugly parts of assessments, how many of these assessments Scott are created, as you know, not in statistically significant ways based on pools of individuals that are not diverse and representative of our population, speaks to the bias and speaks to some really strong limitations that we know exist, not only the ones that I'm sure are highlighted in that documentary you talked about with Myers Briggs, because that's well documented in the literature. Actually, the majority of assessments are not normed on a representative sample. And so we have to be fruitful as users of these tools, with our students. And our authors, the majority of them consistently referenced. So how do we actually help students in the learning process and the CO authoring of their results, not to get the answers that they want, rather situate and contextualize, to manage those limitations, sometimes from the get-go of these assessments, because they were developed in a very limited fashion, and now they're being generalized to all people, regardless of identities, regardless of backgrounds and experiences? And we know that that isn't true. Yeah, we have to deconstruct and reconstruct, often in order to get to the deepest meaning of what the results are.

Scott Allen  14:42  
Well, I'd mentioned that I thought that maybe disk was my first but I was reflecting as you spoke. I think the first one I ever took was it was probably 12 questions. You might know Mary Peterson, right. Hey, Mary. Hi, Mary. And it was wonderful, it was one Assessment, and that we got a flavor of ice cream, we could probably look it up on the internet, and I ended up being chocolate. So I'm probably guessing now in retrospect, maybe the ice cream assessment personality assessment test had not been validated. Chronbach's Alpha may have been an issue. Yes, Marcy? Are you telling me I may not be a chocolate

Marcy Shankman  15:29  
telling you that it may not be as reliable response as you had thought when you first took the assessment? Now listen, we know that there are some really insightful ways of learning about ourselves that are not normed. Yeah, what we also know is, we have to be really careful. So this gets to the bad and the ugly, we have to be really careful about how much meaning we derive from those kinds of I call them Cosmo quizzes. For those of our listeners who remember the Cosmopolitan magazine of the 80s, I'm dating myself, there were always quizzes in there. Well, you know, we got to be careful with what we learn from these light touches, it's not that they can't be helpful is that we have to be really careful about how much meaning we ascribe to them.

Scott Allen  16:17  
I love it. Ralph, what do you think?

Ralph Gigliotti  16:19  
I think that is. So a well-articulated Marcy, and certainly reflects my own and interpretations of the bad and the ugly, in terms of what we learned from the different authors. And then their chapters. The one, the mantra of sort of doing no harm is really what stood out to me. And I don't think any leadership educators, or individuals that are using these assessments, have the intention of doing harm when we're using this pedagogy, right. But I think if they're facilitated inappropriately, or if they're not facilitated in a way that recognizes the unique lived experience of all in each student, who are completing the assessment, it has the potential to do harm, either in constraining learning in minimizing what individuals might take away from their own sort of leadership sense-making, or perhaps even worse, not recognizing some of the barriers that individuals might face or encounter when applying what they learned from these assessments in navigating various leadership roles or structures. So I think for all of those reasons, to think about how we can encourage readers and individuals that are using these assessments, to use them in ways that maximize student learning that maximizes student engagement, and that is really honest with the intention of the assessment,

Scott Allen  17:48  
What if anything comes to mind for you for ugly, that that you came across that just really stood out. And Marcy, this may not be aligned, I saw that you just raised your hand you want to jump in, this may not have to do with what I just asked, but go for it. What do you think?

Marcy Shankman  18:02  
I feel like I'm raising my hand. And it's like, me, me, Scott, one of our listeners. This is how we're trying to not talk over each other because we're trying to be coordinated. So I raised my hand, because what I wanted to add 

Ralph Gigliotti  18:17  
Marcy, and I get really excited about this too. So we would often jump on.

Marcy Shankman  18:26  
So part of what you were just commenting on Scott, and you asked about was where I was heading, which is one of my fears. And it's actually a motive. It was a primary motivation for why I said yes, I wanted to take on this responsibility of editing this book and pulling together this amazing team, I just cannot tell you enough, how terrific Our team of authors is. It's because I've seen over the years too many quick fixes sought by individuals through the use of a tool. Yeah, so there's a problem, let's take an assessment. And then the assessment results will fix the problem, not the best use of a leadership tool. Yeah, we have a team that's dysfunctional. So rather than actually doing the hard work, and having that crucial conversation or that direct confrontation, we've got a team member who's wrestling an issue, which is now negatively impacting the whole team. Let's do a team assessment, and learn about what makes for a high-functioning team and not actually deal with the elephant in the room, which is this problem. Yes, coming from an individual's behavior, where direct feedback and coaching are needed. What I appreciated so much about this learning journey of the last two years, was challenging myself and even challenging our authors to say how do we explain what is an intentional deliberate process is needed in order for a tool to work, there may be a popular one that everyone is using right now. So that's the one you want to pick. How do we interrogate whether that's the right tool? For what purpose and for whom? Because we can get caught up in the leadership industry. And I think the three of us have all talked about how we know we're part of this industry yet. So how do we contribute to the solution rather than being part of the problem? And I'm hopeful that this publication helps us contribute to the solution of how do we best use assessments, ethically and effectively for the desired purposes of improving what leadership looks like?

Scott Allen  20:50  
I love it. Ralph, you're nodding your head.

Ralph Gigliotti  20:52  
Yeah. I wholeheartedly agree with that. And I think to go to your question of the ugly Scott, the book doesn't detail, specifically some of the ugly uses of these assessments necessarily, but I think we all may have encountered in some ways, the abuses or misuses of these tools for purposes that they were not intended or designed for. And I think that's where we see the ugly inaction.

Scott Allen  21:20  
Well, what I would love to hear from each one of you is maybe a reflection that stands out for you, as you think about this project, what was an insight that we maybe haven't covered? Just in how I framed the conversation. But was there something you came across from an author that just made you think, oh, wow, I hadn't framed it or thought of it that way before anything come to mind for you?

Ralph Gigliotti  21:43  
That's an interesting question, the first thing that comes to mind is not a specific author. But I've always been intrigued by and I think this is shaped by the work have been doing at Rutgers on this notion of lateral learning. So although we're approaching leadership development in an academic context, we can learn a tremendous amount from Johnson and Johnson across the street from us, right, even though the context is different. And I think when Marcy and I were initially having conversations about sort of the scope of the book, we were really excited about diversifying the authors so that we could still focus on student leadership development, but learn from applications and other sectors. And I think reading those chapters, in particular, there's a tremendous amount we can learn by looking beyond the walls of the Academy. Yep. And I think, as it relates to assessments and instruments, it certainly lines up with our experience. Awesome.

Marcy Shankman  22:42  
I think for me, what I came to appreciate was the breadth of the tools that are available, and the difficulty in deciding what makes the most sense, and the paralysis that you can experience as a leadership professional when you know that it makes sense to use an assessment. And then you start the process of figuring out Well, so what tool should I use which assessment, the inclination is to go with the flow or go with the crowd? And what I learned was that the field is even bigger than I realized, I thought I knew a lot about what tools are available. I gotta tell you, I learned so much about what tools are actually being used, and in what way? Can you use them, so that we're advancing our knowledge, awareness, and ability to demonstrate leadership? It just overwhelms me how big of a marketplace it is.

Scott Allen  23:48  
Well, I'm interested in knowing if there is a tool or a set of tools that stood out to you, that you think are worth investigating that as you learned about them, you thought, huh, I want to learn more about that work in that perspective, and what these folks have developed, that maybe you hadn't heard of before, either one of you.

Marcy Shankman  24:10  
So, Scott, we tried to be really agnostic in the book and not advocate for any particular tool.

Scott Allen  24:17  
So I'm doing you're doing a and not a but/and to me here. It's another version of it Marcy! I understand where you're going. 

Marcy Shankman  24:29  
What can I say? What can I say?

Scott Allen  24:32  
For listeners? We're good friends. So we like to banter!

Marcy Shankman  24:37  
I can't I think because of that I can't answer your question because I haven't thought about the individual tool as something for me personally to consume. I really spent the majority of my energy on how do we make sure that we are informing your type love our podcasts, the good, the bad, and the ugly. Because every assessment has its limitations. Yeah. Every inventory has its strengths. Every survey can be misused or used well, and so I think I'm literally incapable of answering because I haven't been thinking that way at all. 

Scott Allen  25:21  
I love it. I love it.

Ralph Gigliotti  25:22  
And, and I totally agree with that. I also would add that I spent so much time right before bed just Googling, right? Like all of these assessments and instruments that we're being shared by our authors go beyond the normal ones that most of us are familiar with Strengths, DISC, Hogan, TKI. But there were a whole bunch of other ones that I had never heard of. And I feel like I've been using assessments with some frequency for a while now. So that, yes, the huge marketplace that Marcy described, was reinforced in the chapters.

Scott Allen  26:00  
Yeah. Well, I love it, because we're leaving listeners on a little bit of a cliff hanger. They have to go explore the publication to they do, yes.

Marcy Shankman  26:11  
Okay. Let me just add one more thing. Yeah, yeah, what we were afraid of was that this diverse collection of authors would all write to the same assessments. And therefore, we as editors would have to wrangle one away from one author so that another author could write about it. What I will tell you, Scott, and all of your listeners, we have some overlap, chapter to chapter so I couple of assessments are reviewed by a couple of different authors. We did not have to wrangle at all, and the list was much longer than the half a dozen easily recalled assessments that most leadership professionals think about we probably are covering what would you say Ralph 18, or 20? Nice. Oh, good. And then we have an appendix that refers to another 60?

Ralph Gigliotti  27:10  
Yes. Really? We've revised that several times over. 

Marcy Shankman  27:16  
But yes, over 50. So when we are struggling to answer your question, that's why Yeah, because there was a lot. It's a big marketplace. That's, that's all I can say.

Scott Allen  27:27  
So go ahead and tell listeners the title of the publication, we're going to wind down, I'm going to ask you the traditional question of what you've been reading what stood out lately. Both of you gave me great responses last time that provided hours of education or fun. So what's the title of publication Marcy?

Marcy Shankman  27:46  
I don't remember the format is in the process of editing and final editing. And so now I'm, I'm just sort of laughing because I don't know where we left it off with our editors. Because you know, there's when you edit a book, there's anyone you enter, edit this book in the new direction series. There are senior editors. So it's true. Susan Komivesand Kathy guthrie are shaking their heads right now, like, seriously? Exactly. helped me out here, Ralph, this is why I have a partner.

Ralph Gigliotti  28:23  
Yes, I'm also I'm

Scott Allen  28:25  
for everyone listening right now googling frantically!

Ralph Gigliotti  28:32  
The Effective and Ethical use of Leadership Assessments and Instruments really gets at the heart of the title, we might be off by a couple of words. But that's...

Marcy Shankman  28:42  
This is what happens. Scott, when you ask us, and we aren't done yet, because we're literally in the process of editing. In fact,

Ralph Gigliotti  28:49  
proofs of proofs just arrived this morning at 7 am.

Scott Allen  28:53  
Oh no, no, I totally understand, and Marcy if you want to say to me right now, in all, in all honesty, what is the title of the textbook you wrote with Tony Middlebrooks? Sorry, Tony, I can't tell you. I get some of the words correct, but maybe not the whole thing. So I get it. I get it. Okay. Marcy, the last time you were here, you shared with me the work of James Clear, tiny, atomic little baby habits. Right? BJ Fogg. And I learned that and that was really cool stuff. And Ralph had, I believe, Ralph, you introduced me to the TV show the politician? Yes, on Netflix, which was hours of kind of dark humor, which I enjoyed as well. What stood out for the two of you lately, either, and it's something you've been streaming listening to reading, just consuming.

Ralph Gigliotti  29:43  
I just finished upstream by Dan Heath, which was great. It was a great read and how we can address some of those upskill there it is. Marcy is holding up the book right now. It's really excellent. Right, Marcy. How do we address some of those? Yeah. stream preventative factors to avoid some of the downstream issues that we deal with in organizations of all kinds.

Scott Allen  30:06  
Okay, awesome. Awesome. And he was made to stick.

Ralph Gigliotti  30:12  
Okay, one of the brothers and switch.

Scott Allen  30:15  
Oh, and switch. Okay. Wonderful, wonderful. Marcy. You just held up those books, you're gonna have to come up with something different.

Marcy Shankman  30:21  
Yeah, well, I read that. Sorry, Ralph, I read that months ago.

Ralph Gigliotti  30:27  
I've been editing a book co-editing a book, Marcy.

Marcy Shankman  30:34  
So I'm in this phenomenal book group. And we just finished Adam Grant's new book, think again. Yeah, which I listened to first. And then I read, which is a new approach for me. And I have to share, it was really satisfying to hear it first. And partially, I was so compelled by the content that I knew I had to own it. The process of reading after listening, though, in terms of learning was a really, really cool experience. So if you haven't done it before, I highly encourage listening to a book first, then follow it up with reading it. I did the same thing with the Caste by Isabel Wilkerson. Yeah, which was, if you haven't read that book that is just a lights out powerful it sits with you for a long time. And then the last book that I'll tell you that I'm reading that I'm almost done with is called positive intelligence. Okay. And it talks about the ways in which we get in our own way. It is a fantastic handbook for self-exploration and individual coaching of yourself. And as a coach, it is a phenomenal tool to use with others. And it happens to have one of these great assessments online, that's easy to take and gives you immediate results. So it even wraps us back to the topic of our conversation with you today.

Scott Allen  32:04  
Nice. Okay. For the two of you, thanks for exploring this space. I think it's a really, really cool project, it's really very much needed. And putting a critical eye on that, I think is only a contribution to the work that we do. So thank you for that work. Ralph, Marcy...Thank you for being to time guest on Phronesis. Thank you. Thanks so much. We really, really appreciate your time today. And for listeners, we will put the title and a link to the book in the show notes.

Ralph Gigliotti  32:39  
The real title. Cutting couldn't google it quickly enough.

Scott Allen  32:46  
Take care everybody have a great day. Thanks, Scott. Thanks.

Ralph Gigliotti  32:50  
Thank you so much.

Transcribed by https://otter.ai