Differentiating Difference from Disability: The DYMOND

This transcript is made available as a course accommodation for and is supplementary to this episode / course. This transcript is not intended to be used in place of the podcast episode with the exception of course accommodation. Please note: This transcript was created by robots. We do our best to proof read but there is always a chance we miss something. Find a typo? Email us anytime.

Earn CEUs for this episode

[00:00:00] 

Intro

Kate Grandbois: Welcome to SLP nerd cast your favorite professional resource for evidence based practice in speech, language pathology. I'm Kate grant wa and I'm Amy 

Amy Wonkka: Wonka. We are both speech, language pathologists working in the field and co-founders of SLP nerd cast. Each 

Kate Grandbois: episode of this podcast is a course offered for ashes EU.

Our podcast audio courses are here to help you level up your knowledge and earn those professional development hours that you need. This course. Plus the corresponding short post test is equal to one certificate of attendance to earn CEUs today and take the post test. After this session, follow the link provided in the show notes or head to SLP ncast.com.

Amy Wonkka: Before we get started one quick, disclaimer, our courses are not meant to replace clinical. We do not endorse products, procedures, or other services mentioned by our guests, unless otherwise 

Kate Grandbois: specified. We hope you enjoy 

Announcer: the course. Are you an SLP related [00:01:00] professional? The SLP nerd cast unlimited subscription gives members access to over 100 courses, offered for ashes, EU, and certificates of attendance.

With SLP nerd cast membership, you can earn unlimited EU all year at any time. SLP nerd cast courses are unique evidence based with a focus on information that is useful. When you join SLP nerd cast as a member, you'll have access to the best online platform for continuing education and speech and language pathology.

Join as a member today and save 10% using code nerd caster 10. A link for membership is in the show notes

EpisodeSponsor 1

Kate Grandbois: Hello everyone. Welcome to SLP Nerd Cast. We are really excited today to welcome back two of our favorite guests, and I say favorite because every single time we have you on the show, there is at least one fit of laughter, at least one fit of laughter, and [00:02:00] we learn a ton because you're both brilliant. So we are really excited to welcome Dr.

Doug Peterson and Dr. Trina Spencer. Welcome Trina and Doug. Thank you. Thank you. And yes, we're up for some fun. Thanks for coming back. you guys are here to talk to us about the Diamond assessment. Before we get started, can You please just tell us and our listeners a little bit about yourselves?

Trina Spencer: Go ahead, Doug. 

Doug Petersen: Oh, okay, sure. Yeah. So, um, yeah, I'm Doug Peterson and I'm currently a professor and the Maggie and Dick Scarlet Chair in speech language pathology at the University of Wyoming. Um, and my background goes, I guess this is my 25th year. As a speech language pathologist, I couldn't believe it. Um, so my background is in speech language pathology.

Obviously I worked in the schools for several years before I went, um, back to school to get my PhD with my very dear friend Trina. We weren't friends right when we started, but we became friends very quickly in our program. We were, yeah, we punched it [00:03:00] frenemies. Uh, so yeah. But I worked in, um, Utah and California and Washington and I worked with a lot of children who spoke Spanish primarily, um, and did a lot of work with cultural, um, linguistic diversity and a lot of that you'll see kind of those threats are pervasive and they, they weave their way or woven their way through my life here with all of my research and, and what I focus on.

Um, so that's a little bit about me. I think that's enough though, Trina, I. 

Trina Spencer: I am Trina Spencer and I am currently the director of Juniper Gardens Children's Project, which is in Kansas City, affiliated with the University of Kansas. I'm also a professor in the Department of Applied Behavioral Science and I have courtesy appointments in the Department of Special Education and Speech Language.

I can never remember, it's so long. Speech language, hearing sciences and disorders there. [00:04:00] There we go. Um, yeah, and I used to be a school psychologist and behavior analyst in the New York City area and I also have been doing this for 25 years. Doug and I actually have a lot of like strange commonalities here.

I was in New York, he's in California, and yeah, we met and started our research together in 2006 when we did our PhDs at Utah State University. 

Kate Grandbois: Before we get started, I do need to read our learning outcomes. Learning outcome number one, self-report knowledge gains related to how the diamond reduces test bias related to external factors learning outcome number two, self-report knowledge gains related to how dynamic assessment improves clinical practice learner outcome.

Number three, self-report knowledge gains related to the importance of dynamic assessment for DLD and dyslexia and learner outcome number four, self-report knowledge [00:05:00] gains related to how the diamond can be used for screening as part of eligibility testing.

We've had you on the show before talking about MTSS, narrative intervention. The two of you co-own with others, a business language dynamics group. You've got the cubed.

I'm gonna see if I can remember all of your, all of your really incredible, uh, assessment tools and intervention tools. So story champs, story champs, a, a C, the Cubed and the Pearl. Is that right? And now the diamond, um, for any of you who are listening who have not gone back in time and listened to our previous work with Trina and Doug, I highly recommend it mostly because there are several, at least one product that you have that is free to download and an incredible, it's, I believe it's norm referenced.

Isn't that right? Can you tell us about that really quickly? 

Doug Petersen: That's the cubed assessment and it's primarily a criterion reference general outcome measure. Um, and it's designed to measure both word recognition and language comprehension and the conflation reading comprehension. [00:06:00] Um, it also gives you measures of writing as well.

Um, and that is for preschool through, currently through eighth grade, but we're working on high school. Um, and yeah, that's free to download. There, there are some norms for that test, although it's primarily not really a norm reference test, it's generally to be referenced. The scores were designed generally to be referenced to the criterion, to benchmarks that students are supposed to meet at the end, well actually the beginning, middle, and end of every school year from preschool to eighth grade.

I know you really wanna 

Kate Grandbois: nerd out about the psychometrics. I know we've already had this conversation that you're so excited to talk about those. Sorry, go tr. Go ahead Trina. 

Trina Spencer: I was, I was just gonna say the reason why it's a criterion reference assessment is because that's what gets used or that's the kind of assessment you need for universal screening, beginning and uh, beginning, middle, and end of year to identify kids who need tier two interventions.

And then you need a progress monitoring tool that you can [00:07:00] use with those children who are getting interventions. So something more regular. So in MTSS. Uh, context. It takes care of universal screening and progress monitoring. That's what the, the why you need a criter and reference assessment. 

Doug Petersen: Awesome. And the, and the, I I just wanna add that the, the reason why anybody would care about the cube is because it measures oral language in a, in a way that most people would acknowledge as being valid because it's essentially taking language samples, right?

It's a language sample analysis procedure in addition to all of the word recognition things, reading fluency, but it measures the construct of oral language, which is a unique thing shockingly and shouldn't be, but is a unique thing in the, uh, general education world. 

Trina Spencer: I don't know if you guys know Kate and Amy.

Do you know that, you know, the Cubed has an online platform, a digital platform too. Um, that part's not free, but if you use it, AI [00:08:00] actually translates, or sorry, translates Transcribes, yes. Translates, but transcribes the language sample for you, so. Oh, that's 

Kate Grandbois: awesome. 

Trina Spencer: Yeah, and it's, it's quite accurate with our, with young kids.

So that's kind of, oh, that's awesome. New stuff, you know, new advances since we've spoken about it last. 

Kate Grandbois: Well, we will link all of that in the show notes. Um, for anybody who's new here and listening, it just gives a little bit of context for basically who you two are as researchers, but what you've brought to the field in terms of accessible, unbiased, helpful products, tools, and assessments.

Um, and today we are here to talk about the diamond. Uh, what can you tell us, how did this come about? What's, what's the story with, with this, with this assessment? 

Doug Petersen: Well, you know, the, the idea of dynamic assessment certainly doesn't start with Trina, nor I, um, you know, this, this is something that goes way back in the field of psychology really.

It essentially [00:09:00] originated there. Um, and Liz Pena brought it to the field of speech language pathology. I think, to my knowledge that I think it was her dissertation like in 1992. And it was a brilliant idea. This idea of measuring someone's ability to learn instead of just measuring what they know or what they can do, which is totally confounded by prior experience, exposure, cultural linguistic factors.

Instead, it's just about what your learning potential is. And so you can apply that to a lot of different things. And the diamond applies that measurement to language ability and to reading ability. So the ability to learn language and the ability to learn how to decode. So, um, it's origins with Trina and I though they go back.

Ooh, a long time prior 

Trina Spencer: to our meeting. Yes. 

Doug Petersen: Yeah. Right. Okay. So 

Trina Spencer: I'll, let me, let me tell you my version of that up to that point. Um, so yeah, dynamic assessment originated in psychology, but it was used [00:10:00] as a tool to determine which students should be allowed to go to school. If you, if you have ever heard this term, um, educatable basically children with disabilities were per, uh, were not permitted to go to school unless they could pass this assessment and be determined educatable, right.

So it was used as a, like a gatekeeper to decide who could go to school and who couldn't. The, and that's like from the 50 fifties, I think. Anyway, it has obviously come a long way. And, but you could see that with an origin like that, it kind of fell out of favor for some time. But like what evolved really was like the kind of modern day IQ test.

Out of that. Um, and, but the dynamic aspects of it have really been like dropped, right? So. In, I was in New York as a practicing school psychologist and the kind of issues that I was up [00:11:00] against were, uh, doing assessments like education evaluations with children with significant disabilities, like very, um, you know, high support needs.

Very significant. And, and by law I had to give them an IQ test, right? Which are heavily conflated with verbal skills. And these kids generally could not, you know, communicate typically. And so, but I was responsible for making a recommendation for what type of educational setting that they could be in.

And, um, so, you know, creative and analytical as I was, I guess at the time I did not know about dynamic assessment. I'll say that I did not understand or know it, but from my analytical perspective, I was like, well, what do they really wanna know? They wanna know how this child learns. So I would engage those educational psychology evals.

As a teaching procedure, I would give them that norm reference standardized IQ test. But then I would engage in some learning tasks and I would be, I was looking at things [00:12:00] like, you know, how many prompts did they need to learn something? You know, how resistant were they? Did they engage in problem behavior?

How, what was their attention like, you know, what kind of motivation did they need? Reinforcers, these kinds of things. Those are all dynamic abilities, right? To learn, dynamic abilities to learn. And so that was my experience and when I met Doug, um, when we were PhD students, he had a word for it. It was called dynamic assessment.

I was like, oh, that's kind of cool. Alright, now you can pick up Doug. 

Doug Petersen: Well, uh, so, and, and for me as a speech language pathologist, I was always dealing with questions of, is this a difference? Is this a do disorder? I don't know, um, whether this child is having difficulty on this particular norm reference test because they have a language disorder or because, uh, they have never been exposed to these words before to this testing process before just multiple factors, right?

That, that confound the interpretation of [00:13:00] many norm reference tests. And that was a real problem. And I would test children in both English and Spanish, and often they would do poorly in both of those norm reference tests. And I still didn't know if it was a language disorder because they're still acquiring English and they've, and they've experienced attrition in Spanish, so they're sort of stuck in the middle.

And I, so there's a lot of confusion there about how to actually truly identify whether someone has a. Developmental language disorder. And that's also, so that's one branch that really drove the development of the diamond. And there was another branch, which was all about reading and identifying children really early on.

And like Joe Torgeson showed us and several others that if you do super intensive early intervention on, uh, with children to help them with their word recognition. With phonics, essentially you can make a huge impact, but it requires early identification. And you have this real catch 22 situation where you wanna identify reading disorders [00:14:00] before they ever manifest.

But how do you identify a disorder before it's ever something that you can identify? And the way to do that would be. Dynamic assessment because you're not measuring whether they can read the words or recode or decode, but whether how difficult it is for them to learn. And so both of these ideas of measuring language, learning and decoding learning potential fueled the development of the diamond.

And they were major, the those were major. Um. Problems that I was trying to solve as a speech language pathologist, uh, researcher combo that I am. 

Kate Grandbois: So I know I teased you earlier about being a psychometrics nerd, but I I think I only tease you because you're both brilliant and you've solved, you've solved the riddle I it seems, with this assessment.

So I wonder if you could share what you shared before we started recording related to why the psychometrics of this are important. What is the, [00:15:00] what is the context of the tests that we have to date, uh, in terms of normative samples? And, and why does that matter? Why do, why should working SLPs who are in the trenches with caseloads of 8,000 kiddos care about the psychometrics of, of this test?

Trina Spencer: Mm-hmm. Can I, can I jump in there just before you want to answer that? I know, and I want you to answer that, but I have one other point to maybe bridge this. Is that. The types of dynamic assessment that we were doing in our clinical practice were very informal, right? And they, they vary for every clinician.

And, uh, but that's not scalable, right? It's not something you can, you can like give to your colleagues or do a training on the way in which they were done informally. And so when Doug and I started talking about this, and we identify like things like dyslexia and language disorder that are huge. I mean, you know, upwards of [00:16:00] 20% of the children population, it's like, wait, this is the tool that everyone needs, but how do you replicate it?

How do you make it so you can scale it across thousands of clinicians? And that the answer to that is that you need it to be standardized. Have, and that means standardized administration. So clear administration procedures and clear scoring procedures. And it has to be reliable. In other words, you know, these 10 different clinicians, they've gotta be able to get the same results if they were all testing the same kit or from test to retest time.

So they need to be reliable. And so those were, you know, important, um, psychometric requirements in order for us to scale up, which has been the reason why dynamic assessment has not been adopted in mainstream education or clinical services is because it lacked the reliability and [00:17:00] standardization needed to conform to, uh, the requirements of scaling or, um, you know, uh, state or federal laws.

Okay. Now done. 

Doug Petersen: Right. No, it's that, that was perfect. And I mean, it's a beautiful segue into this question, a pre-question, actually another pre-question. Sorry, Kate, just for a second of, of this idea of like, you know, why hasn't it been adopted widely? And you're right, Trina, the reasons are not because of questions of validity, because all of the research that's been going on for 30, 40 years on dynamic assessment of language and decoding, almost every study has very, very high sensitivity and specificity, very strong accuracy in identifying children who are diverse children, um, accurately, right?

So it's not like people are saying, oh, I don't even know if this is going to work or not, but it's like, how do we make it work? How can we actually make this work across [00:18:00] a large scale? And then in conjunction with that, um, so we did make the choice to norm. This dynamic assessment, but that didn't really affect anything other than just the efforts that we had to go through to obtain a normative sample.

It didn't really affect the sensitivity and specificity of the diamond, which for both identifying, decoding and language is in the nineties, 90 percents. Um. All it did was it just meant that we needed to obtain a normative sample from across the country that was representative of the country. And so that's what we did.

And now the thing that makes it unique, and this is actually finally getting to your question. The thing that makes the diamond unique and its normative sample unique is that we did not exclude any children at all from the normative sample if, except I need to be very clear about this, unless they had any disability or concerns of having a disability.

So the normative sample is a group of [00:19:00] multilingual and monolingual children who are typically developing, quote unquote, right? We define that more specifically. But um, this is really important because you have like one in four children in the US are multilingual or come from a home where it's not just English that's being spoken.

And most of the norm reference tests out there today exclude. Those multilingual children from the normative sample, which means that those norm reference tests should not be administered to those children because there's no normative sample to whom you can reference. Right. And so that's a huge solution to a major problem that we've had in this country, which is the tests that we have are not appropriate for a large percentage of the student body that we have.

Trina Spencer: So they're biased. 

Doug Petersen: Yeah. If you, if that's, they're biased if you administer them to, to those children. That's right. [00:20:00] 

Trina Spencer: Yeah. And I think a lot of clinicians, you know, a lot of SLPs and school psychs, they get that their tools are not very good with anybody who's not like white middle class, you know, American.

Prototype. You know, they know that they want, they want better tools, they need 'em, you know, and they know, I know, I, I know I did. I had the same problems working with the multilingual kiddos that I was testing. I was like, I have no idea if it's because he can't speak English or if he just can't do this task.

You know, I had no idea and I couldn't help him. 

Amy Wonkka: Can you talk to us a little bit, so if you are one of those speech pathologists who has been implementing some degree of dynamic assessment with your caseload, um, can you talk a little bit about how the diamond might be particularly helpful for that clinician who's kind of been doing their own like, non-scalable version of dynamic assessment or trying to, um, what are some features of having the diamond assessment all [00:21:00] organized that are gonna be helpful for that clinician?

Doug Petersen: Well, so you're asking a question about validity and you're asking a question about reliability. And the reality is, is that no test that you administer is exonerated from being reliable or valid. And so you can make up your own dynamic assessment, and that's perfectly fine. But, um, it is not best practice to do so unless you have evidence of validity and reliability.

And that is essentially what the diamond does. It provides that evidence of validity and reliability, um, that we, we've gone through the efforts to obtain that evidence of reliability and validity for this particular dynamic assessment. So that's one major reason to adopt a test like this that's gone through a major rigorous psychometric evaluation instead of just adopting something and making something up on your own.

Trina Spencer: Are you done? I wanna answer that differently. 

Doug Petersen: But [00:22:00] you're finished if you're not done. No, no, no. That's good. You can answer it differently as long as it doesn't contradict me. No, I'm just kidding.

Trina Spencer: Well, I can't promise that, and I don't, I don't think Kate and Amy set that as a condition for me speaking, so, no, of course. As long as 

Kate Grandbois: the gloves stay on. It's fine. 

Trina Spencer: Okay. I wanna, I really do wanna answer her question, right. So, so here's the thing is that the clinic clinicians need, they, they have laws that they have to follow and regulations that tell them, these are the kinds of tests that you need to use to identify or determine eligibility.

So you have to have standard scores and you have to have, you know, you have to have certain things, right? So if they were to just ignore their norm, reference standardized tests, and just do their dynamic assessment informally. That's not gonna cut it, that's not gonna be enough. So usually dynamic assessment is supplemented to those [00:23:00] NRTs nor reference test to provide the actual information that they really need to be able to say, what are the recommendations?

But use those NRTs for identification 'cause they fit the criteria. Right? But with the diamond, this one, and because it's formal and it's been through all the rigorous psychometric analysis, like a traditional norm reference test for reliability and validity, it'll meet the mandate that the clinicians have, right?

And also give them that clinical information that they need, that they really need. Like, how is this person learning? And they'll, it'll do it in less time. So in other words, you can replace all those other things that you do like, like a whole list of a battery of things, and you're gonna get the better information faster.

Amy Wonkka: Those were both such good answers. Do you disagree, Doug? Are you guys fighting? 

Doug Petersen: No. Oh, no. Wasn't I don't disagree at all. No, it, it's okay. You're absolutely [00:24:00] right. Yeah. 

Kate Grandbois: Yeah. Look at, look at that. An argument turned into an agreement on 

Trina Spencer: the air too. He just, he just likes to take that, like he, whenever he can, he likes to talk about reliability and validity and, 

Doug Petersen: well, I'm not gonna retre, I'm not retracting my statement about it's the, the reliability and validity.

That's, that's, I didn't 

Trina Spencer: say that your answer was wrong. I just said I want to answer it differently. 

Doug Petersen: Yeah, it's good. It was a different 

Trina Spencer: perspective. 

Amy Wonkka: Okay. All of that was great. How long does it take? If, if I've, if I have been so because I, 'cause I agree with you as a, as somebody who has worked primarily in schools, we are trying to check all of those boxes, right?

We are trying to determine eligibility. Different states, different entities have different criteria around what you have to do to be able to check that box. Um, and ultimately we're trying to get information to generate meaningful goals and objectives. Um, so if there's one tool that will enable me to both check the box and gather meaningful [00:25:00] implementation information as far as my goals and objectives.

How long, like how long is this assessment? Some of the language batteries that are out through take several sessions to administer, um, what does the diamond look like? 

Doug Petersen: Yeah, so the diamond has, um, you know, three major purposes. One is to help identify developmental language disorder or language disorder.

The other is to help identify children with dyslexia. And then the third purpose is to help, um, identify strengths, weaknesses, and inform intervention targets. And if you, and I guess I should say the fourth purpose is a universal screener, which you can use the diamond for as well. The, um, if you use the diamond just for screening, you wanna screen children, and let's say you wanna identify dyslexia and you want to identify DLD in young children across an entire state.

And you want to push for legislation for not only hearing screening, uh, not only dyslexia screening, but also DLD [00:26:00] screening. Then you can administer the diamond and you can do it in five. Um, six minutes to do that screening for, um, DLD and dyslexia. Um, but if you wanna do a full battery assessment, you wanna do the whole thing, you're looking at maybe 15 minutes to do, um, yeah, probably about 15, maybe 20 minutes to do the whole dyslexia and DLD um, subtest.

Trina Spencer: Yeah. So all you would do is do the main subtest for screening and for diagnosis. But if you're doing a diagnosis or eligibility determination, you would also add the kind of language sample analysis kind of, uh, of activities that are embedded in Diamond. So that would require somebody who has a little bit more expertise, um, who, who can judge Morpho syntax, for example.

Right? We wouldn't want any, you know, first grade teacher doing morpho syntax, but they could certainly do the screening subtest, which are the, the main predictive, uh, subtest. [00:27:00] So 

Amy Wonkka: are you getting standardized scores for both of those? Like if you just run the screener, are you also getting a standardized score?

Yes, they're shaking their heads yes. If you're listening to that. Yeah. 

Doug Petersen: The, we, we really struggle with these terms of screening and diagnosis because, um, what we found is that, okay, so like with most other norm reference tests, by the way, um, they have lots of subtest, but there are only a couple of those subtest that are actually used in the diagnosis of the disorder.

Right. You can see that with lots of the tests out there that are available. Okay. Well that's actually true for the diamond as well. Some of the subtest were designed to help identify disorder and others were more designed to help us identify strengths, weaknesses, and intervention targets. Um, and so, uh.

Yeah, I forgot where I was going with that. 

Trina Spencer: When they get scores, so the strengths, penises and targets, it's not really like a standardized score, but like the scores that lead to that are come from [00:28:00] the standardized scores. 

Doug Petersen: Right. And uh, right, and the Right. The point that I was trying to make is that, um, okay, so with the diamond you need to have the teaching phase, step two score, the behavior scale and the learning scale to help identify DLD.

And you've got 95% sensitivity and I don't know what it is, 93, 91, 91 specificity. Okay. So you have that with those three sub tests, and you obtain a standard score and a percentile rank and the confidence intervals and everything. Now you can do that to do a screener. And you still get the standard score percentile rank, and you're screening every single child with that sensitivity and specificity still, or you're also doing that as an SLP focused on children who've been referred to you to identify disorder sensitivity.

Specificity is the same. And so I really, the question about whether, um, [00:29:00] the, what you do with the test has more to do with its purpose, right? And the interpretation, like the scores are the same. The standard scores and percentile ranks end up being the same, and the accuracy stays the same. You don't lose accuracy just because you've administered it as a universal screener as opposed to as a diagnostic tool.

Trina Spencer: You know, it might help to like unpack why we even have screeners. So like the whole purpose of a screener like this when we're screening for a, at risk for a disability, usually that's because it's too costly in terms of time, personnel and resources to do the diagnostic evaluation to, for every kid in that school.

And so they create a shortened version that can, is not as good in terms of predictability. Right? But it could be at least done. With the masses. So a short, easy version that is not necessarily predictive, but it'll get you something. And it's [00:30:00] supposed to have correlation with the diagnostic, um, results from a full battery, right?

So you're talking about things that are, take a long, long time and are more accurate, but their sensitivity and specificity are still below what the diamond is getting because they're static assessments. And then the, this little teeny easy to use, which could be like a teacher checklist or an SLPA dozen observation or something like that for a screening.

Um, so those were like needed for feasibility reasons. But you decrease your, your, um, classification accuracy for the screening. So with the Diamond, however, we don't, you don't really need a screener because the actual diagnostic tool is as fast as quick and easy as a screener, right? So you don't really need two different versions of this.

Like, why would I spend, you know, an hour doing something [00:31:00] that only is gonna take me 10 minutes? And 10 minutes is sufficient enough to do with every kid in that, that school, from kindergarten to eighth grade? 

Kate Grandbois: Well, and from a clinical it does make sense and from a clinical reality standpoint, right?

Because all of us with, even with the best of intentions, are limited to the time and resources that we have, the support we have from our administrators, workload limitations. Um, and I know we've talked on this podcast before about, about burnout and how, you know, a lot of us are working under conditions that are less than ideal.

So. Everything you're saying, everything from the psychometric properties that are incredibly important to, from a bias stand standpoint or an anti-bias standpoint, to the reality of, of application. I mean this, I'm saying if some oversimplifying something, but this, this sounds very doable, is, I guess what I'm trying to say.

It sounds very doable and it sounds very, very good in terms of evidence, [00:32:00] in terms of accuracy and all of those other really important nerdy things. Is that an accurate summary? 

Trina Spencer: I, I think you're right. But here's the reason why we were able to do it is because Doug and I used to be those busy clinicians that were overworked and we had too many kids to test.

I'm like, heck, I tested 15 kids this week. You know, what do you got? You know that we were that and we're like, there's gotta be a better way. There's gotta be a better way. So when we became researchers, we kind of like committed. Like joint commitment to make it easier for clinicians and teachers to do their jobs.

Right. So we're very persistent on solving that problem. That's the, that's another problem that we are solving with our tools, is trying to make it easier for clinicians and teachers to do their jobs. They are underpaid, overworked, and scientists can do a better job. So we did, 

Doug Petersen: you don't have to lose my two favorite terms, validity and reliability.[00:33:00] 

By, by doing that you, you can, and that's a gift to us from dynamic assessment, from this, this concept of measuring someone's learning potential. That the, there is efficiency, it's not built in, but you can make it efficient. And we were able to, after, what, 15 years? 

Trina Spencer: In 15 years we've been developing the diet.

Yeah. I mean, 

Doug Petersen: we're not talking about something that we just invented in a lab in one year or something. Right. This has taken, this has taken us a long time and, uh. I know we haven't talked about this yet, but I just, I, I do want to say that you don't, with dynamic assessment, you don't lose the validity when you administer this test to children have very, very limited English.

Um, or who come from whatever culture you wanna talk about. It doesn't matter. And in fact, we almost felt like we were sort of wasting our time developing a normative sample that represented children from all across the country. Because every time we would administer the dynamic assessment to one [00:34:00] group of children or another group of children, the um, the sensitivity and specificity stayed consistently high.

It didn't matter. And so in, in the manual, you'll see that we administered this tested children who come from lower SES, middle high SES backgrounds, children who are multilingual, children who are monolingual children have who, who only have an MLU of two words in English. Um, children in Japan who aren't even in the United States.

And the sensitivity and specificity stay the same in the nineties. And that's because again, you're not measuring those. Those variables, those variables don't influence external factors, per se, someone's ability to learn, which is really sort of an innate thing that you are born with, right? This, this potential to learn language and to learn to decode.

Amy Wonkka: I wonder if that's like the perfect segue into our clients who are [00:35:00] having challenges with learning those pieces. Right? So, so I guess maybe you can talk to us a little bit about DLD and dyslexia in general. Um, but also why you feel like dynamic assessment is so particularly important to those student populations or client populations.

Trina Spencer: Well, I, I'm gonna say one thing and then I'm gonna pass it off to Doug 'cause he likes this topic and I, I gotta let him talk when he can. So. My one thing is that dynamic assessment is relevant to all learners because it's a learning assessment. It's about learning. So it doesn't matter what the content is, you can create a dynamic assessment for it.

Right? It's just to formalize an instrument, you have to specify your content. Right. And we chose, we chose, um, language learning and decoding learning because they affect a large portion of the US student population. Like these are problems that many, many, [00:36:00] many kids have, and they're constant problems that the school psychologists and SLPs are experiencing.

Okay. Go ahead, Doug. 

Doug Petersen: Yeah. Well, so if we talk about big picture. Right. So the, and that's what you're doing Trina. You're like, there are so many children that need our help, and not all of those children have dyslexia and not all of them have DLD. They still need our help, by the way, which brings us into another topic of like MTSS, but the thing about dynamic assessment is that it helps us identify those children really early on.

So currently, there are even some states that say that you can't administer a screener for dyslexia until the child is like in first grade, and that's because they're thinking from a static perspective that you have to wait for someone to fail before you can see if they're failing. Right. It is totally discrepancy model.

Yeah. Right. A [00:37:00] discrepancy model. Or even literally, even if you didn't, even if you didn't reference the iq, you still have to wait until they're struggling. Sufficiently and disastrously enough to where you can say, oh yeah, this person definitely has a problem. Look at them. They're floundering. It's too late now.

It's too late. You can't do early identification or prevention. You're not preventing anything when you wait for them to flounder and to fail. And so the dynamic assessment is something that allows you to come in and identify the problem before it ever really manifests. And the power of that is huge, right?

You, it's just the same exact idea that you have with, with Universal hearing screening and with, um, with screening for, um, other things early on. It's that you can identify it before it really becomes a serious issue, and then you apply evidence-based practice to it and, and you can actually change the life of these children because they'll potentially [00:38:00] never even know that it was going to be.

A difficulty for them, right? If you apply, um, explicit systematic phonics instruction and explicit systematic language instruction with these kids. So that's a big picture thing, right? Like dynamic assessment. Why do dynamic assessment? Because you can change the lives of millions of children with dynamic assessment.

And that's, I realize that's big picture. And then with the SLP, like, why would the SLP care about it? Because you're not going to have to ask the question anymore. I wonder if this is a disability or a difference, and I don't know how to tell, and I don't know if this child actually should be receiving specialized services from me or not.

And dynamic assessment can really help us differentiate those. 

Trina Spencer: Can I add the school psychologist, the school psychologist perspective is, is, does this child have a reading disorder or do they have dys? Teia, right? Dys. Teia is, you know, [00:39:00] like a. I don't know a comical term to suggest that the teaching that they received hasn't been that explicit, systematic, and effective in the past.

But so often we're waiting for these kids to fail and then going, oh, they have a disability when in fact they might just have had poor instruction. And if anybody's been following the science of reading movement and the dismantling of three queuing, we know that our kids have been getting crappy instruction, especially around, uh, decoding.

And so we've got a lot of kids who look like they have a reading disorder, when in fact they likely just had poor reading instruction. And had we identify them properly upfront, right, we would be able to tell which ones really truly do have, um, have a difficulty learning that or just need better instruction.

Doug Petersen: I really want to bounce off of that if I can just for a second. 'cause I love that Trina, and [00:40:00] it also applies to language. And you said that, you know, these kids have been receiving, you know, quote unquote crappy instruction with, with decoding, with word recognition. And um, and that's very true. And the, the ship is being righted.

Right. The course is changing there, which is so incredibly awesome. And of course we know the science of reading also includes all of the upper strands and scarboroughs rope, right. With, with language and language comprehension. And there's also a serious issue with the language assessment and intervention instruction that children are getting in the schools as well.

If you really apply this idea of structured literacy, systematic, explicit, cumulative diagnostic, and then you ask yourself, how much are we doing that with language? And then also our, our assessments. For identifying children, actually measuring the construct of language. We are falling severely short in those areas as well, and dynamic assessment helps us with that.

[00:41:00] It helps us identify those children who need immediate, immediate, intensive instruction intervention. You don't have to wait. You can give it to them right away. And then other children who do well on the diamond, who do well on the language subtest of the diamond yet who are still not meeting curriculum expectations.

It's true they don't have a disability, but you know that they still need help, but probably won't need to go all the way to tier three or tier four to receive that help. 

Trina Spencer: Yep. There's nothing more frustrating from a clinical perspective than being forced to watch these kids fail. 'cause you know, a, they're not getting a proper dose of instruction or intensity of instruction, and you don't have the right tools.

Right. But they are clearly failing and you, you can't tell why or nor are you in control of the why. Right. But if you have a diagnostic. Dynamic assessment that can really, [00:42:00] truly identify the kids with disabilities who need that. Then the other kids that are still struggling. That's a teaching problem. 

Doug Petersen: it's really hard for Trina and I to talk about just the diamond because it falls into a bigger picture, right?

It's like you identify the children and then you do systematic, explicit language instruction with those children. And we've obviously developed something that helps you do that right with story champs. And then you also monitor their progress frequently to make sure that they're making adequate progress and to adjust the intervention as necessary.

And that's why we designed the cube also to to monitor. Their progress over time. So it is hard for us to keep the conversation very constrained, just dynamic assessment. 

Trina Spencer: But that's because the problems are huge. They're systemic and you need multiple like integrated solutions to make sure that you're hitting all aspects of the problem.

I mean, you said they're writing the ship around decoding and I'm like, well the only way they're gonna write the ship around language [00:43:00] comprehension is by using the tools that we've developed. 'cause there really is very little out there that do that. Do these things. 

Doug Petersen: There's a 

Trina Spencer: lot of things around decoding, hardly anything around language comprehension.

Doug Petersen: Yeah. That's a very self-serving statement. But it's so true though. I mean, people will look at the NAP assessment, right? And, and they're so frustrated right now because they've been working on explicit systematic phonics instruction now for a while. And these districts are expecting that to have a huge impact on student reading comprehension performance.

And unfortunately, the majority of children are actually having difficulty with reading comprehension because they have difficulty with the understanding the complex language that they're decoding. And so if until we address the language issue, we're gonna continue to be frustrated because obviously you have to address all of the strands of the reading rope to actually improve reading comprehension.

You can't just pick one little strand that you love so much and, uh, think you've solved the problem. [00:44:00] 

Trina Spencer: And the, the numbers of kids who are actually struggling with reading comprehension related to language is about 85% of them. 

Kate Grandbois: That's a high 

Trina Spencer: number. It is a very high number. Mm-hmm. This one particular study, I'll just give you more specifics.

In this one particular study that kids were, you know, um, economically disadvantaged, right? Of, of the a hundred percent of the kids who, of this group who had reading comprehension difficulties, 85% of them had only language comprehension difficulties. And another 10% had, um, decoding and language comprehension problems.

So that's like, there's very few kids who just have decoding difficulties who are having reading comprehension problems. 

Doug Petersen: And I'm, I'm sure that Amy and Kate are like, well, how, how do we get this back to the diamond? [00:45:00] 

Trina Spencer: No, 

Amy Wonkka: no, 

Trina Spencer: no. I'm 

Kate Grandbois: actually, I'm just shaking your notes. No, no, I, I've jumped in a few times because I am, I am kind of, you know, we're talking about the diamond, which is a very, you know, boots hit the ground, rubber meets the road solution, but you've kind of helped us zoom out to this 30,000 foot view of this problem.

And I, I think it honestly helps put the diamond in context, right, of why tools like this are needed, who they serve, how they help us from a practical level. But I am, I'm kind of sitting there reflect, I'm sitting here reflecting on Trina, something that you said related to the context and the systemic piece of this.

What made me think about it was something you said about using. A tool like this to identify the kids who have decoding issues, but then also looking at the kids that frustrated feeling of these kids are not making progress, but they, they don't have an issue with decoding. Mm-hmm. And then what do you do with that?

You go to, you [00:46:00] go to who, right? Like you go rat on your fellow teacher down the hall. I mean, what do you do with that information? I mean, it's just so complicated. 

Trina Spencer: Well, I'm gonna tell you what the ideal would be, right? Here's what we want the world to do, right? And, and so, so often, Doug and I think about these things in revolutionary terms, right?

We're not trying to create tools that fit into old systems. We're trying to create tools that move new systems, force the systems to evolve. Okay? So here's the ideal. Let's say you do the diamond with all the kids in the grade. Right? And you can identify, these are the kids who are struggling, who, who, who actually have a disorder, right?

There's a difference, right? They're struggling and disorder, right? Because you could be struggling for lots of reasons that are not related to a disorder. So here are the kids with a disorder. We're gonna give them that immediate help right now, right? All those other kids who came up as low or struggling, [00:47:00] right?

Those data then get used to aim professional development and support coaching and additional resources into the right places, right? So let's say you've got first grade and this one particular class has a ton of language difficulty that is not disorder, right? And you look and you're like, oh my gosh, like, you know, 20 of these kids come from a non-English speaking household.

So are you gonna do the same thing for that class as you would with the, the class down the street that's, you know, one or two kiddos that speak a different language? No, you're gonna do something different for them because their data look differently. So if you're screening and you're, and you're using good tools that differentiate that, you can then be, use a scalpel to, uh, identify what you do about it, right?

Then you can actually tailor [00:48:00] your interventions, your professional development, your resources to the right places, which ultimately saves schools resources, reduces the amount of expenditures on waste. And, well, this kid doesn't really have a disability, but he is not gonna get this help any other way. So let's just make him in special education, which of course is an equity issue of over identification of culturally and linguistically diverse kids in special education.

Sorry, I. Do you see how excited I get? 

Doug Petersen: What's the ous allocation of resources? And that applies also to the SLP, right? If you're actually truly identifying children with developmental language disorder or language disorder concomitant, if you're actually truly validly identifying those children, then the SLPs caseload will be constrained to around 7% of the population.

It's not going to be a huge, overwhelming caseload that has a lot of children on it that may or may not have a disorder. We don't know. And that will also free up the [00:49:00] SLP to radiate their influence across the entire system because yes, they're working with children with DLD and they care very deeply about those children.

But then they also can care about all of the children by talking to teachers and other educators about how to facilitate language across the whole system. I. 

Trina Spencer: And I think for school psychologists and other educators, they become more aware that language comprehension, that they're not doing as much for language comprehension.

Right. And, but if we don't assess it at all, nobody's paying attention to it. They don't know. The SLPs know, and the teachers are like, no, no, it's language. Give it to them. Like there's, there's a lot more we can do around language. 

Kate Grandbois: I, uh, making a parallel kind of comment here. So there's, there's a lot of literature out there about, out there about change management and impacting change within organizations.

And a lot of that literature points to effective change coming about from a champion, a champion of change. One [00:50:00] person in the system who, who. Makes the assessment of what's wrong in the system. And now when I say system, we're talking about school systems, but this literature is about like saw mills. Okay?

So this is like across organizations and business at large, all right? Factories, you know, any, any business. And what's really interesting or what I'm hearing is that, you know, what that literature would say is that effective assessment of what changes need to happen will lead to effective change if there is a champion to implement that change.

And what I'm hearing is that speech language pathologists, IE you who are listening, could use a tool like this to create that effective measurement, find the holes, find the classrooms, take it to the administrators, you know, push that information out and be those champions of change within your district.

And I mean, guys, this is like the Spider-Man of tests. I feel like this is a Marvel level impact. This is, I mean, it is cool. It's so [00:51:00] awesome. 

Trina Spencer: That's great. I am. You explained that very well, and you're exactly right. You're, you're identifying the SLPs as that potential champion. Um, but I wouldn't discard your school psychologist either.

Go find a friend. Yes, all of 

Kate Grandbois: them. And your teachers too. I, I'm just 

Trina Spencer: speaking to this audience. I know, I know. But I would encourage SLPs to not give up on their school psychs. Right. They're in the same situation. They're, they got too many kids to test, you know, whatever. And they need, they need to understand what's really happening here.

And the science of reading is an opportunity. And more people knowing about the upper strands of Scarborough's Rope is an opportunity to really keep them engaged around these issues. And that together you might be able to bring that data to those upper level decision makers. You know, maybe more impactfully, you know, find a friend to champion with you.

Doug Petersen: And it can be data driven too, right? That's one of the things that, something like the diamond does is it provides you, [00:52:00] with my two favorite words, the validity and reliability of a particular test. So it's not, you don't have to take anyone's word for it, just go to the technical manual and read what we have in there.

This is not our opinion, this is the data that we've collected. And I do, I do wanna say also, I, I love what you said, Kate, about SLPs in particular. With, uh, with our speech language pathologist, being a change maker doesn't mean you take all of it on yourself and that you're the one who's doing all of the assessments and all of the intervention.

It just means you're the one who's drawing attention to the issue and the solution to the problem. And speech language pathologists have a huge, huge piece of the puzzle when it comes to helping children, and that is our knowledge of language assessment and intervention, which almost no other educators have.

They don't have the formal instruction, education on assessment of language or the treatment of language like we have. And it is the key to [00:53:00] so many of the issues that our children are having in school. So, yeah, 

Trina Spencer: if, if I would give one piece of advice, I want every SLP it, every conversation with their school psychologists or their administrators, or the teachers to, to insert this phrase, language is literacy, language is literacy, language is literacy.

And if they keep poking that in every conversation, somebody's eventually gonna go, Hey, what do you mean by that? Right. There's your opportunity. Just say it. Language is literacy. 

Amy Wonkka: I'm gonna zig zag us a little bit because both of you have mentioned Scarborough's reading rope. Can you give us like asking, I don't know what that is.

Okay, so can you give us a quick, like five minute rundown, because reading is important and it incorporates both oral language skills and written language skills, which we've talked about a lot. So maybe just give us a little like rundown on that. 

Doug Petersen: Trina disappears. Like, I don't know if that's my cue that she wants me to, 

Amy Wonkka: she literally just 

Trina Spencer: ran away.

I literally [00:54:00] have three drinks on my desk and they're all empty, so, oh, you were gonna, I'm go get, I'm gonna go get a refill. 

Doug Petersen: I thought you were maybe gonna go grab some pipe cleaners and be like, hall Scarborough back in the day and like twine the pipe cleaners together to, I thought she was gonna grab a visual.

I'm not. So, uh, well, I mean, so. You can sort of start really zoomed out with a simple view of reading, which is this idea that word recognition times language comprehension equals reading comprehension. And it is a product, right? Start there, but that doesn't tell you enough. So, um, there's more to it, right?

What do you need to do to be a, um, a strong reader, decoder, and what do you need to have? Strong language comprehension and language comprehension. Comprehension is not a single unitary construct, nor is word recognition. They are built up with multiple aspects and features. And so, um, the upper strands of Scarborough's rope.[00:55:00] 

Are all about those things that you need to be, um, to have strong oral language to successfully understand what you decode. So background knowledge and syntax and, um, vocabulary and discourse structures and so forth. And then verbal reasoning. 

Trina Spencer: You just missed that one. Verbal reasoning. Oh 

Doug Petersen: wait. Oh, verbal reasoning.

Yeah. Inferencing. Like 

Trina Spencer: inferencing. 

Doug Petersen: Mm-hmm. Yeah. Thank you. And then the lower strands are about word recognition, phonemic awareness and phonics and things like that. So that's, that's Scarborough's Rope. And then I just want to do a little plug for Duke and Cartright 2021 and Duke and Cartright also acknowledge that there are other factors also that we should consider that lead to successful reading comprehension, like whether you're even paying attention to what you're.

Reading because you could have amazingly fluent decoding and you could have absolutely adequate language comprehension to understand what you're reading. But if you are not paying attention to it, if, if your executive function is not there, then [00:56:00] that's also a problem. So I. Oh, and here's a, oh yeah. Are, are we like on video?

Here's a picture of Scarborough's robe right here. 

Trina Spencer: I think it was for, it was for Kate and Amy. I don't think the listeners are gonna see this, but we might be able to add it to the resources, right, Kate? Yep. We can 

Kate Grandbois: add, we could add this image to the show notes and to the landing page. So for anybody who is listening, it's essentially, uh, imagine an image of a rope that is frayed at the end.

And as where it's fraying, you can see the different strands of the rope kind of coming apart. But each strand of the rope is labeled with background knowledge, vocabulary, language structure, verbal reasoning, literacy knowledge. Those parts of the rope are under a category called language comprehension, and then another strand of the rope.

Labeled word recognition. Uh, other smaller strands, I'm really botching this, but our, uh, phonological awareness decoding and site recognition. So it's probably much easier to look at [00:57:00] it than listen to my garbled description, but we will put it on our website and put a link in the show notes. 

Amy Wonkka: And just from a clarification standpoint, I believe the cube looks at all of these components, right?

Does the diamond also, or is the diamond just looking at some, some strands of the rope as it were? 

Trina Spencer: Yeah, that's a really great question. The cube obviously mean you can hear from this conversation. We base all of our tools in the most updated science about it, about all of these things. So yes, our assessments, the cubed strands very, very easily link to those things.

Um, but the diamond does as well. Right. I mean, you, for predictive validity, you don't need to measure everything, right? You have to identify the pieces that have the greatest prediction, um, which, so otherwise it would, it could be long, you know, if [00:58:00] you're assessing everything. Um, but Doug, why don't you go into more details about the, the identifying the strengths and weaknesses in the targets.

Doug Petersen: Yeah, so, um, so with the diamond, you are essentially collecting a language sample. And so if you want to measure language, then, um, obtaining an analyzing a language sample would be a really great place to start because you're actually measuring someone's language in a meaningful context, hopefully. And this particular case with the diamond, you're having children retell a story, which is something that children do all the time.

So, um, so what, what strands of the rope do you. Measure when you are measuring a or or collecting a language sample? Well, just about every one of those strands because you're measuring the construct of language. Um, but like Trina said, with the Diamond, the focus is still on learning and it's not what is, what is your current vocabulary?

What do [00:59:00] you currently use with syntax? What's your syntax like? Right? It's instead, let me teach you something new that you don't know yet with language, and I'm gonna see how easy or how difficult it is for you to learn that. And so it's just a different purpose. And so that's why it's not so hyperfocused on, on isolating each of the strands, but, but with identifying strengths and weaknesses and intervention targets, you of course can do that through the language sample analysis that we have in the diamond.

Trina Spencer: Yeah. And that was great. Thank you. Sorry. We, we, we also have many studies that tell us which of all of those language comprehension and which of all of those, uh, word recognition strands are the greatest predictors. And those are our subtest. So we have a dynamic assessment of decoding, which is that pH blending of written words.

Okay. So that's the lower strand. That's the dynamic as assessment of decoding. So that particular, [01:00:00] not phonemic awareness, not an oral, uh, auditory skill or, uh, like recognizing. Words by sight. That wasn't, that's not what's in it. It's the dynamic assessment of decoding. And then the other major subtest is called the dynamic assessment of narrative discourse.

So in the upper strands that falls under literacy knowledge, which is really about, uh, text structures and understanding the patterns of a large unit or language sample, like a story or uh, uh, some sort of expository content. And so that narrative discourse, we've tons of research. We have several, um, publications, even not even with our tools that tell us that narrative discourse is the best predictor of language learning disabilities.

Um, so those are the two major sub tests that you would do if you were doing it for screening or for eligibility determination. And then you can take it further and analyze the language sample for vocabulary [01:01:00] for, uh, syntax or morph syntax issues. And, um. 

Doug Petersen: Ential word learning. 

Trina Spencer: Inferential. Word learning.

There we go. 

Kate Grandbois: Yeah. This has all been incredibly helpful. Um, I wonder if you know, we're gonna link all of these references and resources in the show notes. You've taken us on a journey of like 30,000 feet down to how, you know, some, how to use 10 minutes to, to deliver to, to give this assessment. And it's all been really eye-opening.

What would you say to anyone listening? What would you say would be their best next step? I have to assume that at least one person is listening to this and is feeling inspired. I am especially about this idea of collecting assessment data about your program and using that information to push the envelope.

What would you recommend an SLP who's listening, who is feeling inspired do on Monday when they go into work?

Doug Petersen: Well, that's a great, that's a [01:02:00] great question. 

Trina Spencer: Slip in the phrase languages literacy conversation. That's all I got. 

Doug Petersen: Be a change. Be a change agent. I, I think, I think that, uh, I don't want someone to go out and just buy the diamond because. Whatever, because of somebody told them to. I, I would personally love for people to look under the hood that would excite me more than anything for them to look and see what the sensitivity and specificity is, to look at the reliability, to look at the way that we did standard error of measurement and maybe they don't even care.

And how the confidence intervals are calculated. Those things mean so much though about whether the test is actually something that's gonna be useful to you and I, and I suppose that not everybody is going to do that or wants to do that. Um, and so then I guess my next, my next challenge really would be to, um.

Take [01:03:00] dynamic assessment and embrace it because the research evidence is there to support its use across multiple purposes. Right. And so I, I, and, and I don't want to push someone to just go buy the diamond. Um, that's, that does not feel like the sort of thing that I would do as a researcher, but I would advocate strongly for the use of dynamic assessment.

It is clearly something that is evidence-based. 

Trina Spencer: I also think that like a single SLP can. Begin, or probably is already in a community of practice with other SLPs or, you know, an opportunity to speak with their colleagues about some of these things and maybe identify some of the, the problems that they're already having.

Maybe over identifying or waiting too long for certain, you know, subgroups of kiddos before we diagnose. Or maybe the group of SLPs [01:04:00] say, Hey, we're kind of shut out out of the literacy initiatives in our schools and we don't wanna be so there, whatever that problem is, that that might be the way that group comes together and, and identifies a path forward where they can, a, contribute greater to the literacy initiatives, which you know, are happening in your elementary schools.

And there's a ton of attention being paid there. So. SLPs have a place at that table and they can, they can certainly inform the directions of those, um, literacy initiatives if they are aware of the tools that is gonna help their, not just them, but their entire team to move forward and do that better. So 

Kate Grandbois: thank you both so much for being here.

We always learn so much from you to interview. To anyone listening, all of the resources are listed in the show notes. Trina and Doug, thank you again so much. 

Doug Petersen: Thank you for having us. It was so much fun. Yeah. Thanks for 

Trina Spencer: letting us. [01:05:00] That was fun. We didn't even argue that much. You're like so passive 

Doug Petersen: these days.

Yeah, we used to really get into it. I'm sorry. We Next time? 

Kate Grandbois: Next time. Next time. Okay. 

thank you so much for joining us in today's episode, as always, you can use this episode for ASHA CEUs. You can also potentially use this episode for other credits, depending on the regulations of your governing body. To determine if this episode will count towards professional development in your area of study.

Please check in with your governing bodies or you can go to our website, www.slpnerdcast.com all of the references and information listed throughout the course of the episode will be listed in the show notes. And as always, if you have any questions, please email us at info@slpnerdcast.com

thank you so much for joining us and we hope to welcome you back here again soon.

.