Unpacking the Role of Generative AI for Knowledge Management with Keith Berg

Transcript

Pete Wright:

Hello, everybody, and welcome to Connected Knowledge from Upland Software on TruStory FM. I’m Pete Wright. Creating knowledge is hard. Creating that knowledge and then putting it in the hands of your frontline agents when they need it is an order of magnitude harder. It is not a surprise, then, that evaluating generative AI tools has become a part of the job for technical leaders in the call center and support space. This week on the show, we have one of our own leaders to help us out. Keith Berg is the senior vice president and general manager of Contact Center Productivity Solutions, and he’s going to lead us through the stack. What can we expect from these new tools? Can it really make our agents work better? And most importantly, can it be trusted? Keith Berg, welcome to the show.

Keith Berg:

Hey, Pete. How are you?

Pete Wright:

I’m very good because, as I told you before we started recording, I am eager for this conversation.

Keith Berg:

Yeah, me too.

Pete Wright:

Edge of my seat kind of stuff. Oh, I’m really excited about this space and what these AI tools can do for us in this space. But let’s start backing it up a little bit. First, tell us a little bit about what some of the challenges are that you’re thinking some of these tools can to help us around in call centers right now, especially in the knowledge management space.

Keith Berg:

Sure. Pete, I’ve been in the knowledge management area for over 20 years. I could write a book about it. And there’s lots of reasons I’ve seen knowledge management work and not work for companies. We have a diagram we usually like to show customers that outlines the key components that make up a knowledge program, and one of those components is the content itself. Having good knowledge and the right volume of knowledge is one of the key contributors to knowledge management done right. Even in that area, there’s lots of examples of it working well and not working well. Not enough personnel. You’ve got bad governance around what it is that you’re writing. There’s resistance to change. Poor classification systems when you set your knowledge up in the first place. So, I really do think that one of the biggest impediments to knowledge is just how do you get that content created? How do you get that out to people, and to maintain that?

What we’ve heard from a number of customers is, we know we need to invest in knowledge, but we don’t have the people or the budget to create the content that’s required, or our agents don’t have time to be creating and updating knowledge. They’re taking phone calls. Or we’ve got lots of subject matter expertise, and they have deep domain expertise, but we can’t commit to giving them all this training and such. Frankly, they don’t know how to write in the right tone for the audience. That’s not what they were hired for. It’s just not their skill set. So, I really do think that there’s lots of reasons knowledge management can work, be successful, and not successful. The content, though, I think, is really the foundation of the house.

Pete Wright:

That seems to be, to me, what is so intriguing about some of these AI tools. It feels like a leveler because, as you say, it’s very expensive to do knowledge management at scale. Am I right in thinking that some of these tools could unlock some of the benefits that large organizations have had all along for smaller companies?

Keith Berg:

Yeah, I definitely think so. You and I were joking earlier. You’ve got AI and how you think about that in your personal life, and then you’ve got it in the business world. In the business world, I think that this is the single most impactful thing to happen to knowledge management since Microsoft created this visual folder structure in Windows. We were like, “Oh, now I can see how you could classify things.”

Pete Wright:

Yeah.

Keith Berg:

I think that this is literally the next big advancement for knowledge management.

Pete Wright:

So, let’s transition to some of the specific tools. Now, we’ve started talking about ChatGPT. It seems to be the one from OpenAI that has a lot of media attention right now. Is it ready for business?

Keith Berg:

I think it is. I do feel like we’ve leapfrogged the AI that existed a little bit when we’ve gotten generative. When we think about predictive AI, which has been around for years, that’s been great at helping users find things more easily by using machine learning to help predict future user search activity, things of that nature.

Pete Wright:

That’s the transparency of it. Apple’s saying machine learning all along to help you figure out who’s in your photos.

Keith Berg:

Sure.

Pete Wright:

That’s AI. Let’s understand that we have different terms for it. It’s been around for a long, long, long time.

Keith Berg:

Sure. We’ve had it in our products even for years. There’s easy ways to drive the efficiencies of author behavior with knowledge suggesting keywords, tagging, or metadata based on what’s happened before and other types of input. So, that is what it is, and I still think that there’s loads of expansion that can happen in that predictive AI world still. But yes, we’ve leapfrogged now, we’re on a generative AI, and its ability to create content opens a whole new world of opportunity for knowledge management. If we fast-forward even a few years, I believe it’s going to be as expected a feature as spell check is today.

Pete Wright:

Right.

Keith Berg:

Remember when spell check was you reread what you typed and then you had to use white auto-correction tape if you needed to fix your misspellings?

Pete Wright:

Oh, man.

Keith Berg:

Right?

Pete Wright:

I remember everything about it. I remember what it smelled like. I remember it on my fingers, under my fingernails, please.

Keith Berg:

There you go. So I think generative AI is going to be that assistant to knowledge authors down the road that helps them get started with writing a new article, approving articles, creating articles that are specific to the channel that they need to be delivered on because not all things are created equal. Ultimately, I could see it also picking up on new call trends and proactively suggesting knowledge to authors that they should create based on some spike in volume that’s been coming in. Maybe we haven’t gotten calls about in a specific call center to look forward.

Pete Wright:

Talk to me a little bit about, again, if we’re going to project a little bit a couple of years out, or maybe I’m speaking in the present and just don’t know it, the idea that a call center operator can receive an inbound call and not know the answer to a specific question that ends up being potentially more complicated. The act of sitting down and typing into a call center bot based on this technology that has already ingested, synthesized, and interpreted, for lack of a better word, the content of our knowledge base, can give that call center manager a real-time response to read to the customer. Is that happening? Is that a thing that is going to happen? What’s next down that road to make operators jobs easier?

Keith Berg:

Yeah, I think the technology is already there. Many of the call center systems, the core systems themselves, already have the ability to listen in and proactively pop information up to that agent who may have taken an inbound call or even people on the sales side making outbound calls who can actually read the sentiment of what somebody’s saying and automatically guide that agent through, “Ask this, do that. Oh, they’re interested in this.” So, I think that stuff is already there.

Now I will say where the real timing nature of it is concerned. In the example I gave, we have a net new problem. Maybe there’s an outage in one of our systems that’s causing people to call in at this moment. I still think we need to have that human in the loop. I don’t think this AI, and I don’t think anybody should be thinking that this generative AI should be left unmonitored, that it should be getting out to the consumer or even the agent without some vetting. There still needs to be this human in the loop in this process. There’s just too much at risk for customers brands, I think, to just let AI take it all over.

Pete Wright:

There was a recent three-part series on the podcast, radio show Planet Money on AI. By chance, have you heard it?

Keith Berg:

I have not, no.

Pete Wright:

Forgive my digression here as I recount another podcast, but interesting enough, the premise of the thing is, the reporters wanted to see if AI could do their whole job. So they generated a voice based on one of their other historical reporters. They had the AI essentially ingest a number of research papers on the subject that they were looking at. They then went to the authors of that research paper, had the AI generate a number of questions, and then read the questions without telling the authors that they were AI. At the end, the author said this, and this blew the whole gambit. The author said, “You see, this is why AI won’t replace humans because humans are the only ones capable of thinking of as thoughtful and provocative questions as you just read to us, the questions that had been generated by AI.”

So the reporters started laughing, and they said, “We have to tell you, we have to come straight. We did this.” And the authors of this paper were suitably dismayed and shocked. But good sports, obviously. I’ll put the link in the show notes. It is really worth listening to this if you’re curious about what these technologies can do. But it goes back to this area of, I think, trust that you said we need humans in the loop. We’re seeing more and more examples of humans not necessarily being needed in the loop. At what point do we trust these systems to be giving answers to be accurate, to be authentic, to be truly represented of the brand? What’s it going to take culturally to get us over the hump?

Keith Berg:

Yeah. I think it does require a massive amount of data.

Pete Wright:

Yeah.

Keith Berg:

The benefit of that example you gave was that the corpus was very discreet.

Pete Wright:

Very discreet.

Keith Berg:

Right?

Pete Wright:

Yeah.

Keith Berg:

In the world of call centers, it could be very, very broad. We’re dealing with a very broad corpus, but I think it will get there. I think what we think about and suggest is that your agents are the Guinea pigs in this process. You can provide information to them and you can rely on them, and as they do in just traditional knowledge management today, when I’m looking at something, “All right, does that actually answer what this person asked me?”

Pete Wright:

Yeah.

Keith Berg:

So you’ve still again got a human in the loop in the agent looking at information and saying, “Oh yeah, that looks right. That looks appropriate to me.” That in and of itself to me is going to help train our understanding about whether the generative AI for each customer, and I do think each customer’s corpus is going to drive how quickly you can get to this kind of self-sufficiency, but it’s going to help drive our understanding of that and help us see how quickly we can get toward more full hands off automation.

Pete Wright:

Another digression, what are you using this stuff for, just personally? I know you’re so invested in it day to day. This is all you do and think about at work. I’m sure when you go home, you just put your head under a pillow and listen to some yawny music.

Keith Berg:

Nah.

Pete Wright:

That’s right.

Keith Berg:

My wife and I are both in technology, and I feel like two or three times a day we’re constantly just saying to each other, “I don’t know, ask ChatGPT.” It feels like you could almost apply it to anything. I think it could be used for just about anything, even if it’s not just to get something to then copy and paste it somewhere but to generate some thought-provoking momentum for you on a topic where you’re starting at a blank screen.

Pete Wright:

Also, I write in part of my other life, and the recent release, as we write this, of ChatGPT-4 has updated their… Or it’s Claude Plus from Anthropic, has updated their number of texts that you can have it analyzed. I uploaded just about 60,000 words of a work in progress and had it build me essentially a narrative character map, a summary of where every character is at any given time based on what it has interpreted, and it was spot on. I’ve never seen anything like it. It felt like I had taken hours and hours and hours to do, to read, and write a timeline assessment of where these characters are, and it happened in about 15 minutes. It was extraordinary.

Keith Berg:

That’s incredible. That’s incredible. It’s incredible. I think one of the fun things about dealing with GPT clients themselves is, you ask the question and then you see it start to come back and you’re like, “What’s it going to say? Where’s it going?” So I think that real-timey nature of it is really intriguing on a personal use front for sure.

Pete Wright:

On a Turing test front, the more conversational I am with the client, the better the results are. I find that it’s still a bit of a mystery to me. Even though I understand in broad strokes how the technology works, the fact that it’s talking to me in a way that could legitimately fool me is directly related to my experience and my fascination with how this can be used in the frontline for knowledge management workers, right?

Keith Berg:

Yeah.

Pete Wright:

Yeah.

Keith Berg:

That iterative, “Oh, you gave me a response. Now let me drill in here. Let me re-clarify here.” Where you can get to a truly curated output is really just amazing, and the speed at which it happens is just a little unbelievable.

Pete Wright:

Let’s talk specifically. We’ve got, Upland just introduced the AI knowledge assistant. Let’s hear how great it is.

Keith Berg:

Yeah, sure. For years, our strength has been in the tool sets that are used by the authors to create and manage that knowledge. Over the years, the end-user side of things and search has become a little bit more commodity. I think everyone’s gotten good at trying to provide a Google-like experience to someone who’s trying to consume knowledge, but there are still those challenges about empowering the authors to create and maintain that knowledge. So we’ve really focused our predictive AI activities in the past, as well as our generative ones now, on authors. How do we help them create knowledge? I mentioned that concept before, of looking at a blank page, that’s really hard to write something that’s new. For many of our customers, where they might launch a new product and have to start with something new or they’re a brand new customer who’s literally starting from zero, that blank page is an issue.

So being able to create knowledge initially from there offers huge savings and speed to get productive with knowledge, improving the quality we’re investing in. How do you take something that you have today and ask generative AI to make it better? Part of that as well has to do with writing in different voices. Having experts write knowledge for experts is one thing. How do you then transform that same thing into something that you want your customer to try and do in a self-serve manner? You don’t just publish the same article to them. You need to put a voice to that. I had my daughter use GPT for something recently that might have been getting out of a speeding ticket, and I told her-

Pete Wright:

That is a laugh line. Please tell me.

Keith Berg:

I said, “In the prompt, make sure you say you want it written in the voice of an 18-year-old.”

Pete Wright:

Yeah.

Keith Berg:

Very important, right?

Pete Wright:

Yeah.

Keith Berg:

Again, on the consumer side, you want to make sure that you’re being spoken to or being presented with something in the right voice for you. So those are great examples of it. Structure and flow, a lot of knowledge bases, you might have 20 people contributing to it. Some people like to use bullets, some people like to number things. Some people like lots of pictures. That inconsistency in the way that knowledge is created can come off very disjointed within an organization and to your customers as well. So you can also use it to improve and standardize that structure and flow in order to make your whole knowledge base a lot more cohesive. So we’re really focusing with our knowledge assistant on how do you make those knowledge authors more effective on day one and how do you help them with the maintenance activities around the knowledge base.

Pete Wright:

So you have already opened the door to this, so I’m going to go ahead and ask you to read your crystal ball again. We’ve talked about predictive AI leading to generative AI. What’s next on the AI horizon for Upland?

Keith Berg:

I think we’re going to get to the point where customers are going to want their applications… Where they’re going to want to bring their own AI to their applications. Customers have a lot of stuff behind the firewall or in specific repositories. It’s their CRM system, it’s their Office 365, and that’s going to be their treasure trove of information that says how the organization works.

Pete Wright:

That seems to be where we are now. Everything we’re hearing now is companies saying, “Don’t use ChatGPT right now.” And that, I think, is the reason because it’s open.

Keith Berg:

Exactly. But they’re all thinking that they want to put generative AI on top of their own stuff in order to expose more insight into it. So I think those customers who then want SaaS solutions to be able to benefit from that without having to push it outside of that level of security or replicate it within all of these other SaaS-based tools are going to need to open the door and allow customers to actually dip into their AI API to get to their information. So I think that’s where we’re going to eventually get.

Pete Wright:

You started talking about that AI years are equivalent to dog years. Everything’s just happening much faster in the AI space. It seems like we just figured out how AI is impacting us in the last eight months. It feels like it’s going to be another eight months, like another breath, and we’re going to be able to be training our own internal corpus under AI technologies.

Keith Berg:

Yeah.

Pete Wright:

Is that what we’re thinking, that timescale?

Keith Berg:

I think so. I think people like Microsoft are going to drive that stuff with Copilot. They’re going to put that AI on top of the stuff that we already know and hold near and dear to us and make that available everywhere. I think one of the things that we’re cautious about in working through all these generative AI topics and plans are to not get too comfortable. Because, like you said, with the speed at which this can happen, I may know what I know today. It may be different three or four months from now. We talk about the growth of it. I can’t remember the source of it, but I had seen a research paper done by someone that showed the revenue in generative AI in ’21 and ’22 and then in ’23.

Pete Wright:

For the listeners, the hand gestures are significant that Keith is just giving me here.

Keith Berg:

So the spending on that topic just went from… Or not the spending on the topic, the revenue associated with generative AI literally went from zero to billions in technology, and the technology life cycle is overnight, right?

Pete Wright:

Yeah.

Keith Berg:

So it’s not going to stop. It’s not going to stop. And I had seen some recent press that there’s been a drop in subscription levels on ChatGPT service, as an example. It’s the first time that’s happened since the boom. But it was all associated with those people going to other GPT services. It wasn’t like people say, “Ah, this thing’s not working.”

Pete Wright:

Boom was six months ago, you all. You’re partaking in the miracle of flight here. Let’s be patient.

Keith Berg:

Exactly.

Pete Wright:

I think I like the nod to Copilot Clippy. We hardly knew ye. Let’s think of where we could be with Clippy. This has been great, Keith. Where do you want to point people if they want to go learn more about what Upland is doing with AI? Do you have a website you want to point them to specifically, or shall I just put the main site in the notes for you?

Keith Berg:

I think our corporate site is uplandsoftware.com. You can find all about our knowledge products that are there and we’ve got lots of information there as well around our generative AI feature set and some of the things that we’re working on. So it’s probably just the corporate site and whatever we can attach to the recording here.

Pete Wright:

If anybody is looking for a suitable prompt whisperer, is your daughter available for hire, consulting, or hanging out a shingle yet?

Keith Berg:

Funny enough, my kids are both in college and are both doing part-time AI new jobs this summer.

Pete Wright:

Surprising.

Keith Berg:

No one.

Pete Wright:

Keith,-

Keith Berg:

Nope.

Pete Wright:

… thank you so much for hanging out with me today. This has been a fantastic conversation. I hope it’s not the last one. I have a feeling you’re going to have more to talk about.

Keith Berg:

Yeah.

Pete Wright:

Let’s say in about eight months.

Keith Berg:

Yeah, or next week.

Pete Wright:

Or next week.

Keith Berg:

It could be that quick. So it’s been a pleasure, Pete. Thank you.

Pete Wright:

Oh, thank you. Everybody else, make sure you head over to uplandsoftware.com. Specific links will be in the show notes. You can get to know us a little bit better. On behalf of Keith Berg, I’m Pete Wright. We’ll see you next time right here on Connected Knowledge.