Exploring AI Audio – Part 1: Fundamentals & Essential AI Audio Tools | 54

WEBVTT
00:00:00.280 --> 00:00:04.432
Associations tend to have infrequent but deep engagement, which is not bad.
00:00:04.432 --> 00:00:12.268
They just haven't created a value proposition that justifies their existence in the lives of their members on a daily or weekly basis.
00:00:12.268 --> 00:00:18.250
Welcome to Sidecar Sync, your weekly dose of innovation.
00:00:18.250 --> 00:00:27.213
If you're looking for the latest news, insights and developments in the association world, especially those driven by artificial intelligence, you developments in the association world, especially those driven by artificial intelligence, you're in the right place.
00:00:27.213 --> 00:00:35.273
We cut through the noise to bring you the most relevant updates, with a keen focus on how AI and other emerging technologies are shaping the future.
00:00:35.273 --> 00:00:38.665
No fluff, just facts and informed discussions.
00:00:38.665 --> 00:00:42.822
I'm Amit Nagarajan, chairman of Blue Cypress, and I'm your host.
00:00:42.822 --> 00:00:51.087
Welcome to the Sidecar Sync, your source for all things artificial intelligence intersected with the world of associations.
00:00:51.087 --> 00:00:53.512
My name is Amit Nagarajan.
00:00:54.402 --> 00:00:55.826
And my name is Mallory Mejiaz.
00:00:56.640 --> 00:00:58.045
And we are your hosts.
00:00:58.045 --> 00:01:00.552
Now we have a real treat for you.
00:01:00.552 --> 00:01:18.694
In this episode and the one that will follow, we're going to be talking all about audio, a modality that has enormous potential with artificial intelligence, ways to communicate with people in just truly incredible new ways, and we've broken it up into two episodes.
00:01:18.694 --> 00:01:23.036
The first one, which you're listening to right now, is all about the foundation.
00:01:23.036 --> 00:01:32.283
It's about audio and the current tools as of late 2024 that we're really watching, that are foundational to thinking about your audio strategy with AI.
00:01:32.283 --> 00:01:40.468
And then the second half of this episode, part two, will air a week later, and that episode will go into specific use cases and specific tools.
00:01:40.468 --> 00:01:48.769
So that's what we have in store for you today, and before we get into the meat of the episode, let's take a moment to hear a word from our sponsor.
00:01:49.632 --> 00:01:58.784
Introducing the newly revamped AI Learning Hub, your comprehensive library of self-paced courses designed specifically for association professionals.
00:01:58.784 --> 00:02:10.413
We've just updated all our content with fresh material covering everything from AI prompting and marketing to events, education, data strategy, ai agents and more.
00:02:10.413 --> 00:02:22.052
Through the Learning Hub, you can earn your Association AI Professional Certification, recognizing your expertise in applying AI specifically to association challenges and operations.
00:02:22.052 --> 00:02:31.893
Connect with AI experts during weekly office hours and join a growing community of association professionals who are transforming their organizations through AI.
00:02:31.893 --> 00:02:37.973
Sign up as an individual or get unlimited access for your entire team at one flat rate.
00:02:37.973 --> 00:02:45.846
Start your AI journey today at learnsidecarglobalcom at learnsidecarglobalcom.
00:02:45.866 --> 00:02:47.288
Amit, how's it going today?
00:02:48.312 --> 00:02:50.329
It's going great, you know.
00:02:50.329 --> 00:02:57.610
I know we're recording this a little bit earlier than usual, since we will both be at Digital Now next week and we will not have time to record the usual pods.
00:02:57.610 --> 00:03:02.912
So we're recording this two-part series today, just ahead of Digital Now, and I'm looking forward.
00:03:02.912 --> 00:03:05.167
I'm flying tomorrow and looking forward.
00:03:05.167 --> 00:03:08.590
I don't ever really look forward to flying, but I'm looking forward to getting to DC.
00:03:08.590 --> 00:03:09.602
How about you?
00:03:10.204 --> 00:03:10.787
Indeed.
00:03:10.787 --> 00:03:14.098
Yeah, we are filming this early, so right now it's the.
00:03:14.098 --> 00:03:15.401
It's the Friday before Digital Now.
00:03:15.401 --> 00:03:22.586
By the time you all hear this episode, digital Now will be over, which at this point in time I can't really see the light at the end of the tunnel.
00:03:22.586 --> 00:03:28.379
But it's crazy to think that in just under a week this will all be wrapped up and I'll miss it so deeply.
00:03:28.379 --> 00:03:29.986
But yes, I'm excited to get to DC.
00:03:29.986 --> 00:03:31.506
I leave tomorrow as well.
00:03:31.506 --> 00:03:37.072
I think I get there in the afternoon and then we'll kind of kick things off Sunday morning with setup.
00:03:38.020 --> 00:03:44.954
Mallory, I was asking you right before we started recording if you are an aisle seat or a window seat person and you're starting to tell me.
00:03:45.820 --> 00:03:46.584
I am.
00:03:46.584 --> 00:03:50.106
You know, even though I'm on the shorter side, I do think I'm an aisle seat person.
00:03:50.106 --> 00:03:52.729
I think you're also an aisle seat person, amit.
00:03:52.729 --> 00:03:56.191
I think I just don't like how to crawl over people.
00:03:56.191 --> 00:04:06.502
So for me that makes the most sense, and I feel like I always have a ton of stuff in my hands as well that might just be a me problem but like a coffee cup and a purse and a backpack.
00:04:06.502 --> 00:04:11.824
So I feel like when I have the aisle seat, I can set all of that down and then put my luggage up and I'm comfortable.
00:04:11.824 --> 00:04:12.407
What about you?
00:04:13.270 --> 00:04:16.038
I am normally an aisle seat person.
00:04:16.038 --> 00:04:22.586
However, if I'm on an overnight flight, I like the window, because then I can settle in and not be disturbed by anyone else.
00:04:22.586 --> 00:04:28.932
But, um, you know, I have a good friend who is actually a middle seat person.
00:04:29.232 --> 00:04:32.214
No, yes, couldn't be.
00:04:32.214 --> 00:04:33.115
What does that mean?
00:04:33.115 --> 00:04:34.197
What does that mean?
00:04:34.740 --> 00:04:38.271
So this guy is interested in people.
00:04:38.271 --> 00:04:53.380
He's a student of humanity, you could say, and he's retired and a brilliant guy, and he loves to talk to random people and hear their stories, and so he chooses middle seats so that he has two people to talk to.
00:04:55.644 --> 00:04:59.132
Yeah, that sounds just like you, amit.
00:04:59.132 --> 00:05:01.541
I feel like you must be the same way right On flights.
00:05:01.862 --> 00:05:06.324
Oh yeah, you know me, I'm the social butterfly just talking to lots of random people all the time.
00:05:06.564 --> 00:05:07.427
Oh my gosh.
00:05:07.427 --> 00:05:11.692
Well, after this recording, you're going to have to tell me who your friend is so I can make sure.
00:05:11.692 --> 00:05:17.172
No, I'm kidding that I'm not next to him on a flight, but typically on flights I am putting on the headphones.
00:05:17.172 --> 00:05:20.103
I'm really not trying to have conversations but, you know.
00:05:20.103 --> 00:05:23.788
Every now and then you do meet someone interesting, so it's important to be open.
00:05:24.228 --> 00:05:25.690
I appreciate that you ought to enjoy this guy.
00:05:25.690 --> 00:05:28.055
This guy is a brilliant, brilliant guy.
00:05:28.055 --> 00:05:30.603
He's a PhD in mechanical engineering.
00:05:30.603 --> 00:05:39.541
He went on to have a tremendous career in the computing industry and then beyond that, he actually went on to do stuff in healthcare, and on top of all that he's a standup comedian.
00:05:39.762 --> 00:05:42.767
So um, well then, maybe, maybe I will.
00:05:42.767 --> 00:05:44.230
Maybe we should bring him on the pod, Amit.
00:05:44.230 --> 00:05:45.232
Maybe that's what should happen.
00:05:45.232 --> 00:05:45.692
Yeah, maybe we should.
00:05:45.713 --> 00:05:46.293
That would be fun.
00:05:47.899 --> 00:06:01.932
Well, in any event, yeah, people come in lots of varieties and a lot of our expressiveness and intelligence is conveyed through audio, so text loses out on a lot of what we are able to communicate.
00:06:01.932 --> 00:06:07.810
In fact, if you took the transcript of this conversation and simply read it, it wouldn't quite have the same effect.
00:06:07.810 --> 00:06:11.745
So I'm really excited about this episode, or these two episodes, I should say.
00:06:11.745 --> 00:06:11.925
Now.
00:06:11.925 --> 00:06:21.824
You and I have been talking about audio as a modality and how exciting it is to have democratized access at scale to unlimited audio consumption and audio creation.
00:06:21.824 --> 00:06:23.810
So can't wait to get into this with you.
00:06:24.600 --> 00:06:43.708
Yep, as you said, in this part one episode we will be kind of laying the foundation to make sure we all understand what we mean when we're talking about AI and audio, and then we'll talk about some tools that we are keeping a close eye on Amit, I know some that you're a big fan of and then in part two, we will talk about specific use cases for associations.
00:06:43.788 --> 00:06:53.432
So to kick off today's part one talking about foundations, audio is fundamentally more information dense than text, like Amit just mentioned.
00:06:53.432 --> 00:07:05.146
Think about how much meaning we derive from a single spoken sentence, not just the words, but the speaker's emotional state, their level of confidence, their geographical background, even their physical state.
00:07:05.146 --> 00:07:11.173
A simple okay can mean drastically different things depending on tone, pitch and timing.
00:07:11.173 --> 00:07:24.391
We instantly understand if someone is tired, excited, skeptical or sincere all from subtle variations in their voice, and this richness of information makes audio both powerful and challenging for AI to process.
00:07:24.391 --> 00:07:31.105
I think back just to the top of this episode when I said oh, amit, that sounds just like you, right, talking about the middle seat thing.
00:07:31.105 --> 00:07:36.564
If an AI had read that conversation text only, they probably wouldn't have picked up on my sarcasm.
00:07:37.165 --> 00:07:37.565
Exactly.
00:07:38.266 --> 00:07:42.500
At its core, ai audio processing involves three main capabilities.
00:07:42.500 --> 00:07:48.593
First, their speech to text, where AI converts spoken words into written text, like transcription.
00:07:48.593 --> 00:07:55.153
Then there's text-to-speech, where AI generates spoken words from text, creating synthetic voices.
00:07:55.153 --> 00:08:03.410
Finally, there's speech-to-speech, where AI can transform audio directly, like changing a voice or translating between languages.
00:08:03.410 --> 00:08:04.411
In real time.
00:08:04.411 --> 00:08:07.535
These capabilities combine in real time.
00:08:07.535 --> 00:08:09.697
These capabilities combine in fascinating ways.
00:08:09.697 --> 00:08:15.132
Some AI voice interfaces convert your speech to text, process that text and then convert its response back to speech.
00:08:15.132 --> 00:08:26.983
The real breakthrough has been in speech-to-speech transformation, which can modify audio directly instead of going through text, preserving crucial layers of meaning that make human speech so rich.
00:08:26.983 --> 00:08:31.855
So, amit, you have been talking about audio for a long time.
00:08:31.855 --> 00:08:40.226
I think this is something even I don't experiment with quite as often, or you have to remind me to try out Advanced Voice with ChatGPT, which we'll talk about.
00:08:40.226 --> 00:08:47.631
Do you recall any wow moments that you've had recently, or maybe all the way back to a few years ago, when you first started using this?
00:08:48.500 --> 00:08:58.908
You know, going way back to when I was a kid, I remember the very first home computer we had that was a real computer was a Mac and it was the old old days of the Mac.
00:08:58.908 --> 00:09:02.604
This did not have a hard drive, it had one meg of RAM and it had a floppy drive.
00:09:02.604 --> 00:09:11.806
But on that computer there was a little program where it could speak very, very crudely a sentence that you put in and it sounded horrendous.
00:09:11.806 --> 00:09:16.432
But it was just amazing that a computer could actually speak based on any text that you put in.
00:09:16.432 --> 00:09:18.006
It was super slow and super terrible.
00:09:18.700 --> 00:09:22.851
It'd be fun actually to dig up an old, early Mac text-to-speech demo.
00:09:22.851 --> 00:09:33.101
I bet there's something on YouTube we could splice that in here to illustrate.
00:09:33.101 --> 00:09:42.389
But what really got my attention was, you know really, the natural way that speech sounds when you generate it 11 Labs specifically, which I know we'll talk about in the next section or the next part of this two-part episode.
00:09:42.389 --> 00:09:49.990
But that particular product blew my mind when I first used it because it really sounded like a person.
00:09:49.990 --> 00:10:10.514
The quality, the tone, everything it was just right on and it's getting better and better and so that's really what got my attention is just that basic text-to-speech direction and to me that opens up a whole bunch of opportunity, because you know so much of what we want to do is hear as opposed to read.
00:10:10.514 --> 00:10:16.610
Reading is great too, but like being able to hear anything that just opens up possibilities, so that got me excited.
00:10:17.471 --> 00:10:17.831
For sure.
00:10:17.831 --> 00:10:21.970
And then you were also an avid user of Otter maybe still are where you would go.
00:10:21.970 --> 00:10:22.912
Speech to text, right?
00:10:23.480 --> 00:10:23.900
That's right.
00:10:23.900 --> 00:10:27.048
Yeah, I mean the both of the Ascend books that we've worked on together.
00:10:27.048 --> 00:10:50.394
You know those books I've contributed a lot of content through just walking around New Orleans like a crazy person just talking to my phone with ideas, and then Otter does a good job with transcription and I combine that with taking that audio I'm sorry, taking the transcript of that audio into something like a chat GPT or a cloud and then using it to kind of distill the ideas down and generate better versions of them.
00:10:50.394 --> 00:11:17.904
But you know, I think there is a lot lost have now with ChatGPT's advanced voice mode and many other models that are natively multimodal where there's no translation layer, where the model understands speech as well as video and images natively, and that is going to produce stunning breakthroughs.
00:11:19.568 --> 00:11:25.691
Yep, I did a little test this morning with advanced voice, but I'll talk about that in the next section, so I don't jump the gun here.
00:11:25.691 --> 00:11:31.113
But I feel like people are still getting comfortable with conversing with AI.
00:11:31.113 --> 00:11:39.870
I'm curious why you think that is and if you feel like that's a mistake and that people should be really prioritizing that way of interacting with AI.
00:11:40.919 --> 00:11:43.682
You know, I think there's so the people that are using AI.
00:11:43.682 --> 00:11:53.927
Ai, we have to remember, is a tiny fraction of the world still, so we have a lot of people all over the planet who still don't have access to this type of technology, and that's going to change.
00:11:53.927 --> 00:12:08.677
And you think about the form factor of technology in most places, it's the mobile phone and, soon to be, probably wearables, whether it's watches, glasses, something else, and so audio can fit on any size device.
00:12:08.677 --> 00:12:32.124
You know, you can have audio on a phone or even on a watch and an earbud, and so the beautiful thing about that is the portability, the mobility that creates, as well as the accessibility, because there are a lot of people who don't read, people who don't read, and so, when you think about it in terms of access, you think about democratizing access to the power of AI, whether it's for consumer services like financial literacy or for healthcare delivery.
00:12:32.124 --> 00:12:38.432
So, on that end of it, it's, I think, critically important to get speech right and, of course, be truly multilingual.
00:12:38.432 --> 00:13:05.326
There's a lot of work happening, particularly in languages that don't have large bodies of existing text or audio or video on the internet, and so to be able to preserve those languages, but also to enable them from an AI perspective, even if there's only say, 100,000 people in the world who speak that particular language, and a lot of times those are also the parts of the world where people do a lot more talking and listening than they do writing or reading, and so that's one of the reasons I think audio is so important.
00:13:05.326 --> 00:13:14.648
But also to me, ultimately, there's just a richness in audio that is different than what you get from text.
00:13:14.648 --> 00:13:15.870
Like the way I think about it.
00:13:15.910 --> 00:13:18.706
If I'm trying to learn a subject, I love audio.
00:13:18.706 --> 00:13:23.442
I listen to podcasts constantly and I also love having phone calls with people.
00:13:23.442 --> 00:13:24.886
When I'm on the move.
00:13:24.886 --> 00:13:36.504
I don't like talking to people on the phone or in Zoom calls other than this one this is great but I don't like Zoom calls or phone calls when I'm stationary normally, partly because I get distracted and I just move on to other things in my brain.
00:13:36.904 --> 00:13:48.427
Where I'm moving and I'm walking down the street talking to someone, I don't multitask because I'm just focused on not getting run over by a New Orleans driver and talking to the person, so I find myself more focused.
00:13:48.427 --> 00:13:53.211
But there's just kind of a richness to that right in terms of that audio conversation with the person.
00:13:53.211 --> 00:14:06.131
But if I want to actually go and you know, quote, unquote, put pen to paper to actually get the real work done, I'm working in text, I'm working with the computer and maybe it's text plus audio side by side.
00:14:06.131 --> 00:14:12.649
In our prior episode we talked about Claude and the computer use and that's text plus computer use.
00:14:12.649 --> 00:14:27.885
But if Claude also had a real-time equivalent like advanced voice mode, imagine talking to Claude and Claude's working on your computer and Claude's talking back while working parallel with you and there's also text going, so more like working with another person, essentially.
00:14:28.827 --> 00:14:34.375
Okay, as a funny side note, if you have ever talked to Amit on the phone, normally you're right.
00:14:34.375 --> 00:14:37.539
Amit, it sounds like there's trains in the background of what.
00:14:37.539 --> 00:14:40.605
Sometimes when I talk to you, but it's because you're always on a walk, which.
00:14:40.605 --> 00:14:41.586
I think is a good way to be.
00:14:42.890 --> 00:14:56.774
If I'm talking to someone on a phone call, I'm almost always on a walk because I like to stretch my legs and I find that it's just more focused, more focused In terms of comfort level.
00:14:56.774 --> 00:14:58.100
Getting back to your question, you know I don't know what to say.
00:14:58.100 --> 00:15:00.793
I think the people who are earlier adopters of AI will get comfortable with new things pretty quickly.
00:15:00.793 --> 00:15:22.879
I think that for a lot of people who haven't touched AI maybe they've heard of it on the news or something, but they haven't themselves interacted with it they're going to start getting interactions with AI just in their daily lives, because many organizations are going to put in place AI voice systems to talk to their customers and provide way better customer service than people have ever experienced, and in some cases they might not even disclose that it's an AI.
00:15:22.879 --> 00:15:27.100
I think it's important that you do that if you use AI, but anywhere you use it, you should disclose it.
00:15:27.100 --> 00:15:33.719
But I think the experience consumers will have in that respect will go up and then, all of a sudden, people will just assume it.
00:15:39.409 --> 00:15:40.354
It's just like right now going through life.
00:15:40.354 --> 00:15:45.755
If you didn't have your mobile phone with you for a day, the feeling of loss and the feeling of disconnectedness you would have compared to what you had before access to Google, access to the internet generally.
00:15:45.755 --> 00:15:53.759
If you have access to an intelligent assistant through voice, you're just talking to this thing all day long and then it goes away.
00:15:53.759 --> 00:15:57.917
You realize how dependent you are on it, which is both good and bad, I guess.
00:15:57.917 --> 00:16:06.873
I guess the point is that I think there will be a very natural adoption for voice, because that's just what we do as a species is we like to talk and listen.
00:16:08.537 --> 00:16:09.359
Yeah, I don't know why.
00:16:09.359 --> 00:16:29.831
I just get the sense that there's a bit of discomfort around it, and I think it might be because we are so accustomed to text when we're looking for information, whether we're texting someone we know or we're on Google searching something or looking in our email or on Microsoft Teams, and so I think the idea of going to voice when you need information, it's just a little bit of a mental shift.
00:16:30.514 --> 00:16:32.659
I think that's right and you know it is for me as well.
00:16:32.659 --> 00:16:40.057
Partly it's that you know this disbelief that a computer can really do such a good job.
00:16:40.057 --> 00:16:55.039
Even those voice assistants or voice products that have been out for a while, like Siri and Alexa or Google Assistant, have been so terrible historically that when you do use them you're like, yeah, I have really low expectations, like Siri is going to totally screw up this text message, but I'm driving and I don't want to crash my car, so let me see if Siri can send this text message for me.
00:16:55.039 --> 00:16:56.822
So there's that whole side to it.
00:16:56.822 --> 00:17:01.980
But I think that our perceptions and our biases, they're self-reinforcing and they're very powerful.
00:17:01.980 --> 00:17:04.351
Actually, just today I posted something on LinkedIn.
00:17:04.351 --> 00:17:12.294
I had this really interesting experience in the last two days where I was talking to a colleague about a technical problem.
00:17:12.835 --> 00:17:14.436
And this other person asked me a question.
00:17:14.436 --> 00:17:17.883
They said, hey, does this database do this one thing?
00:17:17.883 --> 00:17:19.990
And I said, no, I wish it had that feature.
00:17:19.990 --> 00:17:22.556
And I'm like, wait a second.
00:17:22.556 --> 00:17:25.963
I've been using this database product for like 30 years.
00:17:25.963 --> 00:17:28.374
I wonder if they've added that feature since.
00:17:28.374 --> 00:17:30.660
I bothered to check, turns out.
00:17:30.660 --> 00:17:31.402
So I started working.
00:17:31.430 --> 00:17:40.800
This is Microsoft SQL Server, turns out I started working with Microsoft SQL Server right after they bought the database engine from Sybase, which is like the mid-90s, 1995, in version 6.0.
00:17:40.800 --> 00:17:43.756
And I've been working with it for a long time.
00:17:43.756 --> 00:17:44.960
Right, I know it pretty well.
00:17:44.960 --> 00:17:48.901
Well, back in 6.0 and 6.5, they didn't have this feature.
00:17:48.901 --> 00:18:00.349
But in version 7, which came out in 1998, they added this feature and I haven't known about it for, you know, 26 years so, and I'm pretty good at that and I'm also pretty good at adapting At least I think I am.
00:18:01.132 --> 00:18:07.115
But we all get sucked into these loops where we're telling ourselves that and I'm like, wait a second, is that true, right?
00:18:07.115 --> 00:18:08.221
So asking yourself that.
00:18:08.221 --> 00:18:23.278
Anyway, the reason that I think is relevant to your question is we have these self-reinforcing behavioral loops that cause us to feel a certain way at the same time, because there's so many consumers and users of products that haven't come online yet at all or haven't come online with AI.
00:18:23.278 --> 00:18:30.215
I don't know that it's going to be an issue at scale, because the number of people that are currently using this stuff is a tiny fraction of the people who will use this stuff.
00:18:31.619 --> 00:18:33.063
That is a really good example.
00:18:33.063 --> 00:18:37.836
I feel like not similar at all, but I feel like I have that experience with teams even.
00:18:37.836 --> 00:18:42.377
I'll just be like it would be so nice if stumble upon this new app within, like the Microsoft suite.
00:18:42.377 --> 00:18:45.510
I'm like, oh, oh, I guess I could have done that the whole time.
00:18:45.510 --> 00:18:52.433
So just having to kind of keep searching, always keep asking those critical questions of yourselves and of the technology we use.
00:18:53.395 --> 00:18:53.997
Yeah, it's like the.
00:18:53.997 --> 00:18:54.999
Is this still true?
00:18:54.999 --> 00:18:59.534
What are my deeply held beliefs about what AI can and cannot do as of right now?
00:18:59.534 --> 00:19:02.740
And then you say, well, is that still true?
00:19:02.740 --> 00:19:10.192
And some people who had initial this is actually something I do regularly is I talk to executives who said, oh yeah, you know, we know we were.
00:19:10.192 --> 00:19:11.734
We're really on top of AI.
00:19:12.296 --> 00:19:15.982
We tried chat GPT back in February of 2023.
00:19:15.982 --> 00:19:17.332
It was right after it came out.
00:19:17.332 --> 00:19:19.240
We're really proud of ourselves for checking it out.
00:19:19.240 --> 00:19:21.609
We were all over it and it sucked.
00:19:21.609 --> 00:19:24.778
It's so bad, it was so full of shit.
00:19:24.778 --> 00:19:26.582
You know, it was on and on and on about how bad it was.
00:19:26.582 --> 00:19:28.574
I'm like, agreed, it was terrible.
00:19:28.574 --> 00:19:29.978
Have you tried it since then?
00:19:29.978 --> 00:19:31.480
Like no, of course not, it was terrible.
00:19:31.480 --> 00:19:39.472
It's like, well, ordinarily, in like a year and a half, a software product wouldn't have been, you know, dramatically different, maybe a little bit different.
00:19:39.472 --> 00:19:44.978
And it's not an unreasonable assumption to say, oh, I tried Excel in early 2023 and I hated it.
00:19:44.978 --> 00:19:45.842
It was just terrible.
00:19:45.842 --> 00:19:48.830
And now it's late 2024 and I didn't try Excel again.
00:19:48.830 --> 00:19:51.575
I still feel like crap from having tried it back in early 23.
00:19:51.575 --> 00:19:55.500
But this is so different and it's changing so fast, right?
00:19:55.500 --> 00:19:58.444
So you have to readdress or reevaluate ideas.
00:19:58.444 --> 00:20:07.020
You can't hold these kinds of assumptions like I was doing for 26 years and expect to be effective in what you do.
00:20:08.171 --> 00:20:12.555
That's so funny that it came out in version 6.5, you said, or version 7?
00:20:12.555 --> 00:20:19.881
Yeah, Wow, so you were just on the brink of having this feature that, I'm assuming, would have been pretty helpful in the past 26 years.
00:20:20.471 --> 00:20:21.214
I would say so.
00:20:21.214 --> 00:20:24.087
Yeah, it would have saved me a lot of time and it's pretty cool.
00:20:24.087 --> 00:20:29.903
It's a minor thing, but minor things that you do thousands of times over a lot of years end up adding up.
00:20:30.412 --> 00:20:31.095
Right, right.
00:20:31.095 --> 00:20:34.007
I always remind on the Intro to AI webinar.
00:20:34.007 --> 00:20:48.276
I remind everyone that same fact with Midjourney, because these AI powered image generators were pretty rough I feel like back in 2022, in terms of the ridiculous images that they were creating and they were so bad and everyone thought it was so funny.
00:20:48.276 --> 00:20:55.082
But they're pretty good now, and so I always tell people even if you tried it last year, try it again, because these things are improving consistently.
00:20:55.829 --> 00:21:26.271
And that is definitely true with audio, and that's what part of what excites me so much is that there's a lot of investment going into audio as a modality, not just with the specialized tools that are out there, which we'll talk about shortly, but also with the general AI models that are becoming truly multimodal, where Gemini, claude and, of course, openai are all multimodal models natively and some of them haven't exposed that capability from a consumer perspective, where you can't talk to Gemini yet, but you will be able to soon because they're multimodal.
00:21:26.271 --> 00:21:34.621
The real-time side of it is just really hard nut to crack and OpenAI, I think, has done a good job with that, but everyone's going to have that within six to 12 months.
00:21:36.431 --> 00:21:43.624
Well, now that we've got this common foundation and understanding of audio and AI, I do want to talk about some tools that we've been keeping a close eye on.
00:21:43.624 --> 00:21:54.125
One of those, amit, is already mentioned on this podcast, and that is Eleven Labs, which features advanced speech synthesis with natural sounding, context awareaware intonation and emotion.
00:21:54.125 --> 00:22:06.459
It also features voice cloning technology, which allows users to create custom voices from short audio samples, and AI dubbing capabilities that translate speech while maintaining original voice characteristics.
00:22:06.459 --> 00:22:14.077
I have actually not tried out 11 Labs myself, which is terrible, but I've seen all of your demos so I feel like I know what it's capable of.
00:22:14.077 --> 00:22:24.500
Next is Chat2BT's advanced voice mode, which is real-time natural conversations with the ability to handle interruptions and respond fluidly.
00:22:24.500 --> 00:22:41.576
It features emotional recognition and response, allowing the AI to detect and adapt to the user's tone and emotional cues, and there's native speech understanding, like Amit mentioned, using GPT-4-0, eliminating the need for separate speech-to-text and text-to-speech models.
00:22:41.576 --> 00:22:53.759
Now, I did give this a brief test this morning because I had not tried it out for myself and I tried to sound really angry and ask it do I seem mad or do I seem happy or angry right now?
00:22:53.759 --> 00:22:55.436
And it wasn't quite getting it.
00:22:55.436 --> 00:23:00.796
So I don't know if maybe I was pushing it a little too hard or if I didn't sound angry enough, but just worth sharing with all of you.
00:23:01.980 --> 00:23:08.540
Next, I want to talk about hugging face speech to speech, which we have covered on a previous podcast episode which we will link in the show notes.
00:23:08.540 --> 00:23:21.796
Basically, speech to speech features automatic language detection option for seamless multilingual communication and it's real-time, so it allows for immediate responses and audio format in various languages.
00:23:21.796 --> 00:23:26.801
And then, finally, google Notebook LM, which we've also covered on a previous pod.
00:23:26.801 --> 00:23:36.231
It creates podcast-style discussions based on uploading content, transforming written material into engaging conversations between two AI hosts.
00:23:36.231 --> 00:23:49.798
And now added feature since that original podcast episode is that users can now provide specific instructions to guide the AI hosts to focus on particular topics or adjust the expertise level for the intended audience.
00:23:49.798 --> 00:23:54.692
So, amit, I know you're a big fan of 11 Labs, but also Advanced Voice.
00:23:54.692 --> 00:23:59.340
Which of these do you find yourself going back to over and over?
00:24:00.182 --> 00:24:01.243
I use them for different things.
00:24:01.243 --> 00:24:07.767
So with Advanced Voice mode and ChatGPT, I'm just talking to ChatGPT a lot either.
00:24:07.767 --> 00:24:09.988
If I'm driving or walking, you know same thing.
00:24:09.988 --> 00:24:19.330
It's like I'm having these detailed conversations and it's way more fluid, as you mentioned, than the voice mode they had for the last 12 months, where you could speak to the AI and then what?
00:24:19.351 --> 00:24:29.392
it would do is it would take that speech, turn it into text, submit the text prompt for you get a response and as the response was coming in, it would start synthesizing speech from that, which was actually quite useful.
00:24:29.392 --> 00:24:40.917
The regular, the prior speech mode they had, or voice mode, was quite good, but it was kind of like you know, submitting your little speech and then waiting for a little speech to come back and it still had a lot of utility, but now it's way more conversational.
00:24:41.069 --> 00:24:43.819
I have also found it to not be the best at picking up on tone.
00:24:43.819 --> 00:24:50.537
The demos we've seen talk about like picking up on sarcasm a little bit, but I don't know that it's going to be great at that.
00:24:50.537 --> 00:24:54.580
Demos tend to also be kind of cherry picked, so I think there's some ways to go for that.
00:24:54.580 --> 00:25:14.142
But what it is good at doing is actually listening if you interrupt and there's a lot of utility just in that one feature, because a lot of times it starts going down a path explaining something I'm like no, no, no know, doing whatever kind of work I'm doing.
00:25:14.142 --> 00:25:25.999
Eleven Labs I have used it a fair bit for dubbing If I want to record a video and I want to put the video out there.
00:25:25.999 --> 00:25:26.663
But I don't necessarily want my own voice.
00:25:26.663 --> 00:25:32.676
Let's say I'm, you know, discussing some technical thing or whatever and I'm like I want the brand to have a consistent voice, like you're mentioning and we'll talk about in the second part.
00:25:32.676 --> 00:26:02.442
You know, I can go and take a video of a screen share where I'm presenting on a topic and then I can strip out my audio and save that as a WAV file and then upload that to 11 Labs and do speech-to-speech dubbing with whatever voice I want, and other people on our team could also do the same thing with different videos that they've been assigned and then afterwards, like you, can do this speech to speech dubbing and then all the videos have the same professional voice from you know, kind of a it's not a person, but there's something valuable about that potentially.
00:26:02.442 --> 00:26:03.574
So I think it's interesting.
00:26:03.574 --> 00:26:12.336
The Hugging Face speech to speech stuff I'm a big fan of because it's a toolkit that allows you to develop things and there's many other ways to do what they've done.
00:26:12.336 --> 00:26:17.063
They just put together a full stack for this With advanced voice mode from OpenAI.
00:26:17.063 --> 00:26:29.811
There is an API for it, but it's somewhat of a sealed API, meaning there's not much control you have over it, whereas the Hugging Face stuff allows you a lot more granularity as a developer to build your own solutions with their tool set.
00:26:29.811 --> 00:26:31.917
So I think that's interesting.
00:26:32.098 --> 00:26:49.420
So I'm kind of all over the place on this stuff and Google Notebook LM my favorite use case for that is, once again, I like to walk around, so if you want to find me in New Orleans, just look around either in the early morning or in the evening and I'm probably out on the street walking around but Google's Notebook LM.
00:26:49.480 --> 00:26:50.542
I just used it last night.
00:26:50.542 --> 00:26:53.954
I got this long email from one of the companies that we made an investment in.
00:26:53.954 --> 00:27:04.785
One of the people in that company sent this long email with had a five-page PDF attached to it proposing this new project, and then another board member had replied to it.
00:27:04.785 --> 00:27:24.559
And so I took the PDF and the email and I took the reply and I put them all into Notebook LM and I said and I was doing this while I was getting ready to go for my walk, I'm like and I hit generate audio overview and then I had an eight minute podcast that got me up to speed and I came back and then I replied to the email really quickly and actually included the audio clip saying hey, this is what I turned it into, cause I didn't feel like reading your email.
00:27:25.369 --> 00:27:28.016
No way that is so smart.
00:27:28.016 --> 00:27:30.200
To me, that's a really good idea, do you feel?
00:27:30.200 --> 00:27:32.665
Did you read the email just in case to confirm?
00:27:32.665 --> 00:27:33.827
No, you listened to that.
00:27:33.827 --> 00:27:37.517
You felt like you got it Well, because this is the thing where it's like it's going to turn into a conversation.
00:27:37.690 --> 00:27:40.494
So if I missed a little bit of detail, it's a low risk kind of scenario.
00:27:40.494 --> 00:27:46.296
I wouldn't do that for like oh, here's a legal contract where I'm like buying a house, let me have a notebook LM on this.
00:27:46.730 --> 00:27:48.721
Although that would be interesting too.
00:27:56.170 --> 00:27:58.194
Yeah, a contract, fun and interactive, the way notebook LM does, you know, I think.
00:27:58.194 --> 00:28:15.680
One thing I would point out and maybe we'll play a little clip of this is you know, we have been doing a lot of experimenting around audio with one of our AI agents, which is Betty, and since this podcast is going to air after digital, now we can play a little clip of this because we're going to introduce it at digital now on stage as part of my keynote.
00:28:16.069 --> 00:28:25.804
But Betty is going to have a voice mode, and so Betty is known for being this accurate knowledge agent that works on top of the association's content corpus.
00:28:25.804 --> 00:28:51.551
So you have your association's vast array of prior publications and content, and Betty learns it, and today Betty has been interacting with members through text, and now, with the voice mode that is not yet available but will be in the early part of next year, your members will be able to have conversations with Betty through a really advanced voice interface, and so the clip that we'll play here now has just a little taste of what that might be like.
00:28:51.551 --> 00:28:54.278
Hey, betty, what's going on?
00:28:57.769 --> 00:28:59.477
Hey there, exciting stuff in the association world.
00:28:59.477 --> 00:29:08.076
Digital Now is coming up and it's the place to be for top association leaders, keynotes, workshops and networking galore.
00:29:08.309 --> 00:29:08.893
That's awesome.
00:29:08.893 --> 00:29:09.695
When is the conference?
00:29:11.652 --> 00:29:19.403
Digital Now is happening from October 27 to 30, 2024 at the omni shoreham hotel in washington dc.
00:29:19.403 --> 00:29:24.366
Ready to dive into some game-changing content yeah, I am, and you know what?
00:29:24.426 --> 00:29:25.791
here's a surprise for you.
00:29:25.791 --> 00:29:34.397
I hope you're excited because you are actually on stage right now during my opening keynote, so say hi to everyone hello everyone.
00:29:35.138 --> 00:29:41.458
It's a thrill to be here at Digital Now sharing the stage with all of you visionary association leaders.
00:29:41.458 --> 00:29:48.522
Let's dive into some cutting-edge ideas and make this conference unforgettable, Ready to shape the future.
00:29:49.670 --> 00:29:54.682
Hey Betty, listen, some of the folks in the room are probably wondering the best way to get started with AI.
00:29:54.682 --> 00:30:01.289
They're a lot of times perhaps hesitant because their organizations don't have a ton of technical resources.
00:30:01.289 --> 00:30:05.056
So let me know what you think and can you speed it up a little bit, because I'm a little bit impatient.
00:30:06.840 --> 00:30:08.935
Absolutely no need for hesitation.
00:30:08.935 --> 00:30:22.343
Here's a quick start Dedicate just 15 minutes a day to learning about AI, follow AI thought leaders on LinkedIn, subscribe to AI newsletters and listen to the Sidecar Sync podcast.
00:30:22.630 --> 00:30:23.654
Well, that's a great idea.
00:30:23.654 --> 00:30:31.059
Mallory and I, in particular, are 100% on board with everyone listening to the Sidecar Sync podcast that we co-host every single week.
00:30:31.059 --> 00:30:32.756
Well, listen, betty, we've got to go.
00:30:32.756 --> 00:30:36.699
This keynote isn't super long and I'm going to get pulled off stage pretty soon.
00:30:36.699 --> 00:30:37.724
Thanks so much for your help.
00:30:37.724 --> 00:30:42.220
We're really excited about your voice mode and your help for all associations as their knowledge agent.
00:30:44.526 --> 00:30:45.326
Thanks for having me.
00:30:45.326 --> 00:30:48.070
I'm thrilled to be part of your AI journey.
00:30:48.070 --> 00:30:54.780
Enjoy the rest of Digital Now and remember the future of associations is bright with innovation.
00:30:56.144 --> 00:30:57.968
It's a pretty impressive clip, Amit.
00:30:57.968 --> 00:31:01.613
I wanted to ask you about it, but I wasn't sure if we were going to talk about it on the pod.
00:31:01.613 --> 00:31:02.073
But you're right.
00:31:02.073 --> 00:31:04.936
Since it's after Digital, Now it's out there in the open.
00:31:04.936 --> 00:31:07.500
Can I ask what model you're using for this?
00:31:09.684 --> 00:31:10.027
there in the open.
00:31:10.027 --> 00:31:10.911
Can I ask what model you're using for this?
00:31:10.911 --> 00:31:12.398
So under the hood, we use a few different things and this is a prototype.
00:31:12.398 --> 00:31:12.961
It's an early version of it.
00:31:12.961 --> 00:31:14.827
It's more than a prototype, but it's an early version.
00:31:15.289 --> 00:31:24.277
So the conversation I was having was with an underlying API from OpenAI, using their real-time API, but also using some additional components that went with it.
00:31:24.277 --> 00:31:45.713
So right now, the OpenAI real-time API has the lowest latency, and so that latency is important because research suggests that between 200 to 400 milliseconds, or between two-tenths and four-tenths of a second, is the amount of time people feel is natural for the gap between when one speaker stops and the next speaker responds.
00:31:45.713 --> 00:31:50.096
So if it's more than about a quarter of a second, then it feels as though there's an artificial delay.
00:31:50.096 --> 00:31:56.215
It's like playing you know old school telephone game when there was long distancing and waited a second or two for the response to start coming back in.
00:31:57.505 --> 00:32:08.156
But you know, the thing is, I think, that even though they have the advantage there, there's and there are some really amazing things that OpenAI has done, but it's very, very expensive and it also is extremely inflexible.
00:32:08.156 --> 00:32:14.520
So real-time API from OpenAI will get better and will probably be both affordable and very flexible at some point.
00:32:14.520 --> 00:32:20.451
At the moment it's not, so we're really prototyping the use cases with that tool and then we're planning on building.
00:32:20.451 --> 00:32:25.333
We are building with another tool set that's directly integrated with Betty's actual knowledge base.
00:32:25.333 --> 00:32:28.601
So it's a combination of things that we're using.
00:32:28.601 --> 00:32:37.891
We're kind of being open-minded about, like, which model to actually plug in, but at the moment it's a mixture of GPT-4.0 and their real-time API on top of that.
00:32:38.652 --> 00:32:38.893
Nice.
00:32:38.893 --> 00:32:42.189
So the native GPT-4.0 speech-to-speech that's powering there.
00:32:42.269 --> 00:32:45.065
Yes, we're using what is from the consumer perspective.
00:32:45.065 --> 00:32:46.948
It is advanced voice mode.
00:32:46.948 --> 00:32:54.319
In ChatGPT we have essentially used the same underlying technology for what we're calling the Betty voice preview.
00:32:54.319 --> 00:32:56.432
This is not the final product.
00:32:56.432 --> 00:32:58.733
This is a preview of the idea behind the use case.
00:32:58.733 --> 00:33:07.981
The reason we're doing this is because sometimes you have to really illustrate a concept fully before people understand the utility from it, before people understand the utility from it.
00:33:07.981 --> 00:33:15.873
And actually, betty, the whole journey for the last two years with that product has been very much like that, where you first had to kind of explain to people the reason why it would be valuable to unlock the knowledge of your association.
00:33:15.873 --> 00:33:21.951
To begin with, some people immediately got it, but a lot of people need to see it to understand the value creation, what that utility is.
00:33:21.951 --> 00:33:25.768
And then, with voice, it's again part of what you had said earlier.
00:33:25.768 --> 00:33:26.808
People use the text.
00:33:26.808 --> 00:33:28.250
What's the point of having voices?
00:33:28.250 --> 00:33:32.255
Is it just kind of like a neat party trick or is there utility?
00:33:32.255 --> 00:33:34.076
And I think there's tremendous utility.
00:33:34.076 --> 00:33:38.721
It's obviously our position on it and so showing people that through the preview is our goal.
00:33:38.721 --> 00:33:46.393
It's going to get better and better and over the course of the next several months as we work on the development and get something ready to go.
00:33:46.393 --> 00:33:48.997
I think people are going to be blown away with what they can do.
00:33:49.116 --> 00:33:57.336
Here's the bottom line, though whether it's Betty or something else, people in the world are interacting with AI through voice more and more.
00:33:57.336 --> 00:33:59.028
They're going to expect that from you.
00:33:59.028 --> 00:34:01.192
You are a customer service organization.
00:34:01.192 --> 00:34:22.085
At the end of the day, people are going to expect immediate, real-time access, and not to some bot thing that only knows like wrote answers, but to an actual, human-like, 24-7, 365 total expert on your association who knows not only all of your policy and product stuff and membership renewal rules and all that, but an expert on your actual domain.
00:34:22.085 --> 00:34:25.686
That's what people are going to expect from the association, and that's certainly the opportunity.
00:34:25.686 --> 00:34:33.467
That may not be tomorrow, it may not even be at Digital Now 2025, when consumer expectations are fully aligned with that statement, but it's soon.
00:34:33.467 --> 00:34:34.590
It's very, very soon.
00:34:36.172 --> 00:34:39.307
I think this conversation sets us up really well for part two.
00:34:39.307 --> 00:34:44.027
But I do have a question for you, amit Do you find that a lot of associations have mobile apps?
00:34:44.027 --> 00:34:48.235
Because I do think kind of the utility of something like that would be on your phone.
00:34:48.235 --> 00:34:50.358
So I'm just curious if you have any insight on that.
00:34:50.965 --> 00:35:23.813
The only mobile app use case that I have seen successful over time in associations historically has been event apps, where people download the event app, like they will for digital now, when they come to the event and it's a great experience to be able to see the sessions that you're going to organize yourself, maybe connect with other people, and then after the event they delete it and that's it, because there hasn't been a reason for people to want to interact with an association frequently enough to justify the existence of an app.
00:35:23.813 --> 00:35:32.668
Even if you say, oh, we're going to put all of our most commonly used resources in an app and make it really pretty and make it easy to download, the problem is that people don't engaged in that.
00:35:32.668 --> 00:35:55.990
So most of the time associations tend to have infrequent but deep engagement, which is not bad.
00:35:55.990 --> 00:36:04.862
They just haven't created a value proposition that justifies their existence in the lives of their members on a daily or weekly basis.
00:36:04.862 --> 00:36:06.289
The newsletter would be one exception to that.
00:36:06.289 --> 00:36:07.092
A lot of times.
00:36:07.092 --> 00:36:11.512
If newsletters are done well, then that can be something people look forward to every day or every week.
00:36:11.764 --> 00:36:14.530
But my belief is coming back to your point about apps.
00:36:14.530 --> 00:36:18.449
If you have like, why would you download an app Like versus like.
00:36:18.449 --> 00:36:19.954
I know you're a user of Canva.
00:36:19.954 --> 00:36:23.025
I use another product called GitHub for software development a lot Like.
00:36:23.025 --> 00:36:24.208
Why would you download the app?
00:36:24.208 --> 00:36:28.195
It's because you use it a lot and you want a better experience than the mobile device.
00:36:28.195 --> 00:36:31.702
Otherwise, you just go to the mobile website version.
00:36:31.702 --> 00:36:32.244
There's no point.
00:36:32.244 --> 00:36:36.092
Why would you download an online banking app versus go to the website?
00:36:36.092 --> 00:36:37.476
The experience is probably better.
00:36:37.476 --> 00:36:39.648
Maybe it's more secure too in that case.
00:36:39.728 --> 00:36:50.989
But going back to the apps for associations, I think mobile app for an association would be awesome if there was a reason to use it frequently, and the reason to use it frequently is it helps me do my job.
00:36:50.989 --> 00:36:52.634
It helps me in my life.
00:36:52.634 --> 00:37:02.777
It doesn't help the association, like you know, no matter, no matter amount of gamification or anything else is going to draw me into engagement formats that are not value accretive to me.
00:37:02.777 --> 00:37:07.809
So that's why the knowledge assistant concept, I think, is like a dead obvious one in terms of like.
00:37:07.809 --> 00:37:13.760
If I like, if I can help my members on a daily basis in their day-to-day work, that's an enormous opportunity.
00:37:13.760 --> 00:37:17.094
It's an untapped opportunity because no one's doing this yet.
00:37:17.094 --> 00:37:22.936
So associations are in a perfect position to go and crush it in their respective domains.
00:37:22.936 --> 00:37:23.938
It's exciting in my mind.
00:37:25.666 --> 00:37:29.315
Well, that is a perfect way to end part one of the AI and audio series.
00:37:29.315 --> 00:37:34.737
In part two, we will be exploring some association use cases, maybe with some mobile apps, so stay tuned.
00:37:36.846 --> 00:37:38.873
Thanks for tuning into Sidecar Sync this week.
00:37:38.873 --> 00:37:40.458
Looking to dive deeper?
00:37:40.458 --> 00:37:48.677
Download your free copy of our new book Ascend Unlocking the Power of AI for for associations at ascendbookorg.
00:37:48.677 --> 00:37:52.735
It's packed with insights to power your association's journey with AI.
00:37:52.735 --> 00:38:00.211
And remember, sidecar is here with more resources, from webinars to boot camps, to help you stay ahead in the association world.
00:38:00.211 --> 00:38:02.211
We'll catch you in the next episode.
00:38:02.211 --> 00:38:05.894
Until then, keep learning, keep growing and keep disrupting.