Vinton Feature Spotlight: New LLM Options

Transcript

Right. We’re like. Hello? Hello. Hello, Luca. It’s been a while. And how are you today? It has been a while. Hello, Mina. Yeah, I’m doing very well. I’m doing very well. Thank you. I’m excited about, you know, today’s conversation. It’s, We’ll dive into a bit technical details of how large language models work and different types there are.

And I know that this is something that you love talking about… We’re not going to dive into very technical details. I think we’re going to, you know, because I’d be lost right? But we better know from a user experience where you’ve experienced and how you yes, love it, love how it increases your productivity when you’re using them and you’ve, because we see it every day with our clients as well.

And you personally use them as well. Been doing that. So, you know, so what is today about before we begin, do you want to go over the rules of engagement a bit and why we’re here? So everybody good? Yes. Why not? Let’s refresh. Yes. When? Although I think now everybody should know. Yes. So if you don’t know and you’re wondering what we’re doing, all we are, you know, Luca and Mina, we’re from Nativevideo. Yes. And specifically we’re talking about our main product, which actually names our company, which is called Vinton. And it’s an I know ticker. Now, we spend a lot of time talking about AI in general whereas you know, the Salesforce ecosystem, because we work in that ecosystem. We’re native to the platform.

So we talk about a lot about, you know, what’s happening in that world. We don’t spend enough time maybe talking about ourselves. So we take an opportunity at once a month to dig very deep into our own products. And we select the feature that our clients love, and we talk about it. And today we’re talking about all the different options of like language models that we give – LLMs

Yes, very AI centric, but the options that we provide and why we provide it, what our clients are asking and why they like it, and all of that. Right? Yeah. For anyone who doesn’t know how we use a large language model, maybe we want to tell them how we use it. So if you go to the Vinton tab.

Let me let me share my screen. So I think I know the here present share screen I won’t look at that. Look at this. And I’m sharing my story. So we’re actually in the admin panel. But what you wanted me to show is, this. So this is the end result, of an AI noteaker native to Salesforce.

It drops, conversations that are captured. This is a an in-person conversation. So only audio. This is, what is a there’s a bunch of, video conversation. You can see we also captured phone conversations and the voice notes, and we drop them in Salesforce, both in a library like this, but also at the contact level, the account level, the opportunity level, and so forth.

Now, in order to do that, we captured the recording and we’re going to show you an example. And then we extract the transcript and then we pass that transcript through, a specific set of processes that are all relating to a set of prompts, and they are served to and all of them. So we use generative AI, to create that.

And, you know, you are, at the deep end of our, configuration. So we’re, we’re looking at, you know, which region this, particular demo org is set up at. And, you know, what are the solutions that we use? We talked about transcription Deegram, Whisper, Amazon are available. What we are interested in is, is all the way at the bottom here, which is, which is the insights engine that you are using and the options that are available.

And the ones that we want to talk about today are right here. So there’s a couple of OpenAI native accounts, your account, there’s a couple of OpenAI that we want to talk to you about. And importantly very recently we added also, Gemini, both our account with Gemini and your own account. Yes. Now, why do we do that?

Because as an organization, you can choose between different insight engines that you want to use. Some organizations trust Google more, others trust OpenAI more. That’s, so we feel that it should be your choice which Insight engine you want to go with. And we should not sort of limit the options for you. If it’s something that we can configure for you, then we would obviously want to give you that.

For that insight engines and Gemini and OpenAI are the ones that are best in the market at the moment. We can add more as we go to it because it’s. Yeah, we were talking about the Anthropic. Yes. So we and then it’s really for coding. But you know, this is this is kind of ground level. Most of the things that we do with AI, right.

Because we are elaborating text out of another piece of text. So it’s, it’s easy. But, as, as you were saying, our clients have preferences. And so we’re really talking just to put it, you know, to ground it, we’re talking about these options being the, default option right now for all our clients. Then we give the option to our clients to actually switch keeping the same engine running OpenAI.

Yes. But their own account. And sometimes they have a, fine-tuned version that they want to plug in here, you know, kind of use all the learning and the reason why. And the main difference that our clients will see is that with our approach, so with our own native account, we have commitments to our clients that there is no retention of data and no data is used for training.

That’s the reassurance that we give our clients. Sometimes when they switch to their own account, what they want happening is exactly the opposite. They want their own OpenAI account to learn about them and to keep and re-use some of the information that they’ve, used. You know, they even expose, to fine tune. It’s fine. And sometimes some of our, customers have OpenAI an agreement with OpenAI already, and they would rather use OpenAI across all the different AI systems that this write this, this, that you’re referring to me now is actually, governance, you know, approach rather than necessarily, you know, preferring one or the other or expecting even different results. Their, stance is more about I want to be in control directly with an agreement in place directly with the service provider, in this case, an AI generative AI service provider with an agreement in place that, you know, keeps me safe.

And if anything happens, I don’t take it up with, the Nativevideo team, with Vinton. I actually take it up directly with the system that the intern helps me plug into, which is my own, OpenAI, typically an API account. Now, recently we’ve introduced and I’m jumping all the way down here, we’ve introduced the same exact thing with a default solution that we could switch on for using Gemini or, you know, the Google generative AI, within our own native account.

So again, Nativevideo has an agreement in place of zero retention, zero data used for training with Google for their Gemini. And we can plug to that right? Yes. So if you could just click on this let’s see what we do. Not so so and and then you click on Save. And then I click on save.

Are you sure you want to change configurations. Yes I think so. Yes okay. So we just like that updated our our media configuration. Yes. So now Gemini. Now the insights would be generated using Gemini. Yes. And maybe we could do a little bit of example where we record a call for this org. And another with OpenAI and before.

Yes, before we do that I wanted to explain these to in the middle. Let me zoom back in so that everybody can see what I’m talking about. So when we talk about these two options, the approach is slightly different. Yes. Right. So the reason why we introduce these additional options is to enable a local version of OpenAI. And then you know if earlier it was about governance this is more about compliance and data residency.

Correct. Some of our clients. So so our clients don’t want their data that is generated in a region of the world. And we have, you know, three main regions the US, Europe and Australia. Yes, Australia. Exactly. So for for those regions, clients in that region may not want or may have an agreement in place with, you know, because of compliance, because the agreement that they have with their own clients that their data will not be processed, outside of the region where they reside.

And so we can now localize the OpenAI AI stance and put it in an Azure machine, a, you know, server that runs and, enables therefore, the data to be processed with the full extent of, you know, what you should expect from. Yeah. So this is basically localizing your data. So if you’re in EU your data never leaves EU because we know that at the moment OpenAI only operates directly from the US.

They have their data center. But what we give you is an option to make sure that all of your data is processed within the region that you have selected. Yeah. And if you’ve been following the news, you’ve seen that actually, just earlier this week, OpenAI, launched their first, yeah, full open source. And the the aim, the ultimate goal is exactly the same localizing, you know, making sure that, the full extent of a generative AI can run, but it can run locally.

In the case of the, open source, it would actually it could actually run locally on a local server or even on a local machine on a local computer. You know, they they released the very small version. This is actually one level up because it’s the full OpenAI version, is not the open source. It’s the latest model. Yeah, but it will run locally on a machine that we can manage on your behalf.

So that will be yes or natively account. Or you can actually have a fully personalized one that you run. And again you own the relationship with OpenAI. And sure, so and so in essence your data never leaves your region. Yes. So that’s that’s that’s it’s the end to end processing that you want. Right. So you want, you know, complete like a residency.

Now we’ve just switched to Gemini and we are ready to do to do an example. I just wanted to make sure I have two orgs open here. One is the demo org and one is a training or, and I don’t remember if the other one is on Gemini as well. So we might want to put this one on OpenAI.

What we want to do is to show you, you know, what it means to, you know, again, not expecting dramatic differences. But we will, we would record that an audio conversation from a mobile phone and from, the the Vinton. So this is the training. Yes. So we’re going to record from here, and from my phone.

So give me one second. Is this still running? Okay, so we have 2 or 2 different orgs and, you know, one is from my mobile and the other one is from this desktop. And we are going to record the same conversation. So one is here and the other one is here. Okay. It’s going to be an in-person meeting.

Right? Mina one. And you know, when you do. Oh what is that? Your. Oh are we allowing. Are we sure. Yeah. Oh it’s disconnecting. So what’s up. It’s easy. Is it conflicting? Yeah. No, it’s your desktop is trying to use the microphone of your mobile phone phones that. Oh. So but you, you allowed it. We. So let me know because now it says that I cannot use my microphone here because.

Yeah, I want to use that. No. Yeah. No. Okay. Demos. Demos. It’s because I’m sharing. So this is let’s see if I will do it from scratch. I can have access, but it looks like this now has access and I can’t. Well, let’s see if it works. Guys I think, you know, we can start recording at the same time on both.

So I’m tapping on my mobile phone. Just so you see it. It’s, you know, consent. Are you consenting to being recorded? Mina. Yeah. Yes. Okay. Now, this is just an example, of what an in-person conversation could be regarding labs. So what are the main differences between Gemini OpenAI? Why one or the other? Yeah. So on paper, there are a lot of differences between Gemini and OpenAI.

They are token sizes greater than open Gemini’s, token size is greater than open eyes. So your answer or response would be more elaborate, more descriptive. For Gemini okay. Oh yes. Yes. So you can give more information upfront and get more. Exactly. So if you would like to generate reports out of a call, then Gemini would be the preferred option.

Or if you want precise, in a summary out of a call, then you would probably prefer OpenAI. OpenAI is more emotionally intelligent. Gemini is not as much emotional intelligence because of the training that it has. Emotion. Are you talking about emotion and AI in the same sentence? If it’s mostly trained, yes. But on paper there are a lot of differences between the two models.

But when we have tested it ourselves, we did not see a lot of differences. And apart from the output being more descriptive. Okay, okay. So we’ll see what what this this looks like there’s also a speed difference. But I don’t think we are going to notice. So Gemini is a bit slower as compared to OpenAI. That’s what it says on that model that we use.

What are the models that we use. So we’re using Gemini Flash 2.5 and OpenAI 4.1 which is the latest one. Okay. Yes. So we’re using both. So we’re not using reasoning because that will be very slow. Yes. Will be very more accurate. But we don’t need the level of accuracy. Also because I for a summary or something, when you will, you want a summary out of a call.

You don’t want insights to be reasoned. So it’s not an analytical where you want you want to have reasoning in the summary. You just want to have bullet points that you discuss from the transcript. You don’t want AI to go into the email, just rather than being. Yeah. So maybe in future if we add something that is related to reasoning, we’ll also use the reasoning model.

But at the moment, the insights that we generate reasoning just did not, set in the use case that we have at the moment. Got it. So that’s why we didn’t choose it. Should we stop it and see? Okay. Let’s, you know, stop, stop. So what we are hopefully going to show you is so I’m going to approve on here and see here.

So you’re probably familiar with how Vinton works. This is an example of an in-person conversation. The same thing typically happens. And you see I’m done. So I got my thumb up and now, there will be some processing. Oh, nice. The one in front of us is taking its time while the other one is processing. So this is actually not related to the, the LM.

It’s rather related for sure. To Salesforce and, you know, something hanging in one of our dev or, or demo talks here. Now, what is happening is the audio is being transformed, you know, quickly into a transcript. And that’s the transcript that when we are serving now, ideally to, to different, to different, large language models.

Right. So one will be Gemini and the other one would be OpenAI. Now this is granted that if, you know, the finalizing actually finalizes on the one side of the training org, which is right here, and you can see Mina working, looking behind the scenes to see what’s happening. The, the the prompts are relatively straightforward.

You know, you if you’re familiar with the intent, once again, you you will know that you know, we’re asking for a summary. We’re asking for, some, understanding of the key topics, that were discussed during the conversation. We’re asking it to prepare, a set of minutes. So a number of, different, solutions, different, blocks, of the call, with distracted by the, you know, the processing in front of us typically is, you know, between 20s and 40s.

This is taking definitely much longer to, now, because of that, I was saying, because we’re only, you know, kind of going really at that level, text to text, the expectation is that the, there would be relatively little differences, between, the, the output, that we’re seeing that we will see eventually. We’re going to do another one.

You know what I’m going to do another one. Yeah. Let’s do another one. We can I’ll, I’ll just refresh this page. So we’re going to reach that and, and now we’ll refresh this. And I will do another one and we’re ready to do it again. A little below the direct Italian. The or, you know, in English, probably the curse of the demo.

So let’s do that again. And if it doesn’t work, then yeah. Next time. Now we are recording again. Yes we are. And, I think this is. Yeah. Still an in-person meeting. So we’re expecting, that Vinton will also understand that there are two people speaking. So the second person is me. Yes. Hello. Okay. Hello. So. Well, you know, should we just go ahead now?

You know, we’re totally ruining all this demo. No, no, let’s do something. Okay? Let’s, we can talk about it, or we can test it with an uploaded file, and I can upload a file, and we can check the insights from the uploaded. Yes, yes. I’m wondering, you know, what? What is happening with my with my demo environments here?

Yes. You know, I’m it’s not working. Okay. Now, well, I uploaded in a while while you you look at that, I will stop this and I will go back to processing. So 50s is enough. And this is a proven you know, we saw that finalizing earlier. This is uploading progress. So mobile or I think this demo on mobile and training.

So definitely training is a bit stuck right. Well permission sets did you assign me to. Yes I did I it if you actually have a new user. So we create the yeah we created the new user on the fly and I think you know this is yeah. This is now thinking about both of them. That’s the demo.

Oh that’s okay. Just to what I’ll do is I will upload a file. I’ve just uploaded a file. Okay. And I will upload it. We have a success. Okay. Oh, look at that. And processing. So now. Yeah. So what I was describing earlier. Right. So we have a media transcribed now we’re serving it to the LLM.

And we’re expecting an output. And the output again eventually will be, will be comparable but likely different. You know, we’ll see how different it is. It’s run by Gemini or. Yeah. So you’ve been using Gemini for a while. Are there any differences that you’ve noticed between Gemini and one? This loads we can talk about? Yeah, I think I think what you said, makes sense.

So the, the I have noticed. Oh there we go. The, the we have the previous one ready. So I can actually let me see because I think I have both open here. So I. Yes. So I have both arguments, as you were saying, I think Gemini seems to be, definitely more, more articulated or more.

Yeah. So Vinton Tool Evolution is the second one that we recorded 51 seconds. And then, this was the, the previous one that we, we recorded as well. But if we go to the Vinton tool evolution, this is what he made of our conversation. And, so this is the demo. So this is the original, OpenAI.

Right. So that’s the the 50s one. And then if we look at this, you can see this is much shorter, actually. And it’s, Oh, it might have not picked you up as a, as a second person speaking because I was, I was looking into everything. And the same thing you talking about. Right, right. But we have I think we have Gemini on demo and we have, but we have an eye on the training.

Okay. So I can see the difference. So this is, this is Gemini. Yeah. So you’re looking at. Yes. That’s something that we’ve noticed as well I think because Gemini has a token length of context window, which is 2 million, but OpenAI has 128 K. So you can see it over here as well. So if I, if I go to the previous one is even more elaborate.

Exactly. Rationale for avoiding reasoning model, model selection and speech consideration. So it actually goes into the depth of, you know, highlighting why, you know, the differences that you mentioned about and just, you know, very interesting. Exactly. And then let’s see if you know, anything changes on the the follower female I looks again very elaborate. Yeah. OpenAI 4.1 and you know goes into the level of detail.

So the Gemini providing I have also assigned you two goals on both of these orgs that I’ve just uploaded. These are video calls everybody can see. So if you can refresh the page, this one. Yes. Actually. So in a video call scenario, we can see how a real life video call that could be a conversation with you and a client and how the insights about Salesforce call for the activity.

I mean, two people, that, you know, in a, in an, actual video call, there will be understanding who the people are because we would use the, the participation, you know, people join. This is an uploaded file. We don’t have a calendar invite. So we don’t so we don’t have information. But you can reassign the speakers.

So this is, let’s see the OpenAI. Yes. This is OpenAI. So actually with this. Yeah. You can see it’s, it’s more a lot more elaborate. I think it’s a it’s a preference. Interestingly, the prompt is exactly the same. Right. So we’re using exactly the same prompt. Okay. Can you check the follow up email for the let’s see the follow up email.

See. So click it looks analytical with bullet point. Bullet point with point you know next steps action. And whereas on here a follow up email for this goal much shorter. Yes. And the bullet points. But yeah. And if you read the tone of it it would be more user friendly. And when you go to Gemini will be more, robotic like you can, you can say it’s more to the point with bullet points and with.

Yeah. And it has bold and everything. So it’s more formal in that sense. But OpenAI just does the job. It’s the same prompt in both ends. But, you know, someone could say, but isn’t that controlled via the prompt? Yes. But if you’re using the same prompt, then it comes down to the LLN or large language model that you’re using.

Because if we have the same prompt for both. So if I want to use Gemini. But have a less, you know, maybe shorter or less analytical, I need to stress those points and say, look, yes, use an informal or not too analytical or. Yes, keep it short. Like make sure that we recommend that we box the.

So when you’re using Gemini then when you’re writing a prompt for Gemini, you should know that I need to specifically mention all of these bullet points, and I need to enforce them in the prompt for me to have a specific output. If you do not do that, it would use this because we have the same prompt in both of these things.

Right. So it’s it will be more elaborate, more descriptive. And that absolutely makes sense. I’m cautious. We are, you know, four minutes, from time flies when you’re on a LinkedIn like, doesn’t it? Time flies when when your, your demo, you know, goes on. These also with, with a couple of things but but but but questions.

So this is time for question. I’m actually looking at our love to see questions. So their cost implication to and can we assign assistance to OpenAI, for example, maybe depending on the template type. Okay. So that’s and that certainly the the cost implication. No. Meaning both options are available and they can be interchanged.

It’s really down to the, you know, when it comes to our, account, of course, it’s down to the preference of the client. We would be able to recommend the right one if needed. Now, on the, on on the side of the, your own account, then the cost implications are one. Yes. Worry different. Is that the regular?

You probably. And then Nicholas ask. Then we assign assistance from OpenAI, for example, maybe depending on the template type of meeting. It’s an interesting question. I will need to talk to you, Nicholas, and understand a bit more about assistance and what you mean about assistance. So the use of templates, is independent from the, the user or a specific lab.

Right. The maybe if I interpret you correctly, you’re relating to templates, which is a way to define a context in you know, for those that look at all different contexts and then based on the context, apply specific instructions. So I think you’re asking if you can direct these specific template to be elaborated by a specific I mean another.

That’s a functionality that we don’t have I don’t have it’s quite elaborate, you know would be interesting to see. Yeah. We if you know what the real impact would be. But yeah, I think what we this means is that if I have one template and I need to generate the insights from that using Gemini, and I have because I need to political line for instance.

That’s interesting. That’s an interesting one. Good question because maybe I’ll follow up directly with you for that. It’s a guy who’s been doing OpenAI enterprise account to start with, but they wanted to switch their on Gemini at a later date. So yeah. Very good question. Yes. Very much like with everything else that we do is very modular.

So it can be we, we have a few clients actually, that, are working on their customized version. Lem, you know, typical, approach. You know, those are big projects, so they might not be ready to switch on their own. So the version that we’re looking for is exactly what the question is asking. Right. So and they start with ours.

And then at any point in time they switch to different doctors who are with us for a very long time. And they are with us on their AI journey. And now they’ve had their own enterprise account, and they’re wondering if they can shift and switch. So we do have the opportunity for the, if you would want to now switch to your, Insight Engine, you can always just an endless, very good, very good question.

Right. I think we we are at the half hour. Yes. I hope this was, interesting. We we touched on alarms. We touched on why we implemented different options. And, you know, we keep it open going forward. We are always on the lookout for better transcribers, better gen AI, creations, better ways of serving our clients.

We localize compliance, compliance solutions. So if you have questions about that, you know, don’t hesitate to reach out. We’re very much open and interested and excited about all these themes as we continue building success for our clients, one conversation at a time, right. All right. So what will we talk about next? We don’t know. We don’t know what we don’t know.

Interesting things in the pipeline. We are direction. That’s really exciting. Everything from automation to making sure that. Or from automation to personalization, we’re thinking about everything. So that’s the direction that we’re going in and making sure that we support everybody working. Was using a note taker. So we would want to do all your work for you every week and said in the past, have it gone away.

So that’s basically what we want. Everyone who has a conversation with relevant. Yes, yes, I use a Salesforce because we’re near Salesforce there. We say that I think okay. Well thank you everybody. It was a great session. Hope you enjoy it and we’ll see you next time. See you next time. Bye bye.