The “End-User in the Loop” report interview with Gabriel Karandyšovský

In an era where artificial intelligence is becoming increasingly prevalent in our daily lives, understanding how people interact with and perceive AI technology is crucial. Our recent global study sheds light on this very topic, offering fascinating insights that might surprise you.

The Scope of Our Research

We conducted an extensive survey across eight countries: China, the United States, Brazil, France, Germany, India, Japan, and Spain. With nearly 1,000 respondents, we gathered a rich dataset exploring trust, bias, AI system usage, and user awareness of AI’s potential and limitations.

What We Discovered

Our findings challenge some common assumptions about AI interaction. Here are the key takeaways:

• Function Over Form: Users care more about achieving their goals than perfect linguistic accuracy. They’re focused on results – whether the AI helps them accomplish what they need to do.
• The Trust Factor: Trust and transparency emerged as fundamental pillars of user engagement. Users need to feel confident in the systems they’re interacting with.
• A Reality Check: Here’s where it gets interesting – 77% of users have mistaken AI-generated content for human-created content. Yet, paradoxically, 63% still believe they can spot AI content accurately. This confidence-versus-reality gap presents both challenges and opportunities.

What This Means for Businesses

For companies developing AI-powered products and services, these insights are invaluable. The message is clear: focus on creating transparent, trustworthy systems that help users achieve their goals effectively. Success lies not in perfect AI, but in AI that perfectly serves its users’ needs.

Check our interview with Gabriel Karandyšovský below to learn more about the “End-User in the Loop” report and discover the full implications of our findings.

Stephanie Harris-Yee: So I am here with Gabriel, who is a researcher and a writer, and has recently put out this report around end users in the loop. So, Gabe, tell us, what is this report all about?

Gabriel Karandyšovský: The end user in the loop is a report that sheds the light on the end users, and it’s a subtle bit of wordplay on industry concepts, such as human in the loop or human at the core, which are attracting a lot of attention. And this essentially is our way of showing that the end user is equally important and often missing in decisions made by organizations. A frequent blind spot, knowing what users preferences are, what their behaviors are and this report dives into primarily how users engage with generative AI, whether they trust it, whether they use it and what they make of it and the. The aim or objective here is to help companies make better informed decisions when they are creating or developing products with functionality or putting out generated content in order to create meaningful.

Interactions and deeper interactions with their end users who are at the end of the chain and consuming the products and services.

Stephanie: Okay, so who did you survey for this?

Gabriel: So this is a global reports based on a global survey. We, polled nearly 1000 respondents in 8 countries. So, in alphabetical order, I think we have Brazil, China, France, Germany, India, Japan, China, Spain and the United States, so we had a, we had a fairly, sizable sample. We did a survey of thirteen questions that were aimed to dive into areas such as trust and bias.

Preferences, usage of AI systems, and awareness, whether users are aware of the potential limitations or limitations and risks and benefits of AI. So yes, a global report with a global panel of respondents.

Stephanie: Oh, that’s great. after all of that feedback, what was the biggest takeaway?

Gabriel: There were a few, uh, some of them more surprising than others, but I think the, one that stands out the most, and that is most conducive to discussion or that is the hope that this will have a broader effect is that, things that the language services industry is obsessed about, such as quality, as in linguistic quality, linguistic accuracy, do not matter as much for the end users.

What matters to the end users who are consuming content, so engaging with content in its various formats, but also engaging with Gen AI driven products is the, what matters to them is the end users Is their intent, what is their desired objective? So users come into any type of interaction with an objective.

They want to get recommendations. What have you and a lot of that is language agnostic. So, yeah, language doesn’t matter as much as functionality does and next to that come in underlying sort of questions around things like transparency and trust. If the user is supposed to reach or retain their goal, whatever it may be there needs to be the trust. And that needs to be explicit. That needs to be built over time by brands. So, yeah, those are two of the biggest takeaways.

Stephanie: Okay, so. I hear you say that, okay, language doesn’t matter so much to the end user. So if I am a localization leader, what sort of actionable thing can I do based on these results?

Gabriel: Well, first of all, it is important that I introduce a little bit of nuance here. I’m not saying that we should all quit our jobs and localization services are not important. They are actually important because there are … And this is not news. This is not novel or anything that our research has shown, but.

Um, there are contacts and industries and types of contents where factual and linguistic accuracy are really a question of life and death. And so absolutely, we still have jobs. So that’s that’s the first bit of good news. But otherwise. Otherwise, for leaders and decision makers of localization departments, there are a few things and what as to why this type of information and knowing what end user to want and how they engage with products is important.

First of all, this plays into sort of the ongoing discussions they are having within their organization that is pushing. Yes, we need to implement AI and use it for translating and generating content and knowing what users in different countries do, what their preferences are and how they engage with.

With AI generated content just helps to sort of be more mindful be creates more impactful user experiences for those countries. This isn’t this is just an example, but you wouldn’t be pursuing, a specific tactic or strategy for a market. If you knew that the users there prefer. Just human made content and AI can or shouldn’t be used in those, user interfacing instances.

That’s just one example that I’m sort of making up, but the knowledge of what makes users take in different countries and different markets is essential in order to make informed decisions. So, that’s one. Secondly, a lot of ongoing debate around the use or non use of AI in the language services industry is around questions of ethics, transparency and trust and, that is, those are discussions that are difficult to articulate because you sort of lack the underlying data, like, okay, on what basis do we make, do we decide whether we should, you know, put an AI label or made by AI label next to the content or not.

These should be decisions and discussions and having the data at hand saying like, yeah, users prefer or , would like if there wasn’t made by AI label next to the content, which would help towards transparency and trust. And Yeah. Lastly I, think this is just another, or the latest contributing piece to, as in this research report to making the case that researching and understanding your, user’s intent is extremely important if you’re making any kind of product decisions.

So it, sort of stands as a call to action for, brands of any, type and size. To invest the time to get to the bottom of what they’re what they’re targeted users would like, because that is what creates those mutually beneficial interactions.

Stephanie: Okay. So from this whole report and the different results that you got, was there anything that was really surprising to you?

Gabriel: So one of the more surprising bits of information we found through the research is that, humans make a lot of mistakes when consuming AI generated content. For example, 77 percent of our respondents claim that they have mistaken AI generated content for content made by a human.

That’s a pretty large number. We haven’t necessarily investigated the quality of those interaction as and they were good or bad. But, of course, you know, implicitly from there, you see that or can understand that for nefarious reasons. If you were to use AI for nefarious reasons, this poses a risk and the second sort of follow up data point is that 63 percent of users are still confident in their ability to distinguish content created by AI. So there’s a paradox here. On one hand, users are making mistakes and then they are saying, but that’s fine. Next time we’ll know. A few things are playing out here. Honest is our inherent bias at play that we think we’re going to do better, which may be the case, but also.

This speaks to the importance of having transparent and trustworthy systems. Making mistakes is human, but no one is saying that those mistakes won’t be repeated. And the inherent challenge of that is, from a brand’s perspective, is if the user is engaging with the brand, with the product, with the service in order to attain a specific objective, Well, that the path towards attaining that needs to be as clear and transparent as possible.

There, there is no margin to introduce error or misunderstandings or mistakes. And, um, yeah, there’s work to do in order to make, especially the AI type of systems and the content they generate as, as trustworthy as possible.

Stephanie: Right. Okay. Well thanks so much Gabriel for introducing this report to us and if you would like to read the full thing, just check out on this page, we have a download link and you can see it there. All right. Thank you.