00:00:04: Welcome to the Human and Technology Podcast.

00:00:07: This podcast is for anyone who develops, distributes or uses technology For all those that always have a feeling that technology overwhelms or dominates them... ...for everyone wants know how to deal with technology in everyday life.. ..for anyone want so understand what technology does to us And How we can get our lives back.

00:00:32: this podcast.

00:00:33: as for those who want to make technology sexy.

00:00:37: All the product developers, designers, UX-UI professionals, product managers, CTOs and CEOs... And it is for you!

00:00:47: My name is Dr Peter Oreska.

00:00:50: my friends call me Dr Peter.

00:00:53: I am your host ...and i'm happy that are here.

00:01:03: Hello hello and welcome to this super fresh a new episode of The Human technology podcast.

00:01:11: Those of you that are a bit attentive, I know all our highly attentive will have realized that i did not publish an episode last week although um...I should've done this to keep my bi-weekly rhythm.

00:01:30: Last Thursday we had national holiday here in Germany and Friday Saturday I was in Aachen for my lecture there at the university.

00:01:43: So, decided short-term to not record and publish an episode but move on this week!

00:01:51: And from this weekend... ...I will get back into the bi-weekly rhythm.

00:01:56: so two weeks from now you'll hear another episode.

00:02:03: In our last episode three weeks ago, I talked about the greatest challenges in automotive HMI design.

00:02:13: And that was an episode providing and overview of.

00:02:19: the industry is currently facing automated driving artificial intelligence software defined vehicles.

00:02:31: The new Chinese HMI philosophies, the new Chinese OEMs and basically the question of how humans can still be meaningful meaningfully integrated into increasingly complex technological system.

00:02:54: That episode was intentionally designed as an introduction the starting point of a loose series in which I want to explore some of The individual aspects mentioned above step-by-step and To start with How any topic is better suited than multi modality because at first glance the direction seems obvious more interaction channels, more possibilities.

00:03:32: More natural interaction speed touch gestures haptics gaze control already official intelligence based assistance and modern vehicles are increasingly evolving into multi-modal systems.

00:03:53: The term NUI Natural User Interfaces, I haven't heard this for quite some time.

00:03:58: can anyone make me aware of?

00:04:00: This is still on the discussion.

00:04:01: a couple years ago that was the big thing.

00:04:04: natural user interfaces, NuIs... Haven't heard it for quite sometime.

00:04:12: maybe i'm wrong but this what its heading for intelligence driven, multimodal interaction between humans and technology.

00:04:24: And this is exactly where the real problem begins.

00:04:28: many current concepts appear less well integrated like well orchestrated.

00:04:36: you will hear that term quite sometime in this episode quite orchestrated user experiences.

00:04:44: Today, they are more like a collection of technological possibilities.

00:04:49: New modalities are often added without truly questioning what concrete value that create for humans.

00:05:02: and therefore the central question is not which modalities can we technically integrate but rather really makes sense for humans?

00:05:15: in which situation.

00:05:19: And exactly at this point, multi-modality suddenly becomes far more than just a technological topic.

00:05:27: it is about context and attention about cognitive load but trust safety ultimately the question of what interaction inside the vehicle should feel like because the best modality is not necessarily most modern one.

00:05:52: It's just a one that creates the biggest value for humans in specific situations.

00:06:06: Five topics on todays episode, number one multi-modality is another feature its system architecture.

00:06:15: Number two has the headline context beats technology.

00:06:20: Three, their return of the analogs something I'm talking about in almost any of my keynotes why physical interaction is not disappearing.

00:06:31: number four will be artificial intelligence as the next step for multimodal systems.

00:06:38: And finally number five, the real challenge creating value instead of adding features.

00:06:49: Let's start with number one multi modality is not a feature.

00:06:54: when people talk about multi-modality in an automotive HMI today it often creates the impression that the primary focus of available interaction channels.

00:07:13: Touchscreens are to extend it with speech, gesture controllers added.

00:07:18: haptic feedback is integrated AI based assistant as opposed to connect everything so its feature driven.

00:07:28: This exactly where one the biggest understandings for modern HMI development begins because Multimodality does not automatically mean better user experience.

00:07:41: In many vehicles today systems emerge that feel more like a collection of individual technologies than a consistent interaction concept, different modalities exist side by site but they do not follow a clear common logic.

00:08:04: Speech works differently than touch.

00:08:06: Haptic feedback confirms different things.

00:08:09: then the visual interface functions are accessible through multiple interaction points without any recognizable added value for user, driver or people we're developing.

00:08:27: The result is often not simplification but additional complexity.

00:08:36: Therefore, the real challenge is not to offer as many modalities as possible.

00:08:43: The challenge is to orchestrate them intelligently because different interaction forms have different strength and weaknesses.

00:08:55: Speech can be efficient at work without visual attention.

00:08:58: Haptics can reduce attention demands and support muscle memory.

00:09:04: physical controls provide orientation and safety, they can be used blindly.

00:09:11: Touch works well for explorative or visually driven tasks.

00:09:14: if you have a standing vehicle, a standing car and you want to program some playlists touch is nice at touchscreen.

00:09:24: At the end The final question is not which modalities do we have.

00:09:30: but Which modality creates the greatest value?

00:09:34: The biggest value for humans?

00:09:36: in which context of use.

00:09:40: That is exactly why multi-modality becomes an architectural task, it's no longer about individual features but designing and integrating.

00:09:52: orchestrated system that works consistently reduces cognitive load provides situation dependent support Organizes interaction as naturally and efficiently, as somehow possible.

00:10:13: The quality of multimodal HMIs is not determined by the number of input methods but how well they guide humans through complexity without appearing complex.

00:10:32: And I think that's a core sentence guiding humans through complexity, without appearing complex.

00:10:41: I mean there's tons of complexity in automotive HMIs and automotive infotainment systems.

00:10:46: that is full of complexity.

00:10:51: That making it look easy Making It Look Like It Is Super Simple And Easy To Use.

00:10:58: Thats The Art We Want to Do.

00:11:03: Number Two Context Beats Technology.

00:11:09: One Of the Biggest weaknesses of many multi-model HMI concept lies in the assumption that there is a fundamentally single best form of interaction.

00:11:24: This Is not The case In vehicles because Automotive HMIs do Not exist in A static stable context Of use.

00:11:38: They operate in a highly dynamic environment.

00:11:42: speed, traffic stress driver distraction weather fatigue interaction social interactions inside the vehicle.

00:11:53: All those are components creating a context of use and this is exactly why they same modality can work excellently perfectly nicely smoothly in one situation be completely unsuitable on the next.

00:12:14: Speech is a good example of this.

00:12:18: During navigation or while composing short messages, speed interaction can provide enormous advantages because the driver's eyes can remain on the road hands-on wheel.

00:12:31: so it seems to be comparably safe in noisy environments with poor recognition qualities are some more complex tasks.

00:12:43: Speech may create frustration, it might not be the suitable one.

00:12:50: Another example are touchscreens.

00:12:52: they're excellent for visual exploration and more complex information presentation.

00:12:58: at the same time however They often creates visual and cognitive load high-visual cognitive load while driving.

00:13:09: physical controls on the other hand may initially appear a bit old fashioned But they provide one crucial advantage.

00:13:18: They enable interaction through muscle memory and haptic orientation, you can use them blindly.

00:13:27: There's no conscious visual attention required necessarily And therefore the real challenge of multi-model system lies in understanding context.

00:13:39: not every task requires the same form of interaction.

00:13:43: Not every situation allows the same level of attention, not ever user prefers to say mechanisms.

00:13:51: There are people out there that I want visual others that are more talkative or acoustic oriented.

00:13:58: and we can with a multi-model HMI in car.

00:14:02: you can pick them all up provide them right means off interaction.

00:14:13: good multimodality does not mean maximum freedom of choice, but situation dependent optimization.

00:14:22: The goal is now to offer as many interaction paths as possible.

00:14:27: in parallel the goal is to provide humans drivers In a specific situation with exactly the interaction that is most efficient.

00:14:42: That's the safest.

00:14:43: That is most easy to understand.

00:14:48: The least that it creates the lowest cognitive load in the driver.

00:14:59: To summarize this part, an automotive HMIs context almost always beats technology.

00:15:09: Be aware who your user what they do and their plans are and reflect this in the HMI, make it a multi-model.

00:15:24: And do not focus on geotechnology.

00:15:36: Number three of today is the return of the analog for many years.

00:15:42: Automotive industry followed an apparent clear logic.

00:15:45: The future's digital more displays more surfaces, fewer physical controls.

00:15:51: Yeah that the cockpit was increasingly turned developed into a software centric space and to tablet on wheels onto smartphone or wheels inspired by these consumer products, consumer electronics.

00:16:09: That was The Big North Star for couple of years.

00:16:15: This development has limits.

00:16:18: This is exactly where it comes in, what I usually describe on my keynotes and other podcasts as the revenge of the analog.

00:16:33: That's just where this coming from – we are analogue beings!

00:16:36: In a fully digital world….

00:16:38: We are not really at home.

00:16:43: Technology has changed rapidly.

00:16:46: We humans have basically remained the same in the past ten, twelve, fifteen thousand years.

00:16:54: Our brains our cognition... ...our haptic skills are still based on millions of years of evolutionary development and touchscreens have never been part of our history until twenty years ago yeah?

00:17:11: And we ever ten thousand, fifteen thousands, twenty thousand, seventy thousand years old history as humans.

00:17:18: and ninety-nine percent, ninety nine point nine percent of the time we have not used touchscreens.

00:17:26: And this explains why physical interaction continues to possess... ...to provide enormous advantages on vehicle dashboards.

00:17:43: Physical controls provide characteristics that purely digital interfaces do not have Haptic orientation, muscle memory operation without permanent visual attention immediate mechanical feedback feeling of control and safety.

00:18:00: And all these things are critical inside the vehicle because driving is not like a relaxed smartphone use on your sofa at home or in another train or an aircraft In a car, if you're the responsible driver.

00:18:20: You have a highly dynamic environment and... ...you also experience a constant cognitive load on The inside.

00:18:32: here is the future.

00:18:34: probably lies neither in complete return to the analog which Is not possible with all functionality.

00:18:40: we haven't vehicles Nor does it lie in hundred percent digitization.

00:18:48: So we have a hybrid future.

00:18:49: There is, there's a sweet spot somewhere in the middle and next generation multi-model HMIs will probably succeed exactly where they combine digital flexibility intelligence software and smartly strategically applied physical interactions.

00:19:16: And this changes at the end of day, the role of physical controls.

00:19:20: They are not disappearing they're becoming more strategic so that every function requires a button but some functions benefit from a button.

00:19:33: therefore The real challenge here is Not to eliminate as many physical elements.

00:19:45: The challenge is to understand where analog interaction creates real user value.

00:19:54: So the best technology, often not one that feels least like technology and with all our overwhelming technologies we have that we interact constantly with an interaction that feels less technical will probably feel more real and less digital.

00:20:21: Number four, artificial intelligence... No one gets around artificial intelligence these days.

00:20:29: What role does it play as the next evolutionary step in multimodality?

00:20:34: In multi-modal HMIs AI changes multimodalities fundamentally for first time and technology history systems are emerging that do not simply react to inputs, but begin to interpret the context we're using it.

00:21:02: Who is sitting in front of the HMI?

00:21:08: Where are we driving?

00:21:10: What conditions do we have

00:21:11: outside?".

00:21:12: Plus that AI can predict behavior, personal behavior.

00:21:15: What is he or she gonna do next?

00:21:20: And then dynamically adapt the interaction to contexts, two personal preferences... ...to abilities that users have.

00:21:31: Multi-modality evolves from a static combination of different input and output channels into an adaptive learning nicely orchestrated system.

00:21:49: I believe the potential is enormous, AI can anticipate user intentions.

00:21:55: what will he or she do next?

00:21:57: AI can select suitable interaction forms depending on situation.

00:22:03: Yeah, if I'm driving on an open sunny Autobahn in Germany with not too much traffic.

00:22:09: I can of course rely more On the visual channel for interaction Compared to driving into a city?

00:22:17: I don't know.

00:22:17: and rainy weather and it's getting dark And so we have a serious problem there.

00:22:22: So I cannot any.

00:22:23: I need my eyes on the road they're.

00:22:26: And then so AI can detect this and find out.

00:22:29: okay how much information is needs to be visual on how much can we put as much possible.

00:22:37: I can prioritize information.

00:22:42: is it really necessary?

00:22:43: To receive a call when i am in this situation, adjust the scribe driving into city foreign City with rainy weather and yeah do you need to get a call there or cannot?

00:23:02: but just put it onto the mailbox.

00:23:06: So at the end, AI can help to simplify complex systems and make interaction feel more natural.

00:23:20: just an example a modern vehicle could recognize that driver is on heavy cognitive load.

00:23:29: high speed driving makes precise visual interaction difficult.

00:23:35: certain Information is currently more important than other information and speech could be in this situation, more efficient then touch.

00:23:46: In this specific situation so that the core idea behind This is?

00:23:52: The interface adapts to the human not the other way around.

00:23:56: we need to know everything about humans to design technology.

00:23:59: And there's one of my those are all my claims on this claim as reflected here at this point That with AI adapting HMIs, the interface adapts to the human and not in that other way around.

00:24:17: This of course creates a new challenge.

00:24:20: AI can Not only increase the capabilities of this system but also the perceived complexity And it can create unpredictability.

00:24:33: Yeah!

00:24:33: So the Moray System interprets anticipates automates prepares decisions, the more important become things like transparency consistency trust control over controllability.

00:24:52: And I'm fully aware and i don't have a clear single an obvious solution here that there is a sweet spot somewhere.

00:25:03: And of course, we can do it fully adaptable and then totally confuse the user or make it not adaptable.

00:25:14: Both are probably not the best possible solutions.

00:25:18: but to be honest I don't have a clear idea at the moment where exactly that point may be.

00:25:28: just because interaction feels more natural It does not automatically Any, any how better and easier.

00:25:36: So an AI system that constantly makes suggestion interrupts users make false assumptions That can quickly become more exhausting than a classic interface Constantly changing always adapting.

00:25:51: being faster then I am being smarter Then I'm letting me know that... ...that sucks!

00:25:59: It's not what we want.

00:26:01: Therefore, the central challenge is not to use AI as an solution in itself but a tool for reducing complexity.

00:26:13: So AI-supported multi modality should support instead of dominate.

00:26:19: it should give relief Instead of being overwhelming create orientation instead of uncertainty.

00:26:28: The future of multi-model HMIs does not lie in maximum technical intelligence, but meaningfully applied well orchestrated human centered intelligence on dashboards.

00:26:49: Number five creating values instead off adding features very critical here and the end.

00:27:00: Every discussion about margin modality leads to one simple but decisive question.

00:27:06: What value does it create for users, for drivers?

00:27:11: For humans?

00:27:14: because this is exactly where the central danger of modern automotive HMI lies.

00:27:20: new interaction forms are often introduced Because they're technically possible.

00:27:26: They appear innovative.

00:27:31: Speech, gesture touch haptics AI gaze control.

00:27:35: that sounds pretty cool but from the user's perspective this does not automatically create a better use experience.

00:27:47: if modalities are added just edit brainlessly add complexity often increases instead of being reduced.

00:28:00: The user then does not receive a simpler system but just more paths, more options.

00:28:09: More rules and uncertainty.

00:28:13: users must learn which function works through which channel when speech makes sense When touch becomes necessary Which gesture is recognized And why the systems sometimes react differently than expected.

00:28:26: This Is Not Adding Value.

00:28:31: This is adding cognitive load.

00:28:36: Multimodality must solve problems, not demonstrate technology.

00:28:43: The central task of an automotive HMI Is NOT to appear as modern cool funky spacey whatever as possible.

00:28:52: Its task is to support humans better in concrete use cases In concrete context of using concrete uses usage situation.

00:29:03: And this means a modality only makes sense if it creates A real value, If it really contributes value.

00:29:15: so It must make the interaction faster safer simpler easier to understand More pleasant more focused.

00:29:26: All these are parameters that an improved HMI shall provide.

00:29:33: And if it does not achieve that, its probably not the user experience gain but what is called a feature bloat.

00:29:45: Inside of vehicle.

00:29:46: this becomes extremely critical because the user isn't sitting in front of neutral digital device or on his smartphone with a PC computer.

00:30:02: So the user is moving through a real environment at comparably high speed.

00:30:11: We still have fatalities in traffic, so attention is limited and mistakes have consequences.

00:30:19: people die and die in traffic And every additional interaction competes with The actual driving task.

00:30:28: multi modality must never become an Solution in itself.

00:30:35: The wrong question is what else can we integrate?

00:30:38: Many developers begin from a technological perspective, We have new sensor technology and we can recognize gestures... ...we have a new speech model that we can personalize the system.. ..we make the entire interior more context sensitive And I don't want to turn anyone down.

00:30:57: all of this interesting but it's not right starting point.

00:31:02: The better question is, which human problem are we solving with this specific feature?

00:31:10: combination of features?

00:31:11: Combination of channels.

00:31:14: With this multi modality and Which moment does this Modality really improve users life?

00:31:28: This can mean speech replaces the complex menu structures.

00:31:31: Haptics communicate as a faster warning with less distraction than screens.

00:31:39: A physical controller makes frequently used functions more reliable.

00:31:46: AI reduces information instead of adding more.

00:31:51: The display no longer shows what is technically available, but only what a situationally relevant.

00:32:00: And this... ...is the point when value emerges.

00:32:04: Value emerges through reduction not addition.

00:32:09: A good multi-modal HMI is NOT the system with highest number of input possibilities.

00:32:15: It's the system that offers to user the right interaction in a concrete situation while allowing everything else to fade into the background.

00:32:26: It sounds simple, but this is one of the most complex and most challenging design challenge I see at the moment because real value creation does not just mean adding something it means leaving things away.

00:32:45: Less visual searching, fewer manual layers.

00:32:48: Fewer operating errors, fewer uncertainty less need for explanation lower cognitive load.

00:32:57: and this is exactly where the quality of a good HMIL as it makes complexity manageable without constantly confronting in user with that.

00:33:08: from usability perspective multi modality must first function properly.

00:33:15: reliability, reliability, understandability consistency efficiency.

00:33:23: A speech control system that can only understand occasionally is not any progress.

00:33:28: it does not deliver anything.

00:33:30: a gesture control that produces false alarms false activations is not increasing comfort.

00:33:38: haptic feedback uh...that is unclear map doesn't create safety But all that is not even enough from your ex-perspective.

00:33:50: The additional question, how does the interaction feel?

00:33:54: Does it create trust?

00:33:55: Does provide control?

00:33:56: Does reduce stress?

00:33:59: Does support humans without patronizing them?

00:34:03: This exactly where it's decided whether multi modality experience as a real improvement or merely ask another layer of technical complexity.

00:34:16: humans remain the benchmark.

00:34:20: Therefore, there's a biggest challenge of multi-model agimizes to think consistently from human perspective that it is not the sensor or display and speech model nor feature set nor the CES trade show demonstrator.

00:34:39: The human in concrete situation means what does this person currently trying do?

00:34:45: How much attention how much cognitive capacities are available?

00:34:51: What information is really required?

00:34:54: which interaction can be safely performed at this moment?

00:34:59: what Can reduce cognitive load.

00:35:02: What can on the other side overwhelm, The person?

00:35:08: and how can we create trust?

00:35:12: And once We have answered these questions that we can design meaningful valuable HMI's and do multi modality in an appropriate way.

00:35:30: This only multi-modality is as only valuable if it does not create more interaction but better interaction, It must not serve to make technology visible.

00:35:42: that much served to me complexity less visible.

00:35:47: And therefore, the future of automotive HMI does not lie in addition to more and more channels.

00:35:53: It lies on the intelligence selection prioritization... ...and using this word again orchestration of interaction.

00:36:03: The best multi-modality is NOT recognizable because users have many possibilities.

00:36:11: it's recognizable BECAUSE at end at right moment users receive exactly what they need in this context as this specific personality.

00:36:25: All right, let's come to the end.

00:36:31: ultimately The quality of multi-model HMIs is not determined by a number of integrated technologies but By their intelligent orchestration.

00:36:45: This is exactly where one of the biggest misconceptions many current development lies.

00:36:50: Multimodality is often understood as the addition of new features.

00:36:57: One more input method, one more channel and technology demonstration.

00:37:03: True user centricity does not emerge through technological variety alone.

00:37:10: it emerges when different modalities interact meaningfully complement each other situationally Deliver exactly what humans need in that moment.

00:37:23: Not more, not less.

00:37:27: The the best modality is not the most modern one but the one that creates the greatest value for humans In a specific situation.

00:37:40: That's why multi-modality isn't a collection of individual features But an orchestration Of possibilities not to provide as many interactions as possible but develop a consistent, understandable and relieving overall systems for them.

00:38:03: The future of automotive HMI does NOT belong the vehicles with highest number of modalities... ...but those that best understand when which modality truly makes sense!

00:38:24: That's it for today.

00:38:26: Thank you for spending time with me.

00:38:28: I hope, that You are able to take something With You and do Something For Yourself That Will Be Forever!

00:38:51: Take care and stay healthy.