Welcome to "AigoraCast", conversations with industry experts on how new technologies are transforming sensory and consumer science!
AigoraCast is available on Apple Podcasts, Stitcher, Google Podcasts, Spotify, PodCast Republic, Pandora, and Amazon Music. Remember to subscribe, and please leave a positive review if you like what you hear!
Dr. Qian Janice Wang is an assistant professor of food science at Aarhus University. Her interdisciplinary research focuses on the role of the brain sensory system and its connection to flavour perception and eating behaviour, in order to gain a deeper understanding of why people eat what they do, and to encourage behaviour change for a healthier, more sustainable lifestyle. To achieve this goal, she works to develop new ways to evaluate the consumer experience - from opinion mining to virtual reality to biometric measurements.
Janice completed her PhD studies at Crossmodal Research Laboratory at the University of Oxford, and holds a master’s degree from the MIT Media Lab. Her background spans computer science, interaction design, experimental psychology, and sensory science. She works with all food products but has a special interest and expertise in wine.
Twitter: @rbrainengineer
Transcript (Semi-automated, forgive typos!)
John: Janice, welcome to the show.
Janice: Thank you, John. It's a pleasure to be here.
John: Oh, it's great and wonderful because we have so many shared interests. I think it's really exciting to have you on the show. Kind of reading actually just over your biography. It's really neat to me to see how much out there is in our interests. One of our shared interests is virtual reality. And I know that you're really at the forefront of the virtual reality research within consumer science. So maybe we can start the show with you and bring our listeners up to speed on what you've been working on and what you find interesting within the kind of VR sensory science overlap.
Janice: Yeah. So I should actually start off by saying that my interest in virtual reality really comes from the fact that I have a bachelor's degree in computer science. And when I was doing computer science, I was actually focusing on computer graphics.
John: Oh, really?
Janice: So that's really where my interest in VR started. And then I started combining VR with food about three years ago. And that was because I met one of my collaborators whose name is David Zendle, who works at the University of York. We were actually both in a conference, like a conference association together where we were writing papers. I just happened to sit next to David. He is experts in computer games and virtual reality, but nothing to do with food. And we were just having one of these casual chats and I was like, so you a specialist in VR research? That's interesting. And I study how the environment changes you know food perception. So why don't we do some VR studies? And that's actually how it started. So I know that there is a new trend at the moment of using VR in consumer science, looking at, you know, if we have people drink beer in different environments, you know, in a beach, for example, how does that influence the perception? So I actually approached it from quite a different perspective because I wasn't really thinking of you know, if people change their environments, how do they change the taste? I was actually thinking about what kind of VR offers in terms of making things possible scenarios happen. And that's why my first VR study had to do with not changing the environment, but changing the food itself. So we designed a coffee study where we try to make the coffee look different. So we gave people black coffee to drink. And the point was, okay, what if the coffee looked like it was milky? Because traditionally there's been a lot of studies since the 50's looking at how food color changes people's expected flavor and expected taste intensity, and usually you do something like, oh, you colored this drink, you know, orange or green, and then they'll taste the green drink will tastes like lime, orange drink probably tastes like orange. But you can't really make black coffee look white. That's not really changing the the content. And the other thing that we wanted to do with the VR research was to actually have people drink or taste the food while they're wearing the headset. Because in most of the other studies that I've seen, please correct me if I'm wrong, people are wearing a headset and then they taste something, but they don't really see it in the VR. So there's a slight, you know disjointedness between what I see and what they taste. So with David, what we were trying to do really was to figure out a way of, okay, how do we preserve the continuity somehow so that if you're wearing the headset, you can still see the food that you're tasting. And that's how we were able to change the coffee color. So this kind of mixed reality platform that we came up was to actually give people a coffee mug in the real world, but also to make us 3D model of the mug in VR. And we put a tracker on the bottom of the mug so that when the real mug moves in the physical reality, it also moves in virtual reality. And this, we felt it would give people kind of a sense of continuity. So that when they're tasting the coffee in the real world, they can also see the mug in virtual reality. And then they also see the coffee in VR. And that's how we were able to change the coffee color.
John: Yeah, it's fascinating. So you're really studying this kind of multi-modal or multisensory integration?
Janice: Exactly.
John: Yeah. And so what did you find or what were the kind of key learning from this research?
Janice: So this research I did presented at Pangborn last year, and we have a paper in review at the moment. So what we found, it wasn't quite surprising, right. What we found was that the same black coffee without anything added, when people see it as light brown rather than dark brown, people perceive that as more creamy. And of course, it's not surprising because we usually associate coffee with milk more creamy, so it was more of a proof of concept that we were even able to create this experience. And I should mention one thing about the design, which is something that you never think about until you try to drink something in VR, is that we were using a headset called HTC VIVE Pro. And this is quite a involved headset. It's not like a Google Cardboard. This is big. And when you're trying to drink something while wearing a large headset is that you can't just drink from a mug because what you find is that the top of the mug is going to hit the bottom with your headset. And this is something that you really don't think about until you're trying to do it. So we had a really educational, shall we say, pilot testing experience, which is we figured out that we can't just have people drink from the mug. And we ended up having a design where people were drinking from a straw. Because when you're drinking from the straw, you don't have this collision problem with the mug and the headset. And we did end up having to also model the straw in VR so that you would have to come I guess the congruence with the physical in the virtual experience.
John: Yeah, that's fascinating. Well, there's bunch of places to go with this. Is there more that you would like to share as far as learning from the research?
Janice: So I think that really was the biggest learning is that when you're working with VR, sometimes things all seem obvious until you try to pilot test it for yourself and then you run into problems and then you kind of have to find creative ways to solve it. And I'll just mention one more thing, which is we're also at the moment trying to get away with only evaluating beverages and only being kind of stuck to this quite large VR tracker that we're trying to attach to the bottom of a cup. So we've actually been thinking about is it possible to do this with physical foods and of course, one way is to attach the tracker like to a small plate. But we are actually also looking at this technology called LEAP Motion. I don't know if you've heard of it. It's our hand tracking technology. And because you could also integrate the LEAP motion with the VR headset and you could end up with this kind of hybrid system where you could actually see your own hand in VR. And of course, it doesn't look like your own hand, but you could see this kind of skeletal outline of your hand. So we're thinking about doing that. And then you can actually, you know, track one's hand in VR and track which kind of solid foods interacts with. I mean this is now getting really deeply into computer science territory because we're talking about evaluating collision models of when the hand touches which food and then kind of sensing the touch and then using that to keep track of food. But the benefit of that could be, for example, if you have people eating small cookies or cupcakes, you could change the appearance of whatever they're picking up.
John: Right. Yeah, because if you have a sandwich, you take a bite of the sandwich modeling that bite when one person pulled the sandwich away, that's very complicated.
Janice: Exactly. But if they're eating, say Tapas when there's a one bite thing. Or sushi for example, then, as long as you've got the same shape, it can look like whatever.
John: Right. And in times suppose you keep track. They pick up the item, they put it in their mouth. Now it's gone from the plate.
Janice: Right. And in times suppose you keep track. They pick up the item, they put it in their mouth. Now it's gone from the plate.
John: That's feasible.
Janice: Yeah. And I should clarify that. I mean, this line of research is not something that, you know, you can just bite it off the shelf and then use. So it's probably a bit too custom for if you're a food company and you want to test something, right? I think there's a time and place for if you want to buy a headset that doesn't involve, having a cable connection that doesn't involve, having to manually called up stuff in unity. If you just want to test putting people in different contexts. I think it's fine to even just use like Google Cardboard and do that. But the kind of research I'm doing is probably more into how two different senses integrate. And that's why I'm, you know, using my computer science background and you see my kind of interaction design background to really try to study, okay, can I come up with a more complicated setup? But I'm trying to study how the different senses integrate. So this is probably like a more psychological perspective.
John: Right.
Janice: But I think that the learning from what I do can be very easily applied to a more lightweight consumer context.
John: Yeah, well, I think you're like that people working on race cars, you know, you have race cars out there that are doing really high-end technology. And eventually the formula One technology finds its way into reverse. You know, that like you're learning these lessons and developing technology that eventually will become more widespread.
Janice: I'm just going to say that, yeah, the learning from this can very easily be applied to food design or design and packaging or what music you should play in different environments.
John: Yes. Yeah. Well, that's one thing I love about sensory right is there's always these questions; are we studying people? are we studying the sensory stimuli? And I think the answer is studying both that you know, interaction.
Janice: It's interaction between both. Yeah.
John: Yeah, it's impossible to separate the two. And so, yeah, that's neat that you can oftentimes like in your case, you're using this technology to study people. You’re kind of an experimental psychology. It's kind of similar to, you know, my postdoc in neuroscience and in psychology. And so some of what you're doing reminds me of some of, you know we were interested in category learning. But yeah, this is neat. Okay, so let's talk a little bit about applications because there's obvious applications packaging, right? Because you're going to learn things that are going to suggest manipulations to packaging. So what are some of the big picture lessons you would say that you've learned so far? I mean, you've already touched on one of them, which I think is get started doing some because you never really know the problems you're going to run into when using a new technology based on my own experience.
Janice: Yeah. And it's also very important to do things iteratively because a lot of times you can kind of try and plan the perfect study in your head until you actually get your hands dirty and you realize, you know, what you thought would work perfectly actually doesn't work. In technology that's usually the case. Or you find out that maybe two of the technologies you're using don't really talk together in the same way that you thought it would. And then you have to work on another, you know.
John: Yeah. Continuous improvement. I'm sure you are a better programmer than I am, I'm sure. But like, the way that I write, I almost always you draft something, you write it, you make it better, make it better, and make it better. It's never you know, it's always multiple for me at least. So much better, so much easier to improve something that exists than it is to create something out of nothing.
Janice: Yeah. You know, the funny thing is I used to work in Microsoft as a developer and I feel like the one thing I took away from working as a professional software developer is that everything will change and that you shouldn't write your code to be perfect. You should write your code to be adaptive to change. And I actually feel like that's true not just for coding, but it's also true for designing experiments because even if you have the perfect experiment design, if you want to run it the next time something will change. Either the equipment will be different or you're not going to be in the same room or you can have a better version of XYZ, and it's better to be adaptable and to just think about that from the very beginning.
John: Interesting. Are there other particular lessons that you've learned when it comes to experiment design? Things that you think about to try to avoid being tied to some particular configuration? I mean, what's your thought process when you're designing an experiments?
Janice: Oh, that's a good question. I'm just thinking about what I've done for this VR study, for example, because we've done actually multiple studies with approximately the same setup. So the first time, we were changing the color of the coffee, the second time with one of my master students who just finished recently, we were also using coffee. But this time we weren't changing the coffee itself, but we were more changing the environment. So we tried to look at, okay, what happens if the background color is different and what happens if you integrate music? And in that case, I feel like, that was a good example of really just reducing code, because I was able to just essentially take the same environment, design the same experiment setup we had from the first study, and then just add to it to change the environment.
John: Right.
Janice: And then we have another study that's being planned where we don't change the coffee and we don't change the environment, but then we change the design of the mug.
John: Oh, yeah.
Janice: Again, so I feel like maybe that's a good example of where you can just literally reuse the code, but you just, you know, have a different play for changing things, so you're either changing the coffee or you're changing the environment or you're changing the mug and then you're able to get up and go quite quickly.
John: Again, so I feel like maybe that's a good example of where you can just literally reuse the code, but you just, you know, have a different play for changing things, so you're either changing the coffee or you're changing the environment or you're changing the mug and then you're able to get up and go quite quickly.
Janice: Yeah. It's about being able to, you know, replace certain modules for changing everything else while keeping everything else the same.
John: Right. Now, it makes perfect sense. Yeah. Interesting. Yeah. Now, straws is a topic I'm very interested in. I went to the you know, I don't know if you attend the IFT talks. There was a very good presentation on straws and how manipulating, I see it's really interesting that sort of thing I guess in sensory, the things that intrigue us. But it seems like there's a tremendous amount, a tremendous opportunity when it comes to strong manipulation that the experience of beverage does seem to be pretty heavily impacted by the material the straw is made up. Whether you're talking about paper straw or plastic straw or metal straw. And it's quite interesting. It's something that you know...
Janice: That must really fascinating.
John: Yeah. Because paper straw is a quite a hot topic, right? With all the sustainability stuff.
Janice: Yeah. And we did use paper straws for our coffee study. But I mean in terms of haptics, you know, we're kind of getting a slightly away from VR but I'm very interested in haptics and the different material of the straw. That would completely change your perception because your hand, usually your hand is on the straw. And you know how well that can transmit temperature is also another aspect. Also with it's smooth, it's a rough. Well, that is very interesting.
John: It is, isn't it? Yeah, and apparently copper has some ability to accentuate flavors. This is not really my area of expertise, but apparently like copper as opposed to stainless steel is a different experience. That you have copper straw that there's some extra benefit to a copper straw. So I don't know, something to think about, to look back, to think about as a manipulation is the straw itself. Yeah, but for packaging, obviously, this is a hot topic. I have a friend who is just starting a job at Dr Pepper. She was also in this and she said this is great. I mean you could imagine that you're making a beverage, it might be in your interest to engineer straws that maximize the sensory experience of your product and then push the straws out into the public, right?
Janice: So, you know, I will even go one step further. So I my background is really in how music influences flavor perception. And straw is a very good place to introduce music.
John: Really?
Janice: Not only in terms of, you know, when you're sucking something into the straw, it makes a noise. But also because of phone conductance. If you bite down on the straw, you can actually use that as a way of communicating music. This is going to be funny. I was once offered an opportunity to do something for the Museum of Sex in New York. Where they wanted me to design a straw that would play like a metal straw, that would play music, as you know, drinking a cocktail. Because you can actually do that quite easily. It's kind of like one of those toothbrushes for children where you can also hear music.
John: Oh, interesting.
Janice: Because that's also phone conductance so straws can work the same way. So, you know, maybe we're just giving someone the next new idea. If he can design a straw that hopefully play music when you drink through it.
John: Yeah. That is interesting. I assume, okay so you have to, how does this work? And what do you have to connect something to straw or would it be possible to design a straw that could communicate with a phone or something like? I mean I suppose your straws is playing music and got to be attached to some sort of, I don't know, some sort of where does the sound generation come from when you have this stuff like this?
Janice: Yeah. You'd have to attach something else to the straw as well. Like a transducer where you're actually playing the music.
John: Yeah, interesting. Maybe you can be set up. You can communicate with your phone and now you've got this enhanced sensory experience. That's very interesting. So the other topic that the use of technology to enhance sensory experience. I know you have some interesting thoughts on, you know, you said a museum of sex, so I thought of food porn, the idea of taking pictures of your food before you eat it. So now I know that it's kind of popular to people, you know, you shouldn't take pictures on your phone, you should just focus on the experience or whatever, but I think you have a little bit of a different idea. So let me hear your thoughts.
Janice: Yeah, exactly. So it's funny that you mentioned food porn, because one of my new PhD students is focusing on that for his PhD. And that's because, again, because I have this computer science background, I'm very interested in how people interact with technology. And there's a, I know that a lot of people say, oh put your phone away when you're eating and that you know, watching TV while eating is bad. And we're just being so distracted by digital technology these days. But I know, you know, we've talked about this where I don't think that technology is necessarily good or evil. That it really depends on how you use it. So it's really a personal belief of mine that we can also use technology for "good". So I'm very interested in kind of can we design new ways of interacting with technology so that we can have healthier eating experiences or that we can make more healthy food choices. And that's why we're looking at food porn, because I really think that taking pictures of your food might actually lead us to be more mindful of overeating.
John: Yeah, that's great.
Janice: Yeah. For example, if you know that you're going to share the photo of your food with your friends afterwards maybe you'd feel more self-conscious about what you're eating.
John: Oh, that is kind of accountability.
Janice: Or you might remember better of what you've eaten. So, for example, if you take pictures of your food every day and then you look at it before you decide what you're going to eat next, maybe it'll help you remember, like, oh, I've already eaten XYZ today. Maybe I should not eat so much or, you know, I haven't eaten so many vegetables today. Maybe that's what I should do.
John: That's interesting.
Janice : So you can use it as a memory aid.
John: Right. Yeah. Now that's definitely interesting. Yeah. Because I was kind of thinking this is going to go in the other direction in the sense that if, you know, people are going to ask you about the food, you're going to pay more attention while agreeing. There is this tendency to just basically shovel food down and not even pay attention to it and lose sensory experience that might have been.
Janice: And I feel like when you're, you know, I put food pictures on Instagram. So when I'm taking food pictures, I'm actually often looking for, you know, what is the best angle to take pictures, you know, how to convey through the image, you know, the maximum amount of sensory description, because I'm trying to communicate something about the food to other people. And and in that search, I think it's forcing you to be really mindful of what you're eating.
John: Yeah, that is interesting, you know, because in sensory conferences almost always everyone who's eating will be talking about food, especially if it is a nice arranged dinner. If someone in a sensory conference invites you out to dinner, just say yes, because wherever they go, it is probably be good based on my experience. I mean, I had dinner in like this Brazilian treehouse, you know, like all these awesome places because there's all these foodies, but everyone is talking about food. Whereas I did go to an ASTM for cannabis, I was the liaison for the incoming chair for the sensory evaluation group. And I was invited to be the liaison with the Cannabis Group because like, you know, crossover. And all the cannabis people, they weren't talking about the food at all. They were talking about chemistry or whatever was interesting to them. They were just talking about cannabis stuff. They weren't even paying attention to food. And I really miss my sensory people. There was no discussion of the food. It was very, very strange. So I think that it definitely makes a difference to pay attention to what you're eating. And I could definitely see that the angle. It's really interesting. But Janice, amazingly we are at twenty six minute mark. So this has been a really wonderful conversation. If someone wanted to follow up with you, suppose one of the student of yours or collaborate with you with on some of the things that we've talked about. What would be the best way for them to reach out to you?
Janice: Yeah, so they can find me on LinkedIn or on my university homepage. I'm also on Twitter as rbrainengineer that stands for right brain engineer. Yes. All my information is going to be on the podcast description. So they can just look it up and actually I look for new students quite often. So I think LinkedIn would actually be a really good place.
John: Yeah, great. I mean, so who would be the ideal student for you if someone coming, I mean I suppose that could be computer science? That could be food science?
Janice: Yeah. I mean my students so far are ones from neuroscience, ones from art history in musicology. I have one student from food science and nutrition and another one from finance. So I am really looking for interdisciplinary people.
John: Right.
Janice: Who are open minded and who are willing to kind of explore new things, new methods, new topics.
John: Oh, that's great. Yeah, if I was younger, I would apply to be in your lab to gain more experience. So, this is my last question to my guest, what advice do you have for young sensory scientist someone who's just, you know, just going out into the world of sensory science, what would you say?
Janice: For me it's quite interesting because I never formally studied sensory science. So maybe my perspective is quite different from the other people you've interviewed who actually are from sensory science, because for me, I discovered sensory science quite late during my PhD and I was like, wow, this is a fascinating area. So I think my advice would actually to be for new graduates to think about how they can apply sensory science to non-food products. Because for me, I feel like so much of the methods, like the multivariate method that are very sophisticated. It's the kind of things people use in coding for data science and machine learning. And I'm very interested in how sensory science can be applied to things like sound or images. You know, how do we classify complexity in pictures or how do we evaluate whether something sounds good or not, or how do you break down complex new information? So I think there's so much potential in using what you've learned from a sensory science education in new areas. And I think there's like people actually learn a lot of transferable knowledge and skills with the sensory science degree. So maybe they shouldn't only look at food.
John: Yeah, I agree with that. I mean, Thierry Lageat who is a co-founder of Eurosense, was on the show and he was talking about this as well. To him, it's quite odd that you go to a sensory conference and they'll be like non-food is like a category, you know, it's kind of like if you're thinking about languages like English and non-English or something like, you know, the fact that it's like non-food is like there's an enormous universe of things that are all like many different things. And the fact that they're all put together in one category to us, I think shows to some extent how we are maybe too tied to food and that we should be thinking about other.
Janice: Yes, because sensory perception is applied to everything.
John: Right. Exactly. Yeah. And so there should be you know, they do work with automobile companies. But yeah, I mean, it really is everywhere that it should be, you know, it shouldn't be so limited. I definitely agree with that. I think that's a really good point. Yeah. But Janice, we could easily talk, I'm sure, for quite a while longer, but I've reached the end of the show. So thank you very much for being on the show. Any last comments for our listeners?
Janice: No. I think it's important to just stay open-minded and not think about sensory science as only just a small block, because I do think it has the potential to influence all aspects of life.
John: Totally agree with that. Okay, great. Thanks a lot, Janice.
Janice: Okay, thanks.
John: Okay, that's it. Hope you enjoyed this conversation. If you did, please help us grow our audience by telling your friend about AigoraCast and leaving us a positive review on iTunes. Thanks.
That's it for now. If you'd like to receive email updates from Aigora, including weekly video recaps of our blog activity, click on the button below to join our email list. Thanks for stopping by!