South Korean TV broadcaster MBC recently aired a Korean language documentary that centers on a family’s loss of their young daughter, seven-year-old Nayeon. Using the power of photogrammetry, motion capture, and virtual reality, the team recreated Nayeon for one last goodbye with the family’s mother, Ji-sung.

Like a typical seven-year-old, Nayeon was a spry, playful kid. Then she suddenly fell ill with hemophagocytic lymphohistiocytosis (HLH), a life-threatening disease of severe hyper-inflammation caused by uncontrolled growth of the body’s white blood cells. To Ji-sung, it all first appeared to be a common cold due to Nayeon’s swelling and fever. Her daughter passed away one month later in the hospital.

Called ‘Meeting You’, the documentary goes on to recreate a series of warm memories from Nayeon’s life. Donning an HTC Vive Pro, Vive trackers, and wireless adapter, Ji-sung revisits a virtual version of a park the family would frequent. Nayeon giggles, and cautiously asks her mom if she’s afraid. She wonders why it’s cold outside. Touching hands, they’re both lifted up to a heavenly realm.

Image courtesy IMBC

The mother and the virtual simulacrum experience more happy memories together. It’s Nayeon’s birthday, and honey rice sweets, a birthday cake, and her favorite seaweed soup are all there. Ji-sung puts her down to bed for a nap, and plays with her hair as Nayeon precociously bobbles around.

For non-Korean speakers, using YouTube’s auto-generated translation is basically useless for the nine-minute video. Thankfully, the baked-in Korean subtitles were simple to translate via Google’s camera-based app and were remarkably clear too. Translation or not, the power of Ji-sung’s emotions are intensely human, no matter the language.

Putting aside the obvious exploitation factor of reuniting a mother with her deceased child for television viewers—Nayeon even pulls at the heartstrings by telling her father to stop smoking, and her siblings not to fight so much—recreating a deceased loved one in such high fidelity raises some ethical concerns, and they’re ones we simply don’t have clear answers to yet. Whether conjuring virtual doppelgangers of lost loved ones may one day be considered an unnecessary re-traumatization, or a valid coping mechanism to help overcome tragedy, we just can’t say for now.

SEE ALSO
Making 'The Remedy' – How A Veteran Illustrator Made one of VR's Best Short Films With 'Quill'

Personally, all of it was unsettling to me at first glance, and maybe well outside of what I’d consider healthy. Still, it does seem to have helped Ji-sung to some extent, who carries with her a tattoo of her daughter’s birthday as an indelible reminder. At the family home there are pictures of Nayeon all over the place. Every month the family visits Nayeon’s burial place to leave her favorite toys in remembrance. It’s clear the family isn’t running away from the reality, or trying to forget what was very likely one of the worst things to happen to them either, but in the same breath they aren’t holding too tightly onto the past. Before Ji-sung left for the VR experience, she burned letters and offerings to her child, including a shirt that was too warm for her daughter to wear at the hospital. Integrating those last, vivid virtual experiences of Nayeon into her memory serves as a singular, bittersweet goodbye, one you’d never get in a hospital room. Death oftentimes proves to be frightfully uncinematic like that.

After translating each subtitle, my first cynical instincts to label this a blatantly insensitive puppeteering of a dead child are mostly gone. It’s not perfect, but it makes sense to me on some level.

Creating a Virtual Child

Developed by South Korean startup Vive Studios (no relation to HTC’s Vive Studios), the virtual Nayeon was created over the course of eight months using a variety of techniques. Motion capture not only recorded an adult actor’s movements, but also facial expressions, some of which were acted out based on video and photos of the real-world Nayeon.

High resolution photographs were taken in a 3D capture technique called photogrammetry. Nayeon’s little sister, a spitting image of her older sibling, was used as the basis of the character model.

Image courtesy MBC

Although not apparent in the video above, the studio also added a degree of liveliness to the character by integrating voice recognition and a basic AI, which would let the pair have a basic conversation. Responses were created based off of family interviews and videos.

In the end, revitalizing the image of a deceased person isn’t exactly new, and a bevy of examples come to mind: Fred Astaire dancing with a Dirt Devil, Tupac holograms, and more recently the litany of deepfakes that make you question whether famous actors are still alive or not. And much like those early CG humans and carefully contorted deepfake masks, Vive Studios’ tailor-made VR experience is no doubt impressive for a short while too, but at this point it’s really no more than a carefully orchestrated funerary rite. It’s that point in the future though when AI is capable of automatically conjuring a person based off a compendium of video and photo that we’re waiting to see. Because whether you like it or not, virtual humans are coming, and I think we’ve just taken one step closer.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • Pedro Câmara

    This is one of the most disturbing things I’ve ever seen in VR. But it’s pretty obvious that it was made to be controversial. I guess public broadcasting networks in my country (Brazil) would do this kind of thing too (if they had that much money)

  • Andrew Jakobs

    Having seen some demo’s of virtual characters, I can say that it really is getting to a point where you forget it’s a digital representation. Couple it with a decent AI/bot and you really can have a virtual (girl)friend/companion.. But to me this is just a step up from watching an old video of your child.. for centuries they used drawings, then paintings, then photo’s and later video, then 3d video, and now virtual reality.. It’s just progress..

    • Jan Ciger

      Not sure I would call this progress. There are things in engineering where you need to ask not whether you can do something but whether you should.

      • Biggus Dickus

        Like bringing back dinosaurs.

        • Jistuce

          Hey, I just wanna know what a T-Rex T-Bone tastes like, and there’s nothing wrong with that.
          (The answer, of course, is “just like chicken”)

      • Hivemind9000

        Thank you Mr Goldblum :)

      • Andrew Jakobs

        And that’s where people will differ, you might not think it’s a good idea and other think it IS a good idea. Everybody deals with death a different way. It certainly won’t be suitable or good for everybody.

    • DanDei

      Yes, but progress towards what? Holding onto dead loved ones or finding closure? This is a very sensitive area with deep psychological repercussions and the potential of doing much more harm than good.

      • Hivemind9000

        Or more good than harm. Who can say unless these things are tried and tested (hopefully under sensitive and controlled circumstances with psychological evaluations before/after etc). Losing a child can destroy people’s lives, so it’s worth seeing if these things can help people get closure and move on, don’t you think?

        • DanDei

          Yeah, sensitive and controlled circumstances is a big ask in out times that doesn’t go well with a made for TV moment or the foreseeable business applications of this tech.
          There is a lot of money in this and in a couple of years game engines will have the tools to very easily recreate a VR likeness of a human being that won’t require such extensive work as full body scans, modelling and texturing.
          And then you can buy experiences similar to this with lost loved ones. And because it is a corporate thing they will want to sell more of it. So giving real closure is not your best business interest. Instead your VR-ghost-child might tell you how much it has missed you and that it can’t wait to meet you again in the future… followed by a very sensitive sales talk about that special offer to honor the dead for its next birthdays or funeral anniversaries.

          I think there is real potential for therapeutic application of this tech. But there is also a greedy market that will exploit people’s feelings and make a terribly painfull horror show out of this.

          • Hivemind9000

            You are right – this could easily be exploited for profit, but that is true of almost anything when it comes to capitalism. It will be interesting to see if there are some good/clear results from research into this. It’s going to happen whether we like it or not so any research may help cast an early light onto whether this is good for us or not. As someone who has lost a couple of people very dear to me, and who struggled for years to get past it, this is intriguing to me. If it could help people say goodbye and move on with their lives, I think it will be of great benefit.

          • Molly Weiner

            Why must people use the term “big ask.”

    • Ad

      Yeah we need to ban virtual companions while we still can.

      • Andrew Jakobs

        No, we certainly should not.. Maybe you can easily make friends and have a partner, a lot of people can’t, that’s just a fact, even with extra help they have problems.. And in reality, aren’t we all just biological computers? With how fast AI is progressing, it’s learning just the same way as we humans do, and if that AI can be friends/companions to those people who can’t get it otherwise, who are you to say we should ban virtual companions, who are you to deprive these people of some form of happiness.. And if AI is learning the same way as we do, and having an artificial conscious (as it cannot live outside the computer), isn’t it murder to turn it off?

        • Ad

          No, not at all. And no one has the right to an AI sex slave. Don’t give me that nonsense about “some people…”

          • Andrew Jakobs

            So you consider your partner a sex slave. ah. good to know how sick you are..

          • Ad

            If they are a programmable robot you can buy, sell, and delete, then yes.

          • Jorge Gustavo

            Did I hear AI sex slaves? Where can i find one? Its not for me, of course. My dog is here at my side asking about it

  • kazira

    Im a dude and i just teared up!!!! wow this is hard to watch even though i don’t understand her i feel her pain

    • Jason

      Hey, “dudes” have feelings, too. Nothing wrong with showing them.

      • Jistuce

        DUDEs do indeed have feelings, but I’m impressed that Dynamic Unorganic Digital Entities have working tearducts.

  • Trenix

    I’m sorry but that’s sick. That’s not her daughter and I doubt this will do any good for her psychology in the long run. This is how you wail in depression rather than moving on with a loved one.

    • Hivemind9000

      So you’re an expert in human psychology I see. People should just toughen up and move on eh? Super easy.

      • Trenix

        No it’s better to make them see their fake child in VR and cry and talk to them even though they’re not actually there. Yes, that’s very healthy.

        • Hivemind9000

          Yeah grieving is for pussies, right?

          • Trenix

            There is a time to grief and it certainly isn’t during a time like this.

        • Pyrinder

          When people start talking to shit out of thing air, much less a 3D recreation – you’re fucking crazy. That’s not healthy. That’s fucking crazy. That’s the shit people commit other people to mental asylums over.

          • BorisDecember

            guys, just calm down! ya all aren’t therapist and you’re not Korean based. Y’all white people say asians are different. Yes we are! I don’t know how this therapy will affects on her and what benefits it would bring but we are not there. Your words would be true if we would see her acting in that bad way you say (asylum, trauma and etc) but this girl passed away too fast (went out in a month) and her mother didn’t say goodbye in due course. And no way you can’t stop it in capitalistic world. It would be used in this way too! and those things could not be bad in 100% of the time as previous said

          • Trenix

            I agree, but now what is mentally stable, is much different nowadays.

          • This sounds just anti-VR. So it’s crazy to play video games? It’s crazy to interact with any AI that talks?

          • Pyrinder

            Let me know where it was that I said video games was a problem. Because, I’m sure that wasn’t in there.

    • JezVerona

      As a psychologist, I feel confident in saying that this could be beneficial in some circumstances. We are all different, which means we grieve and heal differently. It is not true that this kind of thing must be bad 100% of the time.

      • Trenix

        Oh really? Care to explain?

        • Jarrod Christman

          It really does depend on each individual person. Perhaps the mother wasn’t able to be there at the end and she obsesses over that moment. In such a case, giving exposure to the daughter again and allowing closure could help. In other cases, re-exposure may certainly be worse. There is no singular solution for something as complex as human psychology. Disclaimer: I am not a psychologist.

          • Trenix

            I’m not a psychologist either, but I did take some psychology courses in college. Having someone believe in a false reality is dangerous. It’s fine if we put on VR googles and know that the world we’re in is fake. However this video goes further, this lady speaks to the fake child in VR as if it’s actually her. This is wrong in so many levels. I have dead relatives and pets, I wouldn’t want to see them again in VR, because I feel as if it’d dishonor them.

            This is the definition of brainwash. Shame on this moron who claims he’s a psychologist. I challenged him because he has no idea of what he’s talking about. He’s the type of “psychologist” that would claim a schizophrenic is fine living in their own little world if he or she doesn’t harm anyone. I know people like that, they make me sick. We must set standards in regards to what is mentally healthy. This isn’t, depression is a mental illness that people must overcome. This isn’t how you go about doing it.

        • Moe Curley

          I think he just did.

    • Sheri Mendon

      I agree, and I have list a son.

    • Andrew Jakobs

      As I always say, Ethics are in the eye of the beholder.. What you might find unethical other people might find ethical and other way round..

      • Ad

        That’s not how ethics works, we literally form ethical standards and then enforce them. One individual doesn’t create these projects or their foundations so no individual can on their own day their okay anyway.

        • Andrew Jakobs

          That’s exactly how ethics works. A group of people deciding that they approve it or not.. And another group that might have a different opinion.. which group is right? Ethics are definitely in the eye of the beholder. who are ‘we’ in the “we literally form ethical standards and then enforce them”, right the “we” who thinks something is wrong or right, not “we” as a complete society, just a small group of people.

          • Ad

            And now the larger group of people need to enforce on this smaller absurd group of people with terrible dangerous ideas.

          • Ryan

            Islam is the largest religion in the world. Do Muslims get to enforce their ideas on smaller groups of people because they think they have “terrible, dangerous ideas”?

          • Ad

            I mean in specific countries some people do, like with all religions. But why is that relevant? Human cloning is banned, so are lots of other red lines in science, tech, and general social norms.

          • Dynastius

            Islam is the 2nd largest religion. (But not by much.) It is projected to become the world’d largest religion, but currently it’s behind Christianity. But I get your point.

      • Gonzax

        Absolutely, agreed!

  • Richard Donczi

    Hmmm..Our application is more sick because of the photoscan and AI feature. :)

    https://www.youtube.com/watch?v=u74YChwm-_Y&t=14s
    http://www.whatisthehybri.com

  • DanDei

    Feels like I was just watching an episode of Black Mirror

  • Biggus Dickus

    This is freaking weird.

  • ra51

    Wow…this hits you right in the chest within the first minute. As a parent, I could imagine what emotional weight your child can bear on you. To lose one is infinitely heartbreaking, but to see her/him rendered in front of you again after knowing they’ve been long gone has gotta be just as heavy. Not sure if this is beneficial or not to the mother – being as you’re basically interacting with virtual entity that you are pretending is your child, however I can see the therapeutical parts also. I dunno…I’m no therapist or doctor so I’ll leave it to them to determine that. It’s like an episode from Black Mirror.

    I will admit though, this is quite intriguing. I’ve had dreams of speaking to my deceased dad many times and I always woke up happy.

  • Immersive Computing

    Very powerful and moving, certainly brought me to tears watching. There are ethical issues, but under the supervision of a clinical psychologist this could be extremely beneficial for some patients dealing with loss.

    VR therapy has been successful over 15 years treating combat veterans with PTSD as part of a carefully managed programme.

    https://onlinedegrees.bradley.edu/blog/how-virtual-reality-therapy-impacts-veterans-with-ptsd/

  • MW

    Stupid. I know that escapism is a new religion. But those kind of experiences are not helpful, but harmful. In coping with loss of loved one the most important thing is understanding and acceptation.
    Creating this type of illusion not for helping the victim but for money and viewing count on YouTube (reaction videos are so popular) is just immortal.

  • recreating a deceased loved one in such high fidelity raises some ethical concerns, and they’re ones we simply don’t have clear answers to yet. Whether conjuring virtual doppelgangers of lost loved ones may one day be considered an unnecessary re-traumatization, or a valid coping mechanism to help overcome tragedy, we just can’t say for now.

    Uhh… yes i can. This is 100% damaging, and if i were the parents i would not be anywhere near grateful for this experience. This is like blackmail with no actual redeemable outcome, like walking into the fire knowing you’ll be burnt. How is this good, helpful or beneficial — whether voluntarily or forced? If there are any folks out there trying to recreate their memories of dead loved ones as VR experiences, i would consider their desire itself ill and a sign of their inability to move on as well. They need help out of the spiral of wishing for something other than reality, not a way to further enable their wish to not heal or face what has come to pass.

    Staying attached to trauma is never a positive thing.

    • kuhpunkt

      Maybe this is what helps them to move on?

  • Jonathan Welch

    This is sick. It’s a grieving mother… You show her a 3D interactive pic of her kid, she’s going to be overwhelmed. It’s not her kid any more than Google is your friend. This is a few VR designers trying to get a headline through the most immoral means I can think of. F### these guys!

    • Gonzax

      Yes and no, to be honest. If I could have a virtual reality recreation of my parents and still be able to see them and hear them around me I think I’d definitely like to have the possibility. It sounds a bit creepy but if it helps you in some way then why not? perhaps one day it will be a way to have your loved ones with you forever.

  • this is inevitable. From 2013: ““Memories with Maya is a story that aims to seed ideas, grounded in hard science, on how AI, AR, and advances in the field of deep learning and cybernetic reconstruction will eventually allow us to virtually resurrect the dead, or allow us the option of substrate-independent minds,” https://www.kurzweilai.net/book-review-memories-with-maya

  • Why are there ninjas cutting onions next to me?

  • Ad

    “Putting aside the obvious exploitation factor of reuniting a mother with her deceased child for television viewers”

    No I don’t think you can… Some extreme things are sometimes done in clinical therapy settings but they usually aren’t televised.

    Some people will not comment because they’re not a doctor… but neither was the network executive who put this on.

  • NooYawker

    I didn’t watch this, this feels wrong.

  • jjthejj

    this is fucked up on sooo many levels.

  • NO one will ever know the pain this woman has suffered or what this experience will do for her .. no one … so people cant say this is damaging for this woman because you havent sufffered a loss of a child your self ..

    • DanDei

      I have also never tried to cure a hurting stomach by repeatedly punching my belly very hard. Because some things are just bad ideas and people with a modicum of common sense can see that without having lived through it. Also people with psychology degrees find this alarming. So their opinion is more informed than yours.

      • well the only person who will know if this was a good thing or a bad thing, is the woman her self.. if shes fine after this experience and it helped her move on .. then theres no harm done.

  • Christopher E. J. Cobb

    I think this is groundbreaking in grieving therapy. Most parents keep their emotions extreamly repressed so as to keep their dying child and siblings from being more traumatized. Unfortunately this emotional act has gone on so long that even after death the parent has a difficult time actually grieving. They put on a show at the funeral because they know people expect them to be grieving but its hollow and afterwards they feel guilt. This gives them a chance to have their own personal process of grieving without judgements.

  • Christopher E. J. Cobb

    My guess is people are more disturbed by the commercialization of the event itself.

  • Dan Lokemoen

    Is this more ghoulish than resurrecting Carrie Fisher’s corpse to rise and act in Star Wars movies? Yeah, a little, but it’s all grotesque.

  • Chae Gyun Kang

    I watched it for an hour, and the whole story was as follows.
    The story focused on the healing process of a person who suddenly lost a child due to illness and still suffers from not forgetting her even though it has been a long time.
    The TV program was not just for VR demonstrations. It was a gift used by the production team as one of the ways to relieve the main character’s pain.
    However, most of all are only paying attention to VR demonstrations and the special circumstances that arise from them.

  • notnamed

    Soon to be reunited with your dearest members who now are virtually available for only $50 at 30 minutes. For more download the dlc, we be releasing for only $20 in the not so far future.

  • Lucas Shepherd

    Weird

  • Liam Mulligan

    The concept of capturing a message from the real loved one before death is fair, but a recreation is soul crushing due to the errors in personality and behaviour that can be introduced in dev. You could mess someone up with a misinterpreted construction. Leaving a goodbye msg or capturing as your kids grow, age etc is alot gentler of a concept with accuracy only leading to aid in grieving.

  • Neelesh Bhagwat

    It’s out there. Just like guns, pornography, vaccination and other inventions of humankind. You will always have the gift of CHOICE. Use it wisely.

  • LuoSKraD

    None is forcing you to do it, the mother agreed to it. If it causes her more grief that’s on her.

  • Pyrinder

    Can’t wait for the next sick shit they’ll come up with next.

    Funeral VR – get to actually BURY YOUR CHILD! Yeah! Because that’ll bring “closure” much faster!

    The fuck is with these assholes? Not a fan of this.

  • Tarryn ItUp

    This is 100% disturbing and just plain cruel and horrible. I could never ever in a million years put myself through such a thing, it would be like ripping them out of my heart for a second time. This project should be shut down and the dead should be respected.

  • Mathew Renfro

    reminds me of the ending to the movie “A.I.” (Steven Spielberg’s)

  • Sylvz Fz

    There is absolutely no way this whole thing could have any long or short term real positive consequences on the psychological state of grieving parents, or really help them overcome loss of a child. At most it could create an illusion of presence which could feel comforting for a little while as a shot of drug, but the next step is for sure worse than it was before. I lost my brother in awful circonstances, my parents have been depressed for the last 20 year.
    Trust me, you don’t want to experience waking up from a dream that makes you think your beloved one is actually still alive, and realize it was just a dream a while after (I don’t mean that you don’t ever want to see him/her in a dream being conscious of the death, but actually trusting that he/she’s still alive).
    It is like reviving a little death one more time. This whole thing is taking a so wrong direction…

  • brubble

    YIKES.

  • dk

    obviously she knew exactly what she was getting into and consented so I don’t see what the big deal is ….and most likely there was even an evaluation from a psychiatrist

    ….the issue for me is …it won’t be that accurate of a representation ….it would be much better if u just have high quality 360 3d video of some events from the kid’s life or maybe something like this https://www.microsoft.com/en-us/mixed-reality/capture-studios where u can move in the virtual space

  • ㅤㅤ

    what the fuck this is so weirdo

  • Kimberle McDonald

    This is why i’ve taken tons of VR footage of my parents using my samsung 360 camera. One day that’s going to seem like a time capsule, where I can visit them again, just as they were.

    • Moe Curley

      Vary touching. Someone should create an inexpensive way dedicated way to do this. Doeasn’t have to be high res future tech will be able to interpolte and creat a perfect representation. People just need to get some data while they are stil with their loved one as you are doing. A “VR Time Capsule”.

  • Jorge Gustavo

    I am a father of a 3 year old girl. Really dont know what to think about this. My first thought: Creepy, very creepy and unsettlling.