Jump to content
 
Sign in to follow this  
AirMouse

Update 1.1.2.0 eyelids getting stuck???

Recommended Posts

I just accepted the update to 1.1.2.0 and now suddenly the eyelids for all of my characters get stuck partially open:

What has changed to make the eyes get stuck now?

How can I change it back?

Thank you.

Edited by AirMouse

Share this post


Link to post
Share on other sites

BUG CONFIRMED!!!!

I just down-graded to 1.1.0.1 and the eyelids work WAY BETTER (like they used to):

So this proves that the new update to 1.1.2.0 is totally broken.

Can you please fix it.?

Edited by AirMouse

Share this post


Link to post
Share on other sites

Also I have tried several times to uninstall/reinstall v1.1.2.0 on my Win10 i9-9900K (dual 2080TI) PC.

I also reboot and tried it again and again. But no matter what I do the new v1.1.2.0 looks absolutely horrible!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

But v1.1.0.1 looks awesome and works perfectly on all of my PCs.

 

Could you please re-evaluate your latest release to find the bug that causes the eyelids to get stuck partially open?

I will continue to use v1.1.0.1 myself but what the heck do I tell my users??? That they should download https://hub.vive.com/en-US/download/VIVE_SRanipalInstaller_1.1.0.1.msi and uninstall v1.1.2.0, then never update?? That's kind of a lame workaround IMHO.

 

This new bug makes my software look awful and I am very embarrassed to show it. Can you please fix this soon???????????????????????

 

@Daniel_Y @Corvus

Share this post


Link to post
Share on other sites

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

Share this post


Link to post
Share on other sites

 

7 hours ago, Daniel_Y said:

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

I just searched through my script and could find no mention of `timestamp` anywhere, so I don't think I am using it.

I have attached my re-write of the SRanipal_AvatarEyeSample.cs script.

I have no clue where to even look to find the bug, things were working perfectly until the update hit and now it's unusable right at a critical time. This has been very frustrating and I can't solve this by my self as I have no idea what to even look for.

Could you please help me understand what I have to change in my scripts so the eyelids will close normally again when using v1.1.2.0?

 

Thank you.

 

//========= Copyright 2018, HTC Corporation. All rights reserved. ===========
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Assertions;
using System;
using Unity.Mathematics;
using UnityEditor;

namespace ViveSR
{
   namespace anipal
   {
      namespace Eye
      {
         public class SRanipal_AvatarEyeSample2 : MonoBehaviour
         {
            [SerializeField] private Transform[] EyesModels = new Transform[0];
            public List<EyeShapeTable> EyeShapeTables;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveUpper;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveLower;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveHorizontal;

            public bool NeededToGetData = true;
            public Dictionary<EyeShape, float> EyeWeightings = new Dictionary<EyeShape, float>();
            private AnimationCurve[] EyebrowAnimationCurves = new AnimationCurve[(int)EyeShape.Max];
            private GameObject[] EyeAnchors;
            private const int NUM_OF_EYES = 2;

            private Armature m_armature;

            //private void Start()
            public void Initialize(Armature armature, Transform leftEye, Transform rightEye)
            {        
               Selection.activeGameObject=this.gameObject;
               if (!SRanipal_Eye_Framework.Instance.EnableEye)
               {
                  enabled = false;
                  return;
               }

               
               
               eyeWashV3 = new Vector3(eyeWash.x, eyeWash.y, 1.0f);

               m_armature = armature;

               EyeShapeTables = armature.eyeTablesList;
               EyesModels = new Transform[] {leftEye, rightEye};

               SetEyesModels(EyesModels[0], EyesModels[1]);
               SetEyeShapeTables(EyeShapeTables);

               AnimationCurve[] curves = new AnimationCurve[(int)EyeShape.Max];
               for (int i = 0; i < EyebrowAnimationCurves.Length; ++i)
               {
                  if (i == (int)EyeShape.Eye_Left_Up || i == (int)EyeShape.Eye_Right_Up)          curves[i] = EyebrowAnimationCurveUpper;
                  else if (i == (int)EyeShape.Eye_Left_Down || i == (int)EyeShape.Eye_Right_Down) curves[i] = EyebrowAnimationCurveLower;
                  else                                                                            curves[i] = EyebrowAnimationCurveHorizontal;
               }
               SetEyeShapeAnimationCurves(curves);

               SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
               UpdateEyeShapes(EyeWeightings);
            }


            public Vector3 directionOverrideLeft = Vector3.forward;
            public Vector3 directionOverrideRight = Vector3.forward;

            bool isLeftEyeActiveLast = false;
            bool isRightEyeActiveLast = false;

            float isLeftEyeActiveTime = 0;
            float isRightEyeActiveTime = 0;

            const float blinkEyeDirectionBlendTime = .25f;

            Vector3 leftEyeDirectionLast = Vector3.zero;
            Vector3 rightEyeDirectionLast = Vector3.zero;

            public void PostUpdate() //must be done after vrik
            {
               if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING &&
                  SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT) return;
               
               if (NeededToGetData)
               {
                  EyeData eyeData = new EyeData();
                  SRanipal_Eye.GetEyeData(ref eyeData);

                  bool isLeftEyeActive = false;
                  bool isRightEyeAcitve = false;
                  if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.WORKING)
                  {
                     isLeftEyeActive = eyeData.verbose_data.left.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                     isRightEyeAcitve = eyeData.verbose_data.right.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                  }
                  else if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT)
                  {
                     isLeftEyeActive = true;
                     isRightEyeAcitve = true;
                  }

                  if (isLeftEyeActive || isRightEyeAcitve)
                  {
                     SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
                     UpdateEyeShapes(EyeWeightings);
                  }
                  else
                  {
                     /*
                     for (int i = 0; i < (int)EyeShape.Max; ++i)
                     {
                        bool isBlink = ((EyeShape)i == EyeShape.Eye_Left_Blink || (EyeShape)i == EyeShape.Eye_Right_Blink);
                        EyeWeightings[(EyeShape)i] = isBlink ? 1 : 0;
                     }
                     */

                     //UpdateEyeShapes(EyeWeightings);
                     //UpdateEyeShapesManalOverride(EyeWeightings);



                     //Debug.Log ("UpdateGazeRayIndexed");
                     //UpdateGazeRayIndexed (directionOverrideLeft, EyeIndex.LEFT);
                     //UpdateGazeRayIndexed (directionOverrideRight, EyeIndex.RIGHT);


                     //return;
                  }

                  isLeftEyeActive = eyeData.verbose_data.left.eye_openness > 0.1f;
                  isRightEyeAcitve = eyeData.verbose_data.right.eye_openness > 0.1f;

                  if (UI_Panel_Tracking.Instance.seperateEyeTrackingToggle.isOn) {
                     //Get the rays for each eye seperately
                     Ray gazeRay = new Ray ();
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {
                        SRanipal_Eye.GetGazeRay ((GazeIndex)eyeIndex, out gazeRay);//, out GazeDirectionLocal);

                        Vector3 direction = Vector3.Scale (gazeRay.direction, eyeWashV3);

                        /*switch ((GazeIndex)eyeIndex) {
                        case GazeIndex.LEFT:
                           direction.x += divergence;
                           direction.y += kappaAngle;
                           break;
                        case GazeIndex.RIGHT:
                           direction.x -= divergence;
                           direction.y += kappaAngle;
                           break;
                        }*/
                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                     }
                  } else {
                     Vector3 GazeOriginCombinedLocal, GazeDirectionCombinedLocal = Vector3.zero;
                     if (SRanipal_Eye.GetGazeRay (GazeIndex.COMBINE, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.LEFT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.RIGHT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     }

                     //Get the rays for each eye seperately
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {

                        var direction = Vector3.Scale (GazeDirectionCombinedLocal, eyeWashV3);  ;

                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                              
                     }     
                  
                  }

                  isLeftEyeActiveLast = isLeftEyeActive;
                  isRightEyeActiveLast = isRightEyeAcitve;
               }
            }


            void ApplyEyeDirection(int eyeIndex, Vector3 direction, bool isLeftEyeActive, bool isRightEyeAcitve) {
               switch ((GazeIndex)eyeIndex) {
               case GazeIndex.LEFT:                           
                  if (isLeftEyeActiveLast != isLeftEyeActive) {
                     if (isLeftEyeActive) {
                        isLeftEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x += divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isLeftEyeActiveTime) {
                     if (isLeftEyeActive) {
                        leftEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isLeftEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime) );

                     leftEyeDirectionLast = Vector3.Lerp (leftEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (leftEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               case GazeIndex.RIGHT:
                  if (isRightEyeActiveLast != isRightEyeAcitve) {
                     if (isRightEyeAcitve) {
                        isRightEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x -= divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isRightEyeActiveTime) {
                     if (isRightEyeAcitve) {
                        rightEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isRightEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime ));
                     rightEyeDirectionLast = Vector3.Lerp (rightEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (rightEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               }
            }

            [Range (-1,1)]
            public float divergence = 0;

            [Range (-1,1)]
            public float kappaAngle = 0;

            public Vector2 eyeWash = new Vector2 (0.5f, 0.2f);
            public Vector3 eyeWashV3;
            
            public void UpdateGazeRayIndexed(Vector3 gazeDirectionLocal, EyeIndex eyesModelsIndex)
            {
               int i = (int)eyesModelsIndex;
               
               Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionLocal);
               //Debug.DrawLine ( EyeAnchors[i].transform.position, target, Color.magenta);
               
               EyesModels [i].LookAt(target, m_armature.eyesForward.up);
               switch (eyesModelsIndex)
               {
                  case EyeIndex.LEFT:
                     m_armature.eyeL.localRotation *= m_armature.eyeL_InitialRotation;// m_armature.eyeL_InitialRotation;
                     break;
                  case EyeIndex.RIGHT:
                     m_armature.eyeR.localRotation *= m_armature.eyeR_InitialRotation;// m_armature.eyeR_InitialRotation;
                     break;
               }
               
            }




            private void OnDestroy()
            {
               DestroyEyeAnchors();
            }

            public void SetEyesModels(Transform leftEye, Transform rightEye)
            {
               if (leftEye != null && rightEye != null)
               {
                  EyesModels = new Transform[NUM_OF_EYES] { leftEye, rightEye };
                  DestroyEyeAnchors();
                  CreateEyeAnchors();
               }
            }

            public void SetEyeShapeTables(List<EyeShapeTable> eyeShapeTables)
            {
               bool valid = true;
               if (eyeShapeTables == null)
               {
                  valid = false;
               }
               else
               {
                  for (int table = 0; table < eyeShapeTables.Count; ++table)
                  {
                     if (eyeShapeTables[table].skinnedMeshRenderer == null)
                     {
                        valid = false;
                        break;
                     }
                     for (int shape = 0; shape < eyeShapeTables[table].eyeShapes.Length; ++shape)
                     {
                        EyeShape eyeShape = eyeShapeTables[table].eyeShapes[shape];
                        if (eyeShape > EyeShape.Max || eyeShape < 0)
                        {
                           valid = false;
                           break;
                        }
                     }
                  }
               }
               if (valid)
                  EyeShapeTables = eyeShapeTables;
            }

            public void SetEyeShapeAnimationCurves(AnimationCurve[] eyebrowAnimationCurves)
            {
               if (eyebrowAnimationCurves.Length == (int)EyeShape.Max)
                  EyebrowAnimationCurves = eyebrowAnimationCurves;
            }
               
            [SerializeField] Vector3 gazeDirectionCombinedLocalManualOverride = Vector3.forward;
            public void UpdateGazeRayManualOverride(Vector3 gazeDirectionCombinedLocal)
            {

               //Debug.Log ("UpdateGazeRay");
               for (int i = 0; i < EyesModels.Length; ++i)
               {
                  Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocalManualOverride);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.cyan);
                  EyesModels[i].LookAt(target, transform.up);
                  EyesModels[i].Rotate(Vector3.right * -90);
               }
            }


            public void UpdateGazeRay(Vector3 gazeDirectionCombinedLocal)
                {
                    for (int i = 0; i < EyesModels.Length; ++i)
                    {
                        Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocal);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.magenta);
                        EyesModels[i].LookAt(target);
                  EyesModels[i].Rotate(Vector3.right * 90);
                    }
                }

            [SerializeField][Range(0,1)] float blinkWeightLeftManalOverride = 0;
            [SerializeField][Range(0,1)] float blinkWeightRightManalOverride = 0;
            [SerializeField][Range(0,1)] float browWeightManalOverride = 0;
            public void UpdateEyeShapesManalOverride(Dictionary<EyeShape, float> eyeWeightings)
            {
               eyeWeightings [EyeShape.Eye_Left_Blink] = blinkWeightLeftManalOverride;
               eyeWeightings [EyeShape.Eye_Right_Blink] = blinkWeightRightManalOverride;

               foreach (var table in EyeShapeTables) {

                  for (int i = 0; i < table.eyeShapes.Length; ++i)
                  {
                     EyeShape eyeShape = table.eyeShapes[i];

                     if (eyeShape == EyeShape.Eye_Left_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightLeftManalOverride;
                     } else if (eyeShape == EyeShape.Eye_Right_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightRightManalOverride;
                     } else {
                        eyeWeightings[eyeShape] = browWeightManalOverride;
                     }

                  }

                  RenderModelEyeShape (table, eyeWeightings);
               }

               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            public void UpdateEyeShapes(Dictionary<EyeShape, float> eyeWeightings)
            {
               foreach (var table in EyeShapeTables) {
                  RenderModelEyeShape (table, eyeWeightings);
               }

               //Update eye lid bones (makehuman rigs use bones for eyelids)
               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            private void RenderModelEyeShape(EyeShapeTable eyeShapeTable, Dictionary<EyeShape, float> weighting)
            {
               
               for (int i = 0; i < eyeShapeTable.eyeShapes.Length; ++i)
               {
                  EyeShape eyeShape = eyeShapeTable.eyeShapes[i];
                  //if (eyeShape > EyeShape.Max || eyeShape < 0) continue;
                           
                  if (eyeShape == EyeShape.Eye_Left_Blink || eyeShape == EyeShape.Eye_Right_Blink) {
                     eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, weighting[eyeShape] * 100f);
                  }
                  //else
                  //{
                  // AnimationCurve curve = EyebrowAnimationCurves[(int)eyeShape];
                  //
                  // eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, curve.Evaluate(weighting[eyeShape]) * 100f); //Crashed here
                  //}
               }

               //Debug.Log("EYE WEIGHT " + weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsLeft (weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsRight (weighting[EyeShape.Eye_Right_Blink]);
            }

            private void CreateEyeAnchors()
            {

               EyeAnchors = new GameObject[NUM_OF_EYES];
               for (int i = 0; i < NUM_OF_EYES; ++i)
               {
                  EyeAnchors[i] = new GameObject();
                  EyeAnchors[i].name = "EyeAnchor_" + i;
                  EyeAnchors [i].transform.SetParent (gameObject.transform);
                  EyeAnchors[i].transform.localPosition = transform.InverseTransformPoint(EyesModels[i].position);
                  //EyeAnchors[i].transform.localRotation = Quaternion.Inverse(transform.rotation) * EyesModels[i].rotation;
                  EyeAnchors[i].transform.forward = Vector3.forward;
                  //EyeAnchors[i].transform.rotation = EyesModels[i].rotation;
                  EyeAnchors[i].transform.localScale = EyesModels[i].localScale;
               }


            }

            private void DestroyEyeAnchors()
            {
               if (EyeAnchors != null)
               {
                  foreach (var obj in EyeAnchors)
                     if (obj != null) Destroy(obj);
               }
            }
         }
      }
   }
}

SRanipal_AvatarEyeSample2.cs

Edited by AirMouse

Share this post


Link to post
Share on other sites

Hi @AirMouse

 

Could you let me know which version of SDK are you using now?

You can check it from SRanipal.dll's detail page.

I have test SR_Runtime 1.1.2.0 with both SDK 1.1.0.1 and 1.1.2.0 and they all works as expected.

 

Sincerely

Jason Lu

 

 

  • Like 1

Share this post


Link to post
Share on other sites

I looked through my entire Unity project but could not locate any SRanipal.dll's "detail page". Could you please explain how to reach it?

And I am pretty sure that I am using an old SDK in my project, but I can not find a download for the latest 1.1.2.0 sdk anywhere.

When I look at the official vive-sranipal-sdk page then I only see SRanipal SDK v1.1.0.1 (556 MB) available.

Where do I download the latest 1.1.2.0 sdk you mention???

I even tried this link but the file does not exist: https://hub.vive.com/en-US/download/SRanipal_SDK_1.1.2.0.zip

I can't wait to try the latest SDK in my project! And I am very hopeful it will fix!!

Thank you very much for your help.

Edited by AirMouse

Share this post


Link to post
Share on other sites

Hi Jason

 

Thanks for that picture, I feel kind of silly that I did not think to check there 😶

Here's what I saw:

CWnrbMU.png

So I think my version is a very old one. I will update to the 1.1.0.1 SDK tonight. 😊

 

But I have one question, you said you tested SR_Runtime 1.1.2.0 with both SDK 1.1.0.1 and SDK 1.1.2.0. But I was not able to find any mention of the 1.1.2.0 SDK? Is it available for download anywhere?

Thanks again for your help. And I am very sorry to keep bothering you.

Edited by AirMouse

Share this post


Link to post
Share on other sites

Please sign in to comment

You need to be a member in order to leave a comment

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×
×
  • Create New...