Jump to content

Update 1.1.2.0 eyelids getting stuck???


Recommended Posts

I just accepted the update to 1.1.2.0 and now suddenly the eyelids for all of my characters get stuck partially open:

What has changed to make the eyes get stuck now?

How can I change it back?

Thank you.

Edited by AirMouse
Link to comment
Share on other sites

BUG CONFIRMED!!!!

I just down-graded to 1.1.0.1 and the eyelids work WAY BETTER (like they used to):

So this proves that the new update to 1.1.2.0 is totally broken.

Can you please fix it.?

Edited by AirMouse
Link to comment
Share on other sites

Also I have tried several times to uninstall/reinstall v1.1.2.0 on my Win10 i9-9900K (dual 2080TI) PC.

I also reboot and tried it again and again. But no matter what I do the new v1.1.2.0 looks absolutely horrible!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

But v1.1.0.1 looks awesome and works perfectly on all of my PCs.

 

Could you please re-evaluate your latest release to find the bug that causes the eyelids to get stuck partially open?

I will continue to use v1.1.0.1 myself but what the heck do I tell my users??? That they should download https://hub.vive.com/en-US/download/VIVE_SRanipalInstaller_1.1.0.1.msi and uninstall v1.1.2.0, then never update?? That's kind of a lame workaround IMHO.

 

This new bug makes my software look awful and I am very embarrassed to show it. Can you please fix this soon???????????????????????

 

@Daniel_Y @Corvus

Link to comment
Share on other sites

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

Link to comment
Share on other sites

 

7 hours ago, Daniel_Y said:

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

I just searched through my script and could find no mention of `timestamp` anywhere, so I don't think I am using it.

I have attached my re-write of the SRanipal_AvatarEyeSample.cs script.

I have no clue where to even look to find the bug, things were working perfectly until the update hit and now it's unusable right at a critical time. This has been very frustrating and I can't solve this by my self as I have no idea what to even look for.

Could you please help me understand what I have to change in my scripts so the eyelids will close normally again when using v1.1.2.0?

 

Thank you.

 

//========= Copyright 2018, HTC Corporation. All rights reserved. ===========
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Assertions;
using System;
using Unity.Mathematics;
using UnityEditor;

namespace ViveSR
{
   namespace anipal
   {
      namespace Eye
      {
         public class SRanipal_AvatarEyeSample2 : MonoBehaviour
         {
            [SerializeField] private Transform[] EyesModels = new Transform[0];
            public List<EyeShapeTable> EyeShapeTables;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveUpper;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveLower;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveHorizontal;

            public bool NeededToGetData = true;
            public Dictionary<EyeShape, float> EyeWeightings = new Dictionary<EyeShape, float>();
            private AnimationCurve[] EyebrowAnimationCurves = new AnimationCurve[(int)EyeShape.Max];
            private GameObject[] EyeAnchors;
            private const int NUM_OF_EYES = 2;

            private Armature m_armature;

            //private void Start()
            public void Initialize(Armature armature, Transform leftEye, Transform rightEye)
            {        
               Selection.activeGameObject=this.gameObject;
               if (!SRanipal_Eye_Framework.Instance.EnableEye)
               {
                  enabled = false;
                  return;
               }

               
               
               eyeWashV3 = new Vector3(eyeWash.x, eyeWash.y, 1.0f);

               m_armature = armature;

               EyeShapeTables = armature.eyeTablesList;
               EyesModels = new Transform[] {leftEye, rightEye};

               SetEyesModels(EyesModels[0], EyesModels[1]);
               SetEyeShapeTables(EyeShapeTables);

               AnimationCurve[] curves = new AnimationCurve[(int)EyeShape.Max];
               for (int i = 0; i < EyebrowAnimationCurves.Length; ++i)
               {
                  if (i == (int)EyeShape.Eye_Left_Up || i == (int)EyeShape.Eye_Right_Up)          curves[i] = EyebrowAnimationCurveUpper;
                  else if (i == (int)EyeShape.Eye_Left_Down || i == (int)EyeShape.Eye_Right_Down) curves[i] = EyebrowAnimationCurveLower;
                  else                                                                            curves[i] = EyebrowAnimationCurveHorizontal;
               }
               SetEyeShapeAnimationCurves(curves);

               SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
               UpdateEyeShapes(EyeWeightings);
            }


            public Vector3 directionOverrideLeft = Vector3.forward;
            public Vector3 directionOverrideRight = Vector3.forward;

            bool isLeftEyeActiveLast = false;
            bool isRightEyeActiveLast = false;

            float isLeftEyeActiveTime = 0;
            float isRightEyeActiveTime = 0;

            const float blinkEyeDirectionBlendTime = .25f;

            Vector3 leftEyeDirectionLast = Vector3.zero;
            Vector3 rightEyeDirectionLast = Vector3.zero;

            public void PostUpdate() //must be done after vrik
            {
               if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING &&
                  SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT) return;
               
               if (NeededToGetData)
               {
                  EyeData eyeData = new EyeData();
                  SRanipal_Eye.GetEyeData(ref eyeData);

                  bool isLeftEyeActive = false;
                  bool isRightEyeAcitve = false;
                  if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.WORKING)
                  {
                     isLeftEyeActive = eyeData.verbose_data.left.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                     isRightEyeAcitve = eyeData.verbose_data.right.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                  }
                  else if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT)
                  {
                     isLeftEyeActive = true;
                     isRightEyeAcitve = true;
                  }

                  if (isLeftEyeActive || isRightEyeAcitve)
                  {
                     SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
                     UpdateEyeShapes(EyeWeightings);
                  }
                  else
                  {
                     /*
                     for (int i = 0; i < (int)EyeShape.Max; ++i)
                     {
                        bool isBlink = ((EyeShape)i == EyeShape.Eye_Left_Blink || (EyeShape)i == EyeShape.Eye_Right_Blink);
                        EyeWeightings[(EyeShape)i] = isBlink ? 1 : 0;
                     }
                     */

                     //UpdateEyeShapes(EyeWeightings);
                     //UpdateEyeShapesManalOverride(EyeWeightings);



                     //Debug.Log ("UpdateGazeRayIndexed");
                     //UpdateGazeRayIndexed (directionOverrideLeft, EyeIndex.LEFT);
                     //UpdateGazeRayIndexed (directionOverrideRight, EyeIndex.RIGHT);


                     //return;
                  }

                  isLeftEyeActive = eyeData.verbose_data.left.eye_openness > 0.1f;
                  isRightEyeAcitve = eyeData.verbose_data.right.eye_openness > 0.1f;

                  if (UI_Panel_Tracking.Instance.seperateEyeTrackingToggle.isOn) {
                     //Get the rays for each eye seperately
                     Ray gazeRay = new Ray ();
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {
                        SRanipal_Eye.GetGazeRay ((GazeIndex)eyeIndex, out gazeRay);//, out GazeDirectionLocal);

                        Vector3 direction = Vector3.Scale (gazeRay.direction, eyeWashV3);

                        /*switch ((GazeIndex)eyeIndex) {
                        case GazeIndex.LEFT:
                           direction.x += divergence;
                           direction.y += kappaAngle;
                           break;
                        case GazeIndex.RIGHT:
                           direction.x -= divergence;
                           direction.y += kappaAngle;
                           break;
                        }*/
                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                     }
                  } else {
                     Vector3 GazeOriginCombinedLocal, GazeDirectionCombinedLocal = Vector3.zero;
                     if (SRanipal_Eye.GetGazeRay (GazeIndex.COMBINE, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.LEFT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.RIGHT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     }

                     //Get the rays for each eye seperately
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {

                        var direction = Vector3.Scale (GazeDirectionCombinedLocal, eyeWashV3);  ;

                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                              
                     }     
                  
                  }

                  isLeftEyeActiveLast = isLeftEyeActive;
                  isRightEyeActiveLast = isRightEyeAcitve;
               }
            }


            void ApplyEyeDirection(int eyeIndex, Vector3 direction, bool isLeftEyeActive, bool isRightEyeAcitve) {
               switch ((GazeIndex)eyeIndex) {
               case GazeIndex.LEFT:                           
                  if (isLeftEyeActiveLast != isLeftEyeActive) {
                     if (isLeftEyeActive) {
                        isLeftEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x += divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isLeftEyeActiveTime) {
                     if (isLeftEyeActive) {
                        leftEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isLeftEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime) );

                     leftEyeDirectionLast = Vector3.Lerp (leftEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (leftEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               case GazeIndex.RIGHT:
                  if (isRightEyeActiveLast != isRightEyeAcitve) {
                     if (isRightEyeAcitve) {
                        isRightEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x -= divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isRightEyeActiveTime) {
                     if (isRightEyeAcitve) {
                        rightEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isRightEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime ));
                     rightEyeDirectionLast = Vector3.Lerp (rightEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (rightEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               }
            }

            [Range (-1,1)]
            public float divergence = 0;

            [Range (-1,1)]
            public float kappaAngle = 0;

            public Vector2 eyeWash = new Vector2 (0.5f, 0.2f);
            public Vector3 eyeWashV3;
            
            public void UpdateGazeRayIndexed(Vector3 gazeDirectionLocal, EyeIndex eyesModelsIndex)
            {
               int i = (int)eyesModelsIndex;
               
               Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionLocal);
               //Debug.DrawLine ( EyeAnchors[i].transform.position, target, Color.magenta);
               
               EyesModels [i].LookAt(target, m_armature.eyesForward.up);
               switch (eyesModelsIndex)
               {
                  case EyeIndex.LEFT:
                     m_armature.eyeL.localRotation *= m_armature.eyeL_InitialRotation;// m_armature.eyeL_InitialRotation;
                     break;
                  case EyeIndex.RIGHT:
                     m_armature.eyeR.localRotation *= m_armature.eyeR_InitialRotation;// m_armature.eyeR_InitialRotation;
                     break;
               }
               
            }




            private void OnDestroy()
            {
               DestroyEyeAnchors();
            }

            public void SetEyesModels(Transform leftEye, Transform rightEye)
            {
               if (leftEye != null && rightEye != null)
               {
                  EyesModels = new Transform[NUM_OF_EYES] { leftEye, rightEye };
                  DestroyEyeAnchors();
                  CreateEyeAnchors();
               }
            }

            public void SetEyeShapeTables(List<EyeShapeTable> eyeShapeTables)
            {
               bool valid = true;
               if (eyeShapeTables == null)
               {
                  valid = false;
               }
               else
               {
                  for (int table = 0; table < eyeShapeTables.Count; ++table)
                  {
                     if (eyeShapeTables[table].skinnedMeshRenderer == null)
                     {
                        valid = false;
                        break;
                     }
                     for (int shape = 0; shape < eyeShapeTables[table].eyeShapes.Length; ++shape)
                     {
                        EyeShape eyeShape = eyeShapeTables[table].eyeShapes[shape];
                        if (eyeShape > EyeShape.Max || eyeShape < 0)
                        {
                           valid = false;
                           break;
                        }
                     }
                  }
               }
               if (valid)
                  EyeShapeTables = eyeShapeTables;
            }

            public void SetEyeShapeAnimationCurves(AnimationCurve[] eyebrowAnimationCurves)
            {
               if (eyebrowAnimationCurves.Length == (int)EyeShape.Max)
                  EyebrowAnimationCurves = eyebrowAnimationCurves;
            }
               
            [SerializeField] Vector3 gazeDirectionCombinedLocalManualOverride = Vector3.forward;
            public void UpdateGazeRayManualOverride(Vector3 gazeDirectionCombinedLocal)
            {

               //Debug.Log ("UpdateGazeRay");
               for (int i = 0; i < EyesModels.Length; ++i)
               {
                  Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocalManualOverride);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.cyan);
                  EyesModels[i].LookAt(target, transform.up);
                  EyesModels[i].Rotate(Vector3.right * -90);
               }
            }


            public void UpdateGazeRay(Vector3 gazeDirectionCombinedLocal)
                {
                    for (int i = 0; i < EyesModels.Length; ++i)
                    {
                        Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocal);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.magenta);
                        EyesModels[i].LookAt(target);
                  EyesModels[i].Rotate(Vector3.right * 90);
                    }
                }

            [SerializeField][Range(0,1)] float blinkWeightLeftManalOverride = 0;
            [SerializeField][Range(0,1)] float blinkWeightRightManalOverride = 0;
            [SerializeField][Range(0,1)] float browWeightManalOverride = 0;
            public void UpdateEyeShapesManalOverride(Dictionary<EyeShape, float> eyeWeightings)
            {
               eyeWeightings [EyeShape.Eye_Left_Blink] = blinkWeightLeftManalOverride;
               eyeWeightings [EyeShape.Eye_Right_Blink] = blinkWeightRightManalOverride;

               foreach (var table in EyeShapeTables) {

                  for (int i = 0; i < table.eyeShapes.Length; ++i)
                  {
                     EyeShape eyeShape = table.eyeShapes[i];

                     if (eyeShape == EyeShape.Eye_Left_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightLeftManalOverride;
                     } else if (eyeShape == EyeShape.Eye_Right_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightRightManalOverride;
                     } else {
                        eyeWeightings[eyeShape] = browWeightManalOverride;
                     }

                  }

                  RenderModelEyeShape (table, eyeWeightings);
               }

               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            public void UpdateEyeShapes(Dictionary<EyeShape, float> eyeWeightings)
            {
               foreach (var table in EyeShapeTables) {
                  RenderModelEyeShape (table, eyeWeightings);
               }

               //Update eye lid bones (makehuman rigs use bones for eyelids)
               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            private void RenderModelEyeShape(EyeShapeTable eyeShapeTable, Dictionary<EyeShape, float> weighting)
            {
               
               for (int i = 0; i < eyeShapeTable.eyeShapes.Length; ++i)
               {
                  EyeShape eyeShape = eyeShapeTable.eyeShapes[i];
                  //if (eyeShape > EyeShape.Max || eyeShape < 0) continue;
                           
                  if (eyeShape == EyeShape.Eye_Left_Blink || eyeShape == EyeShape.Eye_Right_Blink) {
                     eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, weighting[eyeShape] * 100f);
                  }
                  //else
                  //{
                  // AnimationCurve curve = EyebrowAnimationCurves[(int)eyeShape];
                  //
                  // eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, curve.Evaluate(weighting[eyeShape]) * 100f); //Crashed here
                  //}
               }

               //Debug.Log("EYE WEIGHT " + weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsLeft (weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsRight (weighting[EyeShape.Eye_Right_Blink]);
            }

            private void CreateEyeAnchors()
            {

               EyeAnchors = new GameObject[NUM_OF_EYES];
               for (int i = 0; i < NUM_OF_EYES; ++i)
               {
                  EyeAnchors[i] = new GameObject();
                  EyeAnchors[i].name = "EyeAnchor_" + i;
                  EyeAnchors [i].transform.SetParent (gameObject.transform);
                  EyeAnchors[i].transform.localPosition = transform.InverseTransformPoint(EyesModels[i].position);
                  //EyeAnchors[i].transform.localRotation = Quaternion.Inverse(transform.rotation) * EyesModels[i].rotation;
                  EyeAnchors[i].transform.forward = Vector3.forward;
                  //EyeAnchors[i].transform.rotation = EyesModels[i].rotation;
                  EyeAnchors[i].transform.localScale = EyesModels[i].localScale;
               }


            }

            private void DestroyEyeAnchors()
            {
               if (EyeAnchors != null)
               {
                  foreach (var obj in EyeAnchors)
                     if (obj != null) Destroy(obj);
               }
            }
         }
      }
   }
}

SRanipal_AvatarEyeSample2.cs

Edited by AirMouse
Link to comment
Share on other sites

I looked through my entire Unity project but could not locate any SRanipal.dll's "detail page". Could you please explain how to reach it?

And I am pretty sure that I am using an old SDK in my project, but I can not find a download for the latest 1.1.2.0 sdk anywhere.

When I look at the official vive-sranipal-sdk page then I only see SRanipal SDK v1.1.0.1 (556 MB) available.

Where do I download the latest 1.1.2.0 sdk you mention???

I even tried this link but the file does not exist: https://hub.vive.com/en-US/download/SRanipal_SDK_1.1.2.0.zip

I can't wait to try the latest SDK in my project! And I am very hopeful it will fix!!

Thank you very much for your help.

Edited by AirMouse
Link to comment
Share on other sites

Hi Jason

 

Thanks for that picture, I feel kind of silly that I did not think to check there 😶

Here's what I saw:

CWnrbMU.png

So I think my version is a very old one. I will update to the 1.1.0.1 SDK tonight. 😊

 

But I have one question, you said you tested SR_Runtime 1.1.2.0 with both SDK 1.1.0.1 and SDK 1.1.2.0. But I was not able to find any mention of the 1.1.2.0 SDK? Is it available for download anywhere?

Thanks again for your help. And I am very sorry to keep bothering you.

Edited by AirMouse
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...