Brainstorming and some additional research lead to three areas of investigation to solve these problems. Dot products, quaternions, and weight maps.
Using two orientation vectors with the same direction but opposite magnitudes I was able to break the pose reading into two hemispheres. This helped with overshoot, but pose interpolation in the second hemisphere was sub par. This system was also limited to 4 perpendicular poses to prevent overlap. Ultimately it was only improvement under limited circumstances.
Quaternions turned out to be too technical. Although they seemed like the best method for handling twist poses, they were not well suited for art direction. They were eventually implemented as a means to extract the twisting of the joint.
The original pose reader was applied successfully on multiple student films and some personal freelance work, however it had several issues and limitations.
1. If you were to rotate the shoulder 90 degrees so that the hand pointed forward, and create a pose reader and corrective sculpt for that pose, then rotating 110 degrees in that same direction would return a blend shape weight less than 1. This is a result of angle measurements always being positive.
2. The cones often overlapped. This made art direction difficult because transitioning between poses would often trigger multiple blend shapes in undesirable ways.
3. It was tedious to set up. The script required each pose to be created manually. In production, joints that required a pose reader almost always required 4 poses or more.
4. It didn't handle twist, because twist did not effect the vectors whose angles were measured.
Ultimately I settled on using weight maps. By sampling the UV's of a nurbs sphere using the joints world space position, I was able to read a 2d weight map that was used to drive the corrective blend shape weights.
This method solved the overshoot problem, by allowing the user to assign weights at any pose, not just within a cone or hemisphere.
This method allowed for normalized weight which solved the problem of overlapping influence cones.
This method was faster. It automatically created four non-overlapping weight maps for the four directions of rotations posing.
It was easy to customize. The weight maps could be tweaked using the reader itself or bitmap images could be substituted for even greater control.
It worked well with twist. Using the joints twist as a dual driver for a corrective blend shape solved the twisting problem.
Pose Reader V2: Development process
The node network created by the pose reader. Despite being larger than the node network created by the original pose reader it is just as fast and is created all at once rather than pose-by-pose.
The first working test of the approach. In the test a locator and annotation were used to pick the UV position on the sphere, but in the final version the joint itself was used.
This is the result of the hip joint being posed past the cone of influence. Because there is now a non-zero angle between the joint and the pose that defines the cone of influence, the pose reader returns a blend shape weight less than 1. Tis results in an undesirable deformation.
This is an example of the cones of influence overlapping. This can lead to multiple blend shapes being triggered to create undesired deformations.