Search results for “marking menu”

So, it seems like I have been going crazy with marking menus lately. I am really trying to get the most of them, and that would not be much if we could only use them in the viewports, so today we are going to look at how we can construct custom marking menus in maya editors.

Custom marking menus in maya editors - node editor

tl;dr: We can query the current panel popup menu parent in maya with the findPanelPopupParent MEL function, and we can use it as a parent to our popupMenu.

So, there are a couple of scenarios that we need to have a look at, as they should be approached differently. Although, not completely necessary I would suggest you have a look at my previous marking menu posts – Custom marking menu with Python and Custom hotkey marking menu – as I will try to not repeat myself.

Okay, let us crack on. Here are the two different situations for custom marking menus in maya editors we are going to look at.

Modifiers + click trigger

In the viewport these are definitely the easier ones to set up as all we need to do is just create a popupMenu with the specified modifiers – sh, ctl and alt, the chosen button and viewPanes as the parent. When it comes to the different editors, though, it gets a bit trickier.

Let us take the node editor as an example.

If we are to create a marking menu in the node editor, it is a fairly simple process. We do exactly the same as before, but we pass "nodeEditorPanel1" as the parent argument. If you have a node editor opened when you run the popupMenu command, you will be able to use your marking menu in there. The catch is though, that once you close the node editor the marking menu is deleted, so it is not available the next time you open the node editor.

Unfortunately, I do not have a great solution to this problem. In fact, it is a terrible solution, but I wanted to get it out there, so someone can see it, be appalled and correct me.

The second method – Custom hotkey trigger – is much nicer to work with. So, you might want to skip to that one.

What I do is, I create a hotkey for a command that invokes the specific editor (I only have marking menus in the node editor and the viewport) and runs the marking menu code after that. So, for example, here is my node editor hotkey (Alt+Q) runTimeCommand.

mc.NodeEditorWindow()

if mc.popupMenu("vsNodeMarkingMenu", ex=1):
    mc.popupMenu("vsNodeMarkingMenu", dai=1, e=1)
else:
    mc.popupMenu("vsNodeMarkingMenu", p="nodeEditorPanel1Window", b=2, ctl=1, alt=1, mm=1)

mc.setParent("vsNodeMarkingMenu", m=1)

from vsRigging.markingMenus import vsNodeMarkingMenu
reload(vsNodeMarkingMenu)

That means that everytime I open the node editor with my hotkey I also create the marking menu in there, ready for me to use. As, I said, it is not a solution, but more of a workaround at this point. In my case, though, I never open the node editor through anything else than a hotkey, so it kind of works for me.

Then the vsRigging.markingMenus.vsNodeMarkingMenu file is as simple as listing the menuItems.

import maya.cmds as mc

mc.menuItem(l="multiplyDivide", c="mc.createNode('multiplyDivide')", rp="N", i="multiplyDivide.svg")
mc.menuItem(l="multDoubleLinear", c="mc.createNode('multDoubleLinear')", rp="NE", i="multDoubleLinear.svg")
mc.menuItem(l="plusMinusAverage", c="mc.createNode('plusMinusAverage')", rp="S", i="plusMinusAverage.svg")
mc.menuItem(l="condition", c="mc.createNode('condition')", rp="W", i="condition.svg")
mc.menuItem(l="blendColors", c="mc.createNode('blendColors')", rp="NW", i="blendColors.svg")
mc.menuItem(l="remapValue", c="mc.createNode('remapValue')", rp="SW", i="remapValue.svg")
...

A proper way of doing this would be to have a callback, so everytime the node editor gets built we can run our code. I have not found a way to do that though, other than ofcourse breaking apart maya’s internal code and overwriting it, which I wouldn’t go for.

Luckily, creating a custom marking menu bound to a custom hotkey actually works properly and is fairly easy. In fact, it is very similar to the Custom hotkey marking menu post. Let us have a look.

Custom hotkey trigger

Now, when we are working with custom hotkeys we actually run the initialization of the popupMenu everytime we press the hotkey. This means we have the ability to run code before we create the marking menu. Therefore, we can query the current panel and build our popupMenu according to it. Here is an example runTimeCommand, which is bound to a hotkey.

import maya.mel as mel

name = "exampleMarkingMenu"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

parent = mel.eval("findPanelPopupParent")

if "nodeEditor" in parent:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import exampleMarkingMenu
    reload(exampleMarkingMenu)
else:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import fallbackMarkingMenu
    reload(fallbackMarkingMenu)

So, what we do here is, we start by cleaning up any existing versions of the marking menu. Then, we use the very handy findPanelPopupParent MEL function to give us the parent to which we should bind our popupMenus. Having that we check if the editor we want exists in the name of the parent. I could also compare it directly to a string, but the actual panel has a number at the end and I prefer just checking the base name. Then, depending on which panel I am working in at the moment, I build the appropriate custom marking menu.

Don’t forget that you need to create a release command as well, to delete the marking menu so it does not get in the way if you are not pressing the hotkey. It is a really simple command, that I went over in my previous marking menu post.

The obvious limitation here is that we have a hotkey defined and we cant just do ctrl+alt+MMB for example.

Conclusion

So, yeah, these tend to be a bit trickier than just creating ones in the viewport, but also I think there is more to be desired from some of maya’s editors *cough* node editor *cough*, and custom marking menus help a lot.

So, recently I stumbled upon a djx blog blost about custom hotkeys and marking menus in different editors in maya. I had been thinking about having a custom hotkey marking menu, but was never really sure how to approach this, so after reading that post I thought I’d give it a go and share my process.

tl;dr: We can create a runtime command which builds our marking menu and have a hotkey to call that command. Thus, giving us the option to invoke custom marking menus with our own custom hotkeys, such as Shift+W or T for example, and a mouse click.

Disclaimer: I have been having a super annoying issue with this setup, where the “release” command does not always get called, so the marking menu is not always deleted. What this means is that if you are using a modifier like Shift, Control or Alt sometimes your marking menu will still be bound to it after it has been closed. Therefore, if you are using something like Shift+H+LMB, just pressing Shift+LMB will open it up, so you lose the usual add to selection functionality. Sure, to fix it you just have to press and release your hotkey again, but it definitely gets on your nerve after a while.

If anyone has a solution, please let me know.

I have written about building custom marking menus in Maya previously, so feel free to have a look as I will try to not repeat myself here. There I also talked about why I prefer to script my marking menus, instead of using the Marking menu editor, and that’s valid here as well.

So, let us have a look then.

RunTimeCommands

The first thing we need to do is define a runTimeCommand, so we can run it with a hotkey. That is what happens if you do it through the Marking menu editor and set Use marking menu in to Hotkey Editor, as well.

There a couple of ways we can do that.

Hotkey Editor

On the right hand side of the hotkey editor there is a tab called Runtime Command Editor. If you go on that one you can create and edit runTime commands.

Scripting it in Python

If you have multiple marking menus that you want to crate, the hotkey editor might seem as a bit of a slow solution. Additionally, if changes need to be made I always find it more intuitive to look at code in my favourite text editor (which is sublime by the way).

To create a runTime command we run the runTimeCommand function which for some reason does not appear in the Python docs, but I’ have been using maya.cmds.runTimeCommand successfully.

All we need to provide is a name for the command, some annotation – ann, a string with some code – c and a language – cl.

Here is an example

mc.runTimeCommand("exampleRunTimeCommand", ann="Example runTime command", c=commandString, cl="python")

Something we need to keep in mind when working with runTime commands is that we cannot pass external functions to them. We can import modules and use them once inside, but I cannot pass a reference to an actual function to the c flag, as I would do to menuItems for example. That means that we need to pass our code as a string.

Press and release

Now, that we know how to create the runTimeCommands let us see what we need these commands for.

As I mentioned, they are needed so we can access them by a hotkey. What that hotkey should do is initialize our marking menu, but once we release the key it should get rid of it, so it does not interfere with other functions. Therefore we need two of them – Press and Release.

Let us say we are building a custom hotkey marking menu for weight painting. In that case we will have something similar to the following.

  • mmWeightPainting_Press runTimeCommand – to initialize our marking menu
  • mmWeightPainting_Release runTimeCommand – to delete our marking menu

The way we bind the release command to the release of a hotkey is by pressing the small arrow to the side of the hotkey field.

Custom hotkey marking menu - release command hotkey

The Press command

import maya.cmds as mc # Optional if it is already imported

name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

import mmWeightPainting
reload(mmWeightPainting)

So, essentially what we do is every time we press our hotkey, we delete our old marking menu and rebuild it. We do this, because we want to make sure that our latest changes are applied.

Now, the lower part of the command is where it gets cool, I think. We can store our whole marking menu build – all menuItems – inside a file somewhere in our MAYA_SCRIPT_PATH and then just import it from the runTimeCommand as in this piece of code. What this gives us, is again, the ability to really easily update stuff (not that it is a big deal with marking menus once you set them up). Additionally, I quite like the modularity, as it means we can have very simple runTimeCommands not cluttered with the actual marking menu build. This is the way that creating through the Marking menu editor works as well, but obviously it loads a MEL file instead.

So, literally that mmWeightPainting file is as simple as creating all our marking menu items.

import maya.cmds as mc

mc.menuItem(l="first item")
mc.menuItem(l="second item")
mc.menuItem(l="North radial position", rp="N")
...

And that takes care of building our marking menu when we press our hotkey + the specified modifiers and mouse button. What, we do not yet have is deleting it on release, so it does not interfere with the other functionality tied to modifier + click combo. That is where the mmWeightPainting_Release runTimeCommand comes in.

The Release command

## mmWeightPainting_Release runTimeCommand
name = "mmWeightPainting"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

Yep, it is a really simple one. We just delete the marking menu, so it does not interfere with anything else. Essentially, the idea is we have it available only while the hotkey is pressed.

Hotkey

All that is left to be done is to assign a hotkey to the commands. There are a couple of things to have in mind.

If you are using modifiers for the popupMenu command – sh, ctl or alt – then the same modifiers need to be present in your hotkey as otherwise, even though the runTimeCommand will run successfully, the popupMenu will not be triggered.

In the above example

mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

we have specified the sh modifier. Therefore, the Shift key needs to be present in our hotkey.

Also, obviously be careful which hotkeys you overwrite, so you do not end up causing yourself more harm than good.

Conclusion

That’s it, it really is quite simple, but it helps a lot once you get used to your menus. Honestly, trying to do stuff without them feels so tedious afterwards.

I am not sure what it is, but there is something incredibly appealing in optimizing our workflows. I think a lot of it comes from the frustration of repeating the same actions over and over again. When you find a way to optimize that, it feels great. One of the easiest way to improve our rigging workflow is to script a custom marking menu with python. Another one, I have already written about is creating a custom shelf.

tl;dr: I will walk you through scripting your own custom marking menu with python, which is going to be easily shareable, extendable and maintainable. The code can be found here.

Here is how my main marking menu looks.

Custom marking menu in maya with python

I have found it is an immense help to have the commands I use most oftenly either in my marking menu or my shelf. It just saves so much time!

Okay, how do we go about creating one of these?

Well, we have two options – either build it with Maya’s native marking menu editor or script it with python.

The reasons I prefer scripting my marking menus in python are a few.
– The native editor does not give us all the available options for a marking menu, such as submenus.
– Updating from the editor is a pain in the butt.
– The editor does not scale nicely, if you want or need to support multiple marking menus.
– Doing it through the editor is boring.

Obviously, python fixes all these issues for us. Additionally, it is easy to share it with co-workers and keep it in a version control system. Okay, so you are sold now. Let us have a look at how to do it then.

The code that I will be going through is on this gist, but I will go through all of it, if you would rather write it yourself.

Custom marking menu with Python

Initialization

import maya.cmds as mc

MENU_NAME = "markingMenu"

We start very simple with the import of maya.cmds and giving a name to our custom marking menu. Now, the name is not very important because we do not ever see it, but maya does. So, in order for us to be able to update our custom marking menu, we need to be able to access it, and that is why we are giving it a name.

Then we have our markingMenu class. The main reason I went for a class is because we can encapsulate everything we need into it quite logically. I know that some people would prefer to have a function instead, which is absolutely fine, it is really a personal preference. Let’s have a look at the constructor.

Constructor

def __init__(self):

self._removeOld()
self._build()

As you can see, our constructor is very simple. We delete the old version of our custom marking menu if it exists and then we build are our new one in place.

Remove old marking menu

def _removeOld(self):
    if mc.popupMenu(MENU_NAME, ex=1):
    mc.deleteUI(MENU_NAME)

As I said, the reason we need to give a name to our marking menu is to be able to modify it after it is created. In our case, we are not really modifying it, but deleting it instead. We do this, so we have a clean slate for building our new marking menu.

The reason I have added this deleting and then building again functionality is just so we can painlessly make changes to our custom marking menu. For example, if I want to add a new button or change a label, I do not want to have to restart maya or do anything other than just running my code again. Or even easier, just importing my code again. That is why I said earlier, that this setup for a custom marking menu is very easily extendable and maintainable. Additionally, we can add a button to our marking menu which rebuilds it, so we only have to make our changes to the file, save it and then rebuild from within. We will have a look at that in the end of the post.

Building our custom marking menu

Now that we have had a look at preparing for our build let us have a look at the _build() method.

def _build(self):
    menu = mc.popupMenu(MENU_NAME, mm = 1, b = 2, aob = 1, ctl = 1, alt=1, sh=0, p = "viewPanes", pmo=1, pmc = self._buildMarkingMenu)

Another very simple method. I was surprised that maya does not have a specific markingMenu method. Instead, the popupMenu() command is used, which is actually nice, since it is a familiar one if you have worked with menus or shelf popups. Let us look at the arguments.

  • MENU_NAME – quite obviously this one sets the name of our custom marking menu.
  • mm – this one defines this popupMenu as a marking menu.
  • b – this is the mouse button we would like to trigger the marking menu. 1 – left, 2 – middle, 3 – right.
  • aob – allows option boxes.
  • ctl – defines the Ctrl button as a needed modifier to trigger the marking menu.
  • alt – defines the Alt button as a needed modifier to trigger the marking menu.
  • sh – defines the Shift button as a needed modifier to trigger the marking menu.
  • p – the parent ui element. For marking menus, that would usually be “viewPanes”, which refers to all of our view panels.
  • pmo – this flag declares that the command that we pass to the pmc flag should be executed only once and not everytime we invoke the marking menu. If this is false, everytime we trigger our custom marking menu, we will see our menus growing as more and more menuItems will be added.
  • pmc – this is the command that gets called right before the popupMenu is displayed. This is where we need to pass our method that actually builds all our buttons and menus – _buildMarkingMenu.

It is important to notice that when we pass the self._buildMarkingMenu we do not add brackets in the end as that would call the function instead of passing it as as reference.

Additionally, it is also important to think about the button flag and the modifiers. Obviously, Maya already has some marking menus and some functions related to mouse clicks and the ctrl, alt and shift modifiers. Therefore, we need to come up with a combination that does not destroy a function which we actually want to keep. That is why, for my marking menu I use the middle mouse button + alt + ctrl. There was some zooming function bound to that combination I believe, but I never used it so it was safe to override.

Actually building our custom marking menu

So, everything up to this point was to set us up for actually adding our commands to our menu. To be honest, it is as simple as everything we have already seen. Let us have a look at how we do this.

def _buildMarkingMenu(self, menu, parent):

## Radial positioned
mc.menuItem(p=menu, l="South West Button", rp="SW", c="print 'SouthWest'")
mc.menuItem(p=menu, l="South East Button", rp="SE", c=exampleFunction)
mc.menuItem(p=menu, l="North East Button", rp="NE", c="mc.circle()")

subMenu = mc.menuItem(p=menu, l="North Sub Menu", rp="N", subMenu=1)
mc.menuItem(p=subMenu, l="North Sub Menu Item 1")
mc.menuItem(p=subMenu, l="North Sub Menu Item 2")

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'")
mc.menuItem(p=menu, ob=1, c="print 'South with Options'")

## List
mc.menuItem(p=menu, l="First menu item")
mc.menuItem(p=menu, l="Second menu item")
mc.menuItem(p=menu, l="Third menu item")
mc.menuItem(p=menu, l="Create poly cube", c="mc.polyCube()")

The first thing to note is that our method receives menu and parent as arguments. These are passed automatically from the pmc flag on the popupMenu function in the _build() method.

Then we have the actual items in our custom marking menu. I like to split them logically in – radial and list blocks.

Radial positions

The radial positions are defined by the directions on a map – East, West, NorthEast, etc. Have a look at the following image.

Custom marking menu - Radial positions

You can have either commands in these slots or additional subMenus like so.

My personal preference here is to have just single commands instead of additional popups, as to invoke a submenu you need to hover on a position and wait a little bit. And I find that small delay quite frustrating.

So the way we create these radial items is by using the menuItem command. All we have to do is pass our menu argument as the parent (p), define a label (l), a radial position (rp) and the command (c) we want to execute on click.

Notice that if passing functions as commands we pass them without brackets, as that would call them instead. Have a look at the SE radial position for an example. Additionally, the functions need to be able to receive arguments as maya’s ui elements tend to pass some info about themselves to the commands they call. The way we do that is by just adding the *args argument.

def exampleFunction(*args):
    print "example function"

Additionally, we can call maya commands from our items. Have a look at the NE item. It is important to note here that even though we have imported maya.cmds as mc in this file, the commands we pass to our menuItems are going to be called from maya’s python environment. That means, that in order for mc.circle() to work, we need to have imported maya.cmds as mc somewhere in our maya session. You could either run it yourself in the script editor or a better solution would be to add it to your userSetup.py file. That is what I did, since I use a lot of python in maya I just do an import maya.cmds as mc in my userSetup.py.

Submenus

As I said we can have subMenu items, which essentially create another menu when you hover on them. The only thing we do is set the subMenu flag to 1. Have a look at one of these in our N radial position example. We store the menuItem with the subMenu flag in a variable, so we can use it as a parent for our following items. From then on, we just follow the same principle, we list menuItems, with the subMenu variable that we stored as a parent. Additionally, we can have deeper subMenus as well, though I do not think that would be great to work with.

Option boxes

Another very useful addition to our menuItems is adding option boxes. A lot of maya’s menus have those and they are a nice way to add additionall functionality without sacrificing space.

Usually, you would add them to commands that sometimes need their options to be changed, but also you can have different commands bound to them. For example, in my main marking menu I have the joint tool in one of my radial positions. But since I never mess with the options for that tool, in the option box I have a command which creates a joint under every object I have selected. Additionally, it names it with the name of the selected object and assumes it’s transformations.

The way we add an option box is very simple. All we need to do is create another menuItem after the one we want to add the option box to and we set the ob flag to 1. Have a look at the S radial position for an example.

List elements

In addition to our radial positioned items, marking menus have another menu beneath the South radial position. What is cool about this one is that it is an actual menu instead of just a single radial position, so we can have multiple items in it.

The way we create those items is by just specifying the marking menu as a parent. Remember it is passed as the menu argument to our method. So, if we do not specify a radial position the menuItem gets added to that menu which is south of the South position.

Again, as with all other menus, we can have submenus in there, by just setting the submenu flag to 1. My personal preference though is to have as few of these as possible, as I want everything to be easy to grab at first glance.

Icons

I have not added any icons in this example marking menu, but these can easily be added to every menuItem by using the i flag. All you need to do is place your custom icons somewhere in the XBMLANGPATH environment variable. To see what that is on your machine run this MEL command getenv XBMLANGPATH.

Additionally, you can use maya’s native icons, which you can browse through in the shelf editor. If you open that, pick any button on the right and next to the Icon Name field you can find a Browse Maya Icons button.

For example if I want to add maya’s standard icon to the S radial position I would do this.

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'", i="mayaIcon.png")

That results in the following image.

Custom marking menu - icon

And of course you can also pass full paths to icons which are not on maya’s XBMLANGPATH path.

Rebuilding our custom marking menu and loading it on startup

So, now that we know how to build our marking menus, let us have a look at how to actually initialize them with maya and how to rebuild them on the fly when we make changes. For building our marking menu we just initialize our markingMenu() class. For rebuilding we need to do the same thing, but we have several options for how we call our class.

Option 1: Just run it

I would say this is the simplest solution. If you are working in an external editor you can easily copy and paste your modified code in maya’s script editor, run it and you’re done. Since, we have added the rebuilding functionality – delete old and then build new one – the marking menu is updated everytime we run our code and then call the markingMenu() class in the end.

Even better if you have connected your external editor to maya via a commandPort command you can just run the code from there and that’ll update your marking menu as well. (I use this plugin for Sublime to do that.

The downside with this option is that everytime we open maya we will need to run our code. Which means that at some point you will get annoyed with doing it.

Option 2: Add it to your scripts path

Even though the first solution is quite simple this one is a bit nicer to work with. It also takes care of loading our marking menu on startup.

So, what we do here is we add the file containing our code – markingMenu.py – somewhere on our MAYA_SCRIPT_PATH. Again, you can run getenv MAYA_SCRIPT_PATH in MEL to get this path.

What this means is that now we can access our marking menu from within maya. Therefore we have a lot of options on how to build/rebuild our marking menu.

For building it on startup we just need to add the following code inside our userSetup.py.

import markingMenu
markingMenu.markingMenu()

When rebuilding though we need to do a reload statement. That is because when importing a module, python checks whether it is already imported and if so, just uses that instead. So if we make changes and then do import, our changes are not going to be imported. That is why we need to do reload(ourModule).

So in the case of reloading our marking menu, we will need to do the following.

import markingMenu
reload(markingMenu)
markingMenu.markingMenu()

This will ensure that whatever changes we have made to our code will be implemented.

Now that we have this rebuilding code, we can add it wherever we find most appropriate. Generally, I would say either a button on a shelf or from within the marking menu itself. Since a button on a shelf is trivial let us look at adding it to the marking menu.

Reloading from within our custom marking menu

The way we would do this, is just add the following menuItem.

mc.menuItem(p=menu, l="Rebuild Marking Menu", c=rebuildMarkingMenu)

Where the rebuildMarkingMenu refers to the following function.

def rebuildMarkingMenu(*args):
    mc.evalDeferred("""
reload(markingMenu)
markingMenu.markingMenu()
""")

I really don’t like using evalDeferred since it feels a bit dirty, but in this case we need it. The reason we need it is that we are rebuilding our marking menu from within. So if we do not use evalDeferred but directly call our markingMenu(), maya will have to delete the button which we have clicked, and that will error.

Conclusion

So, there we have it, we have built our own custom marking menu with Python. What is more, it is fully scripted, so making changes is very easy. Additionally, we can have that file in a version control system, so we can have a log of our changes. And of course, it is super easy to share with co-workers. The best thing about it, though, is how much time it saves in our day-to-day rigging tasks.

If you have been reading bindpose for a while and have seen my marking menu posts you probably know that I am very keen on getting my workflow as optimized as possible. I am a big fan of handy shelfs, marking menus, hotkeys, custom widgets, etc. The way I see it, the closer and easier our tools are to access the quicker we can push the rig through. That is why today we are having a look at using PySide to install a global hotkey in Maya (one that would work in all windows and panels) in a way where we do not break any of the existing functionality of that hotkey (hopefully).

If you have not used PySide before, do not worry, our interaction with it will be brief and pretty straightforward. I myself am very new to it. That being said, I think it is a great library to learn and much nicer and more flexible than the native maya UI toolset.

Disclaimer: The way I do this is very hacky and dirty and I am sure there must be a nicer way of doing this, so if you have suggestions please do let me know, so I can add it both to my workflow and to this post.

What we want to achieve

So, essentially, all I want to do here is install a global hotkey (PySide calls them shortcuts) on the CTRL + H combination that would work in all Maya’s windows and panels as you would expect it to – Hide selected, but if we are inside the Script editor it would clear the history.

Some of you might think that we can easily do this without PySide, just using maya’s hotkeys, but the tricky bit comes in from the fact that maya’s hotkeys are not functioning when your last click was inside the Script editor’s text field or history field. That means, that only if you click somewhere on the frames of the Script editor would that hotkey get triggered, which obviously is not nice at all.

Achieving it

So, let us have a look at the full code first and then we will break it apart.

from functools import partial
from maya import OpenMayaUI as omui, cmds as mc

try:
    from PySide2.QtCore import *
    from PySide2.QtGui import *
    from PySide2.QtWidgets import *
    from shiboken2 import wrapInstance
except ImportError:
    from PySide.QtCore import *
    from PySide.QtGui import *
    from shiboken import wrapInstance


def _getMainMayaWindow():
    mayaMainWindowPtr = omui.MQtUtil.mainWindow()
    mayaMainWindow = wrapInstance(long(mayaMainWindowPtr), QWidget)
    return mayaMainWindow


def shortcutActivated(shortcut):
    if "scriptEditor" in mc.getPanel(wf=1):
        mc.scriptEditorInfo(clearHistory=1)
    else:
        shortcut.setEnabled(0)
        e = QKeyEvent(QEvent.KeyPress, Qt.Key_H, Qt.CTRL)
        QCoreApplication.postEvent(_getMainMayaWindow(), e)
        mc.evalDeferred(partial(shortcut.setEnabled, 1))


def initShortcut():
    shortcut = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_H), _getMainMayaWindow())
    shortcut.setContext(Qt.ApplicationShortcut)
    shortcut.activated.connect(partial(shortcutActivated, shortcut))

initShortcut()

Okay, let us go through it bit by bit.

Imports

We start with a simple import of partial which is used to create a callable reference to a function including arguments. Then from maya we the usual cmds, but also OpenMayaUI which we use to get a PySide reference to maya’s window.

Then the PySide import might look a bit confusing with that try and except block, but the only reason it is there is because between maya 2016 and maya 2017 they switched PySide versions, and the imports had to change as well. So, what we do is we try to import from PySide2 (Maya 2017) and if it cannot be found we do the imports from PySide (Maya 2016).

Getting Maya’s main window

Even though, Maya’s UI is built entirely by Qt (PySide is a wrapper around Qt), the native elements are not usable with PySide functions. In order to be able to interact with these native bits we need to find a PySide reference to them. In the example for hotkeys we need only the main window, but depending on what you are trying to do you might have to iterate through children in order to find the UI element you are looking for. Therefore this _getMainMayaWindow function has become a boilerplate code and I always copy and paste it together with the imports.

The way it works is, using Maya’s API we get a pointer to the memory address where Maya’s main window is stored in memory. That’s the omui.MQtUtil.mainWindow() function. Then what we do is, using that pointer and the wrapInstance function we create a PySide QWidget instance of our window. That means that we can run any QWidget functions on Maya’s main window. In our hotkey example, though, we only need it to bind the hotkey to it.

The logic of the hotkey

The shortcutActivated function is the one that is going to get called every time we press the hotkey. It takes a QShortcut object as an argument, but we will not worry about it just yet. All we need to know is that this object is what calls our shortcutActivated function.

It is worth mentioning that this function is going to get called without giving Maya a chance to handle the event itself. So, that means that if we have nothing inside this function, pressing CTRL + H will do nothing. Therefore, we need to make sure we implement whatever functionality we want inside of this function.

So, having a look at the if statement, you can see that we are just checking if the current panel with focus – mc.getPanel(wf=1) – is the Script editor. That will return True if we have last clicked either on the frames of the Script editor windows or anywhere inside of it.

Then, obviously, if that is the case we just clear the Script editor history.

If it returns False, though, it means that we are outside of the Script editor so we need to let Maya handle the key combination as there might be something bound to it (In the case of CTRL+H we have the hiding functionality which we want to maintain). So, let us pass it to Maya then.

As I said earlier, Maya does not get a chance to handle this hotkey at all, it is entirely handled by PySide’s shortcut. So in order to pass it back to Maya, what we do is we disable our shortcut and we simulate the key combination again, so Maya can do it’s thing. Once that is done, we re-enable our shortcut so it is ready for next time we press the key combination. That is what the following snippet does.

shortcut.setEnabled(0)
e = QKeyEvent(QEvent.KeyPress, Qt.Key_H, Qt.CTRL)
QCoreApplication.postEvent(_getMainMayaWindow(), e)
mc.evalDeferred(partial(shortcut.setEnabled, 1))

Notice we are using evalDeferred as we are updating a shortcut from within itself.

Binding the function to the hotkey

Now that we have all the functionality ready, we need to bind it all to the key combination of our choice – CTRL + H in our example. So, we create a new QShortcut instance, which receives a QKeySequence and parent QWidget as arguments. Essentially, we are saying we want this key combination to exist as a shortcut in this widget. The widget we are using is the main maya window we talked about earlier.

Then, we use the setContext method of the shortcut to extend it’s functionality across the whole application, using Qt.ApplicationShortcut as an argument. Now the shortcut is activated whenever we press the key combination while we have our focus in any of the maya windows.

Lastly, we just need to specify what we want to happen when the user has activated the shortcut. That is where we use the activated signal of the shortcut (more info on signals and slots) and we connect it to our own shortcutActivated function. Notice that we are using partial to create a callable version of our function with the shortcut itself passed in as an argument.

And that’s it!

Conclusion

Hotkeys, marking menus, shelves, custom widgets and everything else of the sort is always a great way to boost your workflow and be a bit more efficient. Spending some time to build them for yourself in a way where you can easily reproduce them in the next version of Maya or on your next machine is going to pay off in the long run.

I hope this post has shown you how you can override maya’s default hotkeys in some cases where it would be useful, while still maintaining the default functionality in the rest of the UI.

If you know of a nicer way of doing this, please do share it!

So, painting skin weights. It is a major part of our rigging lives and sadly one of the few bits, together with joint positioning, that we cannot yet automate, though in the long run machine learning will probably get us 99% there. Untill then though, I thought I would share some of my tips for painting skin weights with maya’s native tools, since whenever I would learn one of these I felt stupid for not finding it out earlier as, more often than not, it was just so simple.

I am sure a lot of you are familiar with these, but even if you learn just a single new idea about them today, it might boost your workflow quite a bit. Additionally, I know that a lot of you are probably using ngSkinTools and literally everyone I know who works with it says they cannot imagine going back. So I am sure that some of the things I am going to mention are probably already taken care of ngSkinTools, but if you, like me, have not had the chance to adopt it yet, you might find these helpful.

I am going to list these in no particular order, but here is a table of contents.

Contents

  1. Simplifying geometries with thickness and copying the weights
  2. Using simple proxy geometry to achieve very smooth weights interpolation quickly
  3. Duplicate the geometry to get maya default bind on different parts
  4. Copy and paste vertex weights
  5. Use Post as normalization method when smoothing
  6. Move skinned joints tool
  7. Reveal selected joint in the influence list
  8. Some handy hotkeys
  9. Average weights
  10. Copy and paste multiple vertex weights with search and replace
  11. Print weights

So with that out of the way, let us get on with it.

Simplifying geometries with thickness and copying the weights

This one comes in very handy when we are dealing with complex double-sided geometries (ones that have thickness). The issue with them is that when you are painting one side, the other one is left unaffected, so as soon as an influence object transforms the two sides intersect like crazy. That is often the case with clothes and wearables in general.

The really easy way to get around this is to
1. Make a copy of the geometry
2. Remove the thickness from it (when having a good topology it is as simple as selecting the row of faces which creates the thickness and deleting it together with one side of the geo)
3. Paint the weights on that one
4. Copy the weights back to the original geometry

Painting skin weights tips - Using a one sided proxy geometry when working with thickness

Now, a really cool thing that I had not thought of untill recently is that even if I have started painting some weights on the double sided geometry to begin with I can also maintain them, by copying the weights from the original one to the simplified one before painting it, so I have a working base.

That means, that if I have managed to paint some weights on a double sided geometry that kind of work, but the two sides are not behaving 1 to 1, I can create a simplified geo, copy the weights from the original one to the simplified and then copy them back to get the 1 to 1 behaviour I am looking for.

Using simple proxy geometry to achieve very smooth weights interpolation quickly

This one is very similar to the first one, but I use it all the time and not only on double-sided geometries.

Very often there are geometries that have some sort of a detail modeled in them that make it hard for weight painting smooth weights around it.

Consider the following example. Let us suppose that we need this geometry to be able to stretch smoothly when using the .translateX of the end joint.

Tips for painting skin weights in maya - Using a simple geometry to copy weights to models which are hard to smooth weights for.

Doesn’t look great with default skinning, but also if I try to block in some weights and smooth them, it is likely that maya won’t be able to interpolate them nicely. To go around it, I’d create a simple plane with no subdivisions so I can have a very nice smooth interpolation from one edge to the other.

Tips for painting skin weights - Using a simple plane without subdivisions to achieve a smooth weights interpolation for copying to complex geometries.

Copying this back to the initial geometry gives us this.

Tips for painting skin weights - Smooth skinned complex geometry using weights from a simple plane.

Very handy for mechanical bits that have some detail in them and also need to be stretched (happens very often in cartoon animation).

Duplicate the geometry to get maya default bind on different parts

So, very often I have to paint the weights on a part of a geometry to a bunch of new joints while I still need to maintain the existing weights on the rest of it. More often than not, I would be satisfied with maya’s default weights after a bind, but obviously if I do that it will obliterate my existing weights.

What I do in such cases is make a copy of the geometry and smooth bind it to only the new joints. Then I select the vertices on the original geometry that comprise the part I want the new influences in and I use the Copy skin weights from the duplicated one to the selected vertices. If the part is actually separated from the rest of the geometry that should do it, but if it s a more of an organic shape, there is going to be some blending of the new weights with the ones surrounding them.

I could imagine, though, that having the ability to have layers and masks on your skin weights would make this one trivial.

Copy and paste vertex weights

I am guilty of writing my own version of this tool just out of the ignorance of not knowing that this exists. Basically what you can do is select a vertex, use the Copy vertex weights then select another one (or more than one) and use the Paste vertex weights command to paste them. Works cross-geometries as well.

A cool thing about the tool that I wrote to do this is I added a search and replace feature that would apply the weights to the renamed joints. For example if I am copying a vert from the left arm and I want to paste it on the right I would add “L_” to “R_” to my replacement flags.

Use Post as normalization method when smoothing

So, I have met both people who love and who hate post. I think the main reason people dislike it is because they don’t feel comfortable with their weights being able to go above 1.0, but I have to say that sometimes it is very handy. Especially for smoothing. Everyone knows how unpredictable maya’s interactive smoothing is, and that’s understandable since in a lot of cases it is not immediately obvious where should the remaining weights go to.

Smoothing on post is 100% predictable which I think is the big benefit. The way it works is that it smooths out a joint’s influence by itself, without touching any of the other weights. That means that the weights are not normalized to 1.0, but instead of verts shooting off in oblivion post normalizes them for our preview. That is also why it is not recommended to leave skinClusters on Post as the weights are going to be normalized on deformation time which would be slower.

So more often than not my workflow for painting weights would be to block in some harsh rigid weights, then switch to Post and go through the influences one by one flooding them with the Smooth paint operation once or twice.

Move skinned joints tool

I am not sure which version of maya did this tool come in, but I learned of it very recently. Essentially you can select a piece of geo (or a joint) and run the Move skinned joints tool, then you can transform the joint however you like or you can also change the inputs going into it without affecting the geometry, though, you’d have to be careful to not change the tool or the selection as that would go out of the Move skinned joints tool. Ideally any other changes than just moving/rotating them about should be ready to be ran in the script editor.

I would not recommend using this for anything else than just testing out different pivot points. Doing it for actual positioning in the final skinCluster feels dirty to me.

Reveal selected joint in the Paint skin weights tool influence list

Only recently I found out what this button does.

Paint skin weights tool - reveal selected joint in influence list

It scrolls the list of influences to reveal the joint that we have selected, which is absolutely brilliant! Previously, I hated how when I need to get out of the Paint skin weights tool and then get back inside of it, the treeView is always scrolled to the top of the list. Considering that the last selection is maintained, pressing that button will always get you back to where you left off. Even better, echoing all commands gives us the following line of MEL that we can bind to a hotkey.

artSkinRevealSelected artAttrSkinPaintCtx;

Some handy hotkeys

I have learned about some of these way too late, which is a shame, but since then I’ve been using them constantly and the speed increase is immense. I hate navigating my mouse to the tool options just to change a setting or value.

  • CTRL + ALT + C – Copy vertex weights
  • CTRL + ALT + V – Paste vertex weights
  • N + LMB (drag) – Adjust the value you are painting with
  • U + LMB – Marking menu to change the current paint operation (Replace, Add, etc.)
  • ALT + F – Flood surfaces with current values

For more of these head on to the Hotkey editor, then in the Edit hotkeys for combobox go for Other items and open up the Artisan dropdown.

From here on I have added some of the functionalities that I have written for myself, but sadly the code is very messy to be shared. Luckily, it is not hard at all to write your own (and it will probably be much better than mine), but if you are interested, do let me know and I can clean it up and share it at some point.

Average weights

This one I use a lot. What it does is, it goes through a selection of verts and calculates the average weights for all influences and then goes through the selection once more and applies that average calculated weight. Essentially, what this results in is a rigidly transforming collection of verts. Stupidly simple, but very useful when rigging mechanical bits, which should not deform. Also I have used it in the past on different types of tubings and ropes where there are bits (rings, leafs, etc.) that need to follow the main deformation but not deform themselves.

Copy and paste multiple vertex weights with search and replace

In addition to the above mentioned copy and paste vertex weights, I have written a simple function that copies a bunch of vertex weights and then pastes them to the same vertex IDs on a new mesh. It is not very often that we have geometries that are copies of each other, but if we do this tool saves me a lot of time, because I can then just skin one of them, copy the weights for all verts and then paste them to the other geometry using the Search and Replace to adjust for the new influences.

Comes in particularly handy for radially positioned geometries where mirroring will not help us a lot.

Print weights

Quite often I’d like to debug why something is deforming incorrectly, but scrolling through the different influences can get tedious especially if you have a lot of them. So I wrote a small function that finds the weights on the selected vertex and prints them to me.

This is the kind of output I get from it.

#####################################
joint2 : 0.635011218122
joint1 : 0.364988781878
#####################################

As I said, there is a lot of room for improvement. It works only on a single vert at the moment, but I could imagine it being really cool to see multiple ones in a printed table similar to what you would get in the component editor.

What would be even cooler would to use PySide to print them next to your mouse pointer.

Conclusion

Considering that we spend such a big chunk of our time on painting weights we should do our best to be as efficient and effective as possible. That is the reason I wanted to share these, as they have helped me improve my workflow immensely and I hope you would find some value in them as well.

I was so amazed the first time I learned about the script node. Now that I look at it, I feel like it is a necessity, for sure, but back before I had no idea that they, callbacks and scriptJobs existed, I often thought that it would be amazing to have a way of running code at different points of interaction with rigs (and any other maya scene). So when I finally learned about them, naturally, I tried to think of cool things I could do with them as, and today I will have a brief look at some of these alleged cool use cases and some not really cool ones.

Disclaimer: Raffaele over at the Cult of rig has some great videos that go over callbacks, script nodes and a great explanation of how they fit in the overall maya event loop. Once I saw those videos everything started making much more sense, when working with callbacks and script nodes, so I cannot recommend them enough. He goes over the use of script nodes to manage callbacks and I will not be going over that, so definitely have a look at his latest videos.

Additionally, this post will be mostly speculative since a lot of these ideas are just that – ideas that I have not yet tried, but sound like they might be cool.

Also, there are a few examples of unethical behaviour mentioned as potential uses of the script node, so I want to make it clear that I DO NOT encourage them, but instead I want people to be familiar with them, in order to be able to protect themselves.

Table of contents

script nodes in rigging

When I think about script nodes I instantly picture animators opening/referencing scenes and cool stuff happening without them having to touch anything, and yeah, I suppose that is probably the major use case scenario. I wouldn’t imagine myself needing a script node in my files, since I can easily run the code I want to run whenever I want to.

Rigging systems

That being said, though, say we have a rigging system, where we script most of the rig, but we still do certain things in the viewport as well, quite possibly through some cool custom UI we’ve built. That UI will very likely rely on some metadata inside of the scene and probably some message connections between nodes. Wouldn’t it be nice if this UI auto-populates itself and pops up ready to be directly used, without you having to touch anything? Sure, you can probably do that with just pressing a single button after the scene is opened, but I think having it happen automatically feels cooler.

Additionally, I know a lot of people have debug modes of their rigs, which take care of what I am about to say, but if you don’t it might be nice to have a script node remove the restrictions you have added for animators and turn on any diagnostics you have in the scene.

Analytics

Recently, I have been looking into running some minor analytics on rigs and I have to say I see why every marketing person out there praises analytics. It is an eye-opening experience seeing how many unitConversion nodes you have in the scene and maybe plotting that against the performance results you are getting to see if they are slowing you down (they probably are) and by how much (probably not much).

Again, this one is going to be purely speculative, but maybe we can gather some data on our own practices when rigging. Say for example, query the time spent working on a specific rig or maybe even run performance tests on opening or closing the file in debug mode and store that data over time.

script nodes in animation

So, even though, there might be some use cases for script nodes in rigging, I think they can be really useful for automating boring stuff for animators. Whether it is going to be related to building and loading certain UIs, creating certain connections between assets or similarly to the previous paragraph – analytics – we can use the script node to make animator’s and our work slightly easier, more intuitive and informative.

Attaching props

This is probably the best use case of the script node that I know of. I know different places and teams definitely have different approaches to this issue, whether it may be custom tools that are used to bring in and attach props or, god forbid, having all props in a character rig file and using switches to turn them on and off, it is a common aspect of pipelines. I think a nice solution are script nodes. Let’s imagine the following case.

Say we have a character that needs to have all kinds of different weapons, clothes and other wearable props. Let’s suppose all these are their own separate assets, so maintaining them is easier and nicer. The way script nodes would help us in this case is by executing a piece of code that would attach our prop to our character on bringing that prop in a scene where the character exists. That would be a nice and simple solution. Of course, it could lead to some issues, for example, having animators bring the prop in before the character or maybe have multiple instances of the character, so our prop does not know to which it should attach, but generally these issues are either solved or prevented by having some pipeline rules.

Building interfaces

This one is probably more appropriate for freelancers, free rigs and cases where the people working on a project don’t necessarily share a pipeline.

Animators often have to do a lot of animation in a short time. Anything we can do to help them out is going to be great for our production. A nice way we can do that is by giving them easy to use interfaces to speed up their workflow. I know a lot of animators use pickers, so maybe we can use script nodes to build them with all their jazz on the spot. Additionally, they might be like me and love marking menus. Wouldn’t it be great to be able to build it for them on the spot and maybe even show them a message, so they know how to bring it up? Then of course, clean up after ourselves when they close the scene, so we do not mess about with their other work.

script nodes in pipeline

Now even though script nodes can be quite handy for rigging and animation, I think where they really shine is in pipeline. Granted, pipeline would probably use scriptJobs to do most of the things I am going to mention, maybe small teams or even freelancers can sort of simulate a pipeline by using script nodes.

Analytics

I talked about analyzing our behaviour when rigging, but I think it is much more practical to actually analyze the working files of animators and lighters, as the data in there will probably be a bit more interesting. For example, we can run performance tests on certain scenes when opening/closing them and then at the end of a production identify problematic areas, so we can potentially have a look at improving them.

Additionally, this one feels a bit unethical, but if we are upfront about it, it might be cool to store some data on how people interact with our rigs. Create some callbacks on nodes and attributes we are interested in, and have a look at how they are being used. Then when closing the file save that data somewhere we can have a look at it, or in any other way send it to ourselves. We might just find that 50% of the control we are adding to rigs is not actually being utilized.

Licensing

How can we prevent our files from being used by people we do not want to use them? I think “do not let them have our files in the first place” is the reasonable solution, but maybe we cannot entirely prevent that. In cases like that, we can add a level of protection by using script nodes.

Since we can fire a script on opening a scene file, that means that we have some room for licensing our rigs. Now, while I am not entirely sure what the best way for forbidding access to a rig would be, I have a couple of ideas.

Inform us

This first option is not necessarily forbidding anyone from using our rig, but we might be able to setup a way of informing us of unauthorized access to our rig. For example, we can check if the user is unauthorized by looking for a specific environment variable that members of our team would have, looking up maya’s license or maybe even the OS username. If we find that the user should not be using our file we can send the information to ourselves by using a simple HTTP request.

I know this can easily be bypassed, but chances are it might just work in certain cases.

Obviously, this involves an amount of tracking users, which I really DO NOT encourage, but I could imagine certain people in certain cases might want to do that.

Break the rig

If the file is being opened as opposed to referenced we have the ability to break connections, delete files, etc. If that is the case we can easily destroy our rig on opening it, so people cannot even see it.

If it is being referenced though, we do not have that level of control. In cases like this it might be worth having an obscure way of breaking your rig, built into the rig itself. I do not think this could ever be bulletproof, but I think it is certainly possible to make it way too annoying for anyone to mess with it. That being said, it is likely that doing that is going to introduce some overhead in your rigs, so it is not an entirely too graceful way of going about it.

Close or hang maya

I feel like I am getting silly here, but I think that for people that really do not want anyone touching their rigs, they can certainly do a maya.cmds.quit(f=1) in the script node and be done with it. Additionally, if you would like to be extra nasty, I suppose you could do something along the lines of

import maya.cmds as mc

while True:
    mc.warning("All work and no play makes Jack a dull boy!")

Alternatively, if you do not want to be nasty I think the nicest way of denying access would be to just unload or remove the reference, but you would need to have a stable way of finding the reference node. If you cannot do that, you could also do a mc.file(new=1, f=1) to pop out of the scene.

script nodes security

So, escalating from the previous section, I am sure you could imagine that if we could hang and crash maya we probably can do much worse things. Since we are able to run python, even though we have a smaller than the default amount of packages included with maya, we are able to actually execute code entirely unrelated to maya. You know, create/read folders and files, run native OS commands, etc. What if we create and read files from the user’s machine that we really shouldn’t be? Additionally, there are included libraries in Maya that are making it trivial to send data over the internet. You can see where I am going with this.

I do not want to carry on mentioning unethical schemes, but I do want to raise your awareness about the extent to which script nodes can be used. Essentially, everything that is available to the user running maya is available to the script node and therefore to the person that published that file. Which leads us to the next point.

Protection

Now, depending on your OS, the ways that we can protect ourselves from malicious vary, but there are a couple of concepts that we can apply to all of them.

Careful when downloading files

The easiest thing we can do is to not use any files created by other people. Depending on what you are doing in Maya that might be okay, but I for example, very often open popular rigs to see how have they done certain things and how can I improve on top of them.

If you like me do occasionally load up files from the web, there is another check you could do. Open the .ma file with a text editor and check if there are script nodes in there. Since text editors can have a hard time with larger files, you can always write a small python function to check that for you. If the file is a binary file, though, you are out of luck and there is no way to check what is inside of it other than opening it.

Permissions

The obvious one. If the user running maya does not have access to a certain directory, maya doesn’t and hence the script node doesn’t. Please DO NOT run maya as a super user or administrator as that would obviously give maya access to anything.

The practical way of doing it I would say is to create a specific user which is going to run Maya and give it permissions only under a specific directory where you would store all your Maya work. That way, everything else is out of reach.

Firewall

Limit Maya’s access to the internet. That can be a reasonable solution, but Autodesk probably wouldn’t be happy with it, as that way you cannot sign into your Autodesk account.

Conclusion

As you can see script nodes can be really powerful in terms of use cases in a proper production or freelancing and also powerful in malicious ways. I like to believe, though, that people in CG are generally quite intelligent and nice, so therefore I can safely say I am not troubled by the potential exploits.

I remember the first time I tried to set up a seamless IK FK switch with Python vividly. There was this mechanical EVA suit that I was rigging for a masterclass assignment at uni given by Frontier. The IK to FK switching was trivial and there were not many issues with that, but I had a very hard time figuring out the FK to IK one, as I had no idea what the pole vector really is and also, my IK control was not oriented the same way as my FK one.

Im sure that throughout the web there are many solutions to the problem, but most of the ones I found were in MEL and some of them were a bit unstable, because they were relying too much on the xform command or the rotate one with the ws flag, which I am assuming causes issues sometimes when mapping from world space to relative, where a joint will have the exact same world rotation, so it looks perfectly, but if you blend between IK and FK you can see it shifting and then coming back in place. That’s why I decided to use constraints to achieve my rotations, which seems to be a simple enough and stable solution.

EDIT: It seems like even with constraints it is possible to get that issue in the case where the IK control is oriented differently. What fixes is though is switching back and forth once more.

Here is what we are trying to achieve

Seamless IK FK swich demo

Basically, there is just one command for the seamless IK FK Switch, which detects the current kinematics and switches to the other one maintaining the pose. I have added the button to a custom marking menu for easier access.

So, in order to give you a bit of a better context I have uploaded the example scene that I am using, so you can have a look at the exact structure, but feel free to use your own scene with IK/FK blending setup. The full code (which is very short anyway) is in this gist and there are three scene files in here for each version of our setup. The files contain just a simple IK/FK blending system, on which we can test our matching setup, but with different control orientations.

It is important to understand the limitations of a seamless IK FK switch before we dive in. Mainly, I am talking about the limited rotation of the second joint in the chain, as IK setups allow for rotations only in one axis. What this means is that if we have rotations in multiple axis on our FK control for that middle joint (elbow, knee, etc.) the IK/FK matching will not work properly. All this is due to the nature of inverse kinematics.

Also, for easier explaining I assume we are working on an arm and hand setup, but obviously the same approach would work for any IK/FK chain.

We will consider three cases:
All controls and joints are oriented the same
IK Control oriented in world space
IK Control and IK hand joint both oriented in world

Again, you do not have to use the same file as I do as it is just an example, but it is important to be clear on the existing setup. We assume that we have an arm joint chain – L_arm01_JNT > L_arm02_JNT > L_arm03_JNT and a hand joint chain – L_hand01_JNT > L_hand02_JNT with their correspondent IK and FK chains – L_armIk01_JNT > …, L_armFk01_JNT > …, etc. These two chains are blended via a few blendColors nodes for translate, rotate and scale into the final chain. The blending is controlled by L_armIkFk_CTL.fkIk. Then we have a simple non-stretchy IK setup, but obviously a stretchy one would work in the same way. Lastly, the L_hand01_JNT is point constrained to L_arm03_JNT and we only blend the rotate and scale attributes on it, as otherwise the wrist becomes dislocated during blending, because we are interpolating linearly translation values.

Now that we know what we have to work with, let us get on with it.

Seamless IK FK Switch when everything shares orientation

So, in this case, all of our controls and joints have the exact same orientation in both IK and FK. What this means is that essentially all we need to do to match the kinematics is to just plug the rotations from one setup to the other. Let’s have a look. The scene file for this one is called ikFkSwitch_sameOrient.ma

IK to FK

This one is always the easier setup, as FK controls generally just need to get the same rotation values as the IK joints and that’s it. Now, initially I tried copying the rotation via rotate and xform commands, but whenever a control was rotated a bit too extreme these would cause flipping when blending between IK and FK, which I am assuming is because these commands have a hard time converting the world space rotation to a relative one, causing differences of 360 degrees. So, even though in full FK and full IK everything looks perfect, in-between the joint rotates 360 degrees. Luckily, maya has provided us with constraints which have all the math complexity built in. Assuming you have named your joints the same way as me we use the following code.

mc.delete(mc.orientConstraint("L_armIk01_JNT", "L_armFk01_CTL"))
mc.delete(mc.orientConstraint("L_armIk02_JNT", "L_armFk02_CTL"))
mc.delete(mc.orientConstraint("L_handIk01_JNT", "L_handFk01_CTL"))

mc.setAttr("L_armIkFk_CTL.fkIk", 0)

As I said, this one is fairly trivial. We just orient each of our FK controls to match the rotations of the IK joints. Then in the end we change our blending control to FK to finalize the switch.

FK to IK

Now, this one was a pain the first time I was trying to do it, because I had no idea how pole vectors worked at all. As soon as I understood that all we need to know about them is that they just need to lie on the same plane as the three joints in the chain, it became easy. So essentially, we need to place the IK control on the FK joint to solve the end position. And then to get the elbow (or whatever your mid joint is representing) to match the FK, we just place the pole vector control at the exact location of the corresponding joint in the FK chain. So, we get something like this.

mc.delete(mc.parentConstraint("L_handFk01_JNT", "L_armIk_CTL"))
mc.xform("L_armPv_CTL", t=mc.xform("L_armFk02_JNT", t=1, q=1, ws=1), ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Now even though this does the matching job perfectly it is not great for the animators to have the control snap at the mid joint location as it might go inside the geometry, which is just an unnecessary pain. What we can do is, get the two vectors from arm01 to arm02 and from arm03 to arm02 and use them to offset our pole vector a bit. Here’s the way we do that.

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

So, since xform returns lists in order to subtract them to get the vectors we just loop through them and subtract the individual elements. If you are new to the shorter loop form in Python have a look at this. Then once we have the two vectors we add 75% of them to the position of the arm02 FK joint and we arrive at a position slightly offset from the elbow, but still on the same plane, thus the matching is still precise. Then our whole FK to IK code would look like this

mc.delete(mc.parentConstraint("L_handFk01_JNT", "L_armIk_CTL"))

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Seamless IK FK switch when the IK control is oriented in world space

Now, in this case, the orientation of the IK control is not the same as the hand01 joint. I think in most cases people go for that kind of setup as it is much nicer for animators to have the world axis to work with in IK. The scene file for this one is called ikFkSwitch_ikWorld.ma.

The IK to FK switch is exactly the same as the previous one, so we will skip it.

FK to IK

So, in order to get this to work, we need to do the same as what we did in the previous case, but introduce an offset for our IK control. How do we get this offset then? Well, since we can apply transformations only on the controls, we need to calculate what rotation we need to apply to that control in order to get the desired rotation. Even though, we can calculate the offsets using maths and then apply them using maths, we might run into the same issue with flipping that I discussed in the previous case. So, instead, a much easier solution, but somewhat dirtier is to create a locator which will act as our dummy object to orient to.

Then, in our case where only the IK control is oriented differently from the joints, what we need to do is create a locator and have it assume the transformation of the IK control. The easiest way would be to just parent it underneath the control and zero out the transformations. Then parent the locator to the L_handFk01_JNT, as that’s the one that we want to match to. Now wherever that handFk01 joint goes, we have the locator parented underneath which shares the same orientation as our IK control. Therefore, just using parentConstraint will give us our matching pose. Assuming the locator is called L_hand01IkOfs_LOC all we do is this.

mc.delete(mc.parentConstraint("L_hand01IkOfs_LOC", "L_armIk_CTL"))

This will get our wrist match the pose perfectly. Then we apply the same code as before to get the pole vector to match as well and set the IK/FK blend attribute to IK.

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Seamless IK FK switch when the IK control and joint are both oriented in world space

Now, in this last scenario, we have the handIk01 joint oriented in world space, as well as the control. The reason you would want to do this again is to give the animators the easiest way to interact with the hand. In the previous case, the axis of the IK control do not properly align with the joint which is a bit awkward. So a solution would be to have the handIk01 joint oriented in the same space as our control, so the rotation is 1 to 1 and it is a bit more intuitive. The scene for this one is ikFkSwitch_ikJointWorld.ma and it looks like this.

It is important to note that the IK joint is just rotated to match the position of the control, but the jointOrient attributes are still the same as the FK one and the blend one.

Seamless IK FK Switch with IK control and joint oriented in world space
IK FK Switch with IK control and joint oriented in world space

So again, going from IK to FK is the same as before, we are skipping it. Let us have a look at the FK to IK.

FK to IK

This one is very similar to the previous one, where we have an offset transform object to snap to. The difference is that now instead of having that offset be calculated just from the difference between the IK control and the FK joint, we also need to adjust for the existing rotation of the IK joint as well. So, we start with our locator the same way as before – parent it to the IK control, zero out transformations and parent to the handFk01 joint. And then, the extra step here is to apply the negative rotation of the IK joint to the locator in order to get the needed offset. So, this calculation would look like this.

ikRot = [-1 * mc.xform("L_handIk01_JNT", ro=1 ,q=1)[i] for i in range(3)]
mc.xform("L_hand01IkOfs_LOC", ro=ikRot, r=1)

We just take the rotation of the IK joint and multiply it by -1, which we then apply as a relative rotation to the locator.

And then again, as previously we just apply the pole vector calculation and we’re done.

Conclusion

So, as you can see, scripting a seamless IK FK switch is not really that complicated at all, but if you are trying to figure it out for the first time, without being very familiar with rigging and 3D maths it might be a bit of a pain. Again, if you want to see the full code it is in this gist.