python

When it comes to rigging in a production or many productions the name of the game is reusability and automation. In that sense rigging is very similar to traditional software development, as we also have to build systems that are easily maintainable, extensible and reusable. There are a few ways to achieve that in rigging and today I am going to look at some of them. The reason I wanted to write this is that rigging systems are one of the big things I wished I had learned earlier.

Disclaimer: This post will probably come out very opinionated and those opinions are going to be based on my own experience, which really, is not that extensive. Additionally, even though there are going to be informative bits, I am writing this more of a way to share my thoughts instead of trying to teach anything as again, I am not really qualified in any way to do that.

Also, please bear in mind that the larger portion of this post is going to be very speculative since I am talking about tools that I have not really used myself.

A rigging system

I just want to briefly go over what I mean by rigging systems. Essentially, everything that takes a model and produces a rig out of it is a rigging system to me. That means if you just rig an entire thing manually you are the rigging system. If you use a tool similar to maya’s HumanIK that is your rigging system. That means that any system that by following a set of instructions can on it’s own produce a rig is a rigging system. Other than just building actual node networks, rigging systems should provide easy ways to save and load a bunch of properties and settings such as deformer weights, blendshapes, control shapes, etc.

Rigging system types

With the definition out of the way we can have a look at the types of rigging systems out there.

Auto rigging tool

Disclaimer: This is going to be the most speculative portion of this post, as I have never used one of these solutions, other just having a brief look at them.

The auto rigging tool is a rigging system which takes care of everything we talked above by providing you with some sort of a guiding system to define the proportions and often the type of a rig (biped, quadruped, wings, .etc) and then using these guides, it builds node networks which become rig components.

Some examples of auto rigging tools are maya’s HumanIk which I mentioned above and also a popular non-Autodesk one is Rapid Rig.

There are a lot of them online and also they are usually a big part of rigging showreels as it seems that every rigger who starts learning scripting, goes for an auto rigging solution at some point. Including me.

Now, the problem that I have with this kind of rigging systems is the lack of extensibility. I have not seen an auto rigging tool with any sort of an API so far. That means that, for every rig you built you have only a limited number of available components (that number might be a large one but still limited). What I mean by that, there is probably only one arm component available and even though there might be many options on how to build that arm, there is a chance you are not going to find what you are looking for and would like to insert your own logic somewhere in that component, but there is no way to do that.

Mentioning the many options brings me to the next issue I see with auto rigging tools – performance and clutter. The way I see it, the more options you want to support inside of just one single component, the more clutter you would introduce in the logic of that component, in order to accommodate those options. Additionally, if everything is happening behind the scenes, I have no other way of knowing what the tool creates than just opening the node editor and having a look at the networks, which as you could imagine is not going to be fun on large rigs. That opaqueness scares me, as I would not know about potential node clutter introduced in my scene.

That brings me to my next point about auto rigging tools, which is the fact that everything is stored baked down in the scene. What I mean by that is, the auto rigging tool might give you some options for rebuilding parts of the rig after they have been created, but ultimately everything we store is baked into node networks. Yes we can save our weights and maybe some properties outside of the scene file, but these would be things that go on top of the auto rig product. There is no way to store how we actually constructed that rig. Then if I need to change the position/orientation of a joint, how do I go about that? What if, god forbid, the proportions of the model have changed? Do I delete everything and rebuild it? And if I have to do that, what happens with the parts of the rig that I have added on top of the auto rig, do I need to manually rebuild them as well?

The last thing I want to mention about auto rigging tools is UI. I mean, it is usually bloody horrible. I think it is probably all down to the native UI tools that maya gives us, which all feel very clunky. They just don’t seem to work for anything as complex as a rigging system. All of the auto rigging tools I have seen make an extensive use of tightly packed buttons, checkboxes and text fields in loads of collapsible sections or tabs, which just doesn’t seem to cut it in 2017. Again, I think the main issue there is that Maya’s native tools are just not enough to build anything more intuitive. That being said, PySide has been available for a while.

Maya Human IK rigging system UI

So, if you are going to be building an auto rigging tool, please keep in mind the following ideas can improve your work with that rigging system a lot

  • creating some sort of an API for easily extending/modifying the functionality of the tool, mainly by creating new or editing existing components
  • storing information about the actual building of the rig instead of just the baked down version of it, in order to enable you to easily rebuild it when changes need to happen
  • create a more intuitive UI than what is already out there. Node graph maybe?

Rigging framework

Going beyond auto rigging tools, we have rigging frameworks. Those are systems which do not necessarily have any high levels components such as arms/legs/spines built into them, but instead they provide you with the tools to create such components and save them for later use. The only system of this type that I know of is mGear. Incidentally, it does actually provide a high level modular system called Shifter that does give you everything that an auto rigging tool would. The good thing here, though, is that using the actual framework you can build your own modular rigging system and extend and modify it a lot.

Now, since I have never actually used it take everything I say with a grain of salt, but from what I understand, similarly, to an auto rigging tool you would build everything in the viewport and then save it as a baked down version. I do not know how easy or difficult rebuilding components is, but if anything has been built on top of them it would have to be also manually rebuilt.

What I really like about mGear, though, is the open source aspect of it. The fact that you can grab it and build out of it a rigging system that would suit your needs perfectly, is amazing.

Guidable modules

Now, this one I think is the only system that you can create yourself without any scripting. Even though, it might be a bit slower to work with, I think in terms of results you would be able to get everything you get out of an auto rigging tool.

So, what I mean by guidable modules is, say, storing a rigged IK chain with all the controls on it in a file and then when you need an IK chain in a rig you bring the one from the file. The rigged IK chain would have some sort of guides (usually just locators) that would be able to reposition the joints and stretch the chain without actually introducing any values on your controls and also the stretch values would be recalculated so there is no actual stretch on the chain, but instead the modified version becomes the default state.

I know that sounds a bit weird and it probably won’t sit well with many of you, since referencing/importing files is often considered dirty because of all the stuff that gets carried across. That being said, if you clean up after yourself, that would not be an issue.

Additionally, if you are referencing the components in the scene, you can modify the components themselves and the changes would be carried across all of your rigs utilizing that component.

What is more, the same idea can easily be applied to a rigging framework or an auto rigging tool you built yourself, so it removes the issue which arises when changes need to be made.

Rigging framework with modular rigging system through an API

Now, the last rigging system I want to talk about is one that combines aspects of all previously mentioned ones, but in a way where extendability, maintainability and reusability are all taken care of. That comes at the expense of not having a UI, having rules and conventions and requiring a thorough programming knowledge.

The way an user interacts with this rigging system is entirely through an API. The way the rigs are stored is in actual building information rather than a baked down version, which means that every time you open the rig it is built on the spot, making changes incredibly easy and non-destructive. And the way the components are created is through an actual Python class, benefiting from all the benefits (and unfortunately negatives) of object oriented programming.

So, the rigging process with this sort of a system is going to be writing code. Of course, we still need to paint weights and store control shapes, but these are easily saved and loaded back in. Here is an example of what a rig might look like in this system.

loadModel()
loadGuides()

initializeRiggingStructure() # Creates the boilerplate for a rig - top/god node, etc.

spine = bodyCommands.spine(spineChain)
arm = bodyCommands.arm(armChain, spine.end)

... ## Build all the components you would want

loadWeights()
loadControlShapes()

The loadGuides() function refers to a file which contains all our guides, similar to the previous rigging system. In this one, though, it is up to you what sort of guides you use. For example, for an arm you would just draw out your arm chain and the module will take it from there.

This is the rigging system that I like using. It feels much more intuitive to me as I do not feel any restrictions from anything, be that UI, pre-built components that I don’t have access to, etc. If I want a slightly different module I can just inherit from the old one and make my changes. If there is a model change I just need to reposition my guides and run my code.

The main downside of it is that it might get a while a new person to get used to such a system. Having a nice documentation and examples would help a lot. Another thing, people might feel uncertain about is the complete lack of UI, but again for me it is liberating to not be constrained by buttons, text field, etc.

Conclusion

I am very happy with the current rigging system I am using described in the previous section. That being said, though, I cannot help but think of things I would like to see in a rigging system.

For starters, let us go back at UI. Even though, I feel great about being able to do whatever I want with the code, having an UI for certain things would be much quicker. What ideally I would like to do is be able to have both working at the same time. Whatever I write needs to be reflected in the UI and, the harder bit, whatever I do in the UI I need to be reflected somewhere in my build file, so next time I build the rig it comes with the changes made from the UI as well. Having an UI modifying my code, though, does not sound amazing, so we need a different way of handling that, which could potentially be metadata. The one issue I have with relying too much on metadata is that it is not immediately obvious what is going on.

Another thing I would really like to see at some point is some sort of a rigging system standard where riggers around the world can exchange components and general bits from the rigs with each other. To be honest, though, I am both excited and worried about something like this, as introducing a standard might significantly hinder innovation.

The big thing that lies in the future, though, is getting a higher level of interactiveness while rigging. Complex builds using the rigging system from the last section can take minutes to build, which means that for every single change I make in the guides file or the build I will need to wait a lot to actually see the result. That is making the process a lot more obscure, when you just need to keep changing stuff in order to hit the right values. Imagine, though, that we have all that building happening in real-time. Say I have the guides opened in one window, the build file in my text editor and I have the product in another window. Ideally, what I would like is by moving something in my guides file, to trigger a rebuild of that “dirtied” portion of the build which would result in the changes applied in my third window without actually deforming my model.

I am saying this lies in the future, though, aspects of it are already taken care by the guidable components method described above. That being said, that full level of interactiveness is what I would ideally like to achieve.

As I said in the beginning, a lot of these are just my own speculations, which means that I am still trying to figure most of this out. That is why, I would love to hear your tips, opinions and ideas on rigging systems, so please do share them!

If you have been reading bindpose for a while and have seen my marking menu posts you probably know that I am very keen on getting my workflow as optimized as possible. I am a big fan of handy shelfs, marking menus, hotkeys, custom widgets, etc. The way I see it, the closer and easier our tools are to access the quicker we can push the rig through. That is why today we are having a look at using PySide to install a global hotkey in Maya (one that would work in all windows and panels) in a way where we do not break any of the existing functionality of that hotkey (hopefully).

If you have not used PySide before, do not worry, our interaction with it will be brief and pretty straightforward. I myself am very new to it. That being said, I think it is a great library to learn and much nicer and more flexible than the native maya UI toolset.

Disclaimer: The way I do this is very hacky and dirty and I am sure there must be a nicer way of doing this, so if you have suggestions please do let me know, so I can add it both to my workflow and to this post.

What we want to achieve

So, essentially, all I want to do here is install a global hotkey (PySide calls them shortcuts) on the CTRL + H combination that would work in all Maya’s windows and panels as you would expect it to – Hide selected, but if we are inside the Script editor it would clear the history.

Some of you might think that we can easily do this without PySide, just using maya’s hotkeys, but the tricky bit comes in from the fact that maya’s hotkeys are not functioning when your last click was inside the Script editor’s text field or history field. That means, that only if you click somewhere on the frames of the Script editor would that hotkey get triggered, which obviously is not nice at all.

Achieving it

So, let us have a look at the full code first and then we will break it apart.

from functools import partial
from maya import OpenMayaUI as omui, cmds as mc

try:
    from PySide2.QtCore import *
    from PySide2.QtGui import *
    from PySide2.QtWidgets import *
    from shiboken2 import wrapInstance
except ImportError:
    from PySide.QtCore import *
    from PySide.QtGui import *
    from shiboken import wrapInstance


def _getMainMayaWindow():
    mayaMainWindowPtr = omui.MQtUtil.mainWindow()
    mayaMainWindow = wrapInstance(long(mayaMainWindowPtr), QWidget)
    return mayaMainWindow


def shortcutActivated(shortcut):
    if "scriptEditor" in mc.getPanel(wf=1):
        mc.scriptEditorInfo(clearHistory=1)
    else:
        shortcut.setEnabled(0)
        e = QKeyEvent(QEvent.KeyPress, Qt.Key_H, Qt.CTRL)
        QCoreApplication.postEvent(_getMainMayaWindow(), e)
        mc.evalDeferred(partial(shortcut.setEnabled, 1))


def initShortcut():
    shortcut = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_H), _getMainMayaWindow())
    shortcut.setContext(Qt.ApplicationShortcut)
    shortcut.activated.connect(partial(shortcutActivated, shortcut))

initShortcut()

Okay, let us go through it bit by bit.

Imports

We start with a simple import of partial which is used to create a callable reference to a function including arguments. Then from maya we the usual cmds, but also OpenMayaUI which we use to get a PySide reference to maya’s window.

Then the PySide import might look a bit confusing with that try and except block, but the only reason it is there is because between maya 2016 and maya 2017 they switched PySide versions, and the imports had to change as well. So, what we do is we try to import from PySide2 (Maya 2017) and if it cannot be found we do the imports from PySide (Maya 2016).

Getting Maya’s main window

Even though, Maya’s UI is built entirely by Qt (PySide is a wrapper around Qt), the native elements are not usable with PySide functions. In order to be able to interact with these native bits we need to find a PySide reference to them. In the example for hotkeys we need only the main window, but depending on what you are trying to do you might have to iterate through children in order to find the UI element you are looking for. Therefore this _getMainMayaWindow function has become a boilerplate code and I always copy and paste it together with the imports.

The way it works is, using Maya’s API we get a pointer to the memory address where Maya’s main window is stored in memory. That’s the omui.MQtUtil.mainWindow() function. Then what we do is, using that pointer and the wrapInstance function we create a PySide QWidget instance of our window. That means that we can run any QWidget functions on Maya’s main window. In our hotkey example, though, we only need it to bind the hotkey to it.

The logic of the hotkey

The shortcutActivated function is the one that is going to get called every time we press the hotkey. It takes a QShortcut object as an argument, but we will not worry about it just yet. All we need to know is that this object is what calls our shortcutActivated function.

It is worth mentioning that this function is going to get called without giving Maya a chance to handle the event itself. So, that means that if we have nothing inside this function, pressing CTRL + H will do nothing. Therefore, we need to make sure we implement whatever functionality we want inside of this function.

So, having a look at the if statement, you can see that we are just checking if the current panel with focus – mc.getPanel(wf=1) – is the Script editor. That will return True if we have last clicked either on the frames of the Script editor windows or anywhere inside of it.

Then, obviously, if that is the case we just clear the Script editor history.

If it returns False, though, it means that we are outside of the Script editor so we need to let Maya handle the key combination as there might be something bound to it (In the case of CTRL+H we have the hiding functionality which we want to maintain). So, let us pass it to Maya then.

As I said earlier, Maya does not get a chance to handle this hotkey at all, it is entirely handled by PySide’s shortcut. So in order to pass it back to Maya, what we do is we disable our shortcut and we simulate the key combination again, so Maya can do it’s thing. Once that is done, we re-enable our shortcut so it is ready for next time we press the key combination. That is what the following snippet does.

shortcut.setEnabled(0)
e = QKeyEvent(QEvent.KeyPress, Qt.Key_H, Qt.CTRL)
QCoreApplication.postEvent(_getMainMayaWindow(), e)
mc.evalDeferred(partial(shortcut.setEnabled, 1))

Notice we are using evalDeferred as we are updating a shortcut from within itself.

Binding the function to the hotkey

Now that we have all the functionality ready, we need to bind it all to the key combination of our choice – CTRL + H in our example. So, we create a new QShortcut instance, which receives a QKeySequence and parent QWidget as arguments. Essentially, we are saying we want this key combination to exist as a shortcut in this widget. The widget we are using is the main maya window we talked about earlier.

Then, we use the setContext method of the shortcut to extend it’s functionality across the whole application, using Qt.ApplicationShortcut as an argument. Now the shortcut is activated whenever we press the key combination while we have our focus in any of the maya windows.

Lastly, we just need to specify what we want to happen when the user has activated the shortcut. That is where we use the activated signal of the shortcut (more info on signals and slots) and we connect it to our own shortcutActivated function. Notice that we are using partial to create a callable version of our function with the shortcut itself passed in as an argument.

And that’s it!

Conclusion

Hotkeys, marking menus, shelves, custom widgets and everything else of the sort is always a great way to boost your workflow and be a bit more efficient. Spending some time to build them for yourself in a way where you can easily reproduce them in the next version of Maya or on your next machine is going to pay off in the long run.

I hope this post has shown you how you can override maya’s default hotkeys in some cases where it would be useful, while still maintaining the default functionality in the rest of the UI.

If you know of a nicer way of doing this, please do share it!

So, it seems like I have been going crazy with marking menus lately. I am really trying to get the most of them, and that would not be much if we could only use them in the viewports, so today we are going to look at how we can construct custom marking menus in maya editors.

Custom marking menus in maya editors - node editor

tl;dr: We can query the current panel popup menu parent in maya with the findPanelPopupParent MEL function, and we can use it as a parent to our popupMenu.

So, there are a couple of scenarios that we need to have a look at, as they should be approached differently. Although, not completely necessary I would suggest you have a look at my previous marking menu posts – Custom marking menu with Python and Custom hotkey marking menu – as I will try to not repeat myself.

Okay, let us crack on. Here are the two different situations for custom marking menus in maya editors we are going to look at.

Modifiers + click trigger

In the viewport these are definitely the easier ones to set up as all we need to do is just create a popupMenu with the specified modifiers – sh, ctl and alt, the chosen button and viewPanes as the parent. When it comes to the different editors, though, it gets a bit trickier.

Let us take the node editor as an example.

If we are to create a marking menu in the node editor, it is a fairly simple process. We do exactly the same as before, but we pass "nodeEditorPanel1" as the parent argument. If you have a node editor opened when you run the popupMenu command, you will be able to use your marking menu in there. The catch is though, that once you close the node editor the marking menu is deleted, so it is not available the next time you open the node editor.

Unfortunately, I do not have a great solution to this problem. In fact, it is a terrible solution, but I wanted to get it out there, so someone can see it, be appalled and correct me.

The second method – Custom hotkey trigger – is much nicer to work with. So, you might want to skip to that one.

What I do is, I create a hotkey for a command that invokes the specific editor (I only have marking menus in the node editor and the viewport) and runs the marking menu code after that. So, for example, here is my node editor hotkey (Alt+Q) runTimeCommand.

mc.NodeEditorWindow()

if mc.popupMenu("vsNodeMarkingMenu", ex=1):
    mc.popupMenu("vsNodeMarkingMenu", dai=1, e=1)
else:
    mc.popupMenu("vsNodeMarkingMenu", p="nodeEditorPanel1Window", b=2, ctl=1, alt=1, mm=1)

mc.setParent("vsNodeMarkingMenu", m=1)

from vsRigging.markingMenus import vsNodeMarkingMenu
reload(vsNodeMarkingMenu)

That means that everytime I open the node editor with my hotkey I also create the marking menu in there, ready for me to use. As, I said, it is not a solution, but more of a workaround at this point. In my case, though, I never open the node editor through anything else than a hotkey, so it kind of works for me.

Then the vsRigging.markingMenus.vsNodeMarkingMenu file is as simple as listing the menuItems.

import maya.cmds as mc

mc.menuItem(l="multiplyDivide", c="mc.createNode('multiplyDivide')", rp="N", i="multiplyDivide.svg")
mc.menuItem(l="multDoubleLinear", c="mc.createNode('multDoubleLinear')", rp="NE", i="multDoubleLinear.svg")
mc.menuItem(l="plusMinusAverage", c="mc.createNode('plusMinusAverage')", rp="S", i="plusMinusAverage.svg")
mc.menuItem(l="condition", c="mc.createNode('condition')", rp="W", i="condition.svg")
mc.menuItem(l="blendColors", c="mc.createNode('blendColors')", rp="NW", i="blendColors.svg")
mc.menuItem(l="remapValue", c="mc.createNode('remapValue')", rp="SW", i="remapValue.svg")
...

A proper way of doing this would be to have a callback, so everytime the node editor gets built we can run our code. I have not found a way to do that though, other than ofcourse breaking apart maya’s internal code and overwriting it, which I wouldn’t go for.

Luckily, creating a custom marking menu bound to a custom hotkey actually works properly and is fairly easy. In fact, it is very similar to the Custom hotkey marking menu post. Let us have a look.

Custom hotkey trigger

Now, when we are working with custom hotkeys we actually run the initialization of the popupMenu everytime we press the hotkey. This means we have the ability to run code before we create the marking menu. Therefore, we can query the current panel and build our popupMenu according to it. Here is an example runTimeCommand, which is bound to a hotkey.

import maya.mel as mel

name = "exampleMarkingMenu"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

parent = mel.eval("findPanelPopupParent")

if "nodeEditor" in parent:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import exampleMarkingMenu
    reload(exampleMarkingMenu)
else:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import fallbackMarkingMenu
    reload(fallbackMarkingMenu)

So, what we do here is, we start by cleaning up any existing versions of the marking menu. Then, we use the very handy findPanelPopupParent MEL function to give us the parent to which we should bind our popupMenus. Having that we check if the editor we want exists in the name of the parent. I could also compare it directly to a string, but the actual panel has a number at the end and I prefer just checking the base name. Then, depending on which panel I am working in at the moment, I build the appropriate custom marking menu.

Don’t forget that you need to create a release command as well, to delete the marking menu so it does not get in the way if you are not pressing the hotkey. It is a really simple command, that I went over in my previous marking menu post.

The obvious limitation here is that we have a hotkey defined and we cant just do ctrl+alt+MMB for example.

Conclusion

So, yeah, these tend to be a bit trickier than just creating ones in the viewport, but also I think there is more to be desired from some of maya’s editors *cough* node editor *cough*, and custom marking menus help a lot.

So, recently I stumbled upon a djx blog blost about custom hotkeys and marking menus in different editors in maya. I had been thinking about having a custom hotkey marking menu, but was never really sure how to approach this, so after reading that post I thought I’d give it a go and share my process.

tl;dr: We can create a runtime command which builds our marking menu and have a hotkey to call that command. Thus, giving us the option to invoke custom marking menus with our own custom hotkeys, such as Shift+W or T for example, and a mouse click.

Disclaimer: I have been having a super annoying issue with this setup, where the “release” command does not always get called, so the marking menu is not always deleted. What this means is that if you are using a modifier like Shift, Control or Alt sometimes your marking menu will still be bound to it after it has been closed. Therefore, if you are using something like Shift+H+LMB, just pressing Shift+LMB will open it up, so you lose the usual add to selection functionality. Sure, to fix it you just have to press and release your hotkey again, but it definitely gets on your nerve after a while.

If anyone has a solution, please let me know.

I have written about building custom marking menus in Maya previously, so feel free to have a look as I will try to not repeat myself here. There I also talked about why I prefer to script my marking menus, instead of using the Marking menu editor, and that’s valid here as well.

So, let us have a look then.

RunTimeCommands

The first thing we need to do is define a runTimeCommand, so we can run it with a hotkey. That is what happens if you do it through the Marking menu editor and set Use marking menu in to Hotkey Editor, as well.

There a couple of ways we can do that.

Hotkey Editor

On the right hand side of the hotkey editor there is a tab called Runtime Command Editor. If you go on that one you can create and edit runTime commands.

Scripting it in Python

If you have multiple marking menus that you want to crate, the hotkey editor might seem as a bit of a slow solution. Additionally, if changes need to be made I always find it more intuitive to look at code in my favourite text editor (which is sublime by the way).

To create a runTime command we run the runTimeCommand function which for some reason does not appear in the Python docs, but I’ have been using maya.cmds.runTimeCommand successfully.

All we need to provide is a name for the command, some annotation – ann, a string with some code – c and a language – cl.

Here is an example

mc.runTimeCommand("exampleRunTimeCommand", ann="Example runTime command", c=commandString, cl="python")

Something we need to keep in mind when working with runTime commands is that we cannot pass external functions to them. We can import modules and use them once inside, but I cannot pass a reference to an actual function to the c flag, as I would do to menuItems for example. That means that we need to pass our code as a string.

Press and release

Now, that we know how to create the runTimeCommands let us see what we need these commands for.

As I mentioned, they are needed so we can access them by a hotkey. What that hotkey should do is initialize our marking menu, but once we release the key it should get rid of it, so it does not interfere with other functions. Therefore we need two of them – Press and Release.

Let us say we are building a custom hotkey marking menu for weight painting. In that case we will have something similar to the following.

  • mmWeightPainting_Press runTimeCommand – to initialize our marking menu
  • mmWeightPainting_Release runTimeCommand – to delete our marking menu

The way we bind the release command to the release of a hotkey is by pressing the small arrow to the side of the hotkey field.

Custom hotkey marking menu - release command hotkey

The Press command

import maya.cmds as mc # Optional if it is already imported

name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

import mmWeightPainting
reload(mmWeightPainting)

So, essentially what we do is every time we press our hotkey, we delete our old marking menu and rebuild it. We do this, because we want to make sure that our latest changes are applied.

Now, the lower part of the command is where it gets cool, I think. We can store our whole marking menu build – all menuItems – inside a file somewhere in our MAYA_SCRIPT_PATH and then just import it from the runTimeCommand as in this piece of code. What this gives us, is again, the ability to really easily update stuff (not that it is a big deal with marking menus once you set them up). Additionally, I quite like the modularity, as it means we can have very simple runTimeCommands not cluttered with the actual marking menu build. This is the way that creating through the Marking menu editor works as well, but obviously it loads a MEL file instead.

So, literally that mmWeightPainting file is as simple as creating all our marking menu items.

import maya.cmds as mc

mc.menuItem(l="first item")
mc.menuItem(l="second item")
mc.menuItem(l="North radial position", rp="N")
...

And that takes care of building our marking menu when we press our hotkey + the specified modifiers and mouse button. What, we do not yet have is deleting it on release, so it does not interfere with the other functionality tied to modifier + click combo. That is where the mmWeightPainting_Release runTimeCommand comes in.

The Release command

## mmWeightPainting_Release runTimeCommand
name = "mmWeightPainting"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

Yep, it is a really simple one. We just delete the marking menu, so it does not interfere with anything else. Essentially, the idea is we have it available only while the hotkey is pressed.

Hotkey

All that is left to be done is to assign a hotkey to the commands. There are a couple of things to have in mind.

If you are using modifiers for the popupMenu command – sh, ctl or alt – then the same modifiers need to be present in your hotkey as otherwise, even though the runTimeCommand will run successfully, the popupMenu will not be triggered.

In the above example

mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

we have specified the sh modifier. Therefore, the Shift key needs to be present in our hotkey.

Also, obviously be careful which hotkeys you overwrite, so you do not end up causing yourself more harm than good.

Conclusion

That’s it, it really is quite simple, but it helps a lot once you get used to your menus. Honestly, trying to do stuff without them feels so tedious afterwards.

So, if you have been rigging for a while you have probably felt annoyed by having to create and adjust control shapes every time you build a new rig. You have probably found also that mirroring just the shape of a control or copying it to another one is a bit too tedious. There are some scripts and tools online to help you with this, such as the classic comet menu and the mz_ctrlcreator, but they do not offer all the functions we need and also extending them is not very practical. So, let us write our own control shape manager.

tl;dr I am going to walk you through the process of creating your own control shape manager, but if you would rather just use the final code you can find it here. If you would prefer it combined into one large file, you can grab it from here.

Here is a quick demo of some of the features we are going to look at.

Control shape manager in Maya - Demo

Table of contents

  1. Introduction
  2. Part 1: Control Shape Manager
  3. Getting and setting shapes
  4. Saving and loading
  5. Additional
  6. Part 2: Control Shape Functions
  7. Getting lists for the UI
  8. Assigning shapes and colours
  9. Saving to library
  10. Copying and pasting shapes
  11. Flipping shapes
  12. Rebuilding
  13. Part 3: Simpe UI
  14. Conclusion

Introduction

What we want is a python package that allows us to load and save control shapes to a library, copy and paste them to multiple other controls, change colours, flip them, mirror them, etc.

The full code can be found here. I have built it as a package with a few different modules, to be a bit clearer and nicer to maintain. I have also combined everything into one file as well, if anyone wants to just grab it and use it immediately. What we are going to do here though, is go through the code and learn how to build our own control shape manager, because it is much nicer when you actually understand how it works, as then you can extend it and adjust it to suit your needs. For example I have built upon this a bit more in my pipeline, so I can save and load shape versions for each control on a rig, so I can easily rebuild them when I am making changes.

Part 1: Control Shape Manager

We are going to be using a couple of commands from the Maya API, but if you are not very familiar with it, worry not I will explain what each function does. You can always read up on it on the Autodesk docs page or if you prefer more of a tutorial approach have a look at Chad Vernon’s Maya API web page.

Getting and setting shapes

Let us start with the two most important functions – getShape() and setShape().

def getShape(crv=None):
    crvShapes = validateCurve(crv)

    crvShapeList = []

    for crvShape in crvShapes:
        crvShapeDict = {
            "points": [],
            "knots": [],
            "form": mc.getAttr(crvShape + ".form"),
            "degree": mc.getAttr(crvShape + ".degree"),
            "colour": mc.getAttr(crvShape + ".overrideColor")
        }
        points = []

        for i in range(mc.getAttr(crvShape + ".controlPoints", s=1)):
            points.append(mc.getAttr(crvShape + ".controlPoints[%i]" % i)[0])

        crvShapeDict["points"] = points
        crvShapeDict["knots"] = utils.getKnots(crvShape)

        crvShapeList.append(crvShapeDict)

    return crvShapeList

What this function does is, it gets all the data from a nurbsCurve node that we need to rebuild that curve down the line. We are going to look at the validateCurve() function a bit later, but it essentially checks if the curve we have passed is actually a valid curve and if so returns the shape node.

A list is initialized here which will later be populated with dictionaries for each shape node on the curve in order to work with compound curves.

The crvShapeDict is where the actual data is stored. All of the keys in the dictionary are just the needed data for building a curve. If you do not know what the knots and degree are when it comes to curve, you can read up on it here, but it is not necessary. We will be thinking of them as the essential building blocks of a curve.

You can see that very easily we can get the form, degree and colour ones as they are just attributes on the nurbsCurve node.

To get the points what we need to do is loop through all of the controlPoints of the curve. Initially, I was just using the cv attribute, but it does not work with closed curves, as the cvs are just representation of these points, so we can interact with them, but under the hood maya changes them a bit and they are stored in the controlPoints attribute. So, we just get the number of control points using the s flag on the getAttr command and we store each point in a list.

Now, for the knots initially I used this snippet from Serge Scherbakov, but it does not work with closed curves. I could have gone in and tried to create my own function to do that, but then maya has made it easy for us to get the knots from the API, so I thought I would just use that.

def getKnots(crvShape=None):
    mObj = om.MObject()
    sel = om.MSelectionList()
    sel.add(crvShape)
    sel.getDependNode(0, mObj)

    fnCurve = om.MFnNurbsCurve(mObj)
    tmpKnots = om.MDoubleArray()
    fnCurve.getKnots(tmpKnots)

    return [tmpKnots[i] for i in range(tmpKnots.length())]

The first part of this function deals with getting an API reference to our curve. It basically, adds the passed in crvShape to a virtual selection (without actually selecting anything in the viewport) and gets an MObject from it. That’s the base class in the API and from there we can cast it to the type we actually need – MFnNurbsCurve. Then we create an empty MDoubleArray, which we populate from the curve with the getKnots function. And that’s it. Lastly, we return it as a python list, just so we can interact with it easier.

And with that we have a list of dictionaries containing all the necessary information for rebuilding that curve.

Let’s look at setting the shape now. What is nice about this code is that if you understand how the getShape() works, the setShape() is going to be trivial. The one thing I do not like about this code is that we are not assigning the data to the existing curve, but we delete it and create a new one in place. This could cause issues if there are any connections to or from the shape node, but you can always store and rebuild those. I have not yet found a way around it though.

def setShape(crv, crvShapeList):
    crvShapes = validateCurve(crv)

    oldColour = mc.getAttr(crvShapes[0] + ".overrideColor")
    mc.delete(crvShapes)

    for i, crvShapeDict in enumerate(crvShapeList):
        tmpCrv = mc.curve(p=crvShapeDict["points"], k=crvShapeDict["knots"], d=crvShapeDict["degree"], per=bool(crvShapeDict["form"]))
        newShape = mc.listRelatives(tmpCrv, s=1)[0]
        mc.parent(newShape, crv, r=1, s=1)

        mc.delete(tmpCrv)
        newShape = mc.rename(newShape, crv + "Shape" + str(i + 1).zfill(2))

        mc.setAttr(newShape + ".overrideEnabled", 1)

        if "colour" in crvShapeDict.keys():
            setColour(newShape, crvShapeDict["colour"])
        else:
            setColour(newShape, oldColour)

We go through the same call to validateCurve() as before and then we store the "overrideColor" of the curve, so we can apply it back after we rebuild the shape. It is important to note that the colour is the one assigned to the first shape child of the curve. And since we have everything we need from the old shapes – the colour – we delete them.

Then for each shape in the list we just use our points, knots, degree and form data from the dictionary to build a new curve with the mc.curve() command. The per flag refers to periodic and basically defines whether our curve is one whole or does it have a start and an end. A bit more info about periodic curves in here.

Once we have created the new shape we parent it to the crv object with the r=1 and s=1 flags for mc.parent() to define that we are working with shapes and to maintain their relative positions. We then can rename the new shape according to our convention. Lastly, we just reapply the colour or we get it from the dictionary.

As I said these two are the most important functions as they are dealing with the actual data. Now that we have them in place we can give them a quick test. Create a nurbsCurve with whatever shape you want. Then let’s create a simple circle and copy the first shape to the circle. Assuming that the first curve is called curve1 and the circle is nurbsCircle1 run the following snippet.

shapeDict = getShape("curve1")
setShape("nurbsCircle1", shapeDict)

I realize this is not very exciting as there are easier ways to do this, but the cool thing is when we start saving and loading them.

Saving and loading

We have been looking only at the manager.py file for know. In the utils.py we have a few more functions mainly dealing with the saving and loading of json data. Loading and saving JSON files is a very popular and fairly trivial python task, but let’s deconstruct it.

def loadData(path=None):
    if os.path.isfile(path):
        f = open(path, "r")
        data = json.loads(f.read())
        f.close()
        return data
    else:
        mc.error("The file " + path + " doesn't exist")

def saveData(path=None,
             data=None):
    if validatePath(path):
        f = open(path, "w")
        f.write(json.dumps(data, sort_keys=1, indent=4, separators=(",", ":")))
        f.close()
        return 1
    return 0

For loading we start by checking if the file exists and if it does, we use python’s open function to get the raw data and we pass it to a json.loads() function to convert the raw data to a Python dict object.

When saving, we are doing the same thing but instead of reading and converting from raw data to a dict we are passing a dict to the json.dumps() function which serializes our dictionary to JSON and then we write it to the file. You will notice that there is a call to another validation function – validatePath().

def validatePath(path=None):
    if os.path.isfile(path):
        confirm = mc.confirmDialog(title='Overwrite file?',
                                   message='The file ' + path + ' already exists.Do you want to overwrite it?',
                                   button=['Yes', 'No'],
                                   defaultButton='Yes',
                                   cancelButton='No',
                                   dismissString='No')
        if confirm == "No":
            mc.warning("The file " + path + " was not saved")
            return 0
    return 1

All we do here is check if the file we are trying to save already exists and if so gives the option to overwrite it or cancel the save process.

Now that we know how our dictionary data is being load and saved, we just need to have a wrapper function in our manager module to load and save to the defined shape library directory.

Before looking at those though, you need to have the SHAPE_LIBRARY_PATH set at the top of the file. Keep in mind that if the path does not exist, Python will not create it for us but error out.

def loadFromLib(shape=None):
    path = os.path.join(SHAPE_LIBRARY_PATH, shape + ".json")
    data = utils.loadData(path)
    return data

What we do here is define the path to the file we want to load and use the loadData function we talked about to load the actual dictionary.

def saveToLib(crv=None,
              shapeName=None):
    crvShape = getShape(crv=crv)
    path = os.path.join(SHAPE_LIBRARY_PATH, re.sub("\s", "", shapeName) + ".json")
    for shapeDict in crvShape:
        shapeDict.pop("colour", None)
    utils.saveData(path, crvShape)

Then when saving we use re.sub("s", "", shape) in order to strip spaces from the name as they can cause issues and pass the path to the saveData() function. Also, we get rid of the colour keys as we want to save only the shape of the curve.

Additional commands

The rest of the functions in the module are fairly self-explanatory.

def validateCurve(crv=None):
    if mc.nodeType(crv) == "transform" and mc.nodeType(mc.listRelatives(crv, c=1, s=1)[0]) == "nurbsCurve":
        crvShapes = mc.listRelatives(crv, c=1, s=1)
    elif mc.nodeType(crv) == "nurbsCurve":
        crvShapes = mc.listRelatives(mc.listRelatives(crv, p=1)[0], c=1, s=1)
    else:
        mc.error("The object " + crv + " passed to validateCurve() is not a curve")
    return crvShapes

The validateCurve() function just checks if we have passed a valid curve and if so it returns the nurbsCurve shape nodes to work with. Otherwise it errors.

Then we have the colour functions which are just simple wrappers around mc.getAttr() and mc.setAttr() commands to interact with the "overrideColor" attribute of shapes.

def setColour(crv, colour):
    if mc.nodeType(crv) == "transform":
        crvShapes = mc.listRelatives(crv)
    else:
        crvShapes = [crv]
    for crv in crvShapes:
        mc.setAttr(crv + ".overrideColor", colour)

def getColour(crv):
    if mc.nodeType(crv) == "transform":
        crv = mc.listRelatives(crv)[0]
    return mc.getAttr(crv + ".overrideColor")

Part 2: Control Shape Functions

Now that we have our core functionality in place we can stop here and just use the code we have so far through our script editor, which is absolutely fine, but is not very scalable and not really user friendly. Additionally, we are still lacking the mirroring and flipping functionality, so let us create a functions.py file which will act as a wrapper to our manager module. The reason we would want this is to prevent messing about with our manager too much and provide a higher level control so we can literally only care about using the tool instead of how it works. Altogether, it is much nicer working with simple short functions. Okay, let us go through the functions.py commands that help us interact with the manager.

Getting lists for the UI

def getAvailableControlShapes():
    lib = manager.SHAPE_LIBRARY_PATH
    return [(x.split(".")[0], functools.partial(assignControlShape, x.split(".")[0])) for x in os.listdir(lib)]

def getAvailableColours():
    return [("index" + str(i).zfill(2), functools.partial(assignColour, i), "shapeColour" + str(i).zfill(2) + ".png") for i in range(32)]

These two functions are mainly here to help us later when we are going to build some sort of UI for our manager. Essentially they return lists of tuples containing the names, commands and in the case of getAvailableColours() images of the available shapes and colours. These are going to be used when building menus that look similar to the following.

Control shape manager - Shapes dropdown
Shapes
Control shape manager - Colours dropdown
Colours

Notice that the second item in the tuple is a functools.partial() call. For more info refer to the docs, but briefly it allows us to get a reference to a function with added arguments as well. So the first argument is a function and then we have a number of arguments which are going to be provided to the function as *args. Let’s have a look at the functions themselves to see how this works.

Assigning shapes and colours

def assignColour(*args):
    for each in mc.ls(sl=1, fl=1):
        manager.setColour(each, args[0])

def assignControlShape(*args):
    sel = mc.ls(sl=1, fl=1)
    for each in sel:
        manager.setShape(each, manager.loadFromLib(args[0]))
    mc.select(sel)

So, both these functions receive *args as an argument, which means that we can provide lots of arguments and they are going to be passed to the function as a list which we can acces by args[n]. In the previous paragraph, we saw that we pass these functions and a single argument to the functools.partial, which means that the first element of args is going to be the second argument of the functools.partial() code. So in the case of functools.partial(assignColour, i), we are going to receive a call equivalent to assignColour(i).

Additionally, keep in mind if these functions that we are defining here are meant to be used from a maya UI, and a lot of the buttons in maya are passing arguments to their commands, so we need to have the *args, because otherwise the functions will error.

Notice that we reselect our initial selection at the end of the function. We will do this in all functions that call the setShape() one, because the creation of the curve inside of it deselects our selection and instead selects the newly created curve, which is not very intuitive.

Saving to library

def saveCtlShapeToLib(*args):
    result = mc.promptDialog(title="Save Control Shape to Library",
                             m="Control Shape Name",
                             button=["Save", "Cancel"],
                             cancelButton="Cancel",
                             dismissString="Cancel")
    if result == "Save":
        name = mc.promptDialog(q=1, t=1)
        manager.saveToLib(mc.ls(sl=1, fl=1)[0], name)
    rebuildUI()

As we said the goal here is to make interacting with our control shape manager as smooth as possible. Therefore, we create a wrapper to our saveToLib() command to let us add a name in a nice and familiar dialog. In the end we are calling the rebuildUI() function which we will look at the end of this part, but the reason it is here is that every time we save a new control shape we would like the UI to be rebuild, in order for the menu containing all of our shapes to be up to date.

Copying and pasting shapes

def copyCtlShape(*args):
    global ctlShapeClipboard
    ctlShapeClipboard = manager.getShape(mc.ls(sl=1, fl=1)[0])
    for ctlShape in ctlShapeClipboard:
        ctlShape.pop("colour")

def pasteCtlShape(*args):
    sel = mc.ls(sl=1, fl=1)
    for each in sel:
        manager.setShape(each, ctlShapeClipboard)
    mc.select(sel)

As we saw previously, it is really easy to copy and paste shapes with the manager alone, but to provide a quick and easy interface these two functions seem to do a good job. Essentially, we are creating a global variable and store the selected shape dictionary inside of it. Again we pop the “colour” key, as we just want to copy the shape. Then we just use the setShape() function on all selected controls with that global variable.

Flipping shapes

Then there are a few functions for flipping the shapes. It’s a bit of a pain to have to do that manually, but it is really easy to scale the points by -1 through script so let’s have a look at the _flipCtlShape() function. You will notice that there are a few more functions for flipping – flipCtlShape(), flipCtlShapeX(), flipCtlShapeY() and flipCtlShapeZ(). They all just make a call to the _flipCtlShape() one, but with different arguments, so we will just look at that one.

def _flipCtlShape(crv=None, axis=[-1, -1, -1]):
    shapes = manager.getShape(crv)
    newShapes = []
    for shape in shapes:
        for i, each in enumerate(shape["points"]):
            shape["points"][i] = [each[0] * axis[0], each[1] * axis[1], each[2] * axis[2]]
        newShapes.append(shape)
    manager.setShape(crv, newShapes)
    mc.select(crv)

All we do in this one, is just go through each CV and scale it’s x, y and z coordinates by -1 in order to flip the shape. The above mentioned other functions just call this one with the axis set to [-1, 1, 1] for x, [1,-1,1] for y, etc.

I skipped the mirrorCtlShapes() function earlier, because I wanted to already have the flip one in place as we are going to be using it again.

def mirrorCtlShapes(*args):
    sel = mc.ls(sl=1, fl=1)
    for ctl in sel:
        if ctl[0] not in ["L", "R"]:
            continue
        search = "R_"
        replace = "L_"
        if ctl[0] == "L":
            search = "L_"
            replace = "R_"
        shapes = manager.getShape(ctl)
        for shape in shapes:
            shape.pop("colour")
        manager.setShape(ctl.replace(search, replace), shapes)
        _flipCtlShape(ctl.replace(search, replace))
    mc.select(sel)

The bulk of the code here is really for defining the search and replace strings. Since the naming convention that I use is SIDE_NAME_NODETYPE my search and replace strings vary between “L_” and “R_”. Have a look at your convention and modify these strings to make it work. Once they are defined, all we do is copy the shape from the current side to the other one and once done, flip it in all axis. In my pipeline, I have made it so this function does not work with a selection, but instead goes through all my left controls and mirrors them to the right. It is just because I always work from left to right, so I do not need this functionality.

Rebuilding

def rebuildUI(*args):
    mc.evalDeferred("""
import controlShapeManager
reload(controlShapeManager)
""")

Lastly, there is a simple function to rebuild the UI. All it does is import the package, as the way I have set it up is that importing just builds the UI which in turn makes the references to all the needed functions. The UI example that I give is very primitive, but obviously you can replace this code with one that will work with your own UI. Keep in mind that it is best to use the mc.evalDeferred() command as otherwise, the rebuild might error as it is being called from the UI that needs to be rebuilt.

Part 3: Simple UI

Now that we have all functions that we need we can build an UI to interact with them. Since everybody has a different pipeline for rigging at place, I am hesitant to suggest any specific way of handling that UI. One might prefer it in a window, other a tool menu or others yet a shelf button like I do. So I have added a very simple shelf button build in the managerUI.py to demonstrate how would we go about it. Additionally, remember how when generating the lists for the available colours we had a third item in the tuple for an image? You can get these here. They are just images of solid colour, corresponding to the index of the overrideColor attribute.

For a more comprehensive intro to building shelves with buttons and popups have a look at my Building custom maya shelves post.

Let’s have a look at it then.

import maya.cmds as mc


# Local import
import functions
reload(functions)

SHELF_NAME = "Custom"
ICON_PATH = "C:/PATH_TO_ICONS"

if SHELF_NAME and mc.shelfLayout(SHELF_NAME, ex=1):
    children = mc.shelfLayout(SHELF_NAME, q=1, ca=1) or []
    for each in children:
        try:
            label = mc.shelfButton(each, q=1, l=1)
        except:
            continue
        if label == "ctlShapeManager":
            mc.deleteUI(each)

    mc.setParent(SHELF_NAME)
    mc.shelfButton(l="ctlShapeManager", i="commandButton.png", width=37, height=37, iol="CTL")
    popup = mc.popupMenu(b=1)
    mc.menuItem(p=popup, l="Save to library", c=functions.saveCtlShapeToLib)

    sub = mc.menuItem(p=popup, l="Assign from library", subMenu=1)

    for each in functions.getAvailableControlShapes():
        mc.menuItem(p=sub, l=each[0], c=each[1])

    mc.menuItem(p=popup, l="Copy", c=functions.copyCtlShape)
    mc.menuItem(p=popup, l="Paste", c=functions.pasteCtlShape)

    sub = mc.menuItem(p=popup, l="Set colour", subMenu=1)

    for each in functions.getAvailableColours():
        mc.menuItem(p=sub, l=each[0], c=each[1], i=ICON_PATH + each[2])

    mc.menuItem(p=popup, l="Flip", c=functions.flipCtlShape)
    mc.menuItem(p=popup, l="Mirror", c=functions.mirrorCtlShapes)

So what happens here is we import the functions.py file which in turn imports the manager.py which then imports the utils.py file. Then there are the two variables – SHELF_NAME and ICON_PATH for declaring the shelf to add the button to and the path to the icons. Then we check if a button with the same name already exists in the shelf and if it does we delete it so we can replace it with our new one.

From then on we have simple maya UI commands to build our buttons and menus. If you are not familiar with UIs in maya it is worth having a look at the docs. Essentially, all we do is create a single mc.shelfButton() and we attach a mc.popupMenu() to it. Which then we populate with mc.menuItem()s where the l flag stands for label and the c for command. So there we pass our functions commands. Notice that we are not adding the () after the function name as that would call it and return the output. Instead we want to pass a reference to that function.

Then for the shapes and colours menuItems we add the subMenu flag so they become deeper level menus and we populate them with the results of our functions.getAvailableControlShapes() and functions.getAvailableColours() commands, which results in lists containing the shapes in our library and all 32 available colours.

Conclusion

And that is it. We have built our own control shape manager. With some easy extensions you can improve it to have almost like a version control system for your rigs, so you do not ever have to worry about your control shapes anymore.

Again, here is the link to the repo and the gist containing everything in one file if you would rather just use something quickly.

I am not sure what it is, but there is something incredibly appealing in optimizing our workflows. I think a lot of it comes from the frustration of repeating the same actions over and over again. When you find a way to optimize that, it feels great. One of the easiest way to improve our rigging workflow is to script a custom marking menu with python. Another one, I have already written about is creating a custom shelf.

tl;dr: I will walk you through scripting your own custom marking menu with python, which is going to be easily shareable, extendable and maintainable. The code can be found here.

Here is how my main marking menu looks.

Custom marking menu in maya with python

I have found it is an immense help to have the commands I use most oftenly either in my marking menu or my shelf. It just saves so much time!

Okay, how do we go about creating one of these?

Well, we have two options – either build it with Maya’s native marking menu editor or script it with python.

The reasons I prefer scripting my marking menus in python are a few.
– The native editor does not give us all the available options for a marking menu, such as submenus.
– Updating from the editor is a pain in the butt.
– The editor does not scale nicely, if you want or need to support multiple marking menus.
– Doing it through the editor is boring.

Obviously, python fixes all these issues for us. Additionally, it is easy to share it with co-workers and keep it in a version control system. Okay, so you are sold now. Let us have a look at how to do it then.

The code that I will be going through is on this gist, but I will go through all of it, if you would rather write it yourself.

Custom marking menu with Python

Initialization

import maya.cmds as mc

MENU_NAME = "markingMenu"

We start very simple with the import of maya.cmds and giving a name to our custom marking menu. Now, the name is not very important because we do not ever see it, but maya does. So, in order for us to be able to update our custom marking menu, we need to be able to access it, and that is why we are giving it a name.

Then we have our markingMenu class. The main reason I went for a class is because we can encapsulate everything we need into it quite logically. I know that some people would prefer to have a function instead, which is absolutely fine, it is really a personal preference. Let’s have a look at the constructor.

Constructor

def __init__(self):

self._removeOld()
self._build()

As you can see, our constructor is very simple. We delete the old version of our custom marking menu if it exists and then we build are our new one in place.

Remove old marking menu

def _removeOld(self):
    if mc.popupMenu(MENU_NAME, ex=1):
    mc.deleteUI(MENU_NAME)

As I said, the reason we need to give a name to our marking menu is to be able to modify it after it is created. In our case, we are not really modifying it, but deleting it instead. We do this, so we have a clean slate for building our new marking menu.

The reason I have added this deleting and then building again functionality is just so we can painlessly make changes to our custom marking menu. For example, if I want to add a new button or change a label, I do not want to have to restart maya or do anything other than just running my code again. Or even easier, just importing my code again. That is why I said earlier, that this setup for a custom marking menu is very easily extendable and maintainable. Additionally, we can add a button to our marking menu which rebuilds it, so we only have to make our changes to the file, save it and then rebuild from within. We will have a look at that in the end of the post.

Building our custom marking menu

Now that we have had a look at preparing for our build let us have a look at the _build() method.

def _build(self):
    menu = mc.popupMenu(MENU_NAME, mm = 1, b = 2, aob = 1, ctl = 1, alt=1, sh=0, p = "viewPanes", pmo=1, pmc = self._buildMarkingMenu)

Another very simple method. I was surprised that maya does not have a specific markingMenu method. Instead, the popupMenu() command is used, which is actually nice, since it is a familiar one if you have worked with menus or shelf popups. Let us look at the arguments.

  • MENU_NAME – quite obviously this one sets the name of our custom marking menu.
  • mm – this one defines this popupMenu as a marking menu.
  • b – this is the mouse button we would like to trigger the marking menu. 1 – left, 2 – middle, 3 – right.
  • aob – allows option boxes.
  • ctl – defines the Ctrl button as a needed modifier to trigger the marking menu.
  • alt – defines the Alt button as a needed modifier to trigger the marking menu.
  • sh – defines the Shift button as a needed modifier to trigger the marking menu.
  • p – the parent ui element. For marking menus, that would usually be “viewPanes”, which refers to all of our view panels.
  • pmo – this flag declares that the command that we pass to the pmc flag should be executed only once and not everytime we invoke the marking menu. If this is false, everytime we trigger our custom marking menu, we will see our menus growing as more and more menuItems will be added.
  • pmc – this is the command that gets called right before the popupMenu is displayed. This is where we need to pass our method that actually builds all our buttons and menus – _buildMarkingMenu.

It is important to notice that when we pass the self._buildMarkingMenu we do not add brackets in the end as that would call the function instead of passing it as as reference.

Additionally, it is also important to think about the button flag and the modifiers. Obviously, Maya already has some marking menus and some functions related to mouse clicks and the ctrl, alt and shift modifiers. Therefore, we need to come up with a combination that does not destroy a function which we actually want to keep. That is why, for my marking menu I use the middle mouse button + alt + ctrl. There was some zooming function bound to that combination I believe, but I never used it so it was safe to override.

Actually building our custom marking menu

So, everything up to this point was to set us up for actually adding our commands to our menu. To be honest, it is as simple as everything we have already seen. Let us have a look at how we do this.

def _buildMarkingMenu(self, menu, parent):

## Radial positioned
mc.menuItem(p=menu, l="South West Button", rp="SW", c="print 'SouthWest'")
mc.menuItem(p=menu, l="South East Button", rp="SE", c=exampleFunction)
mc.menuItem(p=menu, l="North East Button", rp="NE", c="mc.circle()")

subMenu = mc.menuItem(p=menu, l="North Sub Menu", rp="N", subMenu=1)
mc.menuItem(p=subMenu, l="North Sub Menu Item 1")
mc.menuItem(p=subMenu, l="North Sub Menu Item 2")

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'")
mc.menuItem(p=menu, ob=1, c="print 'South with Options'")

## List
mc.menuItem(p=menu, l="First menu item")
mc.menuItem(p=menu, l="Second menu item")
mc.menuItem(p=menu, l="Third menu item")
mc.menuItem(p=menu, l="Create poly cube", c="mc.polyCube()")

The first thing to note is that our method receives menu and parent as arguments. These are passed automatically from the pmc flag on the popupMenu function in the _build() method.

Then we have the actual items in our custom marking menu. I like to split them logically in – radial and list blocks.

Radial positions

The radial positions are defined by the directions on a map – East, West, NorthEast, etc. Have a look at the following image.

Custom marking menu - Radial positions

You can have either commands in these slots or additional subMenus like so.

My personal preference here is to have just single commands instead of additional popups, as to invoke a submenu you need to hover on a position and wait a little bit. And I find that small delay quite frustrating.

So the way we create these radial items is by using the menuItem command. All we have to do is pass our menu argument as the parent (p), define a label (l), a radial position (rp) and the command (c) we want to execute on click.

Notice that if passing functions as commands we pass them without brackets, as that would call them instead. Have a look at the SE radial position for an example. Additionally, the functions need to be able to receive arguments as maya’s ui elements tend to pass some info about themselves to the commands they call. The way we do that is by just adding the *args argument.

def exampleFunction(*args):
    print "example function"

Additionally, we can call maya commands from our items. Have a look at the NE item. It is important to note here that even though we have imported maya.cmds as mc in this file, the commands we pass to our menuItems are going to be called from maya’s python environment. That means, that in order for mc.circle() to work, we need to have imported maya.cmds as mc somewhere in our maya session. You could either run it yourself in the script editor or a better solution would be to add it to your userSetup.py file. That is what I did, since I use a lot of python in maya I just do an import maya.cmds as mc in my userSetup.py.

Submenus

As I said we can have subMenu items, which essentially create another menu when you hover on them. The only thing we do is set the subMenu flag to 1. Have a look at one of these in our N radial position example. We store the menuItem with the subMenu flag in a variable, so we can use it as a parent for our following items. From then on, we just follow the same principle, we list menuItems, with the subMenu variable that we stored as a parent. Additionally, we can have deeper subMenus as well, though I do not think that would be great to work with.

Option boxes

Another very useful addition to our menuItems is adding option boxes. A lot of maya’s menus have those and they are a nice way to add additionall functionality without sacrificing space.

Usually, you would add them to commands that sometimes need their options to be changed, but also you can have different commands bound to them. For example, in my main marking menu I have the joint tool in one of my radial positions. But since I never mess with the options for that tool, in the option box I have a command which creates a joint under every object I have selected. Additionally, it names it with the name of the selected object and assumes it’s transformations.

The way we add an option box is very simple. All we need to do is create another menuItem after the one we want to add the option box to and we set the ob flag to 1. Have a look at the S radial position for an example.

List elements

In addition to our radial positioned items, marking menus have another menu beneath the South radial position. What is cool about this one is that it is an actual menu instead of just a single radial position, so we can have multiple items in it.

The way we create those items is by just specifying the marking menu as a parent. Remember it is passed as the menu argument to our method. So, if we do not specify a radial position the menuItem gets added to that menu which is south of the South position.

Again, as with all other menus, we can have submenus in there, by just setting the submenu flag to 1. My personal preference though is to have as few of these as possible, as I want everything to be easy to grab at first glance.

Icons

I have not added any icons in this example marking menu, but these can easily be added to every menuItem by using the i flag. All you need to do is place your custom icons somewhere in the XBMLANGPATH environment variable. To see what that is on your machine run this MEL command getenv XBMLANGPATH.

Additionally, you can use maya’s native icons, which you can browse through in the shelf editor. If you open that, pick any button on the right and next to the Icon Name field you can find a Browse Maya Icons button.

For example if I want to add maya’s standard icon to the S radial position I would do this.

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'", i="mayaIcon.png")

That results in the following image.

Custom marking menu - icon

And of course you can also pass full paths to icons which are not on maya’s XBMLANGPATH path.

Rebuilding our custom marking menu and loading it on startup

So, now that we know how to build our marking menus, let us have a look at how to actually initialize them with maya and how to rebuild them on the fly when we make changes. For building our marking menu we just initialize our markingMenu() class. For rebuilding we need to do the same thing, but we have several options for how we call our class.

Option 1: Just run it

I would say this is the simplest solution. If you are working in an external editor you can easily copy and paste your modified code in maya’s script editor, run it and you’re done. Since, we have added the rebuilding functionality – delete old and then build new one – the marking menu is updated everytime we run our code and then call the markingMenu() class in the end.

Even better if you have connected your external editor to maya via a commandPort command you can just run the code from there and that’ll update your marking menu as well. (I use this plugin for Sublime to do that.

The downside with this option is that everytime we open maya we will need to run our code. Which means that at some point you will get annoyed with doing it.

Option 2: Add it to your scripts path

Even though the first solution is quite simple this one is a bit nicer to work with. It also takes care of loading our marking menu on startup.

So, what we do here is we add the file containing our code – markingMenu.py – somewhere on our MAYA_SCRIPT_PATH. Again, you can run getenv MAYA_SCRIPT_PATH in MEL to get this path.

What this means is that now we can access our marking menu from within maya. Therefore we have a lot of options on how to build/rebuild our marking menu.

For building it on startup we just need to add the following code inside our userSetup.py.

import markingMenu
markingMenu.markingMenu()

When rebuilding though we need to do a reload statement. That is because when importing a module, python checks whether it is already imported and if so, just uses that instead. So if we make changes and then do import, our changes are not going to be imported. That is why we need to do reload(ourModule).

So in the case of reloading our marking menu, we will need to do the following.

import markingMenu
reload(markingMenu)
markingMenu.markingMenu()

This will ensure that whatever changes we have made to our code will be implemented.

Now that we have this rebuilding code, we can add it wherever we find most appropriate. Generally, I would say either a button on a shelf or from within the marking menu itself. Since a button on a shelf is trivial let us look at adding it to the marking menu.

Reloading from within our custom marking menu

The way we would do this, is just add the following menuItem.

mc.menuItem(p=menu, l="Rebuild Marking Menu", c=rebuildMarkingMenu)

Where the rebuildMarkingMenu refers to the following function.

def rebuildMarkingMenu(*args):
    mc.evalDeferred("""
reload(markingMenu)
markingMenu.markingMenu()
""")

I really don’t like using evalDeferred since it feels a bit dirty, but in this case we need it. The reason we need it is that we are rebuilding our marking menu from within. So if we do not use evalDeferred but directly call our markingMenu(), maya will have to delete the button which we have clicked, and that will error.

Conclusion

So, there we have it, we have built our own custom marking menu with Python. What is more, it is fully scripted, so making changes is very easy. Additionally, we can have that file in a version control system, so we can have a log of our changes. And of course, it is super easy to share with co-workers. The best thing about it, though, is how much time it saves in our day-to-day rigging tasks.

So, a few months back I was looking into clean ways of scripting a custom shelf in Maya, that can be easily shared, and I was surprised that there were not many resources on how to go about it. Therefore, I thought I would share the way that I am maintaining my shelf.

tl;dr Grab the code for the shelf base class from here and derive your own shelf class from it, overwriting and populating the build() method with calls to the addButton, addMenuItem, etc. calls to create your own custom maya shelf.

Scripting your own custom shelf in Maya is much easier than you would think, or at least than I thought. For some odd reason, I always assumed that it would involve a lot of messing about with MEL, which I would really rather not. If that is what you thought as well, you will be pleasantly surprised.

Here is what we are trying to achieve.

Custom shelf in maya

Since, we want to keep the code as modular and versatile as possible, I wrote the main functionality in a base class which needs to be extended for each shelf you want to build. The code for the base class with a little example is on github.

Now we will go through it to make see how it works, so it can be extended and modified to include more functionalities.

Constructor

def __init__(self, name="customShelf", iconPath=""):
    self.name = name

    self.iconPath = iconPath

    self.labelBackground = (0, 0, 0, 0)
    self.labelColour = (.9, .9, .9)

    self._cleanOldShelf()
    mc.setParent(self.name)
    self.build()

In the constructor we initialize the variables we make two calls to _cleanOldShelf() and build(), which we will look at in a bit. The name argument is going to be the name of the shelf. It is important to note that if a shelf with this name already exists it will be replaced with this one. The iconPath argument can be used to define a directory from which to get images to be used as icons of the shelf buttons and commands in the popup menus. If it is not defined, you can still use maya’s default icons like commandButton.png for example.

Additionally there are the labelBackground and labelColour variables which can be seen in the following image.

Custom shelf in maya - button colours

The reason I have hardcoded them in the base class is because I think for the sake of consistency all our shelves should have the same style, but if you want to change them, obviously feel free to do it.

And lastly, there is that mc.setParent(self.name) function, which makes sure that whatever ui elements we are going to build in the following build() method, they will be build as children to our shelf. Otherwise, we might end up with buttons breaking the other layouts.

Clean old shelf

Let’s have a look at what the _cleanOldShelf() method does.

def _cleanOldShelf(self):
    if mc.shelfLayout(self.name, ex=1):
        if mc.shelfLayout(self.name, q=1, ca=1):
            for each in mc.shelfLayout(self.name, q=1, ca=1):
                mc.deleteUI(each)
    else:
        mc.shelfLayout(self.name, p="ShelfLayout")

Essentially, we are checking if our shelf already exists by using the mc.shelfLayout(self.name, ex=1) command where self.name is the name of our shelf and ex is the flag for checking existence. If it does we go through all children of the shelf which we get from mc.shelfLayout(self.name, q=1, ca=1) and delete them. That makes sure we start our build() method with a fresh clean shelf.

If the shelf does not exist though, we simply create it, passing "ShelfLayout" as parent, because that is the parent layout of all the shelves in maya. For example, turn on History > Echo all commands in the the script editor and click through some of the shelves. You will see the paths to some of their popupMenus as so
Shelf|MainShelfLayout|formLayout16|ShelfLayout|CurvesSurfaces||popupMenu546

So, knowing that CurvesSurfaces is a name of a shelf, we can deduce that the ShelfLayout is the parent of all shelves.

Then, before we get into the build() method, let’s have a look at some of the methods we are going to use to actually populate our shelf.

Add button

def addButon(self, label, icon="commandButton.png", command=_null, doubleCommand=_null):
    mc.setParent(self.name)
    if icon:
        icon = self.iconPath + icon
    mc.shelfButton(width=37, height=37, image=icon, l=label, command=command, dcc=doubleCommand, imageOverlayLabel=label, olb=self.labelBackground, olc=self.labelColour)

Very simply, this is the method that creates the main buttons in the shelf. Essentially, we just need to pass it a label which is the name of our button, an optional icon and an optional command and doubleCommand which refer to single click and double click. The command is optional, because we might want buttons that just have popup menus. We will look at those in a sec. By default the _null() command, which is at the top of the file is called. The reason I have that command is that if you pass None to the mc.shelfButton command flag, it will error. So the _null() method essentially does nothing. It is important to note that when we pass a command we are not using the brackets after the name (), because that would call the command instead of passing it.

Add menu item

def addMenuItem(self, parent, label, command=_null, icon=""):
    if icon:
        icon = self.iconPath + icon
    return mc.menuItem(p=parent, l=label, c=command, i="")

This method is very similar to the add button one, but instead of adding a button to the shelf, this one adds a menuItem to an existing popupMenu passed as the parent argument. The popupMenu needs to be created manually, as it doesn’t make sense to wrap it in a method if it is just a simple one line command anyway. So, a quick example of how that works is

...
self.addButon("popup")
p = mc.popupMenu(b=1)
self.addMenuItem(p, "popupMenuItem1")
...

where the b=1 stands for left mouse click. This snippet if added inside the build() method will attach a popupMenu with one menuItem to the popup button.

Add sub menu

def addSubMenu(self, parent, label, icon=None):
    if icon:
        icon = self.iconPath + icon
    return mc.menuItem(p=parent, l=label, i=icon, subMenu=1)

Now, I realize that sub menus inside a popup on a shelf button is getting a bit too deep, as the whole point of a shelf is to streamline everything, but I thought I would add it, as even I have used it on a couple of my buttons.

What this method does is, it creates a menuItem which is a menu in itself, attached to an existing popupMenu which is passed as the parent.

The result is something similar to this.

Custom shelf in maya example popup menus

Separators

You know how some of the shelves have separating lines between the buttons. You can achieve that with the mc.separator() command. I usually set the style flag to "none" as I rather having blank space than having lines, but have a look at the docs page for the separator command for more info.

Build

And then finally let us have a look at the build() method.

def build(self):
'''This method should be overwritten in derived classes to actually build the shelf
elements. Otherwise, nothing is added to the shelf.'''
    pass

As the docstring says, this is an empty method that needs to be defined in each shelf we build. The way this works is, we just populate it with addButton(), mc.popup(), addMenuItem() and addSubMenu() calls as we need them.

Example of a custom shelf in maya

Let’s have a look at the simple example in the end of the file.

class customShelf(_shelf):
    def build(self):
        self.addButon(label="button1", command=mc.polyCube)
        self.addButon("button2")
        self.addButon("popup")
        p = mc.popupMenu(b=1)
        self.addMenuItem(p, "popupMenuItem1")
        self.addMenuItem(p, "popupMenuItem2")
        sub = self.addSubMenu(p, "subMenuLevel1")
        self.addMenuItem(sub, "subMenuLevel1Item1")
        sub2 = self.addSubMenu(sub, "subMenuLevel2")
        self.addMenuItem(sub2, "subMenuLevel2Item1")
        self.addMenuItem(sub2, "subMenuLevel2Item2")
        self.addMenuItem(sub, "subMenuLevel1Item2")
        self.addMenuItem(p, "popupMenuItem3")
        self.addButon("button3")
    customShelf()

As you can see, all we do is we create a new class inheriting from the _shelf class. By the way the _ prefix is used to remind us that it is a class that should not be directly accessed.

Then we overwrite the build() method by simply defining it, and we populate it with the necessary shelf elements.

This example results in the image from above. Here is it again.

Commands

I have not demonstrated the use of any commands on these buttons, but all it takes is just passing the name of the function as a command argument. The one tricky bit about it is that the menuItem commands get passed some arguments, so we need to have *args in our function calls to prevent errors. For example

def createPolyCube(*args):
    mc.polyCube()

We would include this command in our shelf by doing self.addButton(label="cube", command=createPolyCube) in our build() method.

For the sake of clarity though, I recommend having a separate file which contains only the functions that are going to be used in the shelf and import them from there. So at the top of the file where are custom shelf is defined, import your shelf functions file with import shelfFunctions or however you decide to package them. I would advise having a reload(shelfFunctions) statement immediately after that, so you can be sure the latest changes to the file have been taken into account.

And then when creating buttons, just do

self.addButton(label="cube", command=shelfFunctions.createPolyCube)

Additionally, you can also pass maya native commands as well. So to have a button that creates a poly cube we would do

self.addButton(label="cube", command=mc.polyCube)

Building our shelf on load up

The way we will include our shelf on load up is to just add it to our userSetup.py file. If you are not familiar with it, take a look at this. I know this one is for MEL, but it is essentially the same thing for Python as well.

So, assuming that our shelf file is in the maya scripts path, in our userSetup.py we can just do

import maya.cmds as mc
import shelf
mc.evalDeferred("shelf.customShelf()")

where shelf is the file containing our shelf and customShelf is the name of the class.

We are using the evalDeferred() command, because maya loads the userSetup.py file as it is starting, so we are not sure if the shelfLayout is already in place as it gets executed. Therefore, evalDeferred() helps us call our script after maya has been initialized.

Conclusion

And that is it, you have built your own shelf. What I really like about it is that if you are collaborating with other people you can add the shelf file to your version control system and in your userSetup.py file just source it from there. That way you can all contribute to it and have it always updated.

If you have any questions, shoot me a message.

I remember the first time I tried to set up a seamless IK FK switch with Python vividly. There was this mechanical EVA suit that I was rigging for a masterclass assignment at uni given by Frontier. The IK to FK switching was trivial and there were not many issues with that, but I had a very hard time figuring out the FK to IK one, as I had no idea what the pole vector really is and also, my IK control was not oriented the same way as my FK one.

Im sure that throughout the web there are many solutions to the problem, but most of the ones I found were in MEL and some of them were a bit unstable, because they were relying too much on the xform command or the rotate one with the ws flag, which I am assuming causes issues sometimes when mapping from world space to relative, where a joint will have the exact same world rotation, so it looks perfectly, but if you blend between IK and FK you can see it shifting and then coming back in place. That’s why I decided to use constraints to achieve my rotations, which seems to be a simple enough and stable solution.

EDIT: It seems like even with constraints it is possible to get that issue in the case where the IK control is oriented differently. What fixes is though is switching back and forth once more.

Here is what we are trying to achieve

Seamless IK FK swich demo

Basically, there is just one command for the seamless IK FK Switch, which detects the current kinematics and switches to the other one maintaining the pose. I have added the button to a custom marking menu for easier access.

So, in order to give you a bit of a better context I have uploaded the example scene that I am using, so you can have a look at the exact structure, but feel free to use your own scene with IK/FK blending setup. The full code (which is very short anyway) is in this gist and there are three scene files in here for each version of our setup. The files contain just a simple IK/FK blending system, on which we can test our matching setup, but with different control orientations.

It is important to understand the limitations of a seamless IK FK switch before we dive in. Mainly, I am talking about the limited rotation of the second joint in the chain, as IK setups allow for rotations only in one axis. What this means is that if we have rotations in multiple axis on our FK control for that middle joint (elbow, knee, etc.) the IK/FK matching will not work properly. All this is due to the nature of inverse kinematics.

Also, for easier explaining I assume we are working on an arm and hand setup, but obviously the same approach would work for any IK/FK chain.

We will consider three cases:
All controls and joints are oriented the same
IK Control oriented in world space
IK Control and IK hand joint both oriented in world

Again, you do not have to use the same file as I do as it is just an example, but it is important to be clear on the existing setup. We assume that we have an arm joint chain – L_arm01_JNT > L_arm02_JNT > L_arm03_JNT and a hand joint chain – L_hand01_JNT > L_hand02_JNT with their correspondent IK and FK chains – L_armIk01_JNT > …, L_armFk01_JNT > …, etc. These two chains are blended via a few blendColors nodes for translate, rotate and scale into the final chain. The blending is controlled by L_armIkFk_CTL.fkIk. Then we have a simple non-stretchy IK setup, but obviously a stretchy one would work in the same way. Lastly, the L_hand01_JNT is point constrained to L_arm03_JNT and we only blend the rotate and scale attributes on it, as otherwise the wrist becomes dislocated during blending, because we are interpolating linearly translation values.

Now that we know what we have to work with, let us get on with it.

Seamless IK FK Switch when everything shares orientation

So, in this case, all of our controls and joints have the exact same orientation in both IK and FK. What this means is that essentially all we need to do to match the kinematics is to just plug the rotations from one setup to the other. Let’s have a look. The scene file for this one is called ikFkSwitch_sameOrient.ma

IK to FK

This one is always the easier setup, as FK controls generally just need to get the same rotation values as the IK joints and that’s it. Now, initially I tried copying the rotation via rotate and xform commands, but whenever a control was rotated a bit too extreme these would cause flipping when blending between IK and FK, which I am assuming is because these commands have a hard time converting the world space rotation to a relative one, causing differences of 360 degrees. So, even though in full FK and full IK everything looks perfect, in-between the joint rotates 360 degrees. Luckily, maya has provided us with constraints which have all the math complexity built in. Assuming you have named your joints the same way as me we use the following code.

mc.delete(mc.orientConstraint("L_armIk01_JNT", "L_armFk01_CTL"))
mc.delete(mc.orientConstraint("L_armIk02_JNT", "L_armFk02_CTL"))
mc.delete(mc.orientConstraint("L_handIk01_JNT", "L_handFk01_CTL"))

mc.setAttr("L_armIkFk_CTL.fkIk", 0)

As I said, this one is fairly trivial. We just orient each of our FK controls to match the rotations of the IK joints. Then in the end we change our blending control to FK to finalize the switch.

FK to IK

Now, this one was a pain the first time I was trying to do it, because I had no idea how pole vectors worked at all. As soon as I understood that all we need to know about them is that they just need to lie on the same plane as the three joints in the chain, it became easy. So essentially, we need to place the IK control on the FK joint to solve the end position. And then to get the elbow (or whatever your mid joint is representing) to match the FK, we just place the pole vector control at the exact location of the corresponding joint in the FK chain. So, we get something like this.

mc.delete(mc.parentConstraint("L_handFk01_JNT", "L_armIk_CTL"))
mc.xform("L_armPv_CTL", t=mc.xform("L_armFk02_JNT", t=1, q=1, ws=1), ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Now even though this does the matching job perfectly it is not great for the animators to have the control snap at the mid joint location as it might go inside the geometry, which is just an unnecessary pain. What we can do is, get the two vectors from arm01 to arm02 and from arm03 to arm02 and use them to offset our pole vector a bit. Here’s the way we do that.

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

So, since xform returns lists in order to subtract them to get the vectors we just loop through them and subtract the individual elements. If you are new to the shorter loop form in Python have a look at this. Then once we have the two vectors we add 75% of them to the position of the arm02 FK joint and we arrive at a position slightly offset from the elbow, but still on the same plane, thus the matching is still precise. Then our whole FK to IK code would look like this

mc.delete(mc.parentConstraint("L_handFk01_JNT", "L_armIk_CTL"))

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Seamless IK FK switch when the IK control is oriented in world space

Now, in this case, the orientation of the IK control is not the same as the hand01 joint. I think in most cases people go for that kind of setup as it is much nicer for animators to have the world axis to work with in IK. The scene file for this one is called ikFkSwitch_ikWorld.ma.

The IK to FK switch is exactly the same as the previous one, so we will skip it.

FK to IK

So, in order to get this to work, we need to do the same as what we did in the previous case, but introduce an offset for our IK control. How do we get this offset then? Well, since we can apply transformations only on the controls, we need to calculate what rotation we need to apply to that control in order to get the desired rotation. Even though, we can calculate the offsets using maths and then apply them using maths, we might run into the same issue with flipping that I discussed in the previous case. So, instead, a much easier solution, but somewhat dirtier is to create a locator which will act as our dummy object to orient to.

Then, in our case where only the IK control is oriented differently from the joints, what we need to do is create a locator and have it assume the transformation of the IK control. The easiest way would be to just parent it underneath the control and zero out the transformations. Then parent the locator to the L_handFk01_JNT, as that’s the one that we want to match to. Now wherever that handFk01 joint goes, we have the locator parented underneath which shares the same orientation as our IK control. Therefore, just using parentConstraint will give us our matching pose. Assuming the locator is called L_hand01IkOfs_LOC all we do is this.

mc.delete(mc.parentConstraint("L_hand01IkOfs_LOC", "L_armIk_CTL"))

This will get our wrist match the pose perfectly. Then we apply the same code as before to get the pole vector to match as well and set the IK/FK blend attribute to IK.

arm01Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk01_JNT", t=1, ws=1, q=1)[i] for i in range(3)]
arm02Vec = [mc.xform("L_armFk02_JNT", t=1, ws=1, q=1)[i] - mc.xform("L_armFk03_JNT", t=1, ws=1, q=1)[i] for i in range(3)]

mc.xform("L_armPv_CTL", t=[mc.xform("L_armFk02_JNT", t=1, q=1, ws=1)[i] + arm01Vec[i] * .75 + arm02Vec[i] * .75 for i in range(3)], ws=1)

mc.setAttr("L_armIkFk_CTL.fkIk", 1)

Seamless IK FK switch when the IK control and joint are both oriented in world space

Now, in this last scenario, we have the handIk01 joint oriented in world space, as well as the control. The reason you would want to do this again is to give the animators the easiest way to interact with the hand. In the previous case, the axis of the IK control do not properly align with the joint which is a bit awkward. So a solution would be to have the handIk01 joint oriented in the same space as our control, so the rotation is 1 to 1 and it is a bit more intuitive. The scene for this one is ikFkSwitch_ikJointWorld.ma and it looks like this.

It is important to note that the IK joint is just rotated to match the position of the control, but the jointOrient attributes are still the same as the FK one and the blend one.

Seamless IK FK Switch with IK control and joint oriented in world space
IK FK Switch with IK control and joint oriented in world space

So again, going from IK to FK is the same as before, we are skipping it. Let us have a look at the FK to IK.

FK to IK

This one is very similar to the previous one, where we have an offset transform object to snap to. The difference is that now instead of having that offset be calculated just from the difference between the IK control and the FK joint, we also need to adjust for the existing rotation of the IK joint as well. So, we start with our locator the same way as before – parent it to the IK control, zero out transformations and parent to the handFk01 joint. And then, the extra step here is to apply the negative rotation of the IK joint to the locator in order to get the needed offset. So, this calculation would look like this.

ikRot = [-1 * mc.xform("L_handIk01_JNT", ro=1 ,q=1)[i] for i in range(3)]
mc.xform("L_hand01IkOfs_LOC", ro=ikRot, r=1)

We just take the rotation of the IK joint and multiply it by -1, which we then apply as a relative rotation to the locator.

And then again, as previously we just apply the pole vector calculation and we’re done.

Conclusion

So, as you can see, scripting a seamless IK FK switch is not really that complicated at all, but if you are trying to figure it out for the first time, without being very familiar with rigging and 3D maths it might be a bit of a pain. Again, if you want to see the full code it is in this gist.