So, it seems like I have been going crazy with marking menus lately. I am really trying to get the most of them, and that would not be much if we could only use them in the viewports, so today we are going to look at how we can construct custom marking menus in maya editors.
tl;dr: We can query the current panel popup menu parent in maya with the findPanelPopupParent MEL function, and we can use it as a parent to our popupMenu.
So, there are a couple of scenarios that we need to have a look at, as they should be approached differently. Although, not completely necessary I would suggest you have a look at my previous marking menu posts – Custom marking menu with Python and Custom hotkey marking menu – as I will try to not repeat myself.
Okay, let us crack on. Here are the two different situations for custom marking menus in maya editors we are going to look at.
In the viewport these are definitely the easier ones to set up as all we need to do is just create a popupMenu with the specified modifiers – sh, ctl and alt, the chosen button and viewPanes as the parent. When it comes to the different editors, though, it gets a bit trickier.
Let us take the node editor as an example.
If we are to create a marking menu in the node editor, it is a fairly simple process. We do exactly the same as before, but we pass "nodeEditorPanel1" as the parent argument. If you have a node editor opened when you run the popupMenu command, you will be able to use your marking menu in there. The catch is though, that once you close the node editor the marking menu is deleted, so it is not available the next time you open the node editor.
Unfortunately, I do not have a great solution to this problem. In fact, it is a terrible solution, but I wanted to get it out there, so someone can see it, be appalled and correct me.
The second method – Custom hotkey trigger – is much nicer to work with. So, you might want to skip to that one.
What I do is, I create a hotkey for a command that invokes the specific editor (I only have marking menus in the node editor and the viewport) and runs the marking menu code after that. So, for example, here is my node editor hotkey (Alt+Q) runTimeCommand.
That means that everytime I open the node editor with my hotkey I also create the marking menu in there, ready for me to use. As, I said, it is not a solution, but more of a workaround at this point. In my case, though, I never open the node editor through anything else than a hotkey, so it kind of works for me.
Then the vsRigging.markingMenus.vsNodeMarkingMenu file is as simple as listing the menuItems.
A proper way of doing this would be to have a callback, so everytime the node editor gets built we can run our code. I have not found a way to do that though, other than ofcourse breaking apart maya’s internal code and overwriting it, which I wouldn’t go for.
Luckily, creating a custom marking menu bound to a custom hotkey actually works properly and is fairly easy. In fact, it is very similar to the Custom hotkey marking menu post. Let us have a look.
Custom hotkey trigger
Now, when we are working with custom hotkeys we actually run the initialization of the popupMenu everytime we press the hotkey. This means we have the ability to run code before we create the marking menu. Therefore, we can query the current panel and build our popupMenu according to it. Here is an example runTimeCommand, which is bound to a hotkey.
import maya.mel as mel
name = "exampleMarkingMenu"
if mc.popupMenu(name, ex=1):
parent = mel.eval("findPanelPopupParent")
if "nodeEditor" in parent:
popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)
from markingMenus import exampleMarkingMenu
popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)
from markingMenus import fallbackMarkingMenu
So, what we do here is, we start by cleaning up any existing versions of the marking menu. Then, we use the very handy findPanelPopupParent MEL function to give us the parent to which we should bind our popupMenus. Having that we check if the editor we want exists in the name of the parent. I could also compare it directly to a string, but the actual panel has a number at the end and I prefer just checking the base name. Then, depending on which panel I am working in at the moment, I build the appropriate custom marking menu.
Don’t forget that you need to create a release command as well, to delete the marking menu so it does not get in the way if you are not pressing the hotkey. It is a really simple command, that I went over in my previous marking menu post.
The obvious limitation here is that we have a hotkey defined and we cant just do ctrl+alt+MMB for example.
So, yeah, these tend to be a bit trickier than just creating ones in the viewport, but also I think there is more to be desired from some of maya’s editors *cough* node editor *cough*, and custom marking menus help a lot.
So, recently I stumbled upon a djx blog blost about custom hotkeys and marking menus in different editors in maya. I had been thinking about having a custom hotkey marking menu, but was never really sure how to approach this, so after reading that post I thought I’d give it a go and share my process.
tl;dr: We can create a runtime command which builds our marking menu and have a hotkey to call that command. Thus, giving us the option to invoke custom marking menus with our own custom hotkeys, such as Shift+W or T for example, and a mouse click.
Disclaimer: I have been having a super annoying issue with this setup, where the “release” command does not always get called, so the marking menu is not always deleted. What this means is that if you are using a modifier like Shift, Control or Alt sometimes your marking menu will still be bound to it after it has been closed. Therefore, if you are using something like Shift+H+LMB, just pressing Shift+LMB will open it up, so you lose the usual add to selection functionality. Sure, to fix it you just have to press and release your hotkey again, but it definitely gets on your nerve after a while.
If anyone has a solution, please let me know.
I have written about building custom marking menus in Maya previously, so feel free to have a look as I will try to not repeat myself here. There I also talked about why I prefer to script my marking menus, instead of using the Marking menu editor, and that’s valid here as well.
So, let us have a look then.
The first thing we need to do is define a runTimeCommand, so we can run it with a hotkey. That is what happens if you do it through the Marking menu editor and set Use marking menu in to Hotkey Editor, as well.
There a couple of ways we can do that.
On the right hand side of the hotkey editor there is a tab called Runtime Command Editor. If you go on that one you can create and edit runTime commands.
Scripting it in Python
If you have multiple marking menus that you want to crate, the hotkey editor might seem as a bit of a slow solution. Additionally, if changes need to be made I always find it more intuitive to look at code in my favourite text editor (which is sublime by the way).
To create a runTime command we run the runTimeCommand function which for some reason does not appear in the Python docs, but I’ have been using maya.cmds.runTimeCommand successfully.
All we need to provide is a name for the command, some annotation – ann, a string with some code – c and a language – cl.
Something we need to keep in mind when working with runTime commands is that we cannot pass external functions to them. We can import modules and use them once inside, but I cannot pass a reference to an actual function to the c flag, as I would do to menuItems for example. That means that we need to pass our code as a string.
Press and release
Now, that we know how to create the runTimeCommands let us see what we need these commands for.
As I mentioned, they are needed so we can access them by a hotkey. What that hotkey should do is initialize our marking menu, but once we release the key it should get rid of it, so it does not interfere with other functions. Therefore we need two of them – Press and Release.
Let us say we are building a custom hotkey marking menu for weight painting. In that case we will have something similar to the following.
mmWeightPainting_Press runTimeCommand – to initialize our marking menu
mmWeightPainting_Release runTimeCommand – to delete our marking menu
The way we bind the release command to the release of a hotkey is by pressing the small arrow to the side of the hotkey field.
The Press command
import maya.cmds as mc # Optional if it is already imported
name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)
So, essentially what we do is every time we press our hotkey, we delete our old marking menu and rebuild it. We do this, because we want to make sure that our latest changes are applied.
Now, the lower part of the command is where it gets cool, I think. We can store our whole marking menu build – all menuItems – inside a file somewhere in our MAYA_SCRIPT_PATH and then just import it from the runTimeCommand as in this piece of code. What this gives us, is again, the ability to really easily update stuff (not that it is a big deal with marking menus once you set them up). Additionally, I quite like the modularity, as it means we can have very simple runTimeCommands not cluttered with the actual marking menu build. This is the way that creating through the Marking menu editor works as well, but obviously it loads a MEL file instead.
So, literally that mmWeightPainting file is as simple as creating all our marking menu items.
import maya.cmds as mc
mc.menuItem(l="North radial position", rp="N")
And that takes care of building our marking menu when we press our hotkey + the specified modifiers and mouse button. What, we do not yet have is deleting it on release, so it does not interfere with the other functionality tied to modifier + click combo. That is where the mmWeightPainting_ReleaserunTimeCommand comes in.
The Release command
## mmWeightPainting_Release runTimeCommand
name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
Yep, it is a really simple one. We just delete the marking menu, so it does not interfere with anything else. Essentially, the idea is we have it available only while the hotkey is pressed.
All that is left to be done is to assign a hotkey to the commands. There are a couple of things to have in mind.
If you are using modifiers for the popupMenu command – sh, ctl or alt – then the same modifiers need to be present in your hotkey as otherwise, even though the runTimeCommand will run successfully, the popupMenu will not be triggered.
So, if you have been rigging for a while you have probably felt annoyed by having to create and adjust control shapes every time you build a new rig. You have probably found also that mirroring just the shape of a control or copying it to another one is a bit too tedious. There are some scripts and tools online to help you with this, such as the classic comet menu and the mz_ctrlcreator, but they do not offer all the functions we need and also extending them is not very practical. So, let us write our own control shape manager.
tl;drI am going to walk you through the process of creating your own control shape manager, but if you would rather just use the final code you can find it here. If you would prefer it combined into one large file, you can grab it from here.
Here is a quick demo of some of the features we are going to look at.
What we want is a python package that allows us to load and save control shapes to a library, copy and paste them to multiple other controls, change colours, flip them, mirror them, etc.
The full code can be found here. I have built it as a package with a few different modules, to be a bit clearer and nicer to maintain. I have also combined everything into one file as well, if anyone wants to just grab it and use it immediately. What we are going to do here though, is go through the code and learn how to build our own control shape manager, because it is much nicer when you actually understand how it works, as then you can extend it and adjust it to suit your needs. For example I have built upon this a bit more in my pipeline, so I can save and load shape versions for each control on a rig, so I can easily rebuild them when I am making changes.
Part 1: Control Shape Manager
We are going to be using a couple of commands from the Maya API, but if you are not very familiar with it, worry not I will explain what each function does. You can always read up on it on the Autodesk docs page or if you prefer more of a tutorial approach have a look at Chad Vernon’s Maya API web page.
Getting and setting shapes
Let us start with the two most important functions – getShape() and setShape().
What this function does is, it gets all the data from a nurbsCurve node that we need to rebuild that curve down the line. We are going to look at the validateCurve() function a bit later, but it essentially checks if the curve we have passed is actually a valid curve and if so returns the shape node.
A list is initialized here which will later be populated with dictionaries for each shape node on the curve in order to work with compound curves.
The crvShapeDict is where the actual data is stored. All of the keys in the dictionary are just the needed data for building a curve. If you do not know what the knots and degree are when it comes to curve, you can read up on it here, but it is not necessary. We will be thinking of them as the essential building blocks of a curve.
You can see that very easily we can get the form, degree and colour ones as they are just attributes on the nurbsCurve node.
To get the points what we need to do is loop through all of the controlPoints of the curve. Initially, I was just using the cv attribute, but it does not work with closed curves, as the cvs are just representation of these points, so we can interact with them, but under the hood maya changes them a bit and they are stored in the controlPoints attribute. So, we just get the number of control points using the s flag on the getAttr command and we store each point in a list.
Now, for the knots initially I used this snippet from Serge Scherbakov, but it does not work with closed curves. I could have gone in and tried to create my own function to do that, but then maya has made it easy for us to get the knots from the API, so I thought I would just use that.
mObj = om.MObject()
sel = om.MSelectionList()
fnCurve = om.MFnNurbsCurve(mObj)
tmpKnots = om.MDoubleArray()
return [tmpKnots[i] for i in range(tmpKnots.length())]
The first part of this function deals with getting an API reference to our curve. It basically, adds the passed in crvShape to a virtual selection (without actually selecting anything in the viewport) and gets an MObject from it. That’s the base class in the API and from there we can cast it to the type we actually need – MFnNurbsCurve. Then we create an empty MDoubleArray, which we populate from the curve with the getKnots function. And that’s it. Lastly, we return it as a python list, just so we can interact with it easier.
And with that we have a list of dictionaries containing all the necessary information for rebuilding that curve.
Let’s look at setting the shape now. What is nice about this code is that if you understand how the getShape() works, the setShape() is going to be trivial. The one thing I do not like about this code is that we are not assigning the data to the existing curve, but we delete it and create a new one in place. This could cause issues if there are any connections to or from the shape node, but you can always store and rebuild those. I have not yet found a way around it though.
We go through the same call to validateCurve() as before and then we store the "overrideColor" of the curve, so we can apply it back after we rebuild the shape. It is important to note that the colour is the one assigned to the first shape child of the curve. And since we have everything we need from the old shapes – the colour – we delete them.
Then for each shape in the list we just use our points, knots, degree and form data from the dictionary to build a new curve with the mc.curve() command. The per flag refers to periodic and basically defines whether our curve is one whole or does it have a start and an end. A bit more info about periodic curves in here.
Once we have created the new shape we parent it to the crv object with the r=1 and s=1 flags for mc.parent() to define that we are working with shapes and to maintain their relative positions. We then can rename the new shape according to our convention. Lastly, we just reapply the colour or we get it from the dictionary.
As I said these two are the most important functions as they are dealing with the actual data. Now that we have them in place we can give them a quick test. Create a nurbsCurve with whatever shape you want. Then let’s create a simple circle and copy the first shape to the circle. Assuming that the first curve is called curve1 and the circle is nurbsCircle1 run the following snippet.
I realize this is not very exciting as there are easier ways to do this, but the cool thing is when we start saving and loading them.
Saving and loading
We have been looking only at the manager.py file for know. In the utils.py we have a few more functions mainly dealing with the saving and loading of json data. Loading and saving JSON files is a very popular and fairly trivial python task, but let’s deconstruct it.
f = open(path, "r")
data = json.loads(f.read())
mc.error("The file " + path + " doesn't exist")
f = open(path, "w")
f.write(json.dumps(data, sort_keys=1, indent=4, separators=(",", ":")))
For loading we start by checking if the file exists and if it does, we use python’s open function to get the raw data and we pass it to a json.loads() function to convert the raw data to a Python dict object.
When saving, we are doing the same thing but instead of reading and converting from raw data to a dict we are passing a dict to the json.dumps() function which serializes our dictionary to JSON and then we write it to the file. You will notice that there is a call to another validation function – validatePath().
confirm = mc.confirmDialog(title='Overwrite file?',
message='The file ' + path + ' already exists.Do you want to overwrite it?',
if confirm == "No":
mc.warning("The file " + path + " was not saved")
All we do here is check if the file we are trying to save already exists and if so gives the option to overwrite it or cancel the save process.
Now that we know how our dictionary data is being load and saved, we just need to have a wrapper function in our manager module to load and save to the defined shape library directory.
Before looking at those though, you need to have the SHAPE_LIBRARY_PATH set at the top of the file. Keep in mind that if the path does not exist, Python will not create it for us but error out.
path = os.path.join(SHAPE_LIBRARY_PATH, shape + ".json")
data = utils.loadData(path)
What we do here is define the path to the file we want to load and use the loadData function we talked about to load the actual dictionary.
Then when saving we use re.sub("s", "", shape) in order to strip spaces from the name as they can cause issues and pass the path to the saveData() function. Also, we get rid of the colour keys as we want to save only the shape of the curve.
The rest of the functions in the module are fairly self-explanatory.
if mc.nodeType(crv) == "transform" and mc.nodeType(mc.listRelatives(crv, c=1, s=1)) == "nurbsCurve":
crvShapes = mc.listRelatives(crv, c=1, s=1)
elif mc.nodeType(crv) == "nurbsCurve":
crvShapes = mc.listRelatives(mc.listRelatives(crv, p=1), c=1, s=1)
mc.error("The object " + crv + " passed to validateCurve() is not a curve")
The validateCurve() function just checks if we have passed a valid curve and if so it returns the nurbsCurve shape nodes to work with. Otherwise it errors.
Then we have the colour functions which are just simple wrappers around mc.getAttr() and mc.setAttr() commands to interact with the "overrideColor" attribute of shapes.
def setColour(crv, colour):
if mc.nodeType(crv) == "transform":
crvShapes = mc.listRelatives(crv)
crvShapes = [crv]
for crv in crvShapes:
mc.setAttr(crv + ".overrideColor", colour)
if mc.nodeType(crv) == "transform":
crv = mc.listRelatives(crv)
return mc.getAttr(crv + ".overrideColor")
Part 2: Control Shape Functions
Now that we have our core functionality in place we can stop here and just use the code we have so far through our script editor, which is absolutely fine, but is not very scalable and not really user friendly. Additionally, we are still lacking the mirroring and flipping functionality, so let us create a functions.py file which will act as a wrapper to our manager module. The reason we would want this is to prevent messing about with our manager too much and provide a higher level control so we can literally only care about using the tool instead of how it works. Altogether, it is much nicer working with simple short functions. Okay, let us go through the functions.py commands that help us interact with the manager.
Getting lists for the UI
lib = manager.SHAPE_LIBRARY_PATH
return [(x.split("."), functools.partial(assignControlShape, x.split("."))) for x in os.listdir(lib)]
return [("index" + str(i).zfill(2), functools.partial(assignColour, i), "shapeColour" + str(i).zfill(2) + ".png") for i in range(32)]
These two functions are mainly here to help us later when we are going to build some sort of UI for our manager. Essentially they return lists of tuples containing the names, commands and in the case of getAvailableColours() images of the available shapes and colours. These are going to be used when building menus that look similar to the following.
Notice that the second item in the tuple is a functools.partial() call. For more info refer to the docs, but briefly it allows us to get a reference to a function with added arguments as well. So the first argument is a function and then we have a number of arguments which are going to be provided to the function as *args. Let’s have a look at the functions themselves to see how this works.
Assigning shapes and colours
for each in mc.ls(sl=1, fl=1):
sel = mc.ls(sl=1, fl=1)
for each in sel:
So, both these functions receive *args as an argument, which means that we can provide lots of arguments and they are going to be passed to the function as a list which we can acces by args[n]. In the previous paragraph, we saw that we pass these functions and a single argument to the functools.partial, which means that the first element of args is going to be the second argument of the functools.partial() code. So in the case of functools.partial(assignColour, i), we are going to receive a call equivalent to assignColour(i).
Additionally, keep in mind if these functions that we are defining here are meant to be used from a maya UI, and a lot of the buttons in maya are passing arguments to their commands, so we need to have the *args, because otherwise the functions will error.
Notice that we reselect our initial selection at the end of the function. We will do this in all functions that call the setShape() one, because the creation of the curve inside of it deselects our selection and instead selects the newly created curve, which is not very intuitive.
Saving to library
result = mc.promptDialog(title="Save Control Shape to Library",
m="Control Shape Name",
if result == "Save":
name = mc.promptDialog(q=1, t=1)
manager.saveToLib(mc.ls(sl=1, fl=1), name)
As we said the goal here is to make interacting with our control shape manager as smooth as possible. Therefore, we create a wrapper to our saveToLib() command to let us add a name in a nice and familiar dialog. In the end we are calling the rebuildUI() function which we will look at the end of this part, but the reason it is here is that every time we save a new control shape we would like the UI to be rebuild, in order for the menu containing all of our shapes to be up to date.
Copying and pasting shapes
ctlShapeClipboard = manager.getShape(mc.ls(sl=1, fl=1))
for ctlShape in ctlShapeClipboard:
sel = mc.ls(sl=1, fl=1)
for each in sel:
As we saw previously, it is really easy to copy and paste shapes with the manager alone, but to provide a quick and easy interface these two functions seem to do a good job. Essentially, we are creating a global variable and store the selected shape dictionary inside of it. Again we pop the “colour” key, as we just want to copy the shape. Then we just use the setShape() function on all selected controls with that global variable.
Then there are a few functions for flipping the shapes. It’s a bit of a pain to have to do that manually, but it is really easy to scale the points by -1 through script so let’s have a look at the _flipCtlShape() function. You will notice that there are a few more functions for flipping – flipCtlShape(), flipCtlShapeX(), flipCtlShapeY() and flipCtlShapeZ(). They all just make a call to the _flipCtlShape() one, but with different arguments, so we will just look at that one.
def _flipCtlShape(crv=None, axis=[-1, -1, -1]):
shapes = manager.getShape(crv)
newShapes = 
for shape in shapes:
for i, each in enumerate(shape["points"]):
shape["points"][i] = [each * axis, each * axis, each * axis]
All we do in this one, is just go through each CV and scale it’s x, y and z coordinates by -1 in order to flip the shape. The above mentioned other functions just call this one with the axis set to [-1, 1, 1] for x, [1,-1,1] for y, etc.
I skipped the mirrorCtlShapes() function earlier, because I wanted to already have the flip one in place as we are going to be using it again.
sel = mc.ls(sl=1, fl=1)
for ctl in sel:
if ctl not in ["L", "R"]:
search = "R_"
replace = "L_"
if ctl == "L":
search = "L_"
replace = "R_"
shapes = manager.getShape(ctl)
for shape in shapes:
manager.setShape(ctl.replace(search, replace), shapes)
The bulk of the code here is really for defining the search and replace strings. Since the naming convention that I use is SIDE_NAME_NODETYPE my search and replace strings vary between “L_” and “R_”. Have a look at your convention and modify these strings to make it work. Once they are defined, all we do is copy the shape from the current side to the other one and once done, flip it in all axis. In my pipeline, I have made it so this function does not work with a selection, but instead goes through all my left controls and mirrors them to the right. It is just because I always work from left to right, so I do not need this functionality.
Lastly, there is a simple function to rebuild the UI. All it does is import the package, as the way I have set it up is that importing just builds the UI which in turn makes the references to all the needed functions. The UI example that I give is very primitive, but obviously you can replace this code with one that will work with your own UI. Keep in mind that it is best to use the mc.evalDeferred() command as otherwise, the rebuild might error as it is being called from the UI that needs to be rebuilt.
Part 3: Simple UI
Now that we have all functions that we need we can build an UI to interact with them. Since everybody has a different pipeline for rigging at place, I am hesitant to suggest any specific way of handling that UI. One might prefer it in a window, other a tool menu or others yet a shelf button like I do. So I have added a very simple shelf button build in the managerUI.py to demonstrate how would we go about it. Additionally, remember how when generating the lists for the available colours we had a third item in the tuple for an image? You can get these here. They are just images of solid colour, corresponding to the index of the overrideColor attribute.
For a more comprehensive intro to building shelves with buttons and popups have a look at my Building custom maya shelvespost.
Let’s have a look at it then.
import maya.cmds as mc
# Local import
SHELF_NAME = "Custom"
ICON_PATH = "C:/PATH_TO_ICONS"
if SHELF_NAME and mc.shelfLayout(SHELF_NAME, ex=1):
children = mc.shelfLayout(SHELF_NAME, q=1, ca=1) or 
for each in children:
label = mc.shelfButton(each, q=1, l=1)
if label == "ctlShapeManager":
mc.shelfButton(l="ctlShapeManager", i="commandButton.png", width=37, height=37, iol="CTL")
popup = mc.popupMenu(b=1)
mc.menuItem(p=popup, l="Save to library", c=functions.saveCtlShapeToLib)
sub = mc.menuItem(p=popup, l="Assign from library", subMenu=1)
for each in functions.getAvailableControlShapes():
mc.menuItem(p=sub, l=each, c=each)
mc.menuItem(p=popup, l="Copy", c=functions.copyCtlShape)
mc.menuItem(p=popup, l="Paste", c=functions.pasteCtlShape)
sub = mc.menuItem(p=popup, l="Set colour", subMenu=1)
for each in functions.getAvailableColours():
mc.menuItem(p=sub, l=each, c=each, i=ICON_PATH + each)
mc.menuItem(p=popup, l="Flip", c=functions.flipCtlShape)
mc.menuItem(p=popup, l="Mirror", c=functions.mirrorCtlShapes)
So what happens here is we import the functions.py file which in turn imports the manager.py which then imports the utils.py file. Then there are the two variables – SHELF_NAME and ICON_PATH for declaring the shelf to add the button to and the path to the icons. Then we check if a button with the same name already exists in the shelf and if it does we delete it so we can replace it with our new one.
From then on we have simple maya UI commands to build our buttons and menus. If you are not familiar with UIs in maya it is worth having a look at the docs. Essentially, all we do is create a single mc.shelfButton() and we attach a mc.popupMenu() to it. Which then we populate with mc.menuItem()s where the l flag stands for label and the c for command. So there we pass our functions commands. Notice that we are not adding the () after the function name as that would call it and return the output. Instead we want to pass a reference to that function.
Then for the shapes and colours menuItems we add the subMenu flag so they become deeper level menus and we populate them with the results of our functions.getAvailableControlShapes() and functions.getAvailableColours() commands, which results in lists containing the shapes in our library and all 32 available colours.
And that is it. We have built our own control shape manager. With some easy extensions you can improve it to have almost like a version control system for your rigs, so you do not ever have to worry about your control shapes anymore.
I am not sure what it is, but there is something incredibly appealing in optimizing our workflows. I think a lot of it comes from the frustration of repeating the same actions over and over again. When you find a way to optimize that, it feels great. One of the easiest way to improve our rigging workflow is to script a custom marking menu with python. Another one, I have already written about is creating a custom shelf.
tl;dr: I will walk you through scripting your own custom marking menu with python, which is going to be easily shareable, extendable and maintainable. The code can be found here.
Here is how my main marking menu looks.
I have found it is an immense help to have the commands I use most oftenly either in my marking menu or my shelf. It just saves so much time!
Okay, how do we go about creating one of these?
Well, we have two options – either build it with Maya’s native marking menu editor or script it with python.
The reasons I prefer scripting my marking menus in python are a few.
– The native editor does not give us all the available options for a marking menu, such as submenus.
– Updating from the editor is a pain in the butt.
– The editor does not scale nicely, if you want or need to support multiple marking menus.
– Doing it through the editor is boring.
Obviously, python fixes all these issues for us. Additionally, it is easy to share it with co-workers and keep it in a version control system. Okay, so you are sold now. Let us have a look at how to do it then.
The code that I will be going through is on this gist, but I will go through all of it, if you would rather write it yourself.
Custom marking menu with Python
import maya.cmds as mc
MENU_NAME = "markingMenu"
We start very simple with the import of maya.cmds and giving a name to our custom marking menu. Now, the name is not very important because we do not ever see it, but maya does. So, in order for us to be able to update our custom marking menu, we need to be able to access it, and that is why we are giving it a name.
Then we have our markingMenuclass. The main reason I went for a class is because we can encapsulate everything we need into it quite logically. I know that some people would prefer to have a function instead, which is absolutely fine, it is really a personal preference. Let’s have a look at the constructor.
As you can see, our constructor is very simple. We delete the old version of our custom marking menu if it exists and then we build are our new one in place.
Remove old marking menu
if mc.popupMenu(MENU_NAME, ex=1):
As I said, the reason we need to give a name to our marking menu is to be able to modify it after it is created. In our case, we are not really modifying it, but deleting it instead. We do this, so we have a clean slate for building our new marking menu.
The reason I have added this deleting and then building again functionality is just so we can painlessly make changes to our custom marking menu. For example, if I want to add a new button or change a label, I do not want to have to restart maya or do anything other than just running my code again. Or even easier, just importing my code again. That is why I said earlier, that this setup for a custom marking menu is very easily extendable and maintainable. Additionally, we can add a button to our marking menu which rebuilds it, so we only have to make our changes to the file, save it and then rebuild from within. We will have a look at that in the end of the post.
Building our custom marking menu
Now that we have had a look at preparing for our build let us have a look at the _build() method.
menu = mc.popupMenu(MENU_NAME, mm = 1, b = 2, aob = 1, ctl = 1, alt=1, sh=0, p = "viewPanes", pmo=1, pmc = self._buildMarkingMenu)
Another very simple method. I was surprised that maya does not have a specific markingMenu method. Instead, the popupMenu() command is used, which is actually nice, since it is a familiar one if you have worked with menus or shelf popups. Let us look at the arguments.
MENU_NAME – quite obviously this one sets the name of our custom marking menu.
b – this is the mouse button we would like to trigger the marking menu. 1 – left, 2 – middle, 3 – right.
aob – allows option boxes.
ctl – defines the Ctrl button as a needed modifier to trigger the marking menu.
alt – defines the Alt button as a needed modifier to trigger the marking menu.
sh – defines the Shift button as a needed modifier to trigger the marking menu.
p – the parent ui element. For marking menus, that would usually be “viewPanes”, which refers to all of our view panels.
pmo – this flag declares that the command that we pass to the pmc flag should be executed only once and not everytime we invoke the marking menu. If this is false, everytime we trigger our custom marking menu, we will see our menus growing as more and more menuItems will be added.
pmc – this is the command that gets called right before the popupMenu is displayed. This is where we need to pass our method that actually builds all our buttons and menus – _buildMarkingMenu.
It is important to notice that when we pass the self._buildMarkingMenu we do not add brackets in the end as that would call the function instead of passing it as as reference.
Additionally, it is also important to think about the button flag and the modifiers. Obviously, Maya already has some marking menus and some functions related to mouse clicks and the ctrl, alt and shift modifiers. Therefore, we need to come up with a combination that does not destroy a function which we actually want to keep. That is why, for my marking menu I use the middle mouse button + alt + ctrl. There was some zooming function bound to that combination I believe, but I never used it so it was safe to override.
Actually building our custom marking menu
So, everything up to this point was to set us up for actually adding our commands to our menu. To be honest, it is as simple as everything we have already seen. Let us have a look at how we do this.
def _buildMarkingMenu(self, menu, parent):
## Radial positioned
mc.menuItem(p=menu, l="South West Button", rp="SW", c="print 'SouthWest'")
mc.menuItem(p=menu, l="South East Button", rp="SE", c=exampleFunction)
mc.menuItem(p=menu, l="North East Button", rp="NE", c="mc.circle()")
subMenu = mc.menuItem(p=menu, l="North Sub Menu", rp="N", subMenu=1)
mc.menuItem(p=subMenu, l="North Sub Menu Item 1")
mc.menuItem(p=subMenu, l="North Sub Menu Item 2")
mc.menuItem(p=menu, l="South", rp="S", c="print 'South'")
mc.menuItem(p=menu, ob=1, c="print 'South with Options'")
mc.menuItem(p=menu, l="First menu item")
mc.menuItem(p=menu, l="Second menu item")
mc.menuItem(p=menu, l="Third menu item")
mc.menuItem(p=menu, l="Create poly cube", c="mc.polyCube()")
The first thing to note is that our method receives menu and parent as arguments. These are passed automatically from the pmc flag on the popupMenu function in the _build() method.
Then we have the actual items in our custom marking menu. I like to split them logically in – radial and list blocks.
The radial positions are defined by the directions on a map – East, West, NorthEast, etc. Have a look at the following image.
You can have either commands in these slots or additional subMenus like so.
My personal preference here is to have just single commands instead of additional popups, as to invoke a submenu you need to hover on a position and wait a little bit. And I find that small delay quite frustrating.
So the way we create these radial items is by using the menuItem command. All we have to do is pass our menu argument as the parent (p), define a label (l), a radial position (rp) and the command (c) we want to execute on click.
Notice that if passing functions as commands we pass them without brackets, as that would call them instead. Have a look at the SE radial position for an example. Additionally, the functions need to be able to receive arguments as maya’s ui elements tend to pass some info about themselves to the commands they call. The way we do that is by just adding the *args argument.
Additionally, we can call maya commands from our items. Have a look at the NE item. It is important to note here that even though we have imported maya.cmds as mc in this file, the commands we pass to our menuItems are going to be called from maya’s python environment. That means, that in order for mc.circle() to work, we need to have imported maya.cmds as mc somewhere in our maya session. You could either run it yourself in the script editor or a better solution would be to add it to your userSetup.py file. That is what I did, since I use a lot of python in maya I just do an import maya.cmds as mc in my userSetup.py.
As I said we can have subMenu items, which essentially create another menu when you hover on them. The only thing we do is set the subMenu flag to 1. Have a look at one of these in our N radial position example. We store the menuItem with the subMenu flag in a variable, so we can use it as a parent for our following items. From then on, we just follow the same principle, we list menuItems, with the subMenu variable that we stored as a parent. Additionally, we can have deeper subMenus as well, though I do not think that would be great to work with.
Another very useful addition to our menuItems is adding option boxes. A lot of maya’s menus have those and they are a nice way to add additionall functionality without sacrificing space.
Usually, you would add them to commands that sometimes need their options to be changed, but also you can have different commands bound to them. For example, in my main marking menu I have the joint tool in one of my radial positions. But since I never mess with the options for that tool, in the option box I have a command which creates a joint under every object I have selected. Additionally, it names it with the name of the selected object and assumes it’s transformations.
The way we add an option box is very simple. All we need to do is create another menuItem after the one we want to add the option box to and we set the ob flag to 1. Have a look at the S radial position for an example.
In addition to our radial positioned items, marking menus have another menu beneath the South radial position. What is cool about this one is that it is an actual menu instead of just a single radial position, so we can have multiple items in it.
The way we create those items is by just specifying the marking menu as a parent. Remember it is passed as the menu argument to our method. So, if we do not specify a radial position the menuItem gets added to that menu which is south of the South position.
Again, as with all other menus, we can have submenus in there, by just setting the submenu flag to 1. My personal preference though is to have as few of these as possible, as I want everything to be easy to grab at first glance.
I have not added any icons in this example marking menu, but these can easily be added to every menuItem by using the i flag. All you need to do is place your custom icons somewhere in the XBMLANGPATH environment variable. To see what that is on your machine run this MEL command getenv XBMLANGPATH.
Additionally, you can use maya’s native icons, which you can browse through in the shelf editor. If you open that, pick any button on the right and next to the Icon Name field you can find a Browse Maya Icons button.
For example if I want to add maya’s standard icon to the S radial position I would do this.
And of course you can also pass full paths to icons which are not on maya’s XBMLANGPATH path.
Rebuilding our custom marking menu and loading it on startup
So, now that we know how to build our marking menus, let us have a look at how to actually initialize them with maya and how to rebuild them on the fly when we make changes. For building our marking menu we just initialize our markingMenu() class. For rebuilding we need to do the same thing, but we have several options for how we call our class.
Option 1: Just run it
I would say this is the simplest solution. If you are working in an external editor you can easily copy and paste your modified code in maya’s script editor, run it and you’re done. Since, we have added the rebuilding functionality – delete old and then build new one – the marking menu is updated everytime we run our code and then call the markingMenu() class in the end.
Even better if you have connected your external editor to maya via a commandPort command you can just run the code from there and that’ll update your marking menu as well. (I use this plugin for Sublime to do that.
The downside with this option is that everytime we open maya we will need to run our code. Which means that at some point you will get annoyed with doing it.
Option 2: Add it to your scripts path
Even though the first solution is quite simple this one is a bit nicer to work with. It also takes care of loading our marking menu on startup.
So, what we do here is we add the file containing our code – markingMenu.py – somewhere on our MAYA_SCRIPT_PATH. Again, you can run getenv MAYA_SCRIPT_PATH in MEL to get this path.
What this means is that now we can access our marking menu from within maya. Therefore we have a lot of options on how to build/rebuild our marking menu.
For building it on startup we just need to add the following code inside our userSetup.py.
When rebuilding though we need to do a reload statement. That is because when importing a module, python checks whether it is already imported and if so, just uses that instead. So if we make changes and then do import, our changes are not going to be imported. That is why we need to do reload(ourModule).
So in the case of reloading our marking menu, we will need to do the following.
This will ensure that whatever changes we have made to our code will be implemented.
Now that we have this rebuilding code, we can add it wherever we find most appropriate. Generally, I would say either a button on a shelf or from within the marking menu itself. Since a button on a shelf is trivial let us look at adding it to the marking menu.
Reloading from within our custom marking menu
The way we would do this, is just add the following menuItem.
I really don’t like using evalDeferred since it feels a bit dirty, but in this case we need it. The reason we need it is that we are rebuilding our marking menu from within. So if we do not use evalDeferred but directly call our markingMenu(), maya will have to delete the button which we have clicked, and that will error.
So, there we have it, we have built our own custom marking menu with Python. What is more, it is fully scripted, so making changes is very easy. Additionally, we can have that file in a version control system, so we can have a log of our changes. And of course, it is super easy to share with co-workers. The best thing about it, though, is how much time it saves in our day-to-day rigging tasks.
So, a few months back I was looking into clean ways of scripting a custom shelf in Maya, that can be easily shared, and I was surprised that there were not many resources on how to go about it. Therefore, I thought I would share the way that I am maintaining my shelf.
tl;drGrab the code for the shelf base class from here and derive your own shelf class from it, overwriting and populating the build() method with calls to the addButton, addMenuItem, etc. calls to create your own custom maya shelf.
Scripting your own custom shelf in Maya is much easier than you would think, or at least than I thought. For some odd reason, I always assumed that it would involve a lot of messing about with MEL, which I would really rather not. If that is what you thought as well, you will be pleasantly surprised.
Here is what we are trying to achieve.
Since, we want to keep the code as modular and versatile as possible, I wrote the main functionality in a base class which needs to be extended for each shelf you want to build. The code for the base class with a little example is on github.
Now we will go through it to make see how it works, so it can be extended and modified to include more functionalities.
In the constructor we initialize the variables we make two calls to _cleanOldShelf() and build(), which we will look at in a bit. The name argument is going to be the name of the shelf. It is important to note that if a shelf with this name already exists it will be replaced with this one. The iconPath argument can be used to define a directory from which to get images to be used as icons of the shelf buttons and commands in the popup menus. If it is not defined, you can still use maya’s default icons like commandButton.png for example.
Additionally there are the labelBackground and labelColour variables which can be seen in the following image.
The reason I have hardcoded them in the base class is because I think for the sake of consistency all our shelves should have the same style, but if you want to change them, obviously feel free to do it.
And lastly, there is that mc.setParent(self.name) function, which makes sure that whatever ui elements we are going to build in the following build() method, they will be build as children to our shelf. Otherwise, we might end up with buttons breaking the other layouts.
Clean old shelf
Let’s have a look at what the _cleanOldShelf() method does.
if mc.shelfLayout(self.name, ex=1):
if mc.shelfLayout(self.name, q=1, ca=1):
for each in mc.shelfLayout(self.name, q=1, ca=1):
Essentially, we are checking if our shelf already exists by using the mc.shelfLayout(self.name, ex=1) command where self.name is the name of our shelf and ex is the flag for checking existence. If it does we go through all children of the shelf which we get from mc.shelfLayout(self.name, q=1, ca=1) and delete them. That makes sure we start our build() method with a fresh clean shelf.
If the shelf does not exist though, we simply create it, passing "ShelfLayout" as parent, because that is the parent layout of all the shelves in maya. For example, turn on History > Echo all commands in the the script editor and click through some of the shelves. You will see the paths to some of their popupMenus as so Shelf|MainShelfLayout|formLayout16|ShelfLayout|CurvesSurfaces||popupMenu546
So, knowing that CurvesSurfaces is a name of a shelf, we can deduce that the ShelfLayout is the parent of all shelves.
Then, before we get into the build() method, let’s have a look at some of the methods we are going to use to actually populate our shelf.
Very simply, this is the method that creates the main buttons in the shelf. Essentially, we just need to pass it a label which is the name of our button, an optional icon and an optional command and doubleCommand which refer to single click and double click. The command is optional, because we might want buttons that just have popup menus. We will look at those in a sec. By default the _null() command, which is at the top of the file is called. The reason I have that command is that if you pass None to the mc.shelfButton command flag, it will error. So the _null() method essentially does nothing. It is important to note that when we pass a command we are not using the brackets after the name (), because that would call the command instead of passing it.
This method is very similar to the add button one, but instead of adding a button to the shelf, this one adds a menuItem to an existing popupMenu passed as the parent argument. The popupMenu needs to be created manually, as it doesn’t make sense to wrap it in a method if it is just a simple one line command anyway. So, a quick example of how that works is
p = mc.popupMenu(b=1)
where the b=1 stands for left mouse click. This snippet if added inside the build() method will attach a popupMenu with one menuItem to the popup button.
Now, I realize that sub menus inside a popup on a shelf button is getting a bit too deep, as the whole point of a shelf is to streamline everything, but I thought I would add it, as even I have used it on a couple of my buttons.
What this method does is, it creates a menuItem which is a menu in itself, attached to an existing popupMenu which is passed as the parent.
The result is something similar to this.
You know how some of the shelves have separating lines between the buttons. You can achieve that with the mc.separator() command. I usually set the style flag to "none" as I rather having blank space than having lines, but have a look at the docs page for the separator command for more info.
And then finally let us have a look at the build() method.
'''This method should be overwritten in derived classes to actually build the shelf
elements. Otherwise, nothing is added to the shelf.'''
As the docstring says, this is an empty method that needs to be defined in each shelf we build. The way this works is, we just populate it with addButton(), mc.popup(), addMenuItem() and addSubMenu() calls as we need them.
Example of a custom shelf in maya
Let’s have a look at the simple example in the end of the file.
As you can see, all we do is we create a new class inheriting from the _shelf class. By the way the _ prefix is used to remind us that it is a class that should not be directly accessed.
Then we overwrite the build() method by simply defining it, and we populate it with the necessary shelf elements.
This example results in the image from above. Here is it again.
I have not demonstrated the use of any commands on these buttons, but all it takes is just passing the name of the function as a command argument. The one tricky bit about it is that the menuItem commands get passed some arguments, so we need to have *args in our function calls to prevent errors. For example
We would include this command in our shelf by doing self.addButton(label="cube", command=createPolyCube) in our build() method.
For the sake of clarity though, I recommend having a separate file which contains only the functions that are going to be used in the shelf and import them from there. So at the top of the file where are custom shelf is defined, import your shelf functions file with import shelfFunctions or however you decide to package them. I would advise having a reload(shelfFunctions) statement immediately after that, so you can be sure the latest changes to the file have been taken into account.
Additionally, you can also pass maya native commands as well. So to have a button that creates a poly cube we would do
Building our shelf on load up
The way we will include our shelf on load up is to just add it to our userSetup.py file. If you are not familiar with it, take a look at this. I know this one is for MEL, but it is essentially the same thing for Python as well.
So, assuming that our shelf file is in the maya scripts path, in our userSetup.py we can just do
import maya.cmds as mc
where shelf is the file containing our shelf and customShelf is the name of the class.
We are using the evalDeferred() command, because maya loads the userSetup.py file as it is starting, so we are not sure if the shelfLayout is already in place as it gets executed. Therefore, evalDeferred() helps us call our script after maya has been initialized.
And that is it, you have built your own shelf. What I really like about it is that if you are collaborating with other people you can add the shelf file to your version control system and in your userSetup.py file just source it from there. That way you can all contribute to it and have it always updated.