During the week, I got a comment on the first post in my Maya matrix nodes series – the matrix constraint one – about using the wtAddMatrix node to achieve the multiple targets with blending weights functionality similar to constraints. I have stumbled upon the wtAddMatrix node, but I think it is the fact that Autodesk have made it very fiddly to work with it – we need to show all attributes in the node editor and we have 0 access to setting the weight plug – that put me off ever using it. That being said, when RigVader commented on that post I decided I will give it a go. Since it actually works quite nicely, today I am looking at blending matrices in Maya.

Disclaimer: I will be using the matrix constraint setup outlined in the post I mentioned, so it might be worth having a look at that one if you have missed it.

tl;dr: Using the wtAddMatrix we can blend between matrices before we plug the output into a matrix constraint setup to achieve having multiple targets with different weights.

Turns out, the wtAddMatrix is a really handy node. It gives us the chance to plug a number of matrices in the .matrixIn plugs of the .wtMatrix array attribute, and give them weight in the .weightIn plug. That, effectively lets us blend between them.

Blending matrices for a matrix constraint setup

So, now that we know we can blend matrices, we just need to figure out exactly what do we need to blend.

Let us first have a look at the simpler case – not maintaining the offset.

Blending matrices - matrix constraint with no offset

The group1 on the graph is the parent of pCube1 and is used just so we convert the world matrix into a relative to the parent matrix, without using the parentInverseMatrix. The reason for that is we do not want to create benign cycles, which Raff sometimes talks about on the Cult of Rig streams. Other than that, everything seems to be pretty straightforward.

Bear in mind, the wtAddMatrix node does not normalize the weights, which means that we could have all of the targets fully influence our object. What is more, you could also push them beyond 1 or negate them, which would result in seemingly odd results, but that might just be what you need in some cases.

Maintaining the offset

Often we need to maintain the offset in order to achieve the desired behaviour, so the way we do that is we resort to the multMatrix node once more. I am not going in detail, as there are already a couple of ways you can do that outlined in the previous post, but let us see how it fits in our graph.

Blending matrices - matrix constraint maintaing offset

The two additional multMatrix nodes let us multiply the local offset for the current target by the world matrix of the current target, effectively constraining the object but also maintaining the initial offset.

Now, however clean and simple it may be, the graph gets to be a bit long. What this means is, it is probably getting a bit slower to evaluate as well. That is why, I thought I would do a bit of a performance test to see if there still is any benefit to using this setup over a parentConstraint.

Performance

The way I usually do my tests is either loop a few hundred times in the scene and build the setup or build it once, save it in a file and then import the file a few hundred times and let it run with some dummy animation. Then I use Maya 2017’s Evaluation toolkit to Run a performance test, which gives us info about the performance in the different evaluation methods – DG, Serial and Parallel. Since, the results vary quite a bit, what I usually do is, run it three times and take the best ones.

In this case, I built the two setups in separate files, both with 2 target objects and maintain offset. Then I ran the tests on 200 hundred imported setups.

So here are the results.

Parent constraint
Playback Speeds
===============
    DG  = 89.8204 fps
    EMS = 20.1613 fps
    EMP = 59.2885 fps
Matrix constraint
Playback Speeds
===============
    DG  = 91.4634 fps
    EMS = 24.6305 fps
    EMP = 67.2646 fps

Bear in mind these tests are done on my 5 years old laptop, so the results you are going to get if you are to repeat this test are going to be significantly better.

As you can see even with the extended graph we are still getting about 7.5 fps increase by using the matrix constraint setup with blending matrices. Considering, we have 200 hundred instances in the scene (which is by no means a large number), that means we have about .0375 fps increase per setup, which in turn means that on every 26 setups we win a frame.

Conclusion

So, there we have it, an even larger part of the parentConstraint functionality, can be implemented by just using matrix nodes. What this means is we can keep our outliner cleaner and get a better performance out of our rigs at the same time, which is a total win win.

Thanks to RigVader for pointing the wtAddMatrix node as a potential solution, it really works quite nicely!

This post is a part of a three post series, where I implement popular rigging functionalities with just using maya’s native matrix nodes.

Rivets are one of those things that blew my mind the first time I learned of them. Honestly, at the time, the ability to stick an object to the deforming components of a geometry seemed almost magical. Although, the more you learn about how geometries work in Maya, the more sense rivets start to make. The stigma around them, though, has always been that they are a bit slow, since they have to wait for the underlying geometry to evaluate and only then can they evaluate as well. And even though that is still the case, it seems that since parallel was introduced the performance has increased significantly.

It is worth trying to simplify and clean rivets up, considering how handy they are for rigging setups like:
– twist distributing ribbons
– bendy/curvy limbs
– sticking objects to geometries after squash and stretch
– sticking controls to geometries
– driving joints sliding on surfaces

and others.

When I refer to the classic rivet or the aimConstraint rivet, it is this one that I am talking about. I have seen it used by many riggers and also lots of lighters as well.

The purpose of this approach is to get rid of the aimConstraint that is driving the rotation of the rivet. Additionally, I have seen a pointConstraint used as well, in order to account for the parent inverse matrix, which would also be replaced by this setup. Even though we are stripping constraints, the performance increase is not very large, so the major benefit of the matrix rivet is a cleaner graph.

TL;DR: We are going to plug the information from a pointOnSurfaceInfo node directly into a fourByFourMatrix node, in attempt to remove constraints from our rigs.

Disclaimer: Bear in mind, I will be only looking at riveting an object to a NURBS surface. Riveting to poly geo would need to be done through the same old loft setup.

Limitations: Since we are extracting our final transform values using a decomposeMatrix node, we do not have the option to use any rotation order other than XYZ, as at the moment the decomposeMatrix node does not support other orders. A way around it, though, is taking the outputQuat attribute and pluging it into an quatToEuler node which actually supports different rotate orders.

Difference between follicle and aimConstraint rivet

Matrix rivet - follicle and classical rivet differences

Matrix rivet - follicle and classic rivet graph

The locator is riveted using an aimConstraint. You can see there is a small difference in the rotations of the follicle and the locator. Why is that?

The classical rivet setup connects the tangentV and normal attributes of a pointOnSurface to the aimConstraint. The third axis is then the cross product of these two. But it seems like the follicle is actually using the tangentU vector for it’s calculations, since we get this difference between the two setups.

Choosing to plug the tangentU into the aimConstraint, instead of tangentV, results in the same behaviour as a follicle. To be honest, I am not sure which one would be preferable. In the construction of our matrix rivet, though, we have full control over that.

Why not follicles?

As I already said, in parallel, follicles are fast! Honestly, for most of my riveting needs I wouldn’t mind using a follicle. The one aspect of follicles I really dislike though, is the fact that it operates through a shape node. I understand it was not meant for rigging, and having the objects clearly recognizable both in the outliner and the viewport is important, but in my case it is just adding up to clutter. Ideally, I like avoiding unnecessary DAG nodes, since they only get in the way.

Additionally, have you had a look at the follicle shape node? I mean, there are so many hair related attributes, it is a shame to use it just for parameterU and parameterV.

Therefore, if we could use a non-DAG network of simple nodes to do the same job without any added overhead, why should we clutter our rigs?

Constructing the matrix rivet

So, the way matrices work in Maya is that the first three rows of the matrix describe the X, Y and Z axis and the fourth row is the position. Since, this is an oversimplification I would strongly suggest having a look at some matrix math resources and definitely watching the Cult of Rig streams, if you would like to learn more about matrices.

What this means to us, though, is that if we have two vectors and a position we can always construct a matrix out of them, since the cross product of the two vectors will give us a third one. So here is how our matrix construction looks like in the graph.

Matrix rivet - constructing the matrix

So, as you can see, we are utilizing the fourByFourMatrix node to construct a matrix. Additionally, we use the vectorProduct node set to Cross Product to construct our third axis out of the normal and the chosen tangent, in this case tangentV which gives us the same result as using the classic aimConstraint rivet. If we choose to use the tangentU instead, we would get the follicle‘s behaviour. Then, obviously we decompose the matrix and plug it into our riveted transform.

Optionally, similar to the first post in this series, we can use the multMatrix node to inverse the parent’s transform, if we so need to. What I usually do, though, is parent them underneath a transform that has it’s inheritTransform attribute turned off, so we can plug the world transforms directly.

It is important to note that in this case we are absolutely sure that the output matrix is orthogonal, since we know that the normal is perpendicular to both tangents. Thus, crossing it with any of the tangents, will result in a third perpendicular vector.

Skipping the vector product

Initially, when I thought of building rivets like this, I plugged the normal, tangentU and tangentV directly from the pointOnSurfaceInfo to the fourByFourMatrix. What this means, is that we have a matrix that is not necessarily orthogonal, since the tangents might very well not be perpendicular. This results in a shearing matrix. That being said though, it was still giving me proper results.

Matrix rivet - skipping the vectorProduct

Then, I added it to my modular system to test it on a couple of characters and it kept giving me steadily good results – 1 to 1 with the behaviour of a follicle or aimConstraint rivet, depending on the order I plug the tangents in.

What this means, then, is that the decomposeMatrix node separates all the shearing from the matrix and thus returns the proper rotation as if the matrix is actually orthogonal.

If that is the case, then we can safely skip the vectorProduct and still have a working rivet, considering we completely disregard the outputShear attribute of the decomposeMatrix.

Since, I do not understand how that shearing is being extracted, though, I will be keeping an eye on the behaviour of the rivets in my rigs, to see if there is anything dodgy about it. So far, it has proved to be as stable as anything else.

Conclusion

If you are anything like me, you would really like the simplicity of the graph, as we literally are taking care of the full matrix construction ourselves. What is more, there are no constraints, nor follicle shapes in the outliner, which again, I find much nicer to look at.

This matrix series has been loads of fun for me to write, so I will definitely be trying to come up with other interesting functions we could use matrices for.

This post is a part of a three post series, where I implement popular rigging functionalities with just using maya’s native matrix nodes.

Calculating twist is a popular rigging necessity, as often we would rather smoothly interpolate it along a joint chain, instead of just applying it at the end of it. The classical example is limbs, where we need some twist in the forearm/shin area to support the rotation of the wrist or foot. Some popular implementations utilize ik handles or aim constraints, but I find them as a bit of an overkill for the task. Therefore, today we will have a look at creating a matrix twist calculator, that is both clean and quick to evaluate.

Other than matrix nodes I will be using a couple of quaternion ones, but I promise it will be quite simple, as even I myself am not really used to working with them.

tl;dr: We will get the matrix offset between two objects – relative matrix, then extract the quaternion of that matrix and get only the X and W components, which when converted to an euler angle, will result in the twist between the two matrices along the desired axis.

Desired behaviour

Matrix twist calculator - desired behaviour
Please excuse the skinning, I have just done a geodesic voxel bind

As you can see what we are doing is calculating the twist amount (often called roll as well from the yaw, pitch and roll notation) between two objects. That is, the rotation difference on the axis aiming down the joint chain.

Limitations

An undesirable effect you can notice is the flip when the angle reaches 180 degrees. Now, as far as I am aware, there is no reasonable solution to this problem, that does not involve some sort of caching of the previous rotation. I believe, that is what the No flip interpType on constraints does. There was one, using an orient constraint between a no roll joint and the rolling joint and then multiplying the resulting angle by 2, which worked in simple cases, but I found it a bit unintuitive and not always predictable. Additionally, most animators are familiar with the issue, and are reasonable about it. In the rare cases, where this issue will be a pain in your production you can always add control over the twisting matrices, so the animators can tweak them.

Something else to keep in mind is to always use the first axis of the rotate order to calculate the twist in, since the other ones might flip at 90 degrees instead of 180. That is why, I will be looking at calculating the X twist, as the default rotate order is XYZ.

With that out of the way, let us have a look at the setup.

Matrix twist calculator

I will be looking at the simple case of extracting the twist between two cubes oriented in the same way. Now, you might think that is too simple of an example, but in fact this is exactly what I do in my rigs. I create two locators, which are oriented with the X axis being aligned with the axis I am interested in. Then I parent them to the two objects I want to find the twist between, respectively. This, means that finding the twist on that axis of the locators, will give me the twist between the two objects.

Matrix twist calculator

Granted, I do not use actual locators or cubes, but just create matrices to represent them, so I keep my outliner cleaner. But, that is not important at the moment.

The relative matrix

Now, since we are going to be comparing two matrices to get the twist angle between them, we need to start by getting one of them in the relative space of the other one. If you have had a look at my Node based matrix constraint post or you were already familiar with matrices, you would know that we can do that with a simple multiplication of the child matrix by the inverse of the parent matrix. That will give us the matrix of the child object relative to that of the parent one.

The reason, we need that is because that relative matrix is now holding all the differences in the transformations between the two objects, and we are interested in exactly that, the difference on the aim axis.

Here is how that would look in the graph.

Matrix twist calculator - relative matrix

The quaternion

So, if we have the relative matrix, we can proceed to extracting the rotation out of it. The thing with rotations in 3D space is that they seem a bit messy, mainly because we usually think of them in terms of Euler angles, as that is what maya gives us in the .rotation attributes of transforms. There is a thing called a quaternion, though, which also represents a rotation in 3D space, and dare I say it, is much nicer to work with. Nicer, mainly because we do not care about rotate order, when working with quaternions, since they represent just a single rotation. What this gives us is a reliable representation of an angle along just one axis.

In practical terms, this means, that taking the X and W components of the quaternion, and zeroing out the Y and Z ones, will give us the desired rotation only in the X axis.

In maya terms, we will make use of the decomposeMatrix to get the quaternion out of a matrix and then use the quatToEuler node to convert that quaternion to an euler rotation, which will hold the twist between the matrices.

Here is the full graph, where the .outputRotateX of the quatToEuler node is the actual twist value.

Matrix twist calculator - full graph

Conclusion

And that is it! As you can see, it is a stupidly simple procedure, but has proved to be giving stable results, which in fact are 100% the same as using an ik handle or an aim constraint, but with little to no overhead, since matrix and quaternion nodes are very computationally efficient.

Stay tuned for part 3 from this matrix series, where I will look at creating a rivet by using just matrix nodes.

This post is a part of a three post series, where I will try to implement popular rigging functionalities by only using maya’s native matrix nodes.

Following the Cult of rig lately, I realized I have been very wasteful in my rigs in terms of constraints. I have always known that they are slower than direct connections and parenting, but then I thought that is the only way to do broken hierarchy rigs. Even though I did matrix math at university, I never used it in maya as I weirdly thought the matrix nodes are broken or limited. There was always the option of writing my own nodes, but since I would like to make it as easy for people to use my rigs, I would rather keep everything in vanilla maya.

Therefore, when Raffaele used the matrixMult and decomposeMatrix nodes to reparent a transform, I was very pleasantly inspired. Since then, I have tried applying the concept to a couple of other rigging functionalities, such as the twist calculation and rivets and it has been giving me steadily good results. So, in this post we will have a look at how we can use the technique he showed in the stream, to simulate a parent + scale constraint, without the performance overhead of constraints, effectively creating a node based matrix constraint.

Limitations

There are some limitations with using this approach, though. Some of them are not complex to go around, but the issue is that this adds extra nodes to the graph, which in turn leads to performance overhead and clutter. That being said, constraints add up to the outliner clutter, so I suppose it might be a matter of a preference.

Joints

Constraining a joint with jointOrient values, will not work, as the jointOrient matrix is applied before the rotation. There is a way to get around this, but it involves creating a number of other nodes, which add some overhead and for me are making it unreasonable to use the setup instead of an orient constraint.

If you want to see how we go around the jointOrient issue just out of curiosity, have a look at the joint orient section.

Weights and multiple targets

Weights and multiple targets are also not entirely suitable for this approach. Again, it is definitely not impossible, since we can always blend the output values of the matrix decomposition, but that will also involve an additional blendColors node for each of the transform attributes we need – translate, rotate and scale. And similarly to the previous one, that means extra overhead and more node graph clutter. If there was an easy way to blend matrices with maya’s native nodes, that would be great.

Rotate order

Weirdly, even though the decompose matrix has a rotateOrder attribute, it does not seem to do anything, so this method will work with only the xyz rotate order. Last week I received an email from the maya_he3d mailing list, about that issue and it seems like it has been flagged to Autodesk for fixing, which is great.

Construction

The construction of such a node based matrix constraint is fairly simple both in terms of nodes and the math. We will be constructing the graph as shown in the Cult of Rig stream, so feel free to have a look at it for a more visual approach. The only addition I will make to it is supporting a maintainOffset functionality. Also, Raffaele talks a lot about math in his other videos as well, so have a look at them, too.

Node based matrix constraint

All the math is happening inside the matrixMult node. Essentially, we are taking the worldMatrix of a target object and we are converting it to relative space by multiplying by the parentInverseMatrix of the constrained object. The decomposeMatrix after that is there to break the matrix into attributes which we could actually connect to a transform – translate, rotate, scale and shear. It would be great if we could directly connect to an input matrix attribute, but that would probably create it’s own set of problems.

That’s the basic node based matrix constraint. How about maintaining the offset, though?

Maintain offset

In order to be able to maintain the offset, we need to just calculate it first and then put it in the multMatrix node before the other two matrices.

Node based matrix constraint - maintain offset

Calculating offset

The way we calculate the local matrix offset is by multiplying the worldMatrix of the object by the worldInverseMatrix of the parent (object relative to). The result is the local matrix offset.

Using the multMatrix node

It is entirely possible to do this using another matrixMult node, and then doing a getAttr of the output and set it in the main matrixMult by doing a setAttr with the type flag set to "matrix". The local matrixMult is then free to be deleted. The reason we get and set the attribute, instead of connecting it, is that otherwise we create a cycle.

Node based matrix constraint - local matrix offset

Using the Maya API

What I prefer doing, though, is getting the local offset via the API, as it does not involve creating nodes and then deleting them, which is much nicer when you need to code it. Let’s have a look.

import maya.OpenMaya as om

def getDagPath(node=None):
    sel = om.MSelectionList()
    sel.add(node)
    d = om.MDagPath()
    sel.getDagPath(0, d)
    return d

def getLocalOffset(parent, child):
    parentWorldMatrix = getDagPath(parent).inclusiveMatrix()
    childWorldMatrix = getDagPath(child).inclusiveMatrix()

    return childWorldMatrix * parentWorldMatrix.inverse()

The getDagPath function is just there to give us a reference to an MDagPath instance of the passed object. Then, inside the getLocalOffset we get the inclusiveMatrix of the object, which is the full world matrix equivalent to the worldMatrix attribute. And in the end we return the local offset as an MMatrix instance.

Then, all we need to do is to set the multMatrix.matrixIn[0] attribute to our local offset matrix. The way we do that is by using the MMatrix‘s () operator which returns the element of the matrix specified by the row and column index. So, we can write it like this.

localOffset = getLocalOffset(parent, child)
mc.setAttr("multMatrix1.matrixIn[0]", [localOffset(i, j) for i in range(4) for j in range(4)], type="matrix")

Essentially, we are calculating the difference between the parent and child objects and we are applying it before the other two matrices in the multMatrix node in order to implement the maintainOffset functionality in our own node based matrix constraint.

Joint orient

Lastly, let us have a look at how we can go around the joint orientation issue I mentioned in the Limitations section.

What we need to do is account for the jointOrient attribute on joints. The difficulty comes from the fact that the jointOrient is a separate matrix that is applied after the rotation matrix. That means, that all we need to do is, in the end of our matrix chain rotate by the inverse of the jointOrient. I tried doing it a couple of times via matrices, but I could not get it to work. Then I resolved to write a node and test how I would do it from within. It is really simple, to do it via the API as all we need to do is use the rotateBy function of the MTransformationMatrix class, with the inverse of the jointOrient attribute taken as a MQuaternion.

Then, I thought that this should not be too hard to implement in vanilla maya too, since there are the quaternion nodes as well. And yeah there is, but honestly, I do not think that graph looks nice at all. Have a look.

Node based matrix constraint - joint orient

As you can see, what we do is, we create a quaternion from the joint orientation, then we invert it and apply it to the calculated output matrix of the multMatrix. The way we apply it is by doing a quaternion product. All we do after that is just convert it to euler and connect it to the rotation of the joint. Bear in mind, the quatToEuler node supports rotate orders, so it is quite useful.

Of course, you can still use the maintainOffset functionality with this method. As I said though, comparing this to just an orient constraint it seems like the orient constraint was performing faster every time, so I see no reason of doing this other than keeping the outliner cleaner.

Additionally, I am assuming that there is probably an easier way of doing this, but I could not find it. If you have something in mind, give me a shout.

Conclusion

Using this node based constrain I was able to remove parent, point and orient constraints from my body rig, making it perform much faster than before, and also the outliner is much nicer to look at. Stay tuned for parts 2 and 3 from this matrix series, where I will look at creating a twist calculator and a rivet by using just matrix nodes.

The classical rivet was a really popular rigging thing a few years ago (and long before that it seems). I am by no means a seasoned rigger, but whenever I would look for facial rigging techniques the rivet would keep coming up. What is more, barely if ever people suggested using follicle to achieve the result, generally because the classical rivet evaluates faster. So, I thought I’d do a maya performance test to compare them.

Table of contents

Prerequisites

I will be looking into the performance of a follicle and a classical rivet, both on a NURBS sphere and on a poly sphere. NURBS because I tend to use a lot of ribbons and poly, because it’s a popular feature for attaching objects to meshes.

I will be using Maya 2017’s Evaluation Toolkit to run the performance test, as it gives nice output for each evaluation method, even though I cannot imagine using anything but parallel.

The way the tests are going to work is, I will create two files, each containing the same geometry with 10 rivets. In one file I will use follicles and in the other the classical setup. The deformation on the geometry will just be keyed vertices and it will be identical for each setup, so we can be sure that the only difference between the two files is the riveting setup.

Then, the test will be done in a new scene where I will reference the file to test a 100 times. For each setup I will run the evaluation manager’s performance test and take the results and compare them.

Okay, let us have a look then.

NURBS

Classical rivet setup

So, the way this one works is I just loop from 1 to 10 and I create a pointOnSurfaceInfo node with parameterU set to iterator * .1 and parameterV set to .5. Then, I plug the output position directly to a locator’s translate attr. Additionally, the output position, normal vector and a tangent vector go into an aimConstraint which constraints the rotation of the locator.

Follicle setup

This one is fairly straightforward, I just created 10 follicles, with parameterU set to iterator * .1 and V to .5.

Results

Bear in mind, EMS refers to serial evaluation and EMP is parallel.

NURBS surface
Classical Rivet
Playback Speeds
===============
    DG  = 13.1694 fps
    EMS = 11.1359 fps
    EMP = 20.7469 fps
-----------------------------
Follicle
Playback Speeds
===============
    DG  = 11.3208 fps
    EMS = 12.6263 fps
    EMP = 27.8293 fps

Even though I expected the follicle to be faster I was surprised by how much. It is important to note that we have 10 * 100 = 1000 rivets in the scene, which is obviously a big number. Therefore, in a more realistic example the difference is going to be more negligible, but still 7.8fps is quite a bit.

What is also quite interesting is that in DG the follicle is slower than the classic rivet. So, the stigma of the old days that the classical rivet is faster, seem to be deserved, but parallel changes everything.

Polygons

Classical rivet setup

So, when it comes to polys the classical rivet gets a bit more complicated, which I would imagine results in a larger slowdown as well. The way this setup works is, we grab 10 couples of edges, which in turn produce 10 surfaces through a loft node. Maintaining history, the nurbs surfaces will follow the poly geometry. So, we can perform the same rivet setup as before on the nurbs.

Follicle setup

On a mesh with proper UVs the follicles are again trivial to set up. We just loop 10 times and create a follicle with the appropriate U and V parameters.

Results
Polygon geometry
Follicle
Playback Speeds
===============
    DG  = 1.7313 fps
    EMS = 3.32005 fps
    EMP = 9.79112 fps
-------------------------
Classical rivet
Playback Speeds
===============
    DG  = 1.05775 fps
    EMS = 1.52022 fps
    EMP = 3.31053 fps

As expected, follicles are again quite a bit faster. I am saying as expected, as not only do we have a riveting setup as in the NURBS case, but also there is the edges and the loft which add to the slowdown. I am assuming, that is why even in DG the classical rivet is still slower.

Conclusion

So, the conclusion is pretty clear – follicle rivets are much faster than classical rivets in the latest maya versions which include the parallel evaluation method.

So, it seems like I have been going crazy with marking menus lately. I am really trying to get the most of them, and that would not be much if we could only use them in the viewports, so today we are going to look at how we can construct custom marking menus in maya editors.

Custom marking menus in maya editors - node editor

tl;dr: We can query the current panel popup menu parent in maya with the findPanelPopupParent MEL function, and we can use it as a parent to our popupMenu.

So, there are a couple of scenarios that we need to have a look at, as they should be approached differently. Although, not completely necessary I would suggest you have a look at my previous marking menu posts – Custom marking menu with Python and Custom hotkey marking menu – as I will try to not repeat myself.

Okay, let us crack on. Here are the two different situations for custom marking menus in maya editors we are going to look at.

Modifiers + click trigger

In the viewport these are definitely the easier ones to set up as all we need to do is just create a popupMenu with the specified modifiers – sh, ctl and alt, the chosen button and viewPanes as the parent. When it comes to the different editors, though, it gets a bit trickier.

Let us take the node editor as an example.

If we are to create a marking menu in the node editor, it is a fairly simple process. We do exactly the same as before, but we pass "nodeEditorPanel1" as the parent argument. If you have a node editor opened when you run the popupMenu command, you will be able to use your marking menu in there. The catch is though, that once you close the node editor the marking menu is deleted, so it is not available the next time you open the node editor.

Unfortunately, I do not have a great solution to this problem. In fact, it is a terrible solution, but I wanted to get it out there, so someone can see it, be appalled and correct me.

The second method – Custom hotkey trigger – is much nicer to work with. So, you might want to skip to that one.

What I do is, I create a hotkey for a command that invokes the specific editor (I only have marking menus in the node editor and the viewport) and runs the marking menu code after that. So, for example, here is my node editor hotkey (Alt+Q) runTimeCommand.

mc.NodeEditorWindow()

if mc.popupMenu("vsNodeMarkingMenu", ex=1):
    mc.popupMenu("vsNodeMarkingMenu", dai=1, e=1)
else:
    mc.popupMenu("vsNodeMarkingMenu", p="nodeEditorPanel1Window", b=2, ctl=1, alt=1, mm=1)

mc.setParent("vsNodeMarkingMenu", m=1)

from vsRigging.markingMenus import vsNodeMarkingMenu
reload(vsNodeMarkingMenu)

That means that everytime I open the node editor with my hotkey I also create the marking menu in there, ready for me to use. As, I said, it is not a solution, but more of a workaround at this point. In my case, though, I never open the node editor through anything else than a hotkey, so it kind of works for me.

Then the vsRigging.markingMenus.vsNodeMarkingMenu file is as simple as listing the menuItems.

import maya.cmds as mc

mc.menuItem(l="multiplyDivide", c="mc.createNode('multiplyDivide')", rp="N", i="multiplyDivide.svg")
mc.menuItem(l="multDoubleLinear", c="mc.createNode('multDoubleLinear')", rp="NE", i="multDoubleLinear.svg")
mc.menuItem(l="plusMinusAverage", c="mc.createNode('plusMinusAverage')", rp="S", i="plusMinusAverage.svg")
mc.menuItem(l="condition", c="mc.createNode('condition')", rp="W", i="condition.svg")
mc.menuItem(l="blendColors", c="mc.createNode('blendColors')", rp="NW", i="blendColors.svg")
mc.menuItem(l="remapValue", c="mc.createNode('remapValue')", rp="SW", i="remapValue.svg")
...

A proper way of doing this would be to have a callback, so everytime the node editor gets built we can run our code. I have not found a way to do that though, other than ofcourse breaking apart maya’s internal code and overwriting it, which I wouldn’t go for.

Luckily, creating a custom marking menu bound to a custom hotkey actually works properly and is fairly easy. In fact, it is very similar to the Custom hotkey marking menu post. Let us have a look.

Custom hotkey trigger

Now, when we are working with custom hotkeys we actually run the initialization of the popupMenu everytime we press the hotkey. This means we have the ability to run code before we create the marking menu. Therefore, we can query the current panel and build our popupMenu according to it. Here is an example runTimeCommand, which is bound to a hotkey.

import maya.mel as mel

name = "exampleMarkingMenu"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

parent = mel.eval("findPanelPopupParent")

if "nodeEditor" in parent:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import exampleMarkingMenu
    reload(exampleMarkingMenu)
else:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import fallbackMarkingMenu
    reload(fallbackMarkingMenu)

So, what we do here is, we start by cleaning up any existing versions of the marking menu. Then, we use the very handy findPanelPopupParent MEL function to give us the parent to which we should bind our popupMenus. Having that we check if the editor we want exists in the name of the parent. I could also compare it directly to a string, but the actual panel has a number at the end and I prefer just checking the base name. Then, depending on which panel I am working in at the moment, I build the appropriate custom marking menu.

Don’t forget that you need to create a release command as well, to delete the marking menu so it does not get in the way if you are not pressing the hotkey. It is a really simple command, that I went over in my previous marking menu post.

The obvious limitation here is that we have a hotkey defined and we cant just do ctrl+alt+MMB for example.

Conclusion

So, yeah, these tend to be a bit trickier than just creating ones in the viewport, but also I think there is more to be desired from some of maya’s editors *cough* node editor *cough*, and custom marking menus help a lot.

So, recently I stumbled upon a djx blog blost about custom hotkeys and marking menus in different editors in maya. I had been thinking about having a custom hotkey marking menu, but was never really sure how to approach this, so after reading that post I thought I’d give it a go and share my process.

tl;dr: We can create a runtime command which builds our marking menu and have a hotkey to call that command. Thus, giving us the option to invoke custom marking menus with our own custom hotkeys, such as Shift+W or T for example, and a mouse click.

Disclaimer: I have been having a super annoying issue with this setup, where the “release” command does not always get called, so the marking menu is not always deleted. What this means is that if you are using a modifier like Shift, Control or Alt sometimes your marking menu will still be bound to it after it has been closed. Therefore, if you are using something like Shift+H+LMB, just pressing Shift+LMB will open it up, so you lose the usual add to selection functionality. Sure, to fix it you just have to press and release your hotkey again, but it definitely gets on your nerve after a while.

If anyone has a solution, please let me know.

I have written about building custom marking menus in Maya previously, so feel free to have a look as I will try to not repeat myself here. There I also talked about why I prefer to script my marking menus, instead of using the Marking menu editor, and that’s valid here as well.

So, let us have a look then.

RunTimeCommands

The first thing we need to do is define a runTimeCommand, so we can run it with a hotkey. That is what happens if you do it through the Marking menu editor and set Use marking menu in to Hotkey Editor, as well.

There a couple of ways we can do that.

Hotkey Editor

On the right hand side of the hotkey editor there is a tab called Runtime Command Editor. If you go on that one you can create and edit runTime commands.

Scripting it in Python

If you have multiple marking menus that you want to crate, the hotkey editor might seem as a bit of a slow solution. Additionally, if changes need to be made I always find it more intuitive to look at code in my favourite text editor (which is sublime by the way).

To create a runTime command we run the runTimeCommand function which for some reason does not appear in the Python docs, but I’ have been using maya.cmds.runTimeCommand successfully.

All we need to provide is a name for the command, some annotation – ann, a string with some code – c and a language – cl.

Here is an example

mc.runTimeCommand("exampleRunTimeCommand", ann="Example runTime command", c=commandString, cl="python")

Something we need to keep in mind when working with runTime commands is that we cannot pass external functions to them. We can import modules and use them once inside, but I cannot pass a reference to an actual function to the c flag, as I would do to menuItems for example. That means that we need to pass our code as a string.

Press and release

Now, that we know how to create the runTimeCommands let us see what we need these commands for.

As I mentioned, they are needed so we can access them by a hotkey. What that hotkey should do is initialize our marking menu, but once we release the key it should get rid of it, so it does not interfere with other functions. Therefore we need two of them – Press and Release.

Let us say we are building a custom hotkey marking menu for weight painting. In that case we will have something similar to the following.

  • mmWeightPainting_Press runTimeCommand – to initialize our marking menu
  • mmWeightPainting_Release runTimeCommand – to delete our marking menu

The way we bind the release command to the release of a hotkey is by pressing the small arrow to the side of the hotkey field.

Custom hotkey marking menu - release command hotkey

The Press command

import maya.cmds as mc # Optional if it is already imported

name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

import mmWeightPainting
reload(mmWeightPainting)

So, essentially what we do is every time we press our hotkey, we delete our old marking menu and rebuild it. We do this, because we want to make sure that our latest changes are applied.

Now, the lower part of the command is where it gets cool, I think. We can store our whole marking menu build – all menuItems – inside a file somewhere in our MAYA_SCRIPT_PATH and then just import it from the runTimeCommand as in this piece of code. What this gives us, is again, the ability to really easily update stuff (not that it is a big deal with marking menus once you set them up). Additionally, I quite like the modularity, as it means we can have very simple runTimeCommands not cluttered with the actual marking menu build. This is the way that creating through the Marking menu editor works as well, but obviously it loads a MEL file instead.

So, literally that mmWeightPainting file is as simple as creating all our marking menu items.

import maya.cmds as mc

mc.menuItem(l="first item")
mc.menuItem(l="second item")
mc.menuItem(l="North radial position", rp="N")
...

And that takes care of building our marking menu when we press our hotkey + the specified modifiers and mouse button. What, we do not yet have is deleting it on release, so it does not interfere with the other functionality tied to modifier + click combo. That is where the mmWeightPainting_Release runTimeCommand comes in.

The Release command

## mmWeightPainting_Release runTimeCommand
name = "mmWeightPainting"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

Yep, it is a really simple one. We just delete the marking menu, so it does not interfere with anything else. Essentially, the idea is we have it available only while the hotkey is pressed.

Hotkey

All that is left to be done is to assign a hotkey to the commands. There are a couple of things to have in mind.

If you are using modifiers for the popupMenu command – sh, ctl or alt – then the same modifiers need to be present in your hotkey as otherwise, even though the runTimeCommand will run successfully, the popupMenu will not be triggered.

In the above example

mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

we have specified the sh modifier. Therefore, the Shift key needs to be present in our hotkey.

Also, obviously be careful which hotkeys you overwrite, so you do not end up causing yourself more harm than good.

Conclusion

That’s it, it really is quite simple, but it helps a lot once you get used to your menus. Honestly, trying to do stuff without them feels so tedious afterwards.

So, if you have been rigging for a while you have probably felt annoyed by having to create and adjust control shapes every time you build a new rig. You have probably found also that mirroring just the shape of a control or copying it to another one is a bit too tedious. There are some scripts and tools online to help you with this, such as the classic comet menu and the mz_ctrlcreator, but they do not offer all the functions we need and also extending them is not very practical. So, let us write our own control shape manager.

tl;dr I am going to walk you through the process of creating your own control shape manager, but if you would rather just use the final code you can find it here. If you would prefer it combined into one large file, you can grab it from here.

Here is a quick demo of some of the features we are going to look at.

Control shape manager in Maya - Demo

Table of contents

  1. Introduction
  2. Part 1: Control Shape Manager
  3. Getting and setting shapes
  4. Saving and loading
  5. Additional
  6. Part 2: Control Shape Functions
  7. Getting lists for the UI
  8. Assigning shapes and colours
  9. Saving to library
  10. Copying and pasting shapes
  11. Flipping shapes
  12. Rebuilding
  13. Part 3: Simpe UI
  14. Conclusion

Introduction

What we want is a python package that allows us to load and save control shapes to a library, copy and paste them to multiple other controls, change colours, flip them, mirror them, etc.

The full code can be found here. I have built it as a package with a few different modules, to be a bit clearer and nicer to maintain. I have also combined everything into one file as well, if anyone wants to just grab it and use it immediately. What we are going to do here though, is go through the code and learn how to build our own control shape manager, because it is much nicer when you actually understand how it works, as then you can extend it and adjust it to suit your needs. For example I have built upon this a bit more in my pipeline, so I can save and load shape versions for each control on a rig, so I can easily rebuild them when I am making changes.

Part 1: Control Shape Manager

We are going to be using a couple of commands from the Maya API, but if you are not very familiar with it, worry not I will explain what each function does. You can always read up on it on the Autodesk docs page or if you prefer more of a tutorial approach have a look at Chad Vernon’s Maya API web page.

Getting and setting shapes

Let us start with the two most important functions – getShape() and setShape().

def getShape(crv=None):
    crvShapes = validateCurve(crv)

    crvShapeList = []

    for crvShape in crvShapes:
        crvShapeDict = {
            "points": [],
            "knots": [],
            "form": mc.getAttr(crvShape + ".form"),
            "degree": mc.getAttr(crvShape + ".degree"),
            "colour": mc.getAttr(crvShape + ".overrideColor")
        }
        points = []

        for i in range(mc.getAttr(crvShape + ".controlPoints", s=1)):
            points.append(mc.getAttr(crvShape + ".controlPoints[%i]" % i)[0])

        crvShapeDict["points"] = points
        crvShapeDict["knots"] = utils.getKnots(crvShape)

        crvShapeList.append(crvShapeDict)

    return crvShapeList

What this function does is, it gets all the data from a nurbsCurve node that we need to rebuild that curve down the line. We are going to look at the validateCurve() function a bit later, but it essentially checks if the curve we have passed is actually a valid curve and if so returns the shape node.

A list is initialized here which will later be populated with dictionaries for each shape node on the curve in order to work with compound curves.

The crvShapeDict is where the actual data is stored. All of the keys in the dictionary are just the needed data for building a curve. If you do not know what the knots and degree are when it comes to curve, you can read up on it here, but it is not necessary. We will be thinking of them as the essential building blocks of a curve.

You can see that very easily we can get the form, degree and colour ones as they are just attributes on the nurbsCurve node.

To get the points what we need to do is loop through all of the controlPoints of the curve. Initially, I was just using the cv attribute, but it does not work with closed curves, as the cvs are just representation of these points, so we can interact with them, but under the hood maya changes them a bit and they are stored in the controlPoints attribute. So, we just get the number of control points using the s flag on the getAttr command and we store each point in a list.

Now, for the knots initially I used this snippet from Serge Scherbakov, but it does not work with closed curves. I could have gone in and tried to create my own function to do that, but then maya has made it easy for us to get the knots from the API, so I thought I would just use that.

def getKnots(crvShape=None):
    mObj = om.MObject()
    sel = om.MSelectionList()
    sel.add(crvShape)
    sel.getDependNode(0, mObj)

    fnCurve = om.MFnNurbsCurve(mObj)
    tmpKnots = om.MDoubleArray()
    fnCurve.getKnots(tmpKnots)

    return [tmpKnots[i] for i in range(tmpKnots.length())]

The first part of this function deals with getting an API reference to our curve. It basically, adds the passed in crvShape to a virtual selection (without actually selecting anything in the viewport) and gets an MObject from it. That’s the base class in the API and from there we can cast it to the type we actually need – MFnNurbsCurve. Then we create an empty MDoubleArray, which we populate from the curve with the getKnots function. And that’s it. Lastly, we return it as a python list, just so we can interact with it easier.

And with that we have a list of dictionaries containing all the necessary information for rebuilding that curve.

Let’s look at setting the shape now. What is nice about this code is that if you understand how the getShape() works, the setShape() is going to be trivial. The one thing I do not like about this code is that we are not assigning the data to the existing curve, but we delete it and create a new one in place. This could cause issues if there are any connections to or from the shape node, but you can always store and rebuild those. I have not yet found a way around it though.

def setShape(crv, crvShapeList):
    crvShapes = validateCurve(crv)

    oldColour = mc.getAttr(crvShapes[0] + ".overrideColor")
    mc.delete(crvShapes)

    for i, crvShapeDict in enumerate(crvShapeList):
        tmpCrv = mc.curve(p=crvShapeDict["points"], k=crvShapeDict["knots"], d=crvShapeDict["degree"], per=bool(crvShapeDict["form"]))
        newShape = mc.listRelatives(tmpCrv, s=1)[0]
        mc.parent(newShape, crv, r=1, s=1)

        mc.delete(tmpCrv)
        newShape = mc.rename(newShape, crv + "Shape" + str(i + 1).zfill(2))

        mc.setAttr(newShape + ".overrideEnabled", 1)

        if "colour" in crvShapeDict.keys():
            setColour(newShape, crvShapeDict["colour"])
        else:
            setColour(newShape, oldColour)

We go through the same call to validateCurve() as before and then we store the "overrideColor" of the curve, so we can apply it back after we rebuild the shape. It is important to note that the colour is the one assigned to the first shape child of the curve. And since we have everything we need from the old shapes – the colour – we delete them.

Then for each shape in the list we just use our points, knots, degree and form data from the dictionary to build a new curve with the mc.curve() command. The per flag refers to periodic and basically defines whether our curve is one whole or does it have a start and an end. A bit more info about periodic curves in here.

Once we have created the new shape we parent it to the crv object with the r=1 and s=1 flags for mc.parent() to define that we are working with shapes and to maintain their relative positions. We then can rename the new shape according to our convention. Lastly, we just reapply the colour or we get it from the dictionary.

As I said these two are the most important functions as they are dealing with the actual data. Now that we have them in place we can give them a quick test. Create a nurbsCurve with whatever shape you want. Then let’s create a simple circle and copy the first shape to the circle. Assuming that the first curve is called curve1 and the circle is nurbsCircle1 run the following snippet.

shapeDict = getShape("curve1")
setShape("nurbsCircle1", shapeDict)

I realize this is not very exciting as there are easier ways to do this, but the cool thing is when we start saving and loading them.

Saving and loading

We have been looking only at the manager.py file for know. In the utils.py we have a few more functions mainly dealing with the saving and loading of json data. Loading and saving JSON files is a very popular and fairly trivial python task, but let’s deconstruct it.

def loadData(path=None):
    if os.path.isfile(path):
        f = open(path, "r")
        data = json.loads(f.read())
        f.close()
        return data
    else:
        mc.error("The file " + path + " doesn't exist")

def saveData(path=None,
             data=None):
    if validatePath(path):
        f = open(path, "w")
        f.write(json.dumps(data, sort_keys=1, indent=4, separators=(",", ":")))
        f.close()
        return 1
    return 0

For loading we start by checking if the file exists and if it does, we use python’s open function to get the raw data and we pass it to a json.loads() function to convert the raw data to a Python dict object.

When saving, we are doing the same thing but instead of reading and converting from raw data to a dict we are passing a dict to the json.dumps() function which serializes our dictionary to JSON and then we write it to the file. You will notice that there is a call to another validation function – validatePath().

def validatePath(path=None):
    if os.path.isfile(path):
        confirm = mc.confirmDialog(title='Overwrite file?',
                                   message='The file ' + path + ' already exists.Do you want to overwrite it?',
                                   button=['Yes', 'No'],
                                   defaultButton='Yes',
                                   cancelButton='No',
                                   dismissString='No')
        if confirm == "No":
            mc.warning("The file " + path + " was not saved")
            return 0
    return 1

All we do here is check if the file we are trying to save already exists and if so gives the option to overwrite it or cancel the save process.

Now that we know how our dictionary data is being load and saved, we just need to have a wrapper function in our manager module to load and save to the defined shape library directory.

Before looking at those though, you need to have the SHAPE_LIBRARY_PATH set at the top of the file. Keep in mind that if the path does not exist, Python will not create it for us but error out.

def loadFromLib(shape=None):
    path = os.path.join(SHAPE_LIBRARY_PATH, shape + ".json")
    data = utils.loadData(path)
    return data

What we do here is define the path to the file we want to load and use the loadData function we talked about to load the actual dictionary.

def saveToLib(crv=None,
              shapeName=None):
    crvShape = getShape(crv=crv)
    path = os.path.join(SHAPE_LIBRARY_PATH, re.sub("\s", "", shapeName) + ".json")
    for shapeDict in crvShape:
        shapeDict.pop("colour", None)
    utils.saveData(path, crvShape)

Then when saving we use re.sub("s", "", shape) in order to strip spaces from the name as they can cause issues and pass the path to the saveData() function. Also, we get rid of the colour keys as we want to save only the shape of the curve.

Additional commands

The rest of the functions in the module are fairly self-explanatory.

def validateCurve(crv=None):
    if mc.nodeType(crv) == "transform" and mc.nodeType(mc.listRelatives(crv, c=1, s=1)[0]) == "nurbsCurve":
        crvShapes = mc.listRelatives(crv, c=1, s=1)
    elif mc.nodeType(crv) == "nurbsCurve":
        crvShapes = mc.listRelatives(mc.listRelatives(crv, p=1)[0], c=1, s=1)
    else:
        mc.error("The object " + crv + " passed to validateCurve() is not a curve")
    return crvShapes

The validateCurve() function just checks if we have passed a valid curve and if so it returns the nurbsCurve shape nodes to work with. Otherwise it errors.

Then we have the colour functions which are just simple wrappers around mc.getAttr() and mc.setAttr() commands to interact with the "overrideColor" attribute of shapes.

def setColour(crv, colour):
    if mc.nodeType(crv) == "transform":
        crvShapes = mc.listRelatives(crv)
    else:
        crvShapes = [crv]
    for crv in crvShapes:
        mc.setAttr(crv + ".overrideColor", colour)

def getColour(crv):
    if mc.nodeType(crv) == "transform":
        crv = mc.listRelatives(crv)[0]
    return mc.getAttr(crv + ".overrideColor")

Part 2: Control Shape Functions

Now that we have our core functionality in place we can stop here and just use the code we have so far through our script editor, which is absolutely fine, but is not very scalable and not really user friendly. Additionally, we are still lacking the mirroring and flipping functionality, so let us create a functions.py file which will act as a wrapper to our manager module. The reason we would want this is to prevent messing about with our manager too much and provide a higher level control so we can literally only care about using the tool instead of how it works. Altogether, it is much nicer working with simple short functions. Okay, let us go through the functions.py commands that help us interact with the manager.

Getting lists for the UI

def getAvailableControlShapes():
    lib = manager.SHAPE_LIBRARY_PATH
    return [(x.split(".")[0], functools.partial(assignControlShape, x.split(".")[0])) for x in os.listdir(lib)]

def getAvailableColours():
    return [("index" + str(i).zfill(2), functools.partial(assignColour, i), "shapeColour" + str(i).zfill(2) + ".png") for i in range(32)]

These two functions are mainly here to help us later when we are going to build some sort of UI for our manager. Essentially they return lists of tuples containing the names, commands and in the case of getAvailableColours() images of the available shapes and colours. These are going to be used when building menus that look similar to the following.

Control shape manager - Shapes dropdown
Shapes
Control shape manager - Colours dropdown
Colours

Notice that the second item in the tuple is a functools.partial() call. For more info refer to the docs, but briefly it allows us to get a reference to a function with added arguments as well. So the first argument is a function and then we have a number of arguments which are going to be provided to the function as *args. Let’s have a look at the functions themselves to see how this works.

Assigning shapes and colours

def assignColour(*args):
    for each in mc.ls(sl=1, fl=1):
        manager.setColour(each, args[0])

def assignControlShape(*args):
    sel = mc.ls(sl=1, fl=1)
    for each in sel:
        manager.setShape(each, manager.loadFromLib(args[0]))
    mc.select(sel)

So, both these functions receive *args as an argument, which means that we can provide lots of arguments and they are going to be passed to the function as a list which we can acces by args[n]. In the previous paragraph, we saw that we pass these functions and a single argument to the functools.partial, which means that the first element of args is going to be the second argument of the functools.partial() code. So in the case of functools.partial(assignColour, i), we are going to receive a call equivalent to assignColour(i).

Additionally, keep in mind if these functions that we are defining here are meant to be used from a maya UI, and a lot of the buttons in maya are passing arguments to their commands, so we need to have the *args, because otherwise the functions will error.

Notice that we reselect our initial selection at the end of the function. We will do this in all functions that call the setShape() one, because the creation of the curve inside of it deselects our selection and instead selects the newly created curve, which is not very intuitive.

Saving to library

def saveCtlShapeToLib(*args):
    result = mc.promptDialog(title="Save Control Shape to Library",
                             m="Control Shape Name",
                             button=["Save", "Cancel"],
                             cancelButton="Cancel",
                             dismissString="Cancel")
    if result == "Save":
        name = mc.promptDialog(q=1, t=1)
        manager.saveToLib(mc.ls(sl=1, fl=1)[0], name)
    rebuildUI()

As we said the goal here is to make interacting with our control shape manager as smooth as possible. Therefore, we create a wrapper to our saveToLib() command to let us add a name in a nice and familiar dialog. In the end we are calling the rebuildUI() function which we will look at the end of this part, but the reason it is here is that every time we save a new control shape we would like the UI to be rebuild, in order for the menu containing all of our shapes to be up to date.

Copying and pasting shapes

def copyCtlShape(*args):
    global ctlShapeClipboard
    ctlShapeClipboard = manager.getShape(mc.ls(sl=1, fl=1)[0])
    for ctlShape in ctlShapeClipboard:
        ctlShape.pop("colour")

def pasteCtlShape(*args):
    sel = mc.ls(sl=1, fl=1)
    for each in sel:
        manager.setShape(each, ctlShapeClipboard)
    mc.select(sel)

As we saw previously, it is really easy to copy and paste shapes with the manager alone, but to provide a quick and easy interface these two functions seem to do a good job. Essentially, we are creating a global variable and store the selected shape dictionary inside of it. Again we pop the “colour” key, as we just want to copy the shape. Then we just use the setShape() function on all selected controls with that global variable.

Flipping shapes

Then there are a few functions for flipping the shapes. It’s a bit of a pain to have to do that manually, but it is really easy to scale the points by -1 through script so let’s have a look at the _flipCtlShape() function. You will notice that there are a few more functions for flipping – flipCtlShape(), flipCtlShapeX(), flipCtlShapeY() and flipCtlShapeZ(). They all just make a call to the _flipCtlShape() one, but with different arguments, so we will just look at that one.

def _flipCtlShape(crv=None, axis=[-1, -1, -1]):
    shapes = manager.getShape(crv)
    newShapes = []
    for shape in shapes:
        for i, each in enumerate(shape["points"]):
            shape["points"][i] = [each[0] * axis[0], each[1] * axis[1], each[2] * axis[2]]
        newShapes.append(shape)
    manager.setShape(crv, newShapes)
    mc.select(crv)

All we do in this one, is just go through each CV and scale it’s x, y and z coordinates by -1 in order to flip the shape. The above mentioned other functions just call this one with the axis set to [-1, 1, 1] for x, [1,-1,1] for y, etc.

I skipped the mirrorCtlShapes() function earlier, because I wanted to already have the flip one in place as we are going to be using it again.

def mirrorCtlShapes(*args):
    sel = mc.ls(sl=1, fl=1)
    for ctl in sel:
        if ctl[0] not in ["L", "R"]:
            continue
        search = "R_"
        replace = "L_"
        if ctl[0] == "L":
            search = "L_"
            replace = "R_"
        shapes = manager.getShape(ctl)
        for shape in shapes:
            shape.pop("colour")
        manager.setShape(ctl.replace(search, replace), shapes)
        _flipCtlShape(ctl.replace(search, replace))
    mc.select(sel)

The bulk of the code here is really for defining the search and replace strings. Since the naming convention that I use is SIDE_NAME_NODETYPE my search and replace strings vary between “L_” and “R_”. Have a look at your convention and modify these strings to make it work. Once they are defined, all we do is copy the shape from the current side to the other one and once done, flip it in all axis. In my pipeline, I have made it so this function does not work with a selection, but instead goes through all my left controls and mirrors them to the right. It is just because I always work from left to right, so I do not need this functionality.

Rebuilding

def rebuildUI(*args):
    mc.evalDeferred("""
import controlShapeManager
reload(controlShapeManager)
""")

Lastly, there is a simple function to rebuild the UI. All it does is import the package, as the way I have set it up is that importing just builds the UI which in turn makes the references to all the needed functions. The UI example that I give is very primitive, but obviously you can replace this code with one that will work with your own UI. Keep in mind that it is best to use the mc.evalDeferred() command as otherwise, the rebuild might error as it is being called from the UI that needs to be rebuilt.

Part 3: Simple UI

Now that we have all functions that we need we can build an UI to interact with them. Since everybody has a different pipeline for rigging at place, I am hesitant to suggest any specific way of handling that UI. One might prefer it in a window, other a tool menu or others yet a shelf button like I do. So I have added a very simple shelf button build in the managerUI.py to demonstrate how would we go about it. Additionally, remember how when generating the lists for the available colours we had a third item in the tuple for an image? You can get these here. They are just images of solid colour, corresponding to the index of the overrideColor attribute.

For a more comprehensive intro to building shelves with buttons and popups have a look at my Building custom maya shelves post.

Let’s have a look at it then.

import maya.cmds as mc


# Local import
import functions
reload(functions)

SHELF_NAME = "Custom"
ICON_PATH = "C:/PATH_TO_ICONS"

if SHELF_NAME and mc.shelfLayout(SHELF_NAME, ex=1):
    children = mc.shelfLayout(SHELF_NAME, q=1, ca=1) or []
    for each in children:
        try:
            label = mc.shelfButton(each, q=1, l=1)
        except:
            continue
        if label == "ctlShapeManager":
            mc.deleteUI(each)

    mc.setParent(SHELF_NAME)
    mc.shelfButton(l="ctlShapeManager", i="commandButton.png", width=37, height=37, iol="CTL")
    popup = mc.popupMenu(b=1)
    mc.menuItem(p=popup, l="Save to library", c=functions.saveCtlShapeToLib)

    sub = mc.menuItem(p=popup, l="Assign from library", subMenu=1)

    for each in functions.getAvailableControlShapes():
        mc.menuItem(p=sub, l=each[0], c=each[1])

    mc.menuItem(p=popup, l="Copy", c=functions.copyCtlShape)
    mc.menuItem(p=popup, l="Paste", c=functions.pasteCtlShape)

    sub = mc.menuItem(p=popup, l="Set colour", subMenu=1)

    for each in functions.getAvailableColours():
        mc.menuItem(p=sub, l=each[0], c=each[1], i=ICON_PATH + each[2])

    mc.menuItem(p=popup, l="Flip", c=functions.flipCtlShape)
    mc.menuItem(p=popup, l="Mirror", c=functions.mirrorCtlShapes)

So what happens here is we import the functions.py file which in turn imports the manager.py which then imports the utils.py file. Then there are the two variables – SHELF_NAME and ICON_PATH for declaring the shelf to add the button to and the path to the icons. Then we check if a button with the same name already exists in the shelf and if it does we delete it so we can replace it with our new one.

From then on we have simple maya UI commands to build our buttons and menus. If you are not familiar with UIs in maya it is worth having a look at the docs. Essentially, all we do is create a single mc.shelfButton() and we attach a mc.popupMenu() to it. Which then we populate with mc.menuItem()s where the l flag stands for label and the c for command. So there we pass our functions commands. Notice that we are not adding the () after the function name as that would call it and return the output. Instead we want to pass a reference to that function.

Then for the shapes and colours menuItems we add the subMenu flag so they become deeper level menus and we populate them with the results of our functions.getAvailableControlShapes() and functions.getAvailableColours() commands, which results in lists containing the shapes in our library and all 32 available colours.

Conclusion

And that is it. We have built our own control shape manager. With some easy extensions you can improve it to have almost like a version control system for your rigs, so you do not ever have to worry about your control shapes anymore.

Again, here is the link to the repo and the gist containing everything in one file if you would rather just use something quickly.

I am not sure what it is, but there is something incredibly appealing in optimizing our workflows. I think a lot of it comes from the frustration of repeating the same actions over and over again. When you find a way to optimize that, it feels great. One of the easiest way to improve our rigging workflow is to script a custom marking menu with python. Another one, I have already written about is creating a custom shelf.

tl;dr: I will walk you through scripting your own custom marking menu with python, which is going to be easily shareable, extendable and maintainable. The code can be found here.

Here is how my main marking menu looks.

Custom marking menu in maya with python

I have found it is an immense help to have the commands I use most oftenly either in my marking menu or my shelf. It just saves so much time!

Okay, how do we go about creating one of these?

Well, we have two options – either build it with Maya’s native marking menu editor or script it with python.

The reasons I prefer scripting my marking menus in python are a few.
– The native editor does not give us all the available options for a marking menu, such as submenus.
– Updating from the editor is a pain in the butt.
– The editor does not scale nicely, if you want or need to support multiple marking menus.
– Doing it through the editor is boring.

Obviously, python fixes all these issues for us. Additionally, it is easy to share it with co-workers and keep it in a version control system. Okay, so you are sold now. Let us have a look at how to do it then.

The code that I will be going through is on this gist, but I will go through all of it, if you would rather write it yourself.

Custom marking menu with Python

Initialization

import maya.cmds as mc

MENU_NAME = "markingMenu"

We start very simple with the import of maya.cmds and giving a name to our custom marking menu. Now, the name is not very important because we do not ever see it, but maya does. So, in order for us to be able to update our custom marking menu, we need to be able to access it, and that is why we are giving it a name.

Then we have our markingMenu class. The main reason I went for a class is because we can encapsulate everything we need into it quite logically. I know that some people would prefer to have a function instead, which is absolutely fine, it is really a personal preference. Let’s have a look at the constructor.

Constructor

def __init__(self):

self._removeOld()
self._build()

As you can see, our constructor is very simple. We delete the old version of our custom marking menu if it exists and then we build are our new one in place.

Remove old marking menu

def _removeOld(self):
    if mc.popupMenu(MENU_NAME, ex=1):
    mc.deleteUI(MENU_NAME)

As I said, the reason we need to give a name to our marking menu is to be able to modify it after it is created. In our case, we are not really modifying it, but deleting it instead. We do this, so we have a clean slate for building our new marking menu.

The reason I have added this deleting and then building again functionality is just so we can painlessly make changes to our custom marking menu. For example, if I want to add a new button or change a label, I do not want to have to restart maya or do anything other than just running my code again. Or even easier, just importing my code again. That is why I said earlier, that this setup for a custom marking menu is very easily extendable and maintainable. Additionally, we can add a button to our marking menu which rebuilds it, so we only have to make our changes to the file, save it and then rebuild from within. We will have a look at that in the end of the post.

Building our custom marking menu

Now that we have had a look at preparing for our build let us have a look at the _build() method.

def _build(self):
    menu = mc.popupMenu(MENU_NAME, mm = 1, b = 2, aob = 1, ctl = 1, alt=1, sh=0, p = "viewPanes", pmo=1, pmc = self._buildMarkingMenu)

Another very simple method. I was surprised that maya does not have a specific markingMenu method. Instead, the popupMenu() command is used, which is actually nice, since it is a familiar one if you have worked with menus or shelf popups. Let us look at the arguments.

  • MENU_NAME – quite obviously this one sets the name of our custom marking menu.
  • mm – this one defines this popupMenu as a marking menu.
  • b – this is the mouse button we would like to trigger the marking menu. 1 – left, 2 – middle, 3 – right.
  • aob – allows option boxes.
  • ctl – defines the Ctrl button as a needed modifier to trigger the marking menu.
  • alt – defines the Alt button as a needed modifier to trigger the marking menu.
  • sh – defines the Shift button as a needed modifier to trigger the marking menu.
  • p – the parent ui element. For marking menus, that would usually be “viewPanes”, which refers to all of our view panels.
  • pmo – this flag declares that the command that we pass to the pmc flag should be executed only once and not everytime we invoke the marking menu. If this is false, everytime we trigger our custom marking menu, we will see our menus growing as more and more menuItems will be added.
  • pmc – this is the command that gets called right before the popupMenu is displayed. This is where we need to pass our method that actually builds all our buttons and menus – _buildMarkingMenu.

It is important to notice that when we pass the self._buildMarkingMenu we do not add brackets in the end as that would call the function instead of passing it as as reference.

Additionally, it is also important to think about the button flag and the modifiers. Obviously, Maya already has some marking menus and some functions related to mouse clicks and the ctrl, alt and shift modifiers. Therefore, we need to come up with a combination that does not destroy a function which we actually want to keep. That is why, for my marking menu I use the middle mouse button + alt + ctrl. There was some zooming function bound to that combination I believe, but I never used it so it was safe to override.

Actually building our custom marking menu

So, everything up to this point was to set us up for actually adding our commands to our menu. To be honest, it is as simple as everything we have already seen. Let us have a look at how we do this.

def _buildMarkingMenu(self, menu, parent):

## Radial positioned
mc.menuItem(p=menu, l="South West Button", rp="SW", c="print 'SouthWest'")
mc.menuItem(p=menu, l="South East Button", rp="SE", c=exampleFunction)
mc.menuItem(p=menu, l="North East Button", rp="NE", c="mc.circle()")

subMenu = mc.menuItem(p=menu, l="North Sub Menu", rp="N", subMenu=1)
mc.menuItem(p=subMenu, l="North Sub Menu Item 1")
mc.menuItem(p=subMenu, l="North Sub Menu Item 2")

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'")
mc.menuItem(p=menu, ob=1, c="print 'South with Options'")

## List
mc.menuItem(p=menu, l="First menu item")
mc.menuItem(p=menu, l="Second menu item")
mc.menuItem(p=menu, l="Third menu item")
mc.menuItem(p=menu, l="Create poly cube", c="mc.polyCube()")

The first thing to note is that our method receives menu and parent as arguments. These are passed automatically from the pmc flag on the popupMenu function in the _build() method.

Then we have the actual items in our custom marking menu. I like to split them logically in – radial and list blocks.

Radial positions

The radial positions are defined by the directions on a map – East, West, NorthEast, etc. Have a look at the following image.

Custom marking menu - Radial positions

You can have either commands in these slots or additional subMenus like so.

My personal preference here is to have just single commands instead of additional popups, as to invoke a submenu you need to hover on a position and wait a little bit. And I find that small delay quite frustrating.

So the way we create these radial items is by using the menuItem command. All we have to do is pass our menu argument as the parent (p), define a label (l), a radial position (rp) and the command (c) we want to execute on click.

Notice that if passing functions as commands we pass them without brackets, as that would call them instead. Have a look at the SE radial position for an example. Additionally, the functions need to be able to receive arguments as maya’s ui elements tend to pass some info about themselves to the commands they call. The way we do that is by just adding the *args argument.

def exampleFunction(*args):
    print "example function"

Additionally, we can call maya commands from our items. Have a look at the NE item. It is important to note here that even though we have imported maya.cmds as mc in this file, the commands we pass to our menuItems are going to be called from maya’s python environment. That means, that in order for mc.circle() to work, we need to have imported maya.cmds as mc somewhere in our maya session. You could either run it yourself in the script editor or a better solution would be to add it to your userSetup.py file. That is what I did, since I use a lot of python in maya I just do an import maya.cmds as mc in my userSetup.py.

Submenus

As I said we can have subMenu items, which essentially create another menu when you hover on them. The only thing we do is set the subMenu flag to 1. Have a look at one of these in our N radial position example. We store the menuItem with the subMenu flag in a variable, so we can use it as a parent for our following items. From then on, we just follow the same principle, we list menuItems, with the subMenu variable that we stored as a parent. Additionally, we can have deeper subMenus as well, though I do not think that would be great to work with.

Option boxes

Another very useful addition to our menuItems is adding option boxes. A lot of maya’s menus have those and they are a nice way to add additionall functionality without sacrificing space.

Usually, you would add them to commands that sometimes need their options to be changed, but also you can have different commands bound to them. For example, in my main marking menu I have the joint tool in one of my radial positions. But since I never mess with the options for that tool, in the option box I have a command which creates a joint under every object I have selected. Additionally, it names it with the name of the selected object and assumes it’s transformations.

The way we add an option box is very simple. All we need to do is create another menuItem after the one we want to add the option box to and we set the ob flag to 1. Have a look at the S radial position for an example.

List elements

In addition to our radial positioned items, marking menus have another menu beneath the South radial position. What is cool about this one is that it is an actual menu instead of just a single radial position, so we can have multiple items in it.

The way we create those items is by just specifying the marking menu as a parent. Remember it is passed as the menu argument to our method. So, if we do not specify a radial position the menuItem gets added to that menu which is south of the South position.

Again, as with all other menus, we can have submenus in there, by just setting the submenu flag to 1. My personal preference though is to have as few of these as possible, as I want everything to be easy to grab at first glance.

Icons

I have not added any icons in this example marking menu, but these can easily be added to every menuItem by using the i flag. All you need to do is place your custom icons somewhere in the XBMLANGPATH environment variable. To see what that is on your machine run this MEL command getenv XBMLANGPATH.

Additionally, you can use maya’s native icons, which you can browse through in the shelf editor. If you open that, pick any button on the right and next to the Icon Name field you can find a Browse Maya Icons button.

For example if I want to add maya’s standard icon to the S radial position I would do this.

mc.menuItem(p=menu, l="South", rp="S", c="print 'South'", i="mayaIcon.png")

That results in the following image.

Custom marking menu - icon

And of course you can also pass full paths to icons which are not on maya’s XBMLANGPATH path.

Rebuilding our custom marking menu and loading it on startup

So, now that we know how to build our marking menus, let us have a look at how to actually initialize them with maya and how to rebuild them on the fly when we make changes. For building our marking menu we just initialize our markingMenu() class. For rebuilding we need to do the same thing, but we have several options for how we call our class.

Option 1: Just run it

I would say this is the simplest solution. If you are working in an external editor you can easily copy and paste your modified code in maya’s script editor, run it and you’re done. Since, we have added the rebuilding functionality – delete old and then build new one – the marking menu is updated everytime we run our code and then call the markingMenu() class in the end.

Even better if you have connected your external editor to maya via a commandPort command you can just run the code from there and that’ll update your marking menu as well. (I use this plugin for Sublime to do that.

The downside with this option is that everytime we open maya we will need to run our code. Which means that at some point you will get annoyed with doing it.

Option 2: Add it to your scripts path

Even though the first solution is quite simple this one is a bit nicer to work with. It also takes care of loading our marking menu on startup.

So, what we do here is we add the file containing our code – markingMenu.py – somewhere on our MAYA_SCRIPT_PATH. Again, you can run getenv MAYA_SCRIPT_PATH in MEL to get this path.

What this means is that now we can access our marking menu from within maya. Therefore we have a lot of options on how to build/rebuild our marking menu.

For building it on startup we just need to add the following code inside our userSetup.py.

import markingMenu
markingMenu.markingMenu()

When rebuilding though we need to do a reload statement. That is because when importing a module, python checks whether it is already imported and if so, just uses that instead. So if we make changes and then do import, our changes are not going to be imported. That is why we need to do reload(ourModule).

So in the case of reloading our marking menu, we will need to do the following.

import markingMenu
reload(markingMenu)
markingMenu.markingMenu()

This will ensure that whatever changes we have made to our code will be implemented.

Now that we have this rebuilding code, we can add it wherever we find most appropriate. Generally, I would say either a button on a shelf or from within the marking menu itself. Since a button on a shelf is trivial let us look at adding it to the marking menu.

Reloading from within our custom marking menu

The way we would do this, is just add the following menuItem.

mc.menuItem(p=menu, l="Rebuild Marking Menu", c=rebuildMarkingMenu)

Where the rebuildMarkingMenu refers to the following function.

def rebuildMarkingMenu(*args):
    mc.evalDeferred("""
reload(markingMenu)
markingMenu.markingMenu()
""")

I really don’t like using evalDeferred since it feels a bit dirty, but in this case we need it. The reason we need it is that we are rebuilding our marking menu from within. So if we do not use evalDeferred but directly call our markingMenu(), maya will have to delete the button which we have clicked, and that will error.

Conclusion

So, there we have it, we have built our own custom marking menu with Python. What is more, it is fully scripted, so making changes is very easy. Additionally, we can have that file in a version control system, so we can have a log of our changes. And of course, it is super easy to share with co-workers. The best thing about it, though, is how much time it saves in our day-to-day rigging tasks.

So, a few months back I was looking into clean ways of scripting a custom shelf in Maya, that can be easily shared, and I was surprised that there were not many resources on how to go about it. Therefore, I thought I would share the way that I am maintaining my shelf.

tl;dr Grab the code for the shelf base class from here and derive your own shelf class from it, overwriting and populating the build() method with calls to the addButton, addMenuItem, etc. calls to create your own custom maya shelf.

Scripting your own custom shelf in Maya is much easier than you would think, or at least than I thought. For some odd reason, I always assumed that it would involve a lot of messing about with MEL, which I would really rather not. If that is what you thought as well, you will be pleasantly surprised.

Here is what we are trying to achieve.

Custom shelf in maya

Since, we want to keep the code as modular and versatile as possible, I wrote the main functionality in a base class which needs to be extended for each shelf you want to build. The code for the base class with a little example is on github.

Now we will go through it to make see how it works, so it can be extended and modified to include more functionalities.

Constructor

def __init__(self, name="customShelf", iconPath=""):
    self.name = name

    self.iconPath = iconPath

    self.labelBackground = (0, 0, 0, 0)
    self.labelColour = (.9, .9, .9)

    self._cleanOldShelf()
    mc.setParent(self.name)
    self.build()

In the constructor we initialize the variables we make two calls to _cleanOldShelf() and build(), which we will look at in a bit. The name argument is going to be the name of the shelf. It is important to note that if a shelf with this name already exists it will be replaced with this one. The iconPath argument can be used to define a directory from which to get images to be used as icons of the shelf buttons and commands in the popup menus. If it is not defined, you can still use maya’s default icons like commandButton.png for example.

Additionally there are the labelBackground and labelColour variables which can be seen in the following image.

Custom shelf in maya - button colours

The reason I have hardcoded them in the base class is because I think for the sake of consistency all our shelves should have the same style, but if you want to change them, obviously feel free to do it.

And lastly, there is that mc.setParent(self.name) function, which makes sure that whatever ui elements we are going to build in the following build() method, they will be build as children to our shelf. Otherwise, we might end up with buttons breaking the other layouts.

Clean old shelf

Let’s have a look at what the _cleanOldShelf() method does.

def _cleanOldShelf(self):
    if mc.shelfLayout(self.name, ex=1):
        if mc.shelfLayout(self.name, q=1, ca=1):
            for each in mc.shelfLayout(self.name, q=1, ca=1):
                mc.deleteUI(each)
    else:
        mc.shelfLayout(self.name, p="ShelfLayout")

Essentially, we are checking if our shelf already exists by using the mc.shelfLayout(self.name, ex=1) command where self.name is the name of our shelf and ex is the flag for checking existence. If it does we go through all children of the shelf which we get from mc.shelfLayout(self.name, q=1, ca=1) and delete them. That makes sure we start our build() method with a fresh clean shelf.

If the shelf does not exist though, we simply create it, passing "ShelfLayout" as parent, because that is the parent layout of all the shelves in maya. For example, turn on History > Echo all commands in the the script editor and click through some of the shelves. You will see the paths to some of their popupMenus as so
Shelf|MainShelfLayout|formLayout16|ShelfLayout|CurvesSurfaces||popupMenu546

So, knowing that CurvesSurfaces is a name of a shelf, we can deduce that the ShelfLayout is the parent of all shelves.

Then, before we get into the build() method, let’s have a look at some of the methods we are going to use to actually populate our shelf.

Add button

def addButon(self, label, icon="commandButton.png", command=_null, doubleCommand=_null):
    mc.setParent(self.name)
    if icon:
        icon = self.iconPath + icon
    mc.shelfButton(width=37, height=37, image=icon, l=label, command=command, dcc=doubleCommand, imageOverlayLabel=label, olb=self.labelBackground, olc=self.labelColour)

Very simply, this is the method that creates the main buttons in the shelf. Essentially, we just need to pass it a label which is the name of our button, an optional icon and an optional command and doubleCommand which refer to single click and double click. The command is optional, because we might want buttons that just have popup menus. We will look at those in a sec. By default the _null() command, which is at the top of the file is called. The reason I have that command is that if you pass None to the mc.shelfButton command flag, it will error. So the _null() method essentially does nothing. It is important to note that when we pass a command we are not using the brackets after the name (), because that would call the command instead of passing it.

Add menu item

def addMenuItem(self, parent, label, command=_null, icon=""):
    if icon:
        icon = self.iconPath + icon
    return mc.menuItem(p=parent, l=label, c=command, i="")

This method is very similar to the add button one, but instead of adding a button to the shelf, this one adds a menuItem to an existing popupMenu passed as the parent argument. The popupMenu needs to be created manually, as it doesn’t make sense to wrap it in a method if it is just a simple one line command anyway. So, a quick example of how that works is

...
self.addButon("popup")
p = mc.popupMenu(b=1)
self.addMenuItem(p, "popupMenuItem1")
...

where the b=1 stands for left mouse click. This snippet if added inside the build() method will attach a popupMenu with one menuItem to the popup button.

Add sub menu

def addSubMenu(self, parent, label, icon=None):
    if icon:
        icon = self.iconPath + icon
    return mc.menuItem(p=parent, l=label, i=icon, subMenu=1)

Now, I realize that sub menus inside a popup on a shelf button is getting a bit too deep, as the whole point of a shelf is to streamline everything, but I thought I would add it, as even I have used it on a couple of my buttons.

What this method does is, it creates a menuItem which is a menu in itself, attached to an existing popupMenu which is passed as the parent.

The result is something similar to this.

Custom shelf in maya example popup menus

Separators

You know how some of the shelves have separating lines between the buttons. You can achieve that with the mc.separator() command. I usually set the style flag to "none" as I rather having blank space than having lines, but have a look at the docs page for the separator command for more info.

Build

And then finally let us have a look at the build() method.

def build(self):
'''This method should be overwritten in derived classes to actually build the shelf
elements. Otherwise, nothing is added to the shelf.'''
    pass

As the docstring says, this is an empty method that needs to be defined in each shelf we build. The way this works is, we just populate it with addButton(), mc.popup(), addMenuItem() and addSubMenu() calls as we need them.

Example of a custom shelf in maya

Let’s have a look at the simple example in the end of the file.

class customShelf(_shelf):
    def build(self):
        self.addButon(label="button1", command=mc.polyCube)
        self.addButon("button2")
        self.addButon("popup")
        p = mc.popupMenu(b=1)
        self.addMenuItem(p, "popupMenuItem1")
        self.addMenuItem(p, "popupMenuItem2")
        sub = self.addSubMenu(p, "subMenuLevel1")
        self.addMenuItem(sub, "subMenuLevel1Item1")
        sub2 = self.addSubMenu(sub, "subMenuLevel2")
        self.addMenuItem(sub2, "subMenuLevel2Item1")
        self.addMenuItem(sub2, "subMenuLevel2Item2")
        self.addMenuItem(sub, "subMenuLevel1Item2")
        self.addMenuItem(p, "popupMenuItem3")
        self.addButon("button3")
    customShelf()

As you can see, all we do is we create a new class inheriting from the _shelf class. By the way the _ prefix is used to remind us that it is a class that should not be directly accessed.

Then we overwrite the build() method by simply defining it, and we populate it with the necessary shelf elements.

This example results in the image from above. Here is it again.

Commands

I have not demonstrated the use of any commands on these buttons, but all it takes is just passing the name of the function as a command argument. The one tricky bit about it is that the menuItem commands get passed some arguments, so we need to have *args in our function calls to prevent errors. For example

def createPolyCube(*args):
    mc.polyCube()

We would include this command in our shelf by doing self.addButton(label="cube", command=createPolyCube) in our build() method.

For the sake of clarity though, I recommend having a separate file which contains only the functions that are going to be used in the shelf and import them from there. So at the top of the file where are custom shelf is defined, import your shelf functions file with import shelfFunctions or however you decide to package them. I would advise having a reload(shelfFunctions) statement immediately after that, so you can be sure the latest changes to the file have been taken into account.

And then when creating buttons, just do

self.addButton(label="cube", command=shelfFunctions.createPolyCube)

Additionally, you can also pass maya native commands as well. So to have a button that creates a poly cube we would do

self.addButton(label="cube", command=mc.polyCube)

Building our shelf on load up

The way we will include our shelf on load up is to just add it to our userSetup.py file. If you are not familiar with it, take a look at this. I know this one is for MEL, but it is essentially the same thing for Python as well.

So, assuming that our shelf file is in the maya scripts path, in our userSetup.py we can just do

import maya.cmds as mc
import shelf
mc.evalDeferred("shelf.customShelf()")

where shelf is the file containing our shelf and customShelf is the name of the class.

We are using the evalDeferred() command, because maya loads the userSetup.py file as it is starting, so we are not sure if the shelfLayout is already in place as it gets executed. Therefore, evalDeferred() helps us call our script after maya has been initialized.

Conclusion

And that is it, you have built your own shelf. What I really like about it is that if you are collaborating with other people you can add the shelf file to your version control system and in your userSetup.py file just source it from there. That way you can all contribute to it and have it always updated.

If you have any questions, shoot me a message.