marking menu

So, it seems like I have been going crazy with marking menus lately. I am really trying to get the most of them, and that would not be much if we could only use them in the viewports, so today we are going to look at how we can construct custom marking menus in maya editors.

Custom marking menus in maya editors - node editor

tl;dr: We can query the current panel popup menu parent in maya with the findPanelPopupParent MEL function, and we can use it as a parent to our popupMenu.

So, there are a couple of scenarios that we need to have a look at, as they should be approached differently. Although, not completely necessary I would suggest you have a look at my previous marking menu posts – Custom marking menu with Python and Custom hotkey marking menu – as I will try to not repeat myself.

Okay, let us crack on. Here are the two different situations for custom marking menus in maya editors we are going to look at.

Modifiers + click trigger

In the viewport these are definitely the easier ones to set up as all we need to do is just create a popupMenu with the specified modifiers – sh, ctl and alt, the chosen button and viewPanes as the parent. When it comes to the different editors, though, it gets a bit trickier.

Let us take the node editor as an example.

If we are to create a marking menu in the node editor, it is a fairly simple process. We do exactly the same as before, but we pass "nodeEditorPanel1" as the parent argument. If you have a node editor opened when you run the popupMenu command, you will be able to use your marking menu in there. The catch is though, that once you close the node editor the marking menu is deleted, so it is not available the next time you open the node editor.

Unfortunately, I do not have a great solution to this problem. In fact, it is a terrible solution, but I wanted to get it out there, so someone can see it, be appalled and correct me.

The second method – Custom hotkey trigger – is much nicer to work with. So, you might want to skip to that one.

What I do is, I create a hotkey for a command that invokes the specific editor (I only have marking menus in the node editor and the viewport) and runs the marking menu code after that. So, for example, here is my node editor hotkey (Alt+Q) runTimeCommand.

mc.NodeEditorWindow()

if mc.popupMenu("vsNodeMarkingMenu", ex=1):
    mc.popupMenu("vsNodeMarkingMenu", dai=1, e=1)
else:
    mc.popupMenu("vsNodeMarkingMenu", p="nodeEditorPanel1Window", b=2, ctl=1, alt=1, mm=1)

mc.setParent("vsNodeMarkingMenu", m=1)

from vsRigging.markingMenus import vsNodeMarkingMenu
reload(vsNodeMarkingMenu)

That means that everytime I open the node editor with my hotkey I also create the marking menu in there, ready for me to use. As, I said, it is not a solution, but more of a workaround at this point. In my case, though, I never open the node editor through anything else than a hotkey, so it kind of works for me.

Then the vsRigging.markingMenus.vsNodeMarkingMenu file is as simple as listing the menuItems.

import maya.cmds as mc

mc.menuItem(l="multiplyDivide", c="mc.createNode('multiplyDivide')", rp="N", i="multiplyDivide.svg")
mc.menuItem(l="multDoubleLinear", c="mc.createNode('multDoubleLinear')", rp="NE", i="multDoubleLinear.svg")
mc.menuItem(l="plusMinusAverage", c="mc.createNode('plusMinusAverage')", rp="S", i="plusMinusAverage.svg")
mc.menuItem(l="condition", c="mc.createNode('condition')", rp="W", i="condition.svg")
mc.menuItem(l="blendColors", c="mc.createNode('blendColors')", rp="NW", i="blendColors.svg")
mc.menuItem(l="remapValue", c="mc.createNode('remapValue')", rp="SW", i="remapValue.svg")
...

A proper way of doing this would be to have a callback, so everytime the node editor gets built we can run our code. I have not found a way to do that though, other than ofcourse breaking apart maya’s internal code and overwriting it, which I wouldn’t go for.

Luckily, creating a custom marking menu bound to a custom hotkey actually works properly and is fairly easy. In fact, it is very similar to the Custom hotkey marking menu post. Let us have a look.

Custom hotkey trigger

Now, when we are working with custom hotkeys we actually run the initialization of the popupMenu everytime we press the hotkey. This means we have the ability to run code before we create the marking menu. Therefore, we can query the current panel and build our popupMenu according to it. Here is an example runTimeCommand, which is bound to a hotkey.

import maya.mel as mel

name = "exampleMarkingMenu"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

parent = mel.eval("findPanelPopupParent")

if "nodeEditor" in parent:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import exampleMarkingMenu
    reload(exampleMarkingMenu)
else:
    popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p=parent, mm=1)

    from markingMenus import fallbackMarkingMenu
    reload(fallbackMarkingMenu)

So, what we do here is, we start by cleaning up any existing versions of the marking menu. Then, we use the very handy findPanelPopupParent MEL function to give us the parent to which we should bind our popupMenus. Having that we check if the editor we want exists in the name of the parent. I could also compare it directly to a string, but the actual panel has a number at the end and I prefer just checking the base name. Then, depending on which panel I am working in at the moment, I build the appropriate custom marking menu.

Don’t forget that you need to create a release command as well, to delete the marking menu so it does not get in the way if you are not pressing the hotkey. It is a really simple command, that I went over in my previous marking menu post.

The obvious limitation here is that we have a hotkey defined and we cant just do ctrl+alt+MMB for example.

Conclusion

So, yeah, these tend to be a bit trickier than just creating ones in the viewport, but also I think there is more to be desired from some of maya’s editors *cough* node editor *cough*, and custom marking menus help a lot.

So, recently I stumbled upon a djx blog blost about custom hotkeys and marking menus in different editors in maya. I had been thinking about having a custom hotkey marking menu, but was never really sure how to approach this, so after reading that post I thought I’d give it a go and share my process.

tl;dr: We can create a runtime command which builds our marking menu and have a hotkey to call that command. Thus, giving us the option to invoke custom marking menus with our own custom hotkeys, such as Shift+W or T for example, and a mouse click.

Disclaimer: I have been having a super annoying issue with this setup, where the “release” command does not always get called, so the marking menu is not always deleted. What this means is that if you are using a modifier like Shift, Control or Alt sometimes your marking menu will still be bound to it after it has been closed. Therefore, if you are using something like Shift+H+LMB, just pressing Shift+LMB will open it up, so you lose the usual add to selection functionality. Sure, to fix it you just have to press and release your hotkey again, but it definitely gets on your nerve after a while.

If anyone has a solution, please let me know.

I have written about building custom marking menus in Maya previously, so feel free to have a look as I will try to not repeat myself here. There I also talked about why I prefer to script my marking menus, instead of using the Marking menu editor, and that’s valid here as well.

So, let us have a look then.

RunTimeCommands

The first thing we need to do is define a runTimeCommand, so we can run it with a hotkey. That is what happens if you do it through the Marking menu editor and set Use marking menu in to Hotkey Editor, as well.

There a couple of ways we can do that.

Hotkey Editor

On the right hand side of the hotkey editor there is a tab called Runtime Command Editor. If you go on that one you can create and edit runTime commands.

Scripting it in Python

If you have multiple marking menus that you want to crate, the hotkey editor might seem as a bit of a slow solution. Additionally, if changes need to be made I always find it more intuitive to look at code in my favourite text editor (which is sublime by the way).

To create a runTime command we run the runTimeCommand function which for some reason does not appear in the Python docs, but I’ have been using maya.cmds.runTimeCommand successfully.

All we need to provide is a name for the command, some annotation – ann, a string with some code – c and a language – cl.

Here is an example

mc.runTimeCommand("exampleRunTimeCommand", ann="Example runTime command", c=commandString, cl="python")

Something we need to keep in mind when working with runTime commands is that we cannot pass external functions to them. We can import modules and use them once inside, but I cannot pass a reference to an actual function to the c flag, as I would do to menuItems for example. That means that we need to pass our code as a string.

Press and release

Now, that we know how to create the runTimeCommands let us see what we need these commands for.

As I mentioned, they are needed so we can access them by a hotkey. What that hotkey should do is initialize our marking menu, but once we release the key it should get rid of it, so it does not interfere with other functions. Therefore we need two of them – Press and Release.

Let us say we are building a custom hotkey marking menu for weight painting. In that case we will have something similar to the following.

  • mmWeightPainting_Press runTimeCommand – to initialize our marking menu
  • mmWeightPainting_Release runTimeCommand – to delete our marking menu

The way we bind the release command to the release of a hotkey is by pressing the small arrow to the side of the hotkey field.

Custom hotkey marking menu - release command hotkey

The Press command

import maya.cmds as mc # Optional if it is already imported

name = "mmWeightPainting"
if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

popup = mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

import mmWeightPainting
reload(mmWeightPainting)

So, essentially what we do is every time we press our hotkey, we delete our old marking menu and rebuild it. We do this, because we want to make sure that our latest changes are applied.

Now, the lower part of the command is where it gets cool, I think. We can store our whole marking menu build – all menuItems – inside a file somewhere in our MAYA_SCRIPT_PATH and then just import it from the runTimeCommand as in this piece of code. What this gives us, is again, the ability to really easily update stuff (not that it is a big deal with marking menus once you set them up). Additionally, I quite like the modularity, as it means we can have very simple runTimeCommands not cluttered with the actual marking menu build. This is the way that creating through the Marking menu editor works as well, but obviously it loads a MEL file instead.

So, literally that mmWeightPainting file is as simple as creating all our marking menu items.

import maya.cmds as mc

mc.menuItem(l="first item")
mc.menuItem(l="second item")
mc.menuItem(l="North radial position", rp="N")
...

And that takes care of building our marking menu when we press our hotkey + the specified modifiers and mouse button. What, we do not yet have is deleting it on release, so it does not interfere with the other functionality tied to modifier + click combo. That is where the mmWeightPainting_Release runTimeCommand comes in.

The Release command

## mmWeightPainting_Release runTimeCommand
name = "mmWeightPainting"

if mc.popupMenu(name, ex=1):
    mc.deleteUI(name)

Yep, it is a really simple one. We just delete the marking menu, so it does not interfere with anything else. Essentially, the idea is we have it available only while the hotkey is pressed.

Hotkey

All that is left to be done is to assign a hotkey to the commands. There are a couple of things to have in mind.

If you are using modifiers for the popupMenu command – sh, ctl or alt – then the same modifiers need to be present in your hotkey as otherwise, even though the runTimeCommand will run successfully, the popupMenu will not be triggered.

In the above example

mc.popupMenu(name, b=1, sh=1, alt=0, ctl=0, aob=1, p="viewPanes", mm=1)

we have specified the sh modifier. Therefore, the Shift key needs to be present in our hotkey.

Also, obviously be careful which hotkeys you overwrite, so you do not end up causing yourself more harm than good.

Conclusion

That’s it, it really is quite simple, but it helps a lot once you get used to your menus. Honestly, trying to do stuff without them feels so tedious afterwards.