An Artist’s Guide - Learning Python Scripting for Animation.
A guide in how to approach learning Python scripting as an artist in an animation pipeline.
You should learn Python scripting, it will be super useful for you!”
It's an advice I’ve heard many times over the years. By people who are technical wizards, and people who have never written a line of code before, but like the idea of it. Sure, it’s a solid advice, it would be super useful for someone in the team to do some scripting, but not me. Let’s recommend it to someone else instead, so I don’t have to.
Despite enjoying the challenge of learning new skills, I’ve always told myself I would be incapable of learning scripting. I am bad at learning languages, and I have a terrible memory. And I considered myself a visual artist, writing code is not me.
And now I’m writing an article on how to learn scripting- ...wait. What happened?
The Problem
In my final year at The Animation Workshop, I worked on two short-film projects; Monachopsis and Forget-Me-Not. The projects had around 110 shots combined, and 440 scene-files.
I desperately needed a faster alternative to "manual labor" for creating, managing, and rendering all the scenes.
Imagine you’re setting up 10 or more shots in Maya. Your assets needs to be referenced into every single scene-file, all with consistent settings and namespaces. And your scene needs AOVs, correct output settings and more.
Doing so by hand would attract human errors as you’d go on auto-pilot, guaranteed to cause problems down the line. It is also harmful to your wrist and schedule.
Every time you update an asset, you’d have to open each scene-file and submit a render to the render-farm. A punishing and time-consuming task that would defeat the purpose of an iterative workflow. Worst case, only opening the scene itself could take a few minutes, depending on the project.
As I was the only person working on Monachopsis, I could not selfishly delegate this busy-work to someone else. Despite having no interest in learning scripting, the alternative was far worse. That’s when I started to learn Python in my spare time during production. In the weeks and months that followed, I had managed to automate most of my pipeline.
It turns out... learning how to script is super, super useful!
How to approach the learning curve
In this article I will guide you through my suggested the learning curve for Python scripting. This is based on my experience learning it, and how I would teach it to myself to get the same results.
While I cover some basics, this is not intended as a tutorial.
There are far better resources out there for that.
My goal is to hopefully make it less scary for others to start learning and getting useful results sooner. I will use a simple pipeline in Autodesk Maya as context for the examples here. But the approach to learning it is similar in other applications for the most part.
Before you start
Before you decide whether scripting is not for you or not, there are three points I wish to emphasize.
You don’t have to learn everything.
A small amount of scripting knowledge will take you a long way.
The first steps into learning scripting does not have to be a big commitment into becoming a hobby-programmer. Think of it as a practical tool in your toolkit, allowing you to approach problems from a different angle.
You don’t have to memorize everything you learn.
Focus on learning how to find the answers when you need them.
While remembering everything is a good advantage, it is more important to know how you find the solutions in the first place. Even if you later forget your solution, you’ll feel comfortable knowing you can find it again later if you need to.
Ask any programmer how often they’ve had to Google the basics while solving an advanced problem. I’ve never met any programmer who, to my knowledge, relied on their memory alone. Google is a great tool, learn to use it.
You need a problem to solve
Everything you learn needs a context to be interesting.
When you’re taught new skills, you’re often taught the solution to a problem you don’t have. This is especially true for scripting.
Your first lesson is often how to make the software say “Hello World!”, which is not super inspiring. A teacher might prove how easy it is to make a polygonal cube in Maya using Python. But for your needs, it isn’t easier than clicking the “make cube”-button.
The basics are not fun to learn, but you need to start somewhere. It does not feel rewarding, as it does not solve a problem you have, and you can’t imagine any uses for it… Yet.
Find a problem to solve first, rather than trying to create new problems to make use of the solutions you’re taught.
Once you have a goal, any otherwise boring topic can become the most interesting part of your day. Every bit of knowledge feels like a treasure clue, as you know it could be a useful step towards the solution.
Level 1 - The First Steps
Be kind to yourself, start simple.
It’s easy to get overwhelmed by everything that goes into building a strong foundation to learn a new craft. You want to do things the right way, because there are so many ways to mess it up. And it’s true, it’s easy to make mistakes.
My advice would be to ignore those fears, and start getting your hands dirty.
Nobody plays their first instrument by reading musical notes.
Nobody makes their first pencil drawing by reading a book on perspective construction.
Unlike a drawing, a script either works, or it doesn’t. This already makes it harder to learn as a beginner and feel rewarded for your efforts. Anything you can do to feel less pressure over succeeding in a particular way, the better.
Have fun with it, as long as it works. The foundations can be taught later. Approach them on a need-to-know basis.
Problem #1: Referencing assets into shots.
In addition to Python, Maya has its own scripting language called “MEL”.
Both languages have access to the same software-commands, but they’re written with different syntax or “grammar”.
I strongly recommend using Python, as it will prove to be much more powerful and flexible later on. But MEL has one big advantage for beginners.
Inside the Script Editor, whenever you perform an action by hand, the action will show up as MEL-code. If you perform your action by hand once, you can simply copy and paste that line of code into a shelf-button. Now that button will do the exact same command with a single click, instead of 20.
Congrats, you’ve made your first functional, time-saving script!
Next time you set up a shot scene by hand, such as referencing assets and changing output settings, copy your actions into a single shelf-button.
A boring and time-consuming task is now reduced down to a single click, with minimal effort. Very satisfying.
(Note: Not all interfaces gives you MEL-commands for free. Like the output resolution in Render Settings. This does not mean they can’t be changed by scripting, but it requires a little more research.)
While MEL is a great starting point for these simple functions…
It is a horrible, horrible language to actually write and use if you want more complex code. So I recommend translating these MEL-commands into Python using the documentation. Once you learn the pattern, it’s easy to do.
Learn to read the documentation.
Familiarize yourself with the documentation, and treat it like a dictionary. It is the best resource you have for learning to manipulate a software using Python. They often provide a lot of example-code that you can copy and edit to fit your needs.
Knowing where you can find the information when you need it is important to learn any new skill. Understanding the documentation is not always as easy as you'd hope for. Sometimes it requires a deeper understanding of some concepts than you currently have. Identifying these weaknesses are tough, but it invites you to research and gain more knowledge.
If the documentation is unclear, search for articles and tutorials explaining the concepts in a different way. Dive into the rabbit whole. Don't get discouraged, your perseverance will be rewarded.
Level 2 - Taking the dive
Learning to write basic Python code.
So far, your scripts have been a series of step-by-step actions. “Do this, then that.”. And you can get pretty far with that.
But your scripts becomes more powerful combined with Python’s built-in functions. Use them to complement the software-specific commands you’ve been using so far.
Here are some things you should learn about to start off with:
- Basic object types
- Functions and variables.
- For-loops, if-statements, string formatting, and dictionaries
Now we can write:
- “For every object I’ve selected, do this to each object.”,
- “If something is true, do this. If not, do that”.
Again, keep it simple. You don’t need to learn a lot of Python’s functions to super-charge your scripts. Explore, and get creative with the tools you have. There's enough here to build a small pipeline for your projects.
Break down complex problems with pseudo-code.
When you’re writing code, what you’re essentially doing is writing very precise step-by-step instructions for a robot to follow. It's only written in a language the computer can understand. Sometimes, a tasks proves too complicated to explain to a robot, (or a team-member). So often you end up doing it yourself instead.
In most cases, the complicated solution is a collection of many simple solutions.
When you write pseudo-code, you write the code as many small, simple steps in your own language. Like a cake-recipe.
Doing so will break down a complex operation down to smaller and more approachable problems you can solve individually.
Once you have your instructions ready, translate each step into Python. Google for a solution to each step, if necessary.
Problem #2: Caching out animation
After importing a character rig into some shots in Problem #1, we want a script to cache out animation as well.
By doing the process manually first, we can identify some new challenges.
- Each shot and asset requires its own export-path.
- We need to know the name of the shot we’re caching out.
- Alembic-caching uses a funky string of commands, instead of arguments.
- That string requires the absolute path of the geometry to export.
- We have to make the export-folder if it does not exist.
- Get the frame-range to export.
That made the task a lot more complicated right away, but don’t panic!
Before we build the whole script, we’ll try to find a solution to each problem first. How we solve each problem requires both a bit of research and creativity. There are multiple ways of approaching the problems, and as you get more advanced, you’ll find more elegant solutions.
Challenge 1: Making export path
The export-path is the same for each file, except the name of the shot.
If we make a template path, we can simply replace the part we need to change, and keep the rest. Googling “python replace part of a string”, gives us the solution:
Challenge 2: Finding shotName
Assuming you have a proper naming convention on your files, your shot scene probably has a “sh0010” or similar in its filename. We can search for a “sh****” in that file path to find the shotname. Each * represents an unknown number. Unknown characters in a string is called a wildcard.
Googling "python search for wildcard in string" gives us a clue to look at "Regular expressions". This is a bit more complicated, but it puts us on the right path.
Challenge 3: Creating command-string
Strings like this can be built by using string-substitution, which inserts variables into a string.
Challenge 4: Getting objectPaths
For each object you export, Alembic needs a string with the object’s absolute path in the scene. Example: “-root |GEO_GRP|body_GEO_01”
This is because two objects might have the same name, so we need a way to identify a unique object. Again, Google “Python Maya get full path”. The top results usually have a good clue. The "cmds.ls"-command with the "long"-argument gives the right path.
Challenge 5: Making folders if they don’t exist.
Search Google for “python make folder if it does not exist”. Sensing a pattern yet?
Challenge 6: Get frame range.
When we manually change the start frame in Render Settings, we get the following MEL-code: setAttr "defaultRenderGlobals.startFrame" 1;
setAttr means “Set Attribute”, so we assume there’s also a “get attribute” command somewhere. "startFrame" is the attribute, and "defaultRenderGlobals" refers to the node the attribute is attached to.
By looking up the Python documentation for Maya, we find the solution:
startFrame = cmds.getAttr(“defaultRenderGlobals.startFrame")
With all the individual challenges solved, we can make a pseudocode based on what we’ve learned. We can then build our full script step by step.
Level 3+ - No turning back
Once you’ve gotten comfortable making more complex scripts, it’s time for a sanity check. Now you need to take a step back to the basics, and establish the foundations you might have skipped earlier. And prepare for scaling up your projects.
Organizing code
As your scripts and tools grow in complexity, you need ways to stay organized using Modules and Packages.
A module is a single text-file containing python-code saved on your computer. The name of a module is the filename excluding the .py/.pyc file-extension.
A package is a collection of modules, stored inside a folder on your computer. The folder must also contain a "__init__.py" file to be recognized as a package.
You can access the code inside these files by using a python importing command:
from myLittlePipeline import shotTools
shotTools.someFunction()
In order for the python-scripts to find these modules and packages, they need to be inside a folder where Python knows to search for files.
In Maya, one of these locations are inside your scripts-folder:
C:\Users\USERNAME\Documents\maya\2018\scripts.
You can specify custom folders for Python to search for scripts in.
I recommend making a Dropbox-folder, so you don’t lose your code.
Consume a lot of surface-level information
One of the challenging things about problem-solving a new craft, is that you don’t know what tools you have available to provide a solution.
If you try to build a house with only a hammer and nail, it would be a very slow and frustrating experience. Even if the voice of reason in the back of our heads tells us “There has to be a better way”, we often ignore it and power through.
Motivated artists have an amazing ability to force a good result with inefficient solutions, using sheer force of willpower and determination. Sometimes to a flaw. It’s only after the house is finished, you discover you had a nail gun and table saw available at your finger-tips the whole time. You just didn’t know it was an option.
When I learn new skills, I try to get an overview over available tools early on. I consume a lot of surface-level information, without actually trying to learn it. It's like browsing the hardware store for power-tools I need for a project. Except the tools are free.
By skimming through lots of tutorials and articles, even for the stuff I don’t need right away. Next time there's a problem, I have more ideas on what to research to find a better solution.
Using an external text editor
Writing and testing code can be a frustrating process inside a software’s script editor. Especially if the software crashes due to testing out faulty code, and you lose the code you’ve written. The script editors are not built for writing more complex scripts.
I strongly recommend using an external text editor as early as possible. They provide auto-completion, diagnostic tools and other features making it easier to write and test code.
Some text editors, such as Sublime Text, offers plugins allowing you to execute code in Maya with a hotkey. If Maya crashes, you still keep your code.
Building interfaces.
Giving your scripts an interface allows you to write more flexible and user-friendly tools, and use them in a team-environment.
Many applications provide Python-commands for building interfaces, offering a good starting point. But I will recommend to eventually learn about writing interfaces using “Qt”.
It is the codebase which many applications are built around under the hood, including Maya and Nuke.
Learning to write Qt-interfaces is another big learning curve, but it allows you to make software-agnostic code. This especially useful if you’re planning on expanding your toolkit to be a bigger pipeline. You can research “Pyside2”and “PyQt5” if you wish to learn more.
As a general rule, you should keep function and interface as separate as possible. No tools should rely on having an interface to function.
For simple tools, the interface is only a more convenient way to access the underlying functions, and display data intuitively.
Problem #3: Refactoring.
Once you progress and learn more efficient solutions to old problems, it is a good idea to refactor your scripts. Refactoring is the process of rewriting existing code, without necessarily changing the overall functionality.
This becomes a crucial step as your projects grow in complexity.
Badly organized code can and will slow you down considerably, perhaps even to a complete stop. In this example, we’ve made some progress by splitting the code into reusable functions. We’ve also made a small asset database using a dictionary. This database contains all the info we need from each asset in order to reference and cache out from a shot.
After putting the script it is own package and module, we can reference and cache the different assets by just writing simple commands in Maya:
The asset-dictionary can later be saved out and edited as a .json-file on your harddrive. This allows your asset-database and pipeline code to live separately. For projects with different assets, you can reuse the code, while accessing a different asset-database.
The next step would be to develop an interface to access and edit the data in your asset-database. Throw in a few buttons to use your referencing and caching functions, and you've developed a basic pipeline! It might sound complicated, but it is a good place to start.
Some additional resources:
The Hitchhiker's Guide to Python &
Chad Vernon - Python Scripting for Maya Artists
Reading through these two websites will teach you everything you need to know to get started. These proved to be invaluable resources to me starting to learn Python.
Dhruv Govil's gitHub
His Github hosts a lot of scripts you can open and study from. He writes comments for every part of the script, making them excellent learning materials.
His video tutorials are also excellent!
Python 2.7 Official Documentation &
Maya Python Command Reference
No but seriously, use the documentation. Bookmark and buy an extra monitor for them.
Conclusion
If you’ve made it this far, I hope you feel encouraged to learn a bit of scripting yourself, if you haven’t already. While I’ve tried to make it seem less intimidating, I will not say it is easy.
The learning process is unlikely to be as straight-forward as this guide implies. It is also important to remember that we all learn differently. You might need to find your own way of learning it, but I hope this guide provides a reassuring starting point.
While it is challenging, it was nowhere near as difficult as I thought it would be either. And the benefits have changed the way I work and solve problems. Learning to script, even just a little, feels incredibly empowering.
After only a few months of learning a small part of Python, my entire pipeline was mostly automated. I could make a new shot from scratch and take it through the entire pipeline to first pass composite, in 15 minutes.
Anything after that, is time spent only making the shot look better.
In the production of Forglemmegei, the main character’s model needed to be fixed after all the animation had finished. This required us to re-export animation caches for all the 60+ shots. A task that would delay the entire team’s schedule and result in more crunch if done by hand.
With Python, I could write a script in 2 hours, and automate the process overnight.