“Why is our new tool not being used?”
This is the number one question I get as a tool design consultant. You just spent a bunch of time, money, and energy into building a new tool and your internal artists designers and other customers just do not want to use it. Why? The answer is usually very simple:
Handoffs
There are too many handoffs! Which means that the problem origin is in product and production. What's not a problem is engineering expertise and user feedback. Those are usually pretty fine.
Basically what happens is: There is an artist and they run into some kind of issue. They say “I need help.”. They need a tool, a workflow change, or something has to be different. So that request goes to an art producer, and that producer goes to a producer meeting, and in that producer meeting a tools producer hears about it, and then the tools producer gives the request to a tools engineer.
At that point does the tools engineer talk to the artist? Usually that doesn't actually happen. There's just a bunch of handoffs. So the tool engineer builds something, and they say: “Here's a new tool!” and then they send it off to the tools producer, the tools producer takes it to producer meeting producer, then the art producer gives to the artist and the artist goes: “This new tool sucks!”
Which is terrible! That feedback then goes to the art producer, the art producer goes back to the production meeting, the producer goes back to tools engineer who says: "What the hell? I did what was asked on the ticket that I got from my producer! Why isn't good enough?” and then a lot of people are frustrated and unhappy.
I would say pretty much +90% of studios work this way. I’ve been in the games industry for 15 years, and a lot of them have this problem of too many handoffs. Again, what's not a problem is engineering expertise and user feedback. The engineers are probably really smart and really good at coding, and the user feedback is totally realistic and appropriate to the realistic issues they are running into.
Handoffs have been a problem for a very long time and not just for tools either. For example Joel Burgess, a level designer on Skyrim, did a great talk at GDC 2013 about a similar problem. In his case with level designers and the art team. So they're building a level, and they're constantly handing it off back and forth to either make the level playable or make the level look really really good, and it caused a lot of issues. So we can kind of look at this image from that talk and we can replace ‘Level Design’ with ‘Users’ and ‘Art Team’ with ‘Tools Team’, and we see the same problem. There’s a bunch of handoffs back and forth.
So what's the solution? We still have to go through the production systems. Most companies still have those. Requests have to be checked, and end up on the right backlog, things like that. But once a ticket has gone through, and we have some kind of work that needs to happen, and it's been agreed upon, then at that point engineer and the artist need be singled out.
I'm just going to rename them engineer and customer in this case for ease of use of explaining. So basically those two people: Get them together. Make them literally sit together if possible. Like, move their desks close to one another.
Then at any point when the engineer is building on that ticket and has an early solution, or some early things to look at, the customer can just come right over and take a look. They can ask: “Hey is this good enough? Is this solving your problem? Are we going in the right direction?” They can talk to each other and really find out what needs to be delivered. What the right thing is. They can iterate towards a solution!
The customer can then go back to their work, and at some point the engineer will have something that's functional. Something they can actually use. Cool, get the customer right over there and literally try it out on the engineer's machine. That's really important because basically at this point there are no PRs yet. There's no weird bugs on another machine, like on the customer’s machine there might be way different different plugins, all kinds of different hardware, a lot of weird stuff might be happening, but we're not dealing with that because everything is on the engineer's machine. This also means there is direct debugging! If there is a crash or an issue or anything like that, we go right back to Visual Studio or Xcode on that engineer's machine. And, there is direct feedback from the customer. They can say: “No, this is not going in the right direction,” or "Oh yeah this is kind of going in the right way! What about this, what about that?”. If you're remote, you can screen share it, or even let the customer take remote control of the engineers's machine.
So the engineer and customer basically go through this process continually building things, trying things out, building things, trying things out, and at some point the customer will use a thing and they will go: “This is nice!”. Awesome, so at that point they split ways again. The engineer goes back to their team and says: “Hey check this PR.” You know, is the code good, do we need to make any changes, etc? We know what we're needing to build and we know what the solution is that the customer requires, so that makes it much easier. The customer goes to all the other customers and says: “Hey this really new cool tool is coming!”. So now you have an advocate inside that team! They will say: “I tried this new tool, it's awesome, so when it comes out please use it!”. Now we're no longer wasting that precious engineering time to build something that the customers don't need, because we know what they need, and we've checked it with a customer.
At that point you can go back to the regular production process. The engineer goes gives the new tool to the producer, the producer goes to producer meeting, the producer meeting to the art producer, and the art producer takes it to the customer, and the customer says: “This is awesome!”.
And everyone lived happily ever after.
Of course there are more factors to this, this is just quick article to explain a particular issue that I've seen many times over my career. There are other things you can also take a look at to make this even better, for example:
Having a clear goal for the tool
Having a test scenario for the tool
Make sure QA can smoke test this tool
And be very careful of that scooter the car image that everyone loves to use.
(I made a separate video & article about that one because it's a longer story.)
Now some questions that often come up when I talk about these things are:
What if everyone super busy?
These artists, level designers, and engineers don't have time to do all this talking. They have to be constantly working.Who has knowledge of the overall scope?
If we have this one tool for this one particular thing, how do we know if it fits within all these other workflows and tools?What if there are multiple tools being built? Do we just constantly sit everyone together with everyone else?
We don't have the space, or there's just too much effort, or it doesn't fit well with the project we're building.
Sure, all those things could happen. If you go back to this idea we've gone through of the loop the artist issue makes, and it has landed on the tools engineers lap, then at that point it can sometimes be really good idea to have another person there too. A tool designer. And that tool designer can then facilitate the actual creation of the right thing.
In this case let's rename them again to engineer and customer. The tool designer now owns the workflow. They own that in the way of ‘This is going to be good and it's going to be nice to use, and it fits within all these other tools we have, and if it doesn’t, it’s my fault.’ So that tool designer gets the engineer and customer together. They talk and talk and talk, and the tool designer figures out what needs to be built to solve the problem best.
Then everyone goes back to their desks. The tool designer then designs the workflow. They understand the scope of all the other workflows too, and the other tools, and what needs to happen for the customer to be happy. The tool designer also has designed many tools before, they know what a good tool looks like, what problems users will frequently run into, etc. Then when they have that initial design, and this is just in Figma, on paper, or even just in text, they again grab the engineer and customer and go over the concept. What is good, what does this do, is this thing easy to build, or hard to build, etc. They ask the engineer if it takes months, or weeks, or just a day to build. And they ask the customer does this solve your problem 70%, 80%, or 100%? And then they all have a general idea of the solution. They will know if this is an easy thing that solves a lot of the problem, or a hard thing that solves little of the problem. They will know what is worthwhile to do, and they can figure something out together if anything needs to change about that concept. Then, if all goes well, it's approved! Awesome we know what we're building.
If there are multiple tools or multiple engineers working on things, then the tool designer can do all the leg work. They can go to the engineers, go through issues or revisions with them, etc. The tool designer knows exactly what the customer wants for this tool or workflow, and the tool designer has the time to do that conversational work. So the customer can keep on doing environment art, or tech design, or whatever it is that they need to be doing. They can keep on doing the work and do not need to be bothered as much because the tool designer is doing all that leg work. The tool designer can also go to the customer if there is a question, something has changed, the engineer found something was too time consuming to do, etc. They make it nice and easy easy to talk to everyone. Which is why especially people at Guerrilla and IO-Interactive, the previous studios that have worked at, know that I've often mentioned: A tool designer not at their desk is a good tool designer.
They are gathering information, they're talking to people, and figuring out what the right thing is to do. In helping those people build the right things they are saving everyone else time. So they do all that work, we see that same iterative loop of trying things out on the engineer’s machine, and then at some point we have a finished tool. Awesome, so everyone goes to the engineers machine machine, because we still have that same bonus of no PRs, trying it out on the engineers machine, all that good stuff. Then, hopefully if everything went well, the customer will say “This is awesome!”. The customer is super happy, and then again they split up.
At this point what is also really helpful is that the tool designer has seen the whole process. They have seen what the engineer did, what the customer did, and they can then document that workflow. They can write everything down so that everyone else in the studio also understands how that tool works, and how those workflows work. Again this is possible as they have the time to do that. The engineer is super busy coding stuff, the customer is super busy building levels, and the tool designer has the time because there's a defined role for that work.
At this point the tool designer gets out of the way again. We go back to this production system of an engineer telling a tools producer about the new tool, the producer takes it to a producer meeting, the producer meeting to the art producer, and then then the customer uses it and is happy.
So when people are asking me: “Why is our new tool not being used?” and they want to solve that problem, then the takeaways are:
Prevent many handoffs.
If you have too many handoffs you're not going to deliver the right thing to the customer.Make an engineer work directly with a customer.
Make sure that they sit together and really look at what's going onIf you have a big scope: Hire a tool designer.
They will solve this problem for you.
Thank you for reading! If you enjoyed this post, you can find my talks here, and my other posts here. You can also subscribe below to get updates when a new post is published. There is also a video version of this article that you can find below.