Figma has won: Notes of the Tool Design Roundtables at GDC 2024
I hosted the Tool Design Roundtables at GDC 2024, and it was great fun to do so. We have a lot of good discussion about many topics folks were wondering about. As I host during the full duration, I do not have the time to write full notes at the same time, but nevertheless I like to write about what I think are the neat highlights of the discussions.
For those who have never attended one: A roundtable is a room filled with mostly game developers, from places like Blizzard, Riot, Epic, Unity, and also indie studios and solo developers, as well as folks from Uber, TikTok, Perforce, etc. Those folks bring up their own questions, and then we discuss them as a group. Thank you all very much for coming, and sharing your experiences.
You can decide to read or watch this article. In the video below I go over the highlights, or you can go further below and read the full notes at your own pace.
Tools Design Roundtable Day 1: Design & UX
A photo of the board after the roundtable ended
What User Experience and User Interface Design Tools do folks use?
I always start out with some quick polls on what software folks are currently using for UX & UI design. I ask who uses Figma, Sketch, Miro, Adobe XD, Balsamiq, Axure, Framer, Illustrator, and any others. Figma won by a huge margin, about 95% of the room. This is very interesting to see as about 5 years ago there was still much more of an even split. Figma is dominating the market, and I think for very good reasons: Figma is easy to use, it is remote friendly, prototyping is made incredibly easy, and it is not that expensive. Congratulations to the folks at Figma!
Best things & Worst things
Then I ask folks what some good things and bad thing about tools design that happened to them in the last year. Two interesting things came up that were very opposite: Folks really like Figma for the prototyping workflow with variables, and for how beautiful something can be made to look quite easily.
However, on the flip side folks mentioned that often the components in Figma may be beautiful, but the actual implemented UI may be way off from this beauty. So you may have something gorgeous being approved in Figma, but then it is unsure if the real tools will actually look like that in the future. This caused some friction in the projects of a few folks.
What User Experience Research tools do folks use, and do they involve AI?
Next we discussed UX research tools, with Dovetail coming up initially, and folks wondering if anyone used AI for their research. The only one that came up a few times was that Miro’s AI tooling is apparently pretty good for collating notes. The idea being that you write down whatever you want wherever you want in Miro board, and then ask the AI to find the connections and collate it for you. Apart from that, no big winners came out of this conversation apart from the old classic: Sit behind folks, and write down notes. Keep asking them why, why, why, why, why, and you can get much better feedback that you can work with.
How do you measure success in Tool UX?
Next, a classic question was asked: How do you measure success? How do you know you have made the right tools? Around the room many good metrics were brought up: Were customers happy? How many of them were using your tools? Were they able to achieve their tasks in time? Were you able to build the tools in time? How much time was saved? How much did quality, or customer reviews of built content, go up? Many of these, and more, can be used depending on what qualifies as a success for your project and studio. Though I felt one was missing near the end, which I think is the most important one: Were you able to deliver your product on time and at the quality necessary? Not the product of the tool, but the product that is made with the tool. So if you develop a videogame with these tools, did a videogame get made on time with the quality that was asked for? Because depending on your studio, you do not ship your tools, you ship your product made with the tools. In that case the final end product is what matters most.
One other thing also came up: Nobody likes using NPS. Folks were being asked to use it within their studios, but nobody thought it really gave them enough information to comfortably take action on. It is good to give you an initial indication of what to look for, but it will never answer your questions directly, though some folks were asked to do that anyway and get conclusions from it, which made folks unhappy.
How can you allow users to edit your UI?
A great question to come up, as I also gave a talk this year about why you should allow your users to edit their tool UI. Nowadays, many tools such as Unity and Unreal allow you to set up your own interface in some way. Though some allow it via editing .ini files, and others by dragging & dropping. And dragging & dropping still wins out. Nearly everyone knows how it works, so it is easy for them to do, and doing some ghosting to show where an panel or object would end up is relatively easy to do while paying incredibly dividends for ease of use. Furthermore, having some defaults and profiles to choose from initially, so folks can use familiar interfaces depending on whether they are used to Maya, Blender, 3dsMax, Houdini, or something else, is helpful. Drag & drop still wins out though!
What are the challenges of a design system?
Someone asked what the challenges are of a design system, and the two main ones that came up from the room are:
Getting it started at all
Upkeep
Another thing that came up was that most of the room did not have a design system for their tools! This really surprised me. I then asked who had a design system for their games’ UI, and many more hands were raised. Considering how as developers we spend 8+ hours a day for 4+ years in our tools, having a design system set up with components that are easy to use and re-use is very helpful. The amount of time and money you can save, as well as how happy you can make your fellow developers, is very useful.
Getting a design system started at all was a big issue though. Getting the funding for someone to really spend a lot of time and energy on one, for the initial creation, seemed to be a big ask for most managers and leads that had to approve something like that. At the same time, once it was built, making sure the upkeep for it was done rigorously, so that folks always used the latest version and could rely on that UI to be the one they had to use, was the other challenge. That kind of upkeep is costly in time. In general the time investment in design systems is the main challenge, but the dividends once it is up and running are very helpful depending on your projects’ needs, scope, and team size. If your team is very large, then this becomes a much easier ask. If your UX team is 2 people, then it may be harder to get that funding and upkeep running.
The Unity UI Toolkit, who uses it, and what for?
Lastly, someone asked who in the room uses the Unity UI Toolkit, and what for. Turns out not many folks were using it. Unity UI and IMGUI still were the ones mostly used.
Tools Design Roundtable Day 2: Production & Strategy
A photo of the board after the roundtable ended
Best things & Worst things
Some folks mentioned being happy that they were finally able to hire a dedicated UX designer for their tools. I am so glad to hear that more and more each year! Back in 2014 and 2015, when me, David Lightbown, and others started talking more publicly about Tool Design having to become a thing in the games industry for our toolsets and our games to be better, there was a lot of hope. Nowadays, in 2024, we are seeing that hope solidify in jobs, happier users, and Tool UX Design no longer being seen as something strange, but a regular job that companies hire for!
Some bad things that came up was proper prioritization still being very hard to do, and that folks found it hard to communicate with remote teams across time zones.
When to use UI and when to use command lines, for features in tools?
The main thing that came out of this conversation is that it always heavily relies on your target audience. The tool itself does not matter as much as the users do. In the end, the goal of the tool is for the users to achieve a goal with it. If they are technical enough, and are comfortable with command lines, then that can work fine. In that case, starting off with scripting tools in which they can create their own tools and workflow can be useful, as they will build what they think they need. But, it is good to keep in mind that the amount of folks in the world who can use command lines is significantly smaller than the amount of folks who can use a simpler user interface with buttons.
Post ship changes by the user: What to do with them?
You have shipped a tool, and the users edit and change it to fit their needs. How do you deal with that? The main thing that came out of this discussion was to learn as much as possible from why users have done the things they have. Some folks in the room did not trust users to make changes, while I had given a talk the day before about why it is important for users to be allowed to make changes.
Changes by users are sometimes like a long-term user research method. You give users the tools, and months later you come back to all kinds of changes. Asking why they have done so, what they wanted to achieve by doing that, and finding out if those changes were intended to be permanent or for the short term is very useful. Maybe the documentation was not clear, and they made changes that were not necessary. Maybe you did not anticipate a particular use case, and so now users had to resolve that themselves. Maybe the project goals changed a lot since you gave them their tools. It is a great learning opportunity.
Empathy for users: How do you get it?
This got a bit of a laugh from the room at first, and it is a good question as many of us deal with trying to get empathy towards users, especially those higher up who have decision making powers of priority. Many folks have natural empathy for what users have to deal with, and some really need to build towards it. How do you convince folks to have that empathy? This mentioned were: Really seeing what users are doing, what they are running into, and what issues they are having, right there as they work.
Many folks in the room mentioned having an open door policy, and office hours, for folks to come in and talk about their experiences, but the problem is that this creates self-selection. Only those willing to walk through the door are the ones you hear about. The biggest thing that came out of this conversation is what we called the ‘Bustin’ Doors Policy’: You have to get out there and see what folks are doing with your tools. How are they using them? Why are they using them? What issues are they running into? What is frustrating them? You have to go and find out. Go join their meetings, go sit next to them, go and talk to them. Do not just sit there and wait until you hear something, as at that point it may be too late already.
Warning dialogs: How do you do them well?
This question was essentially about destructive workflows. What if an action will remove something? How do you make it very clear that something will be deleted? Many users will simply click ‘Yes’ or ‘Ok’ without reading. That is not their fault: It is human nature to do so, especially when we are all swamped in cookie notifications and EULAs that everyone simply accepts to make sure they can continue using the software. So how can you make a warning dialog really grab attention? Some things that came up were red buttons, red text, a shake of the window, clear copy on the button that indicated a point of no return, etc.
But then someone brought up a fantastic point: Why are you being destructive at all? Hard drive space is cheap! Save it somewhere, so that if a user is destructive in some way, they can always undo it somehow. Houdini and Hammer save many backups in a separate backups folder, for example, so that no matter what you do, you can always load a previous version and copy something over that you may have deleted. Which brought the room to a classic point: Don’t have the problem in the first place, and you won’t have to look so hard for a solution. That may sound simplistic, but it is true. As we say in Dutch: “Voorkomen is beter dan genezen.” (Prevention is better than curing.)
Hybrid work: How do we deal with it when designing tools?
One great thing of folks working from home, is that everyone now has a webcam all the time. So if you want to do some user research recording, having folks record their screen and face is super simple. And if folks work for very long hours, a timelapse from hours of work into 10 minutes of video can be a great initial indicator of where folks get stuck, what features they need, and what features they often use.
Another problem that was mentioned was how to stay in touch with your users. Folks talked about making Slack channels for folks to join and talk in, or holding particularly office hours for folks to join in on, but the reactions to doing such things were mostly moot across the room where folks had tried it, and it had been good but not a breakout success. Then, someone brought up the different way of thinking again: Don’t let your users come to you, make sure you come to your users. So instead of an ‘Open door policy’, have a ‘Breaking down the door policy’. Actively burst into meetings and conversations with the folks using your tools. Which, in this day of hybrid work and digital meetings, is much easier. You can essentially teleport from meeting to meeting.
Also, if you need to teach folks how to use your toolsets, then give tutorials in gif or video format. Rarely anyone reads documentation. An image, a gif, or a video, are much faster to transfer knowledge in, especially in remote work places. A gif or video in a Slack channel is much more readily seen than a 5 page document.
How do you know what your Minimum Viable Product (MVP) is?
A classic question: How do you know you are building the right amount, not too much, and that folks are able to do just enough with it? Or that you have just enough to test with to see if your assumptions are correct? You do not want to ship something so minimum that folks will disregard your tools entirely for not having the features they need. The correct, and yet annoying, answer is that: It depends. It depends on your project, your studio, what achievements are necessary for the company to survive, and to thrive, etc. You have to take a deep look at your userbase, and understand what they are able to do, how they are able to do that, and then look at what they need to achieve. What is the minimum you can get away with to make sure they can achieve the base needs? It sounds really basic, but it is in essence all simple hard work. This takes a lot of time, and saves you time later on.
Also, try to find the power users who do not mind ‘bad’ tools as much initially. Those who are able to squint and see what you are trying to do with an initial prototype. Send them early builds, alphas, betas, and screen record what they do. They can be your initial guiding lights if you are finding it difficult to know when good is good enough.
One personal note from me though, is to be very careful about what you qualify as ‘minimum’. It is very easy to just keep cutting and cutting into features, especially over a long time, and not notice you have lost sight of what is really necessary. Be very careful about what counts as minimum. Quite often new tools do not get used by the userbase, and the reason for that is because the new tool does not have feature parity with the old tool. That parity has to exist first. Who is going to trade in their old car for a new car with less features?
Localization: Who does it, how do you do it, and why?
I asked for a raise of hands to see who has localization in their tooling, which we simplified here by talking mostly about languages. Only a few hands went up. Then someone brought up that while they did support other languages, they noticed no one used those features. This was widely agreed on within the room: Most people seemed to use English, and so that is what folks built for and used most of the time.
I have to note here that this conversation happened at a conference in North America. It is difficult for folks in other countries to join this conversation, either due to cost or visa issues. I would really like to do roundtables like this at other conferences in the world to see how different the answers would be.
Tools Design Roundtable Day 3: User Research
A photo of the board after the roundtable ended
Best things & Worst things
Multiple folks were very happy about having telemetry in their toolsets. It made it easier for them to see what users were thinking, what they were using, and thus they could reach out about issues.
Some bad things were that a new tool was made by someone, and folks did not want to use this new tool. They still used the old one. Folks also mentioned it is hard to get feedback on tools: How do you know you are not giving the loudest person in the room all the power for change? We discussed these, and more.
User research tools
This topic ended up being very similar to the one in Day 1: Dovetail was mentioned, and mind map managers were mentioned as well, with Miro’s AI features coming up again as being very useful for quickly collating a lot of thoughts.
Google forms was still the most used tool out of all of them by far, as well as just plainly keeping notes while observing users, and recording users. The best research is the direct observation of user behavior. Asking them question can only get you so far.
Metrics: We use NPS, what do others use, and why?
Many folks used NPS, and so we asked: Do folks like using NPS? And no hands went up.
Which brought us back to something someone mentioned on Day 2: The ‘Busting open doors’ policy. Go find your users! Join their meetings. Hear what they have to say, see what they have to do, and it becomes much easier to see what you need to change for a positive impact.
However, considering everything below a 9 in NPS is considered not good enough, some folks switched over to CSAT instead: Customer Satisfaction Score. In simple terms, do your users feel Positive, Neutral, or Bad about your product? It is a choice between those 3, or sometimes 5 options, and that can give you enough to then look further into. There was also mention of the HEART framework by Google, and the CASTLE frame work of the NNgroup, which you can read up more about here.
In my opinion the most important metric is still: Are you able to ship the content you need, at the quality you need, in the time you need? If no, then you need to change your tools. And if you cannot change your tools to fit that, then your project has to change. That, or you can crash and burn trying to make something you can never make, which is a recipe for burnout and redundancy.
How do you make your users accept change?
This felt like a bit of a harsh question, but it can be very frustrating to make tools that folks then not end up using. The main, and very basic and crude way to solve this is: Make your new tool good enough. I know it sounds very simplified, but your users will not throw away a well working tool for a less well working one. Do you have feature parity? If you have built something, no matter how much effort and time you have put into it, if users do not like it then they will not use it. That is not the users’ fault.
Other things that were mentioned were getting early adopters in who are happy to try some tools out that may not initially be optimal, to build up goodwill that they can then share with others users. Kind of like “Hey, I used this new tool, and it’s quite neat and good!” which can then help others start to use it too.
A fun one that was also mentioned was: Bribery. Give folks cookies if they use your new tools! Set up a playtest of your new tools, and have snacks at the ready. Multiple folks in the room mentioned this as being very effective.
Then, lastly, someone mentioned the negative reinforcement of a time bomb: Every time a user clicks on the old tool, while the new one is still available, it will tell the user the old tool is going away in time. Possibly even showing a date and time, or even launching the new tool instead of the old tool if they do not click a countdown warning fast enough.
That last one seems like a very negative interacting to me, but someone did mention it worked really well for them, because they could go talk to folks who clicked away the warning very fast. So they could find folks who really, really did not want to switch to the new tool. Then you bust open their door, and talk with them to find out what you can do for them.
How do you collate research well?
Someone mentioned they had done a lot of research, collated it, and then nothing happened with it for months. Others mentioned they had also ran into this.
One thing we found that worked around the room was to have the research be extremely easy to reach. Sometimes even in the tool itself. So you have links to Confluence, Miro, or where ever else that contains the research, directly within the tools, so that folks find them much harder to miss. This is of course much easier to do with in-house tools, and not as much with public ones.
The main thing that needs to happen for research to get enacted upon is for higher-ups to see the research, to understand the research, and know they need it to make better decisions. Any way for the research to reach those who can actually make decisions is the really important part that can make a difference. Whether that is via email, presentations, their peers, or their direct contacts. This may sound strange to some. They may ask: “Isn’t it up to the directors and managers to know they need that data, to read that data, and to then act on that data, especially if they requested it?”. In a perfect world, yes, but we do not live in a perfect world. We have to deal with the hand we are dealt, and sometimes that means a lot of managing upwards.
How do you create a culture of design
Someone asked how you can make a good culture of design at a studio. For folks to see, understand, and know that design is necessary for good tools. Which brought us back to the classic of Steve Krug’s Don’t Make Me Think: Above all else, observer your users. Do user tests. If there is a choice between anything else, but it would remove the opportunity user tests, then you should think twice, and instead do your user tests.
Everyone in their life has dealt with bad software at some point. So watching people use tools, especially your own tools, is a great way of creating a culture of design: It is hard to misinterpret a user using tool wrong, especially if the whole team can see that they are using it wrong in a recording.
Someone also mentioned feigning ignorance: Act as if you do not understand a tool. Ask folks to show you how to use it. See how they teach you and how they understand it, and let them see where the issues are. If remote, screen share what is being done.
Also, OBS came up as the best way to record a lot of footage, and timelapse it.
Resources
I have personally set up a page about Tool UX Design resources, which contains books, videos, and talks. Other pages that are helpful, are https://thetoolsmiths.org, especially its Discord where folks talk about tool development all the time, and David Lightbown’s book about Designing the User Experience of Game Development Tools.
If you have more resources that you think are helpful for folks getting into Tool UX Design, feel free to tweet them at me, or mastodon them at me, or bluesky them at me, or even email me about them.
Thank you
A big ‘Thank you!’ for everyone that attended the Tool Design Roundtables at GDC 2024, and a very special thanks for the folks who were able to attend all 3 days. I sincerely appreciate everyone who shared their own stories of both joy and woe as we all strive to create better tools for everyone. Please subscribe below to get updates when a new post is published.