If you are writing your own game, sooner or later you will need some sort of user interface (or graphic user interface = GUI). There are some existing libraries lying around. Probably the most famous is CEGUI, that is also easy to integrate into OGRE 3D engine. A good looking system was also LibRocket, but its development stopped in 2011 (but you can still download and use it without any problem). There are a lot of discussion threads that deal with the eternal problem “What GUI should I use?”.
Not so long ago I faced the very same problem. I needed a library written in C++ that would be able to coexist with OpenGL and DirectX. Another requirement was support for classic (mouse) control as well as touch input. CEGUI seems to be good choice, but the problem is in its complexity and not so great touch support (from what I was told). Most of the time, I just need a simple button, check box, text caption and “icon” or image. With those, you can build almost anything. As you may know, GUI is not just what you can see. There is also functionality, like what happens if I click on a button, if I move mthe ouse over an element etc. If you choose to write your own system, you will have to also implement those and take into account that some of them are only for mouse control and some of them are only for touch (like more than one touch at a time).
Writing a GUI from scratch is not an easy thing to do. I am not going to create some kind of super-trooper complex system that will handle everything. Our designed GUI will be used for static game menu controls and in-game menus. You can show scores, lives and other info with this system. An example of what can be created with this simple system can be seen in figures 1 and 2. Those are actuals screen-shots from engines and games I have created with this.
Our GUI will be created in C++ and will work with either OpenGL or DirectX. We are not using any API-specific things. Apart from plain C++, we also will need some libraries that make things easier for us. One of the best “library” (it's really just a single header file) - FastDelegate - will be used for function pointers (we will use this for triggers). For rendering of fonts, I have decided to use the FreeType2 library. It's a little more complex, but it's worth it. That is pretty much all you need. If you want, you can also add support for various image formats (jpg, png...). For simplicity I am using tga (my own parser) and png (via not so fast, but easy to use LodePNG library).
Ok. You have read some opening sentences and it's time to dig into details.
The most important thing of every GUI system is positioning of elements on your screen. Graphics APIs are using screen space coordinates in range [-1, 1]. Well... it's good, but not really good if we are creating / designing a GUI.
So how to determine the position of an element? One option is to use an absolute system, where we set real pixel coordinates of an element. Simple, but not really usable. If we change resolution, our entire beautifully-designed system will go to hell. Hm.. second attempt is to use a relative system. That is much better, but also not 100%. Let's say we want to have some control elements at the bottom of the screen with a small offset. If relative coordinates are used, the space will differ depending on resolution which is not what we usually want.
What I used at the end is a combination of both approaches. You can set position in relative and absolute coordinates. That is somewhat similar to the solution used in CEGUI.
Steps described above were applied on the entire screen. During GUI design there is a very little number of elements that will meet this condition. Most of the time, some kind of panel is created and other elements are placed on this panel. Why? That's simple. If we move the panel, all elements on it move along with it. If we hide the panel, all elements hide with it. And so on.
What I write so far for positioning is perfectly correct for this kind of situations as well. Again, a combination of relative and absolute positioning is used, but this time the relative starting point is not [0,0] corner of entire screen, but [0,0] of our “panel”. This [0,0] point already has some coordinates on screen, but those are not interesting for us. A picture is worth a thousand words. So, here it is:
That is, along with hard offsets, the main reason why I internally store every position in pixels and not in relative [0, 1] coordinates (or simply in percents). I once calculated pixel position and I don't need to have several recalculations of percents based on an element's parent and real pixel offsets. Disadvantage of this is that if the screen size is changed, the entire GUI needs to be reloaded. But let's be honest, how often do we change the resolution of a graphics application ?
If we are using some graphics API (either OpenGL or DirectX), rendering is done in screen space and not in pixels. Screen space is similar to percents, but has the range [-1,1]. Conversion to the screen space coodinates is done as the last step just before uploading data to the GPU. A pipeline of transforming points to the screen space coordinates is shown on the following three equations. Input pixel is converted to point in range [0, 1] by simply dividing pixel position by screen resolution. From point in [0, 1], we map it to [-1, 1].
If we want to use the GUI without a graphics API, lets say do it in Java, C#... and draw elements into image, you can just stick with pixels.
Anchor System
All good? Good. Things will be little more interesting from now on. A good thing in GUI design is to use anchors. If you have ever created a GUI, you know what anchors are. If you want to have your element stickied to some part of the screen no matter the screen size, that's the way you do it - anchors. I have decided to use a similar but slighty different system. Every element I have has its own origin. This can be one of the four corners (top left - TL, top right - TR, bottom left - BL, bottom right - BR) or its center - C. The position you have entered is than relative to this origin. Default system is TL.
Let's say you want your element always to be sticked in the bottom right corner of your screen. You can simulate this position with TL origin and the element's size. Better solution is to go backward. Position your element in a system with changed origin and convert it to TL origin later (see code). This has one advantage: you will keep user definition of GUI unified (see XML snippet) and it will be more easy to maintain.
All In One
In the following code, you can see full calculation and transformation from user input (eg. from above XML) into internal element coordinate system, that is using pixels. First, we calculate pixel position of the corner as provided by our GUI user. We also need to calculate element width and height (proportions of elements will be discussed further in the next part). For this, we need proportions of the parent - meaning its size and pixel coordinate of TL corner.
For now, we have calculated the pixel position of our reference corner. However, internal storage of our system must be unified, so everything will be converted to a system with [0,0] in TL.
With the above code, you can easily position elements in each corner of a parent element with almost the same user code. We are using float instead of int for pixel coordinate representations. This is OK, because at the end we transform this to screen space coordinates anyway.
Once we established position of an element, we also need to know its size. Well, as you may remember, we have already needed proportions for calculating the element's position, but now we discuss this topic a bit more.
Proportions are very similar to positioning. We again use relative and absolute measuring. Relative numbers will give us size in percents of parent and pixel offset is, well, pixel offset. We must take in mind one important thing - aspect ratio (AR). We want our elements to keep it every time. It would not be nice if our icon was correct on one system and deformed on another. We can repair this by only specifying one dimension (width or height) and the relevant aspect ratio for this dimension.
See the difference in example below:
Both of them will create an element with the same width. Choice a) will always have correct AR, while choice b) will always have the same size in respect of its parent element.
While working with relative size, it is also a good thing to set some kind of maximal element size in pixels. We want some elements to be as big as possible on small screens but its not neccessary to have them oversized on big screens. A typical example will be phone and tablet. There is no need for an element to be extremly big (eg. occupy let's say 100x100 pixels) on a tablet. It can take 50x50 as well and it will be enough. But on smaller screens, it should take as much as possible according to our relative size from user input.
Special care must be taken for fonts. Positioning and proportions differ a little from classic GUI elements. First of all, for font positioning it is often good to put origin into its center. That way, we can center very easily fonts inside parent elements, for example buttons. As mentioned before, to recalculate position from used system into system with TL origin, we need to know the element size.
That is the tricky part. When dealing with text, we are setting only height and the width will depend on various factors - used font, font size, printed text etc. Of course, we can calculate size manually and use it later, but that is not correct. During runtime, text can change (for instance the score in our game) and what then? A better approach is to recalculate position based on text change (change of text, font, font size etc.).
As I mentioned, for fonts I am using the FreeType library. This library can generate a single image for each character based on used font. It doesn´t matter if we have pregenerated those images into font atlas textures or we are creating them on the fly. To calculate proportions of text we don't really need an actual image, but only its size. Problem is size of the whole text we want to display. This must be calculated by iterating over every character and accumulating proportions and spaces for each of them. We also must take care of new lines.
There is one thing you need to count on when dealing with text. See the image and its caption below. Someone may think that answer to “Why?” question is obvious, but I didn't realize that in time of design and coding, so it brings me a lot of headache.
The answer is really simple. There are diacritics marks in the text that are counted in total size. There should also be space for descendents, but they are not used for capitals in the font I have used. Everything you need to take care of can be seen in this picture:
This will be all for now. I have described the basics of positioning and sizing of GUI elements that I have used in my design. There are probably better or more complex ways to do it. This one used here is easy and I have not run across any problems using it.
I have written a simple C# application to speed up the design of the GUI. It uses the basics described here (but no fonts). You can place elements, change their size and image, drag them to see positions of them. You can download the source of application and try it for yourself. But take it as “alpha” version, I have written it for fast prototyping during one evening.
In future parts (don't worry, I have already written them and only doing finishing touches) I will focus on basic types of elements, controls, rendering. That is, after all, one of the important things in GUI :-)
3 May 2014: Initial release
Not so long ago I faced the very same problem. I needed a library written in C++ that would be able to coexist with OpenGL and DirectX. Another requirement was support for classic (mouse) control as well as touch input. CEGUI seems to be good choice, but the problem is in its complexity and not so great touch support (from what I was told). Most of the time, I just need a simple button, check box, text caption and “icon” or image. With those, you can build almost anything. As you may know, GUI is not just what you can see. There is also functionality, like what happens if I click on a button, if I move mthe ouse over an element etc. If you choose to write your own system, you will have to also implement those and take into account that some of them are only for mouse control and some of them are only for touch (like more than one touch at a time).
Writing a GUI from scratch is not an easy thing to do. I am not going to create some kind of super-trooper complex system that will handle everything. Our designed GUI will be used for static game menu controls and in-game menus. You can show scores, lives and other info with this system. An example of what can be created with this simple system can be seen in figures 1 and 2. Those are actuals screen-shots from engines and games I have created with this.
Our GUI will be created in C++ and will work with either OpenGL or DirectX. We are not using any API-specific things. Apart from plain C++, we also will need some libraries that make things easier for us. One of the best “library” (it's really just a single header file) - FastDelegate - will be used for function pointers (we will use this for triggers). For rendering of fonts, I have decided to use the FreeType2 library. It's a little more complex, but it's worth it. That is pretty much all you need. If you want, you can also add support for various image formats (jpg, png...). For simplicity I am using tga (my own parser) and png (via not so fast, but easy to use LodePNG library).
Ok. You have read some opening sentences and it's time to dig into details.
Coordinate System and Positioning
The most important thing of every GUI system is positioning of elements on your screen. Graphics APIs are using screen space coordinates in range [-1, 1]. Well... it's good, but not really good if we are creating / designing a GUI.
So how to determine the position of an element? One option is to use an absolute system, where we set real pixel coordinates of an element. Simple, but not really usable. If we change resolution, our entire beautifully-designed system will go to hell. Hm.. second attempt is to use a relative system. That is much better, but also not 100%. Let's say we want to have some control elements at the bottom of the screen with a small offset. If relative coordinates are used, the space will differ depending on resolution which is not what we usually want.
What I used at the end is a combination of both approaches. You can set position in relative and absolute coordinates. That is somewhat similar to the solution used in CEGUI.
Steps described above were applied on the entire screen. During GUI design there is a very little number of elements that will meet this condition. Most of the time, some kind of panel is created and other elements are placed on this panel. Why? That's simple. If we move the panel, all elements on it move along with it. If we hide the panel, all elements hide with it. And so on.
What I write so far for positioning is perfectly correct for this kind of situations as well. Again, a combination of relative and absolute positioning is used, but this time the relative starting point is not [0,0] corner of entire screen, but [0,0] of our “panel”. This [0,0] point already has some coordinates on screen, but those are not interesting for us. A picture is worth a thousand words. So, here it is:
Figure 3: Position of elements. Black color represents screen (main window) and position of elements within it. Panel is positioned within Screen. Green element is inside Panel. It's positions are within Panel.
That is, along with hard offsets, the main reason why I internally store every position in pixels and not in relative [0, 1] coordinates (or simply in percents). I once calculated pixel position and I don't need to have several recalculations of percents based on an element's parent and real pixel offsets. Disadvantage of this is that if the screen size is changed, the entire GUI needs to be reloaded. But let's be honest, how often do we change the resolution of a graphics application ?
If we are using some graphics API (either OpenGL or DirectX), rendering is done in screen space and not in pixels. Screen space is similar to percents, but has the range [-1,1]. Conversion to the screen space coodinates is done as the last step just before uploading data to the GPU. A pipeline of transforming points to the screen space coordinates is shown on the following three equations. Input pixel is converted to point in range [0, 1] by simply dividing pixel position by screen resolution. From point in [0, 1], we map it to [-1, 1].
pixel = [a, b] point = [pixel(a) / width, pixel(b) / height] screen_space = 2 * point - 1
If we want to use the GUI without a graphics API, lets say do it in Java, C#... and draw elements into image, you can just stick with pixels.
Anchor System
All good? Good. Things will be little more interesting from now on. A good thing in GUI design is to use anchors. If you have ever created a GUI, you know what anchors are. If you want to have your element stickied to some part of the screen no matter the screen size, that's the way you do it - anchors. I have decided to use a similar but slighty different system. Every element I have has its own origin. This can be one of the four corners (top left - TL, top right - TR, bottom left - BL, bottom right - BR) or its center - C. The position you have entered is than relative to this origin. Default system is TL.
Figure 4: Anchors of screen elements
Let's say you want your element always to be sticked in the bottom right corner of your screen. You can simulate this position with TL origin and the element's size. Better solution is to go backward. Position your element in a system with changed origin and convert it to TL origin later (see code). This has one advantage: you will keep user definition of GUI unified (see XML snippet) and it will be more easy to maintain.
<position x="0" y="0" offset_x="0" offset_y="0" origin="TL" /> <position x="0" y="0" offset_x="0" offset_y="0" origin="TR" /> <position x="0" y="0" offset_x="0" offset_y="0" origin="BL" /> <position x="0" y="0" offset_x="0" offset_y="0" origin="BR" />
All In One
In the following code, you can see full calculation and transformation from user input (eg. from above XML) into internal element coordinate system, that is using pixels. First, we calculate pixel position of the corner as provided by our GUI user. We also need to calculate element width and height (proportions of elements will be discussed further in the next part). For this, we need proportions of the parent - meaning its size and pixel coordinate of TL corner.
float x = parentProportions.topLeft.X; x += pos.x * parentProportions.width; x += pos.offsetX; float y = parentProportions.topLeft.Y; y += pos.y * parentProportions.height; y += pos.offsetY; float w = parentProportions.width; w *= dim.w; w += dim.pixelW; float h = parentProportions.height; h *= dim.h; h += dim.pixelH;
For now, we have calculated the pixel position of our reference corner. However, internal storage of our system must be unified, so everything will be converted to a system with [0,0] in TL.
//change position based on origin
if (pos.origin == TL)
{
//do nothing - top left is default
}
else if (pos.origin == TR)
{
x = parentProportions.botRight.X - (x - parentProportions.topLeft.X); //swap x coordinate
x -= w; //put x back to top left
}
else if (pos.origin == BL)
{
y = parentProportions.botRight.Y - (y - parentProportions.topLeft.Y); //swap y coordinate
y -= h; //put y back to top left
}
else if (pos.origin == BR)
{
x = parentProportions.botRight.X - (x - parentProportions.topLeft.X); //swap x coordinate
y = parentProportions.botRight.Y - (y - parentProportions.topLeft.Y); //swap y coordinate
x -= w; //put x back to top left
y -= h; //put y back to top left
}
else if (pos.origin == C)
{
//calculate center of parent element
x = x + (parentProportions.botRight.X - parentProportions.topLeft.X) * 0.5f;
y = y + (parentProportions.botRight.Y - parentProportions.topLeft.Y) * 0.5f;
x -= (w * 0.5f); //put x back to top left
y -= (h * 0.5f); //put y back to top left
}
//this will overflow from parent element
proportions.topLeft = MyMath::Vector2(x, y);
proportions.botRight = MyMath::Vector2(x + w, y + h);
proportions.width = w;
proportions.height = h;
With the above code, you can easily position elements in each corner of a parent element with almost the same user code. We are using float instead of int for pixel coordinate representations. This is OK, because at the end we transform this to screen space coordinates anyway.
Proportions
Once we established position of an element, we also need to know its size. Well, as you may remember, we have already needed proportions for calculating the element's position, but now we discuss this topic a bit more.
Proportions are very similar to positioning. We again use relative and absolute measuring. Relative numbers will give us size in percents of parent and pixel offset is, well, pixel offset. We must take in mind one important thing - aspect ratio (AR). We want our elements to keep it every time. It would not be nice if our icon was correct on one system and deformed on another. We can repair this by only specifying one dimension (width or height) and the relevant aspect ratio for this dimension.
See the difference in example below:
a) <size w="0.1" offset_w="0" ar="1.0" /> - create element of size 10% of parent W b) <size w="0.1" h="0.1" offset_w="0" offset_h="0" /> - create element of size 10% of parent W and H
Both of them will create an element with the same width. Choice a) will always have correct AR, while choice b) will always have the same size in respect of its parent element.
While working with relative size, it is also a good thing to set some kind of maximal element size in pixels. We want some elements to be as big as possible on small screens but its not neccessary to have them oversized on big screens. A typical example will be phone and tablet. There is no need for an element to be extremly big (eg. occupy let's say 100x100 pixels) on a tablet. It can take 50x50 as well and it will be enough. But on smaller screens, it should take as much as possible according to our relative size from user input.
Fonts
Special care must be taken for fonts. Positioning and proportions differ a little from classic GUI elements. First of all, for font positioning it is often good to put origin into its center. That way, we can center very easily fonts inside parent elements, for example buttons. As mentioned before, to recalculate position from used system into system with TL origin, we need to know the element size.
Figure 5: Origin in center of parent element for centered font positioning
That is the tricky part. When dealing with text, we are setting only height and the width will depend on various factors - used font, font size, printed text etc. Of course, we can calculate size manually and use it later, but that is not correct. During runtime, text can change (for instance the score in our game) and what then? A better approach is to recalculate position based on text change (change of text, font, font size etc.).
As I mentioned, for fonts I am using the FreeType library. This library can generate a single image for each character based on used font. It doesn´t matter if we have pregenerated those images into font atlas textures or we are creating them on the fly. To calculate proportions of text we don't really need an actual image, but only its size. Problem is size of the whole text we want to display. This must be calculated by iterating over every character and accumulating proportions and spaces for each of them. We also must take care of new lines.
There is one thing you need to count on when dealing with text. See the image and its caption below. Someone may think that answer to “Why?” question is obvious, but I didn't realize that in time of design and coding, so it brings me a lot of headache.
Figure 6: Text rendered with settings: origin is TL (top left), height is set to be 100% of parent height. You may notice, text is not filling the whole area. Why ?
The answer is really simple. There are diacritics marks in the text that are counted in total size. There should also be space for descendents, but they are not used for capitals in the font I have used. Everything you need to take care of can be seen in this picture:
Figure 7: Font metrics
Discussion
This will be all for now. I have described the basics of positioning and sizing of GUI elements that I have used in my design. There are probably better or more complex ways to do it. This one used here is easy and I have not run across any problems using it.
I have written a simple C# application to speed up the design of the GUI. It uses the basics described here (but no fonts). You can place elements, change their size and image, drag them to see positions of them. You can download the source of application and try it for yourself. But take it as “alpha” version, I have written it for fast prototyping during one evening.
In future parts (don't worry, I have already written them and only doing finishing touches) I will focus on basic types of elements, controls, rendering. That is, after all, one of the important things in GUI :-)
Article Update Log
3 May 2014: Initial release