Quantcast
Channel: GameDev.net
Viewing all 17825 articles
Browse latest View live

Building a First-Person Shooter: Part 1.5 Running, Jumping, & Crouching

$
0
0
At this point making the player run at a faster speed is a simple two step process, we need to detect if the shift key is hit and then multiply movespeed. We start off by adding in two variables to detect if the player is running and a multiplier value for increasing movespeed:

running=false;
runboost = 2.0;

In the UpdateControls function we add in a line to check if the shift key has been hit:

running = Window::GetCurrent()->KeyDown(Key::Shift);

Then in the player Update, multiply movespeed by the runboost factor and increase maxaccel:

//Run if shift key is pressed
if (running) movespeed *= runboost;
if (running) maxaccel *= 2.0;

Your player class should now look like the following:

#include "MyGame.h"

using namespace Leadwerks;

Player::Player()
{
    //Create the entity
    entity = Pivot::Create();
    entity->SetUserData(this);
    
    //Initialize values
    standheight=1.7;
    crouchheight=1.2;
    cameraheight = standheight;
    move = 0.0;
    strafe = 0.0;
    movementspeed = 3.0;
    maxacceleleration = 0.5;
    sensitivity=1.0;
    smoothedcamerapositiony = 0;
    cameraypositionsmoothing = 3.0;
    cameralooksmoothing = 2.0;
    runboost = 2.0;
    running=false;
    
    
    //Set up player physics
    entity->SetPhysicsMode(Entity::CharacterPhysics);
    entity->SetCollisionType(Collision::Character);
    entity->SetMass(10.0);
    
    //Player position
    entity->SetPosition(0,0,0,true);
    
    //Create the player camera
    camera = Camera::Create();
    camera->SetPosition(0,entity->GetPosition().y + cameraheight,0,true);
}

Player::~Player()
{
    if (camera)
    {
        camera->Release();
        camera = NULL;
    }
}

void Player::UpdateControls()
{
    Window* window = Window::GetCurrent();
    Context* context = Context::GetCurrent();
    
    //Get inputs from the controller class
    move = window->KeyDown(Key::W) - window->KeyDown(Key::S);
    strafe = window->KeyDown(Key::D) - window->KeyDown(Key::A);
    running = Window::GetCurrent()->KeyDown(Key::Shift);
    
    //Get the mouse movement
    float sx = context->GetWidth()/2;
    float sy = context->GetHeight()/2;
    
    //Get the mouse position
    Vec3 mouseposition = window->GetMousePosition();
    
    //Move the mouse to the center of the screen
    window->SetMousePosition(sx,sy);
    
    //Get change in mouse position
    float dx = mouseposition.x - sx;
    float dy = mouseposition.y - sy;
    
    //Mouse smoothing
    mousespeed.x = Math::Curve(dx,mousespeed.x,cameralooksmoothing/Time::GetSpeed());
    mousespeed.y = Math::Curve(dy,mousespeed.y,cameralooksmoothing/Time::GetSpeed());
    
    //Adjust and set the camera rotation
    playerrotation.x += mousespeed.y*sensitivity / 10.0;
    playerrotation.y += mousespeed.x*sensitivity / 10.0;
    
    //Prevent inhuman looking angles
    playerrotation.x = Math::Clamp(playerrotation.x,-90,90);
}

//Update function
void Player::Update()
{
    UpdateControls();
    
    float maxaccel = this->maxacceleleration;
    float movespeed = this->movementspeed;
    
    //Run if shift key is pressed
    if (running) movespeed *= runboost;
    
    //Make sure movements are normalized so that moving forward at the same time as strafing doesn't move your character faster
    normalizedmovement.z = move;
    normalizedmovement.x = strafe;
    normalizedmovement = normalizedmovement.Normalize() * movespeed;
    
    
    //Set camera rotation
    camera->SetRotation(playerrotation,true);
    
    //Set player input
    if (running) maxaccel *= 2.0;
    entity->SetInput(playerrotation.y,normalizedmovement.z,normalizedmovement.x,0,false,maxaccel);
    
    //Set the camera position & smooth
    cameraposition = entity->GetPosition();
    camera->SetPosition(cameraposition.x, cameraposition.y + cameraheight, cameraposition.z );
}

Jumping and Crouching


Since jumping and crouching both affect camera height we are going to implement both of them at the same time. Like normal we add in the needed variables into the constructor:

jump = 0.0;
jumpforce = 6.0;
jumpboost=2.0;
crouched = false;

Now we turn our attention to the UpdateControl function. We check if the space key has been hit and then multiply it by a jump force. We also look to see if the ‘C’ button is hit, but only if the player is not in the air:

jump = window->KeyDown(Key::Space) * jumpforce;
if (!entity->GetAirborne())
{
	crouched = window->KeyDown(Key::C);
}

Player Update is our next focus and we begin to set the camera to its proper height. If the player is crouching we increment the camera height by the crouchheight but if the player is standing normally we increment it by standheight:

if (entity->GetCrouched())
{
	cameraheight = Math::Inc(crouchheight,cameraheight,0.1);
}
else
{
	cameraheight = Math::Inc(standheight,cameraheight,0.1);
}

To implement the jumping mechanic we continue our work in the Update function. If the entity is in the air we set jump to 0 to prevent the player from in-air jumps. To achieve the long jump feeling we want from our FPS we decided to boost the player's velocity while in-air:

if (entity->GetAirborne()) //Player is in the air
{
    //if the player is in the air don't let them jump
    jump = 0.0;
}
else //Player is on the ground
{
    //Give an extra boost if jumping
    if (jump>0.0)
    {
        if (velocity.z>movementspeed*0.85)
        {
            normalizedmovement.z *= jumpboost;
            normalizedmovement.z = min(normalizedmovement.z,movementspeed*runboost);
            maxaccel = 10;
        }
    }
}

Now that we have a jump and crouch value we have to remember to change our SetInput function to the correct values:

entity>SetInput(playerrotation.y,normalizedmovement.z,normalizedmovement.x,jump,crouched,maxaccel);

With all the jumping and crouching going on, our camera’s y position will be changing often, this means that we should now smooth the camera’s y movements:

//Set the camera position & smooth
cameraposition = entity->GetPosition();
if (cameraposition.y>smoothedcamerapositiony)
{
    smoothedcamerapositiony = Math::Curve(cameraposition.y, smoothedcamerapositiony, cameraypositionsmoothing / Time::GetSpeed());
    cameraposition.y=smoothedcamerapositiony;
}
else
{
	smoothedcamerapositiony = cameraposition.y;
}
camera->SetPosition(cameraposition.x, cameraposition.y + cameraheight, cameraposition.z );

Our quickly growing player class will now look like so:

#include "MyGame.h"

using namespace Leadwerks;

Player::Player()
{
    //Create the entity
    entity = Pivot::Create();
    entity->SetUserData(this);
    
    //Initialize values
    standheight=1.7;
    crouchheight=1.2;
    cameraheight = standheight;
    move = 0.0;
    strafe = 0.0;
    movementspeed = 3.0;
    maxacceleleration = 0.5;
    sensitivity=1.0;
    smoothedcamerapositiony = 0;
    cameraypositionsmoothing = 3.0;
    cameralooksmoothing = 2.0;
    runboost = 2.0;
    jump = 0.0;
    jumpforce = 6.0;
    jumpboost = 2.0;
    footstepwalkfrequency=400;
    footsteprunfrequency=320;
    footsteptimer=Time::GetCurrent();
    running=false;
    crouched = false;
    landing = false;
    
    //Set up player physics
    entity->SetPhysicsMode(Entity::CharacterPhysics);
    entity->SetCollisionType(Collision::Character);
    entity->SetMass(10.0);
    
    //Player position
    entity->SetPosition(0,0,0,true);
    
    //Create the player camera
    camera = Camera::Create();
    camera->SetPosition(0,entity->GetPosition().y + cameraheight,0,true);
}

Player::~Player()
{
    if (camera)
    {
        camera->Release();
        camera = NULL;
    }
}

void Player::UpdateControls()
{
    Window* window = Window::GetCurrent();
    Context* context = Context::GetCurrent();
    
    //Get inputs from the controller class
    move = window->KeyDown(Key::W) - window->KeyDown(Key::S);
    strafe = window->KeyDown(Key::D) - window->KeyDown(Key::A);
    jump = window->KeyDown(Key::Space) * jumpforce;
    
    if (!entity->GetAirborne())
    {
	    crouched = window->KeyDown(Key::C);
    }
    running = Window::GetCurrent()->KeyDown(Key::Shift);
    
    //Get the mouse movement
    float sx = context->GetWidth()/2;
    float sy = context->GetHeight()/2;
    
    //Get the mouse position
    Vec3 mouseposition = window->GetMousePosition();
    
    //Move the mouse to the center of the screen
    window->SetMousePosition(sx,sy);
    
    //Get change in mouse position
    float dx = mouseposition.x - sx;
    float dy = mouseposition.y - sy;
    
    //Mouse smoothing
    mousespeed.x = Math::Curve(dx,mousespeed.x,cameralooksmoothing/Time::GetSpeed());
    mousespeed.y = Math::Curve(dy,mousespeed.y,cameralooksmoothing/Time::GetSpeed());
    
    //Adjust and set the camera rotation
    playerrotation.x += mousespeed.y*sensitivity / 10.0;
    playerrotation.y += mousespeed.x*sensitivity / 10.0;
    
    //Prevent inhuman looking angles
    playerrotation.x = Math::Clamp(playerrotation.x,-90,90);
}

//Update function
void Player::Update()
{
    UpdateControls();
    
    if (entity->GetCrouched())
    {
    cameraheight = Math::Inc(crouchheight,cameraheight,0.1);
    }
    else
    {
    cameraheight = Math::Inc(standheight,cameraheight,0.1);
    }
    
    float maxaccel = maxacceleleration;
    
    Vec3 velocity = entity->GetVelocity(false);
    float groundspeed = velocity.xz().Length();
    float movespeed = this->movementspeed;
    
    //Run if shift key is pressed
    if (running) movespeed *= runboost;
    
    //Make sure movements are normalized so that moving forward at the same time as strafing doesn't move your character faster
    normalizedmovement.z = move;
    normalizedmovement.x = strafe;
    normalizedmovement = normalizedmovement.Normalize() * movespeed;
    
    if (entity->GetAirborne()) //Player is in the air
    {
        //if the player is in the air don't let them jump
        jump = 0.0;
    }
    else //Player is on the ground
    {
        //Give an extra boost if jumping
        if (jump>0.0)
        {
            if (velocity.z>movementspeed*0.85)
            {
                normalizedmovement.z *= jumpboost;
                normalizedmovement.z = min(normalizedmovement.z,movementspeed*runboost);
                maxaccel = 10;
            }
		}
    }
}

//Set camera rotation
camera->SetRotation(playerrotation,true);

//Set player input
if (running) maxaccel *= 2.0;

entity->SetInput(playerrotation.y,normalizedmovement.z,normalizedmovement.x,jump,crouched,maxaccel);

//Set the camera position & smooth
cameraposition = entity->GetPosition();
if (cameraposition.y>smoothedcamerapositiony)
{
    smoothedcamerapositiony = Math::Curve(cameraposition.y, smoothedcamerapositiony, cameraypositionsmoothing / Time::GetSpeed());
    cameraposition.y=smoothedcamerapositiony;
}
else
{
	smoothedcamerapositiony = cameraposition.y;
}
camera->SetPosition(cameraposition.x, cameraposition.y + cameraheight, cameraposition.z );
}

Virtual Texture Terrain

$
0
0
The Leadwerks 2 terrain system was expansive and very fast, which allowed rendering of huge landscapes. However, it had some limitations. Texture splatting was done in real-time in the pixel shader. Because of the limitations of hardware texture units, only four texture units per terrain were supported. This limited the ability of the artist to make terrains with a lot of variation. The landscapes were beautiful, but somewhat monotonous.

With the Leadwerks 3 terrain system, I wanted to retain the advantages of terrain in Leadwerks 2, but overcome some of the limitations. There were three different approaches we could use to increase the number of terrain textures.
  • Increase the number of textures used in the shader.
  • Allow up to four textures per terrain chunk. These would be determined either programmatically based on which texture layers were in use on that section, or defined by the artist.
  • Implement a virtual texture system like id Software used in the game Rage.
Since Leadwerks 3 runs on mobile devices as well as PC and Mac, we couldn't use any more texture units than we had before, so the first option was out. The second option is how Crysis handles terrain layers. If you start painting layers in the Crysis editor, you will see when "old" layers disappear as you paint new ones on. This struck me as a bad approach because it would either involve the engine "guessing" which layers should have priority, or involve a tedious process of user-defined layers for each terrain chunk.

A virtual texturing approach seemed liked the ideal choice. Basically, this would render near sections of the terrain at a high resolution, and far sections of the terrain at low resolutions, with a shader that chose between them. If done correctly, the result should be the same as using one impossibly huge texture (like 1,048,576 x 1,048,576 pixels) at a much lower memory cost. However, there were some serious challenges to be overcome, so much so that I added a disclaimer in our Kickstarter campaign basically saying "this might not work"..

Previous Work


id Software pioneered this technique with the game Rage (a previous implementation was in Quake Wars). However, id's "megatexture" technique had some serious downsides. First, the data size requirements of storing completely unique textures for the entire world were prohibitive. Rage takes about 20 gigs of hard drive space, with terrains much smaller than the size I wanted to be able to use. The second problem with id's approach is that both games using this technique have some pretty blurry textures in the foreground, although the unique texturing looks beautiful from a distance.


Attached Image: gf8800_trailer_1.jpg


I decided to overcome the data size problem by dynamically generating the megatexture data, rather than storing it on the hard drive. This involves a pre-rendering step where layers are rendered to the terrain virtual textures, and then the virtual textures are applied to the terrain. Since id's art pipeline was basically just conventional texture splatting combined with "stamps" (decals), I didn't see any reason to permanently store that data. I did not have a simple solution to the blurry texture problem, so I just went ahead and started implementing my idea, with the understanding that the texture resolution issue could kill it.

I had two prior examples to work from. One was a blog from a developer at Frictional Games (Amnesia: The Dark Descent and Penumbra). The other was a presentation describing the technique's use in the game Halo Wars. In both of these games, a fixed camera distance could be relied on, which made the job of adjusting texture resolution much easier. Leadwerks, on the other hand, is a general-purpose game engine for making any kind of game. Would it be possible to write an implementation that would provide acceptable texture resolution for everything from flight sims to first-person shooters? I had no idea if it would work, but I went forward anyway.

Implementation


Because both Frictional Games and id had split the terrain into "cells" and used a texture for each section, I tried that approach first. Our terrain already gets split up and rendered in identical chunks, but I needed smaller pieces for the near sections. I adjusted the algorithm to render the nearest chunks in smaller pieces. I then allocated a 2048x2048 texture for each inner section, and used a 1024x1024 texture for each outer section:


Attached Image: terrain.jpg


The memory requirements of this approach can be calculated as follows:

1024 * 1024 * 4 * 12 = 50331648 bytes
2048 * 2048 * 4 * 8 = 134217728
Total = 184549376 bytes = 176 megabytes

176 megs is a lot of texture data. In addition, the texture resolution wasn't even that good at near distances. You can see my attempt with this approach in the image below. The red area is beyond the virtual texture range, and only uses a single low-res baked texture. The draw distance was low, the memory consumption high, and the resolution was too low.


Attached Image: grid.jpg


This was a failure, and I thought maybe this technique was just impractical for anything but very controlled cases in certain games. I wasn't ready to give up yet without trying one last approach. Instead of allocating textures for a grid section, I tried creating a radiating series of textures extending away from the camera:


Attached Image: terr2.jpg


The resulting resolution wasn't great, but the memory consumption was a lot lower, and terrain texturing was now completely decoupled from the terrain geometry. I found by adjusting the distances at which the texture switches, I could get a pretty good resolution in the foreground. I was using only three texture stages, so I increased the number to six and found I could get a good resolution at all distances, using just six 1024x1024 textures. The memory consumption for this was just 24 megabytes, a very reasonable number. Since the texturing is independent from terrain geometry, the user can fine-tune the texture distances to accommodate flight sims, RPGs, or whatever kind of game they are making.


Attached Image: goodterrain.jpg


The last step was to add some padding around each virtual texture, so the virtual textures did not have to be completely redrawn each time the camera moves. I used a value of 0.25 the size of the texture range so the various virtual textures only get redrawn once in a while.

Advantages of Virtual Textures


First, because the terrain shader only has to perform a few lookups each pixel with almost no math, the new terrain shader runs much faster than the old one. When the bottleneck for most games is the pixel fillrate, this will make Leadwerks 3 games faster. Second, this allows us to use any number of texture layers on a terrain, with virtually no difference in rendering speed. This gives artists the flexibility to paint anything they want on the terrain, without worrying about budgets and constraints. A third advantage is that this allows the addition of "stamps", which are decals rendered directly into the virtual texture. This allows you to add craters, clumps of rocks, and other details directly onto the terrain. The cost of rendering them in is negligible, and the resulting virtual texture runs at the exact same speed, no matter how much detail you pack into it. Below you can see a simple example. The smiley faces are baked into the virtual texture, not rendered on top of the terrain:


Attached Image: Image1.jpg


Conclusion


The texture resolution problem I feared might make this approach impossible was solved by using a graduated series of six textures radiating out around the camera. I plan to implement some reasonable default settings, and it's only a minor hassle for the user to adjust the virtual texture draw distances beyond that.

Rendering the virtual textures dynamically eliminates the high space requirements of id's megatexture technique, and also gets rid of the problems of paging texture data dynamically from the hard drive. At the same time, most of the flexibility of the megatexture technique is retained.

Having the ability to paint terrain with any number of texture layers, plus the added stamping feature gives the artist a lot more flexibility than our old technique offered, and it even runs faster than the old terrain. This removes a major source of uncertainty from the development of Leadwerks 3.1 and turned out to be one of my favorite features in the new engine.

Reinforcement Learning for Games

$
0
0
This article assumes that the reader already knows what neural networks are and how they operate.
If you do not know what neural networks are, I recommend trying out the great tutorials at AI Junkie.

Neural networks are often overlooked when considering game AI. This is because they once received a lot of hype but the hype didn't amount to much. However, neural networks are still an area of intense research, and numerous learning algorithms have been developed for each of the 3 basic types of learning: supervised, unsupervised, and reinforcement learning.

Reinforcement learning is the learning algorithm that allows an agent to learn from its environment and improve itself on its own. This is the class of learning algorithms we will focus on in this article. This article will discuss the use of genetic algorithms as well as an algorithm the author has researched for single-agent reinforcement learning. This article assumes that the neural networks are simple integrate and fire, non-spiking sigmoidal activation neural networks.

Genetic Algorithms


The concept


Genetic algorithms are one of the simplest but also one of the most effective reinforcement learning methods. It does have one key limitation though: It has to operate on multiple agents (AI's). Nevertheless, genetic algorithms can be a great tool for creating neural networks via the process of evolution.

Genetic algorithms are part of a broader range of evolutionary algorithms. Their basic operation proceeds as follows:

1. Initialize a set of genes
2. Evaluate the fitnesses of all genes
3. Mate genes based on how well they performed (performing crossover and mutation)
4. Replace old genes with the new children
5. Repeat steps 2 - 4 until a termination criterion is met

The genes can be either a direct encoding of the traits of the AI (neuron weights in the neural network case), or an indirect, "generative" encoding. Evaluating the fitnesses is where most of the processing time is spent, since it involves simulating a phenotype created from each genotype to assert how good it is at finishing the task. The fitnesses are recorded, and are typically scaled such that the lowest fitness is always 0. The reason for this is the mating step. Here, genes are selected based on their fitnesses using a selection function. A popular selection function is the fitness proportional "roulette wheel" selection function, which randomly chooses genes with likelihoods proportional to their fitnesses. Some selection functions such as fitness proportional selection require all positive fitnesses, which is why they are typically rescaled so the lowest is always 0.

When a pair of parents is selected, their genes are crossed over using a crossover function. This function depends on your encoding, but you generally want some traits of each parent to make it into the child without being destructive. After crossover, the genes are also mutated (randomly altered slightly) to help force the algorithm to perform more exploration of possible solutions.

Over time, genotypes will improve themselves, and often will even learn how to exploit faults in whichever system they are operating!

Next, we will discuss a particular gene encoding method for neural networks.

NEAT


The code accompanying performs the genetic algorithm following the NEAT (Neuro-Evolution of Augmenting Topologies) methodology created by Kenneth Stanley. As a result, the neural network genes are encoded by storing connections between neurons as index pairs (indexed into the neuron array) along with the associated weight, as well as the biases for each of the neurons.

This is all the information that is needed to construct a completely functional neural network from genes. However, along with this information, both the neuron biases and connection genes have a special "innovation number" stored along with them. These numbers are unique, a counter is incremented each time an innovation number is assigned. That way, when network genes are being mated, we can tell if connections share a heritage by seeing if their innovation numbers match. These can then be crossed over directly, while the genes without innovation number matches can be assigned randomly to the child neural networks.

This description is lacking in detail, but intends to simply provide an overview of the way the genetic algorithm included in the software package functions.

While this genetic algorithm works very well for many problems, it requires that many agents are simulated at a time rather than one just learning by itself. So, we will briefly cover another method of neural network training.

Local Dopamine Weight Update Rule with Weight and Output Traces


The concept


This method quite possibly has already been invented, but I could not find a paper describing the same method so far. This method applies to how neuron weights are updated when learning in a single agent scenario. This method is entirely separate from network topology selection. As a result, the included software package uses a genetic algorithm to evolve a topology for use with the single agent reinforcement learning system. Of course, one could also simply grow a neural network by randomly attaching new neurons over time.

Anyways, I discovered this technique after a lot of trial and error while trying to find a weight update rule for neural networks that operates using information available at the neuron/synapse level. It therefore could be biologically plausible. The method uses a reward signal, dopamine, to determine when the network should be rewarded or punished. In order to make this method work, one needs to add a output trace (floating point variable) to each neuron, as well as a weight trace to each neuron weight, except for the bias weight. Other than that, one only needs the reward signal dopamine, which can take on any value where positive means reward the network and negative means punish the network (0 is therefore neutral and doesn't do anything). When one has this information, all one needs to do is update the neural network weights after every normal update cycle using the following code (here in C++):

m_output = Sigmoid(potential * activationMultiplier);

m_outputTrace += -traceDecay * m_outputTrace + 2.0f * m_output - 1.0f;

// Weight update
for(size_t i = 0; i < numInputs; i++)
{
	m_inputs[i].m_trace += -traceDecay * m_inputs[i].m_trace + m_outputTrace * (m_inputs[i].m_pInput->m_outputTrace * 0.5f + 0.5f);
	m_inputs[i].m_weight += alpha * m_inputs[i].m_trace * dopamine;
}

// Bias update
m_bias += alpha * m_outputTrace * dopamine;

Where m_output is the output of the neuron, traceDecay is a value that defines how quickly the network forgets (ranges from [0, 1]), alpha is the learning rate, and m_inputs is an array of connections.

This code works as follows:

The output trace is simply an average output over time that decays if left untouched. The weight update simply moves the weight if dopamine is not equal to 0 (it doesn't have perfect fitness yet) in the direction that would cause it to output its average output less often if dopamine was negative or more often if dopamine was positive. However, it does so based on the weight trace, which measures how much a connection has contributed to the firing of the neuron over time, and therefore helps judge how eligible the weight is for a weight update. The bias doesn't use a weight trace, since it is always eligible for a weight update.

This method is able to solve the XOR problem with considerable ease (it easily learns an XOR gate). I tested it with simple feed-forward neural networks (1 hidden layer with 2 neurons), a growing neural cloud, and networks resulting from the NEAT algorithm.

Use in Games?


These methods seem like total overkill for games. But, they can do things that traditional methods can't. For instance, with the genetic algorithm, you can create a physics-based character controller like this one.

The animation was not made by an animator; rather the AI learned how to walk by itself. This results in an animation that can react to the environment directly. The AI in the video was created using the same software package this article revolves around (linked below).

The second discussed technique can be used to have game characters or enemies learn from experience. Enemies can for instance be assigned a reward for how close they get to a player, so that they try to get as close as possible to the player given a few sensory inputs. This method can also be used by virtual pet type games, where you can reward or punish a pet to achieve the desired behavior.

Using the Code


The software package accompanying this article contains a manual on how to use the code.

The software package can be found at: https://sourceforge.net/projects/neatvisualizers/

Conclusion


I hope this article has provided some insight and inspiration for the use of reinforcement learning neural network AI in games. Now get out there and make some cool game AI!

Article Update Log


2 July 2013: Initial release
14 July 2013: Heavy edits

Building a First-Person Shooter: Part 1.6 Sound

$
0
0
For a final touch we are going to add in sound effects. We begin by declaring a few variables to deal with the footstep sound effects times and frequencies:

footsteptimer=Time::GetCurrent();
footstepwalkfrequency=400;
footsteprunfrequency=320;

Next we load in our sound effects which are located in the MyGame/Sound folder. For variety’s sake we have four separate footstep sounds, four landing sounds, and one jumping grunt sound effect:

//Load Sound files
for (int i=0; i<4; ++i)
{
    footstepsound[i] = Sound::Load("Sound/Player/Footsteps/footstep_shoe-softsole_metal_single_01_v0"+String(i+1)+".wav");
}
for (int i = 0; i < 4; ++i)
{
    landsound[i] = Sound::Load("Sound/Player/Landing/footstep_shoe-softsole_metal_jump_0"+String(i+1)+".wav");
}
for (int i = 0; i < 1; ++i)
{
    jumpsound[i] = Sound::Load("Sound/Player/Jumping/grunt_male01.wav");
}

Inside our Update function we had our code to deal with the jumping mechanic, to add in our sound effects we will go back and work inside the section of the if statement where in the player is currently on the ground. We first check if jump is greater than 0, remember that since we are currently on the ground it means that the jump button was just recently hit thus we can now play the jumping grunt sound as the player starts his jump. Please note that we do not play the jump sound in the UpdateControl function when the Space bar is hit because then the sound effect would be played every time the button was pressed even if you were already jumping. The other key thing we do is set the landing variable to be true:

if (jump>0.0)
{
    jumpsound[0]->Play();
    landing = true;
    
    if (velocity.z>movementspeed*0.85)
    {
        normalizedmovement.z *= jumpboost;
        normalizedmovement.z = min(normalizedmovement.z,movementspeed*runboost);
        maxaccel = 10;
    }
}

Directly above the previous code we now add in a check for the landing variable. If the player is landing then we play the landsound and immediately set landing to be false.

//Play landing sounds
if(landing)
{
    landsound[(int)Math::Random(0,3)]->Play();
    landing = false;
}

All that is left to do is implement the footstep sounds. We first need to calculate ground speed , this will help us determine how often to play the footstep sounds:

Vec3 velocity = entity->GetVelocity(false);
float groundspeed = velocity.xz().Length();

After determining the ground speed it is time for the footsteps code. We start off by checking if that a footstep sound is not currently playing by seeing if the current time is greater than the footsteptimer, we also make sure that the player is not jumping, and that the player is actually moving. If all of these cases are true then we check to make sure the player is moving fast enough to warrant a sound effect, so if the groundspeed is greater than 1.0 we play a footstep. Directly after we decide how much to increment the footsteptimer, if the player is running fast we will increment the timer more than if the player is walking, this is determined by whether or not groundspeed has surpassed a little bit more than movement speed:

//Play footsteps
if (Time::GetCurrent()>footsteptimer && jump==0.0 && normalizedmovement.Length()>0.0)
{
    if (groundspeed>1.0)
    {
        int soundindex = (int)Math::Random(0,3);
        footstepsound[soundindex]->Play();
        if (groundspeed>movementspeed*1.2)
        {
            footsteptimer = Time::GetCurrent() + footsteprunfrequency;
        }
        else
        {
            footsteptimer = Time::GetCurrent() + footstepwalkfrequency;
        }
    }
}

We finally have our complete player class. You can move around with the ‘W’ ‘A’ ‘S’ ‘D’ keys, run with Shift, jump with Space, and crouch with ‘C’. Hit F5 and see the game in action:

#include "MyGame.h"

using namespace Leadwerks;

Player::Player()
{
    //Create the entity
    entity = Pivot::Create();
    entity->SetUserData(this);
    
    //Initialize values
    standheight=1.7;
    crouchheight=1.2;
    cameraheight = standheight;
    move = 0.0;
    strafe = 0.0;
    movementspeed = 3.0;
    maxacceleleration = 0.5;
    sensitivity=1.0;
    smoothedcamerapositiony = 0;
    cameraypositionsmoothing = 3.0;
    cameralooksmoothing = 2.0;
    runboost = 2.0;
    jump = 0.0;
    jumpforce = 6.0;
    jumpboost = 2.0;
    footstepwalkfrequency=400;
    footsteprunfrequency=320;
    footsteptimer=Time::GetCurrent();
    running=false;
    crouched = false;
    landing = false;
    
    //Set up player physics
    entity->SetPhysicsMode(Entity::CharacterPhysics);
    entity->SetCollisionType(Collision::Character);
    entity->SetMass(10.0);
    
    //Player position
    entity->SetPosition(0,0,0,true);
    
    //Create the player camera
    camera = Camera::Create();
    camera->SetPosition(0,entity->GetPosition().y + cameraheight,0,true);
    
    //Load Sound files
    for (int i=0; i<4; ++i)
    {
        footstepsound[i] = Sound::Load("Sound/Player/Footsteps/footstep_shoe-softsole_metal_single_01_v0"+String(i+1)+".wav");
    }
    for (int i = 0; i < 4; ++i)
    {
        landsound[i] = Sound::Load("Sound/Player/Landing/footstep_shoe-softsole_metal_jump_0"+String(i+1)+".wav");
    }
    for (int i = 0; i < 1; ++i)
    {
        jumpsound[i] = Sound::Load("Sound/Player/Jumping/grunt_male01.wav");
    }
}

Player::~Player()
{
    if (camera)
    {
        camera->Release();
        camera = NULL;
    }
}

void Player::UpdateControls()
{
    Window* window = Window::GetCurrent();
    Context* context = Context::GetCurrent();
    
    //Get inputs from the controller class
    move = window->KeyDown(Key::W) - window->KeyDown(Key::S);
    strafe = window->KeyDown(Key::D) - window->KeyDown(Key::A);
    jump = window->KeyDown(Key::Space) * jumpforce;
    if (!entity->GetAirborne())
    {
        crouched = window->KeyDown(Key::C);
    }
    running = Window::GetCurrent()->KeyDown(Key::Shift);
    
    //Get the mouse movement
    float sx = context->GetWidth()/2;
    float sy = context->GetHeight()/2;
    
    //Get the mouse position
    Vec3 mouseposition = window->GetMousePosition();
    
    //Move the mouse to the center of the screen
    window->SetMousePosition(sx,sy);
    //Get change in mouse position
    float dx = mouseposition.x - sx;
    float dy = mouseposition.y - sy;
    
    //Mouse smoothing
    mousespeed.x = Math::Curve(dx,mousespeed.x,cameralooksmoothing/Time::GetSpeed());
    mousespeed.y = Math::Curve(dy,mousespeed.y,cameralooksmoothing/Time::GetSpeed());
    
    //Adjust and set the camera rotation
    playerrotation.x += mousespeed.y*sensitivity / 10.0;
    playerrotation.y += mousespeed.x*sensitivity / 10.0;
    
    //Prevent inhuman looking angles
    playerrotation.x = Math::Clamp(playerrotation.x,-90,90);
}

//Update function
void Player::Update()
{
    UpdateControls();
    if (entity->GetCrouched())
    {
        cameraheight = Math::Inc(crouchheight,cameraheight,0.1);
    }
    else
    {
        cameraheight = Math::Inc(standheight,cameraheight,0.1);
    }

    float maxaccel = maxacceleleration;
    Vec3 velocity = entity->GetVelocity(false);
    float groundspeed = velocity.xz().Length();
    float movespeed = this->movementspeed;
    //Run if shift key is pressed
    if (running) movespeed *= runboost;

    //Make sure movements are normalized so that moving forward at the same time as strafing doesn't move your character faster
    normalizedmovement.z = move;
    normalizedmovement.x = strafe;
    normalizedmovement = normalizedmovement.Normalize() * movespeed;
    if (entity->GetAirborne()) //Player is in the air
    {
        //if the player is in the air don't let them jump
        jump = 0.0;
    }
    else //Player is on the ground
    {
        //Play landing sounds
        if(landing)
        {
            landsound[(int)Math::Random(0,3)]->Play();
            landing = false;
        }
        
        //Give an extra boost if jumping
        if (jump>0.0)
        {
            jumpsound[0]->Play();
            landing = true;
            if (velocity.z>movementspeed*0.85)
            {
                normalizedmovement.z *= jumpboost;
                normalizedmovement.z = min(normalizedmovement.z,movementspeed*runboost);
                maxaccel = 10;
            }
        }
        
        //Play footsteps
        if (Time::GetCurrent()>footsteptimer && jump==0.0 && normalizedmovement.Length()>0.0)
        {
            if (groundspeed>1.0)
            {
                int soundindex = (int)Math::Rnd(0,4);
                footstepsound[soundindex]->Play();
                if (groundspeed>movementspeed*1.2)
                {
                    footsteptimer = Time::GetCurrent() + footsteprunfrequency;
                }
                else
                {
                    footsteptimer = Time::GetCurrent() + footstepwalkfrequency;
                }
            }
        }
    }

    //Set camera rotation
    camera->SetRotation(playerrotation,true);
    //Set player input
    if (running) maxaccel *= 2.0;
    entity->SetInput(playerrotation.y,normalizedmovement.z,normalizedmovement.x,jump,crouched,maxaccel);
    //Set the camera position & smooth
    cameraposition = entity->GetPosition();
    if (cameraposition.y>smoothedcamerapositiony)
    {
        smoothedcamerapositiony = Math::Curve(cameraposition.y, smoothedcamerapositiony, cameraypositionsmoothing / Time::GetSpeed());
        cameraposition.y=smoothedcamerapositiony;
    }
    else
    {
        smoothedcamerapositiony = cameraposition.y;
    }
    camera->SetPosition(cameraposition.x, cameraposition.y + cameraheight, cameraposition.z );
}

Characters and Worldbuilding: Analyzing the Strength of Japanese Games

$
0
0
From April 2010 to May 2011, I had the opportunity to study manga (Japanese comics) and video game design at one of Japan's leading art schools, Kyoto Seika University. In total, I lived, worked, and studied in Japan for about four years, and in this article I would like to share what I learned there about game design.

It all started in high school when a friend introduced me to anime (Japanese animation) through Studio Ghibli's "Princess Mononoke.” The characters and creatures were unlike anything I’d seen before, and soon I was borrowing more. From there I started playing Japanese video games, especially Super Nintendo-era role-playing games (RPGs).

The first one I played, chosen at random, was the curiously titled "Chrono Trigger." Little did I know I was picking up one of the most revered games of all time, and, of course, I was hooked. Similar to my interest in Studio Ghibli's animated movies, I was enchanted by the level of imagination that I saw in Japanese RPGs.

An Overlooked Treasure


Eventually I stumbled upon Paladin's Quest, one of the lesser-known Super Nintendo RPGs, released in the US in 1993. I almost skipped the game entirely, however, due to its generic name. I don't know how the translators settled on “Paladin’s Quest,” since the game has nothing to do with paladins; in Japan, it goes by the exciting and mysterious "Lennus: Memory of the Ancient Machine."

Paladin’s Quest turned out to be, in my opinion, the most original and memorable game in an era that was already brimming with innovation. The pastel color scheme and simple visual style seems to have turned off many players, but the novelty only added to the appeal for me; the haunting music, alien plant-life, and novel control scheme came together to create a unique and very engaging experience.


Fig1.png
Fig. 1 "The Magic School," first area of Paladin's Quest


For the Love of the Game


When I found out that Paladin's Quest had a sequel, “Lennus II: The Apostles of the Seals,” I was eager to play; however, the game had never been released in English, and with zero knowledge of Japanese, I couldn’t even make my way out of the first area.

That’s why, in freshman year of college, I enrolled in an "Intensive Japanese" class that culminated in a four-week trip to Kyoto Seika University. (At the time, Kyoto Seika had just made news for being the first school in Japan to offer a major in manga.)

I had a great time there, which led me to transfer to Stanford University to major in East Asian Studies and continue studying Japanese. As soon as I graduated, I moved to Japan to teach English with the Japan Exchange Teaching (JET) Program.

That was when I discovered the blog of Hidenori Shibao, the director of the Lennus series (Paladin's Quest and its sequel), who also worked on Legend of Legaia, a PlayStation RPG. After double-checking my grammar with a Japanese friend, I sent him a message, and we ended up exchanging a number of emails about the Lennus games and game design in general.

Back To Kyoto Seika


After two years with JET, I wanted to return to what drew me to Japan in the first place - popular culture like video games and anime. I had always loved drawing, and I remembered Kyoto Seika and their manga program from my study abroad trip. That’s how I ended up going back in 2010 as a research student in the Story Manga Department. (As a “research student” I took classes like a regular student, but without any grades or diploma upon completion.)

Although Japanese schools tend to be strict about taking classes outside your department, I was able to attend some lectures in a Game Design class with Kenichi Nishi, the director of the cult-hit “Chibi-Robo!” (He was also a field designer for Chrono Trigger.) Nishi walked us through development from idea to execution, and students formed groups to create their own games over the course of the semester.

Through all of this, I learned a lot about the Japanese approach to creating popular culture like manga and video games, and here I would like to share what I learned, focusing on two points: Japanese-style characters and their function in video games, and sekaikan, a term used frequently in reference to video games and other media.

Japanese Characters: More than Just “Cute”


Anyone who's been to Japan can tell you that characters appear everywhere: billboards, TV, clothes, trains, food (and not just the packaging, but often the food contained inside, too) - anywhere you can imagine. You can even find them in situations that in America might be considered slightly inappropriate, such as a flyer I saw that proclaimed something along the lines of "Let's reduce the number of suicides!" with a boy and cute green creature raising their fists in smiling determination.


Fig2.jpg
Fig. 2 The Pokemon Train

Fig3.jpg
Fig. 3 “Hello Kitty” Lunch Box Ideas


In America, Japanese characters like Mario, Hello Kitty, and the various Pokémon are, while popular, sometimes put down for being child-like or simplistic. Personally, I have always had a soft spot for them, but I didn't think they were anything more than an aesthetic style that I happened to like. Little did I know that characters in Japan wield great power, both in terms of money and as an important element in popular media like video games.

The huge demand for and production of characters constitutes what is essentially a character industry; video game, anime, manga, and merchandise companies work together in close coordination to create spin-offs, crossovers, and various products based around popular characters. This is referred to as “media mix” in Japan, or sometimes “transmedia storytelling” in the West. Through these collaborations, a character from a successful manga series, for example, can end up bringing in far more money than sales of the original manga itself.

At Kyoto Seika, my classmates were very much aware that creating popular characters was a crucial part of the manga-making process, and, as you can imagine, skilled character designers in Japan are highly valued. There are artists like Kosuke Fujishima of the “Tales” series of RPGs and Metal Gear’s Yoji Shinkawa (a Kyoto Seika alumnus), who have achieved great fame and recognition thanks in large part to their work designing characters. Artists such as these have their work featured in exhibitions, and can sell expensive art books of their sketches and designs.

The Making of Japanese Characters


What then is special about these characters? What qualities, if any, do they share? Some people may be familiar with the term kawaii, which is usually rendered in English as "cute,” and is often used to describe characters. Actually, it is not as simple as just being “cute”; my understanding of kawaii characters is that they are expressive, endearing, and easy-to-read. Large heads and eyes, simple, colorful designs, and exaggerated emotional reactions are some recurring stylistic elements of kawaii characters.

A less common but also important term is sonzaikan, which literally translates to “the feeling that something exists.” In terms of characters, it means that they seem real- not necessarily that they are just like real people with complex personalities, but more that they feel full of life, and provoke an emotional response from the viewer. In this way, people can feel a personal connection with their favorite characters, almost as if they are real friends.

In practice, how do designers make these kind of characters? Ian Condry, a scholar of Japanese popular culture, describes one example in his paper, “Anime Creativity: Characters and Premises in the Quest for Cool Japan.” There, he interviews m&k, the design team who created the characters for a popular animated show called “Dekoboko and Friends.”


[T]hey developed the characters by “auditioning” about 60 of them, that is, drawing up a wide range and selecting from them. “We avoided average characters, and aimed instead for those who were in some way unbalanced,” he explained... The creators also didn’t start with the visual image of the character, but instead thought in terms of a character’s distinctive flavor (mochiaji) or special skill (tokugi)… “[T]he personality (kyara) precedes the character itself, evoking the feeling of some kind of existence (sonzaikan) or life force (seimeikan)”… When m&k selected characters from among the many they auditioned, they emphasized the extreme: one character is extremely shy, another extremely speedy, another is an elegant older woman who sings traditional sounding songs, another is so big he can’t fit through the door.


In other words, each character is defined by a simple concept, which in turn determines both their behavior and appearance. The result is that, though simplistic, each character feels likable and real, as reflected by the show’s popularity.



Fig4.jpg
Fig. 4 “Dekoboko and Friends”


What Characters Are: Characters in Games
In his game design blog "What Games Are," Tadhg Kelly explains characters’ role and function in video games, and, based on his description, Japanese-style characters seem to me to be especially suitable.

First, it helps to understand his view on the role of story in video games. In his post "On Player Characters and Self Expression," Kelly explains what he calls "storysense":


"Storysense" is an approach to narrative which relies on the creation of an interesting world, a discoverable set of threads and bits of story, a minimalist approach to goal direction, but dispenses with dramatic plot and character development. It treats story as a backing track to the play of the game, and so the player can participate or not as he likes. There is no time given over to extrinsically rewarding the player for being in-character, and the only rewards are literal - just as the game is. There is no elaborate characterization, no attempt to insert unnecessary meaning, and no emoting at the player to try and make him or her feel.


That’s why, he explains in “Character Establishment,” characters in games don't need traditional development through a story:


Establishing character is not the same thing as character development. Character development in a dramatic arc is a long and complex process, but in a game it’s completely at odds with what a world needs to achieve. The art of establishing characters is conveying an impression of who they are in totality, because they are just a part of a portrait.

A game character needs to be established with a light touch, so that it’s the player’s choice to like or loathe at their own pace. Take that away, or foist exposition on the player, and intended feelings of sympathy quickly turn to antipathy or boredom.


As he sums it up in his post on “Character Development,” “the world is what develops and characters are (for the most part) just resources within it.”

Japanese characters, with their impression of being real (sonzaikan), and simple but engaging personalities (like the Dekoboko characters and many others), seem to me to be very well-suited for this kind of role. More so than physically and emotionally realistic characters, simplified and stylized kawaii characters help bring the gameworld to life for the “art brain” without distracting the "play brain" from trying to win with excessive dialogue or cutscenes.

Taking Care of Characters


Given characters’ importance in terms of gameplay and larger financial models, it should come as no surprise that great care is taken in handling famous characters’ image and use. I was personally surprised, however, to discover that there is an entire company, called Warpstar, devoted to managing Kirby. Warpstar collaborated with HAL Laboratory, for example, in the making of “Kirby's Epic Yarn” for the Wii. In that case, designers spent an incredible three months perfecting Kirby’s appearance.

This careful attention to characters can also be seen in interviews with the team who made “Zelda: Skyward Sword.” The designers discuss working to make even the enemies kawaii, to give them "a touch of humanity" that makes you like them even as you want to defeat them.

I’d like to emphasize here that kawaii and sonzaikan are not inseparable, though they often go hand-in-hand in the case of Japan. Kawaii is certainly an important part of what makes Japanese characters feel alive and real, but I imagine that there are other approaches to giving your characters sonzaikan, too.

Sekaikan: Developing Your Game’s “Worldview”


In addition to discussing characters, I would also like to introduce the term sekaikan, which roughly translates to "worldview.” It is, in my opinion, the single-most important word in understanding Japanese game development.

I first encountered sekaikan on Hidenori Shibao's blog in a post about the development of Paladin's Quest, where he explains that his goal was to make the world the main character of the game. Toward that end, he tried to make everything feel novel and new, from the battle menus, which can be navigated solely with the directional pad, to the otherworldly character designs and concept art. He approached all elements of the game, he says, with the goal of creating a good sekaikan.

I soon discovered that other developers and gamers in Japan often use the term in similar ways. But, I wondered, what does it mean for a game to have a good sekaikan? Through interviews, research, and my personal experiences, I have developed a definition of the term, which can be broken down into two parts:

First, sekaikan can refer to the feeling that a world exists beyond what you see onscreen. This is similar to characters’ sonzaikan; the world feels alive and real, even though it may not be a physically accurate simulation of reality.

At other times, people use sekaikan to refer to the creator’s vision for the game’s world, which informs everything from the visual design, to the rules, to the controls, to the story. It is the overall “feel” of the world, created by the the interaction of all these elements.</p>

Having a good sekaikan, then, means that the game feels real in the first sense, but is also interesting and well-done in the second sense. Often this means that the player wants to re-visit the world, whether through replaying the game, playing other games in the series, or through other media.

Sekaikan in the Western World


Given its frequent use in Japan, there is surprisingly little discussion of sekaikan in the English-speaking world. I found only a few mentions of it on the internet at large, here and here.

In “Anime Creativity,” however, Ian Condry touches on sekaikan and its role in the creation of anime, which sheds some light on its importance in video games.

He describes what it was like to sit in on discussions among the production team of an anime series called “Zenmai Zamurai.” What he noticed was that, rather than thinking about the anime in terms of the story- which is the focus for many fans and reviewers - the developers used a "premise + worldview + characters" model. As Condry explains:


More central than the story itself in organizing the collaborative production of anime was… the design of characters, the establishment of dramatic premises that link the characters, and the properties that define the world in which the characters interact. This combination of characters (kyarakutaa), premises (settei), and world-settings (sekaikan) generally came prior to the writing of the story.


Condry goes on to mention that a similar model could be used for games, and although he doesn’t specify what that might be, his research shows the crucial role of sekaikan in the development of popular culture like anime and video games. It is established in the beginning of the process, and it informs secondary elements like the story and visuals.

In addition, although Kelly never uses the term sekaikan on “What Games Are,” he emphasizes points similar to those that Japanese developers consider when addressing sekaikan. First, he essentially offers a definition of sekaikan in his post titled “Worldmakers”:


We often infer more from a game experience than is actually on the screen. We have the capacity to use the game as an imaginative springboard, inferring personality traits, characters, behaviours and a sense of a larger game world beyond even what the developer intended. We make cognitive leaps, little observations and associations that contain the quality of empathy, and so it feels like there is more there than meets the eye.


He goes on to insist that game developers are “worldmakers,” and that “[t]he art of game design is all about the place,” explaining that:


[C]reating a world is a complex task that demands not just imagination, but elegance, form and direction... It is not just geography or art style, and it has very little to do with story. It’s a balance of dynamic, audio-visual and pace-changing elements all intended to achieve a kind of engagement.

Worldmaking is the act of creating a canvass… Much of my book talks about the art of games as worldmaking because I believe that worlds are where the true art of games lies.


To me, this sounds similar to the focus in Japan on the world experienced through a game. Kelly is the only person I am aware of in the Western world with a theory of games that takes this into account and explains the fun of games in a way that applies to both Western and Japanese games, FPSs and RPGs, The Sims and Harvest Moon.

Game Design at Seika


Finally, I wanted to share some practical lessons that I learned in Kenichi Nishi's Game Design class at Kyoto Seika University.

As each student came up with their own idea, he instructed us to assign a theme to our game, such as “getting bigger” or “becoming friends.” At first, I thought it was ridiculous to summarize an entire game in such a way, but I soon realized that, as a simple expression of your “worldview,” it can actually help provide the game with structure and consistency. He said that in Katamari Damacy, for example, the basic theme would be “getting bigger,” which is reflected in the story, visuals, and gameplay.

At the same time, Nishi encouraged us to consider the world in our game from all angles: What is the political system? What is the weather like? What is the source of energy? What type of clothes do people wear? And so on, even if the element in question doesn’t have anything to do with gameplay and won’t appear in the game.

In this vein, he encouraged us to think carefully about what items we would include in the game’s world. In one class, we discussed a student’s idea for a game about a cute character that flies through the sky, collecting hearts while avoiding obstacles like eyemasks. When Nishi questioned her inclusion of eyemasks, however, she didn't have an explanation. Eyemasks, he said, changed everything- they bring in the existence of human beings who wear eyemasks at night, adding a whole world of associations.

His point was that each object changes the user's understanding of and response to the game's world, and so it is dangerous to thoughtlessly include something just because it would make sense in reality or because other games have done the same thing. Each element must fit with your game’s sekaikan in order for the world to feel real and make sense on its own terms.

A Concrete Example: Cafe Murder


Though I have been using phrases like “Japanese characters” and “the Japanese approach to video game development,” I don’t want to imply that these things are fundamentally Japanese or inaccessible to people outside Japan. On the contrary, I think that while Japan has developed these techniques and approaches, anyone in the world can learn from and utilize them.

In fact, as artist and designer for an upcoming iOS game called Cafe Murder, I have had the chance to try to put these concepts into practice over the course of the past year.

Twelve Unique Customers


Cafe Murder, which began as a Kickstarter project, is a restaurant simulation game with a chef who attacks the customers. I built each of the twelve customer characters around a simple concept that defines their appearance and how they behave, similar to what Condry described with “Dekoboko and Friends.” Visually, I tried to make each character colorful and easily recognizable, expressing their emotions in a way that suits their personality and concept.

Frantic Fred, for example, is the “busy” one, a rushed businessman who has to be served right away. Sharkeeta, the wealthy and corpulent connoisseur, represents gluttony, and Pincushion, unlike all the others, will both survive and enjoy a stabbing.


Fig5.jpg
Fig. 5 Frantic Fred

Fig6.jpg
Fig. 6 Sharkeeta

Fig7.jpg
Fig. 7 Pincushion


I also tried to keep in-your-face dialogue to a minimum, while adding a customer feedback menu that can be viewed at the player’s leisure if they choose to read through and learn about each customer’s personality.

Sekaikan, Cafe Murder-style


I started designing Cafe Murder in Game Design class at Kyoto Seika, and at that time I chose the theme “to serve and protect.” Although the basic gameplay revolves around using quick reflexes to catch ingredients and assemble sandwiches, getting good at the game requires understanding and catering to each character’s individual behavior in order to serve sandwiches with maximum efficiency and satisfaction.

Knowing, for example, that you should serve that imperfect sandwich to Fred before he gets too impatient, whereas it is worth taking longer to get Sharkeeta’s sandwich just right because of her generous tips- all while protecting them from the insane chef’s rampages.

In terms of concrete design decisions related to sekaikan, I remember one in particular. When designing the “Speed Shoes” upgrade for Rainy the waitress, I originally began drawing sneakers with Hermes-looking wings on them; I soon realized, however, that this felt too mythological, reminiscent of an action platformer like Kid Icarus.


Fig8.jpg
Fig. 8 Speed Shoes with Hearts


It didn’t feel right to have speedy wing shoes in the world of Cafe Murder, whereas cute, speed-boosting hearts made a lot more sense. Cafe Murder, I realized, is a world where loving care for customers is more powerful than magical wings. If you’d like to check it out for yourself, Cafe Murder is available now in the App Store.

In Conclusion…


In terms of both the approach to characters and the use of sekaikan, I see one overarching theme in Japanese game design: simplification and stylization as a means of creating a sense of reality and expansiveness.

The characters, both in terms of personality and appearance, are simplified and exaggerated, expressive and easy-to-understand. Game-worlds are organized around simple concepts, and items in the world are chosen or rejected with great care.

The effect of these measures, however, is to create the feeling that there is more than meets the eye; characters feel endearing and real, and popular ones are consumed almost endlessly. Similarly, the worlds in games feel alive, and players want to visit them again and again through different media.

I hope I have been able to convey clearly what I learned during my years in Japan. Thank you for reading, and please let me know if you have any comments, criticisms, or insights about the ideas I’ve presented. If you enjoyed the article, feel free to check out my blog about the worlds that games create at gamemakeworld.wordpress.com

3ds Max 2014 Review

$
0
0
Autodesk recently released the latest version of 3ds Max. This latest version, called 3ds Max 2014, is available as both Standard and Design versions. The new version includes a host of new features and several clever improvements to the existing tools.

Among my favorite new features are the Populate tool, which can instantly add crowds of people to the current scene. I'm also quite taken with the new mParticles added to the Particle Flow interface. Other new features and improvements include a set of enhanced menus, an interactive search command feature, vector maps, a great new Perspective Matching tool, support for DirectX 11 and an automatic Mesh Inspector for making sure that saved data files are free of errors.

Enhanced Menus


Although I typically frown on changes to a software's user interface, especially in the menus, but the new enhanced menus in 3ds Max 2014 are implemented as a new workspace that you can switch to. If you want to keep the old menus, then they are still there as another workspace and switching between the two is easy.

The new enhanced menus are better organized and they include icons and fewer submenus. As an example of the better organization, there is a new Objects menu that has all the different objects you can add to a scene in one convenient place. You can also pull any submenu away from the menu bar and make it free standing and instantly accessible.

Instant Crowds with Populate


Imagine that you've spent months modeling a new building and on the final approval, the review team suggests that you add people throughout the building and grounds to give it a lived-in quality. This sounds like a simple addition, but as an architecture expert familiar with open spaces, wiring conduits and heating and cooling ducts, making a person walking realistically is quite a leap from your familiar world.

At this point you have several choices. You could delay the project for six weeks while you find a character animator that can help add the people to the scene or you could simply upgrade to the latest version of 3ds Max that includes the new Populate feature. This new feature will enable you to finish the job in a matter of minutes.

The coolest new feature found in 3ds Max 2014 is the Populate feature. This new tool lets you add animated crowds of people to any scene. Although the results are quite complex, the crowd creation is very easy to use.

Populate lets you drag to create Flow paths where the people walk and Idle areas where crowds of people stand about doing tasks like conversing or talking on a cell phone. Flow paths are created like a simple spline path by clicking at the start and end of each path section. There are settings for the width of the Flow paths and you can edit existing paths. There is also an option to raise or lower Flow path segments to make ramps. Idle areas are created using a brush with modes to add and remove areas from the defined area.

After the Flow and Idle areas are defined, you simply press the Simulate button and a random assortment of pre-built characters automatically appear within those areas. These characters are already animated and begin moving as you scrub the Time Slider. The characters walk along the Flow paths and are animated with different actions while standing still in the Idle areas. For both areas, you can set the Density and the mix of males and females.

If you see repeating characters, you can select a character and use the Regenerate Selected button to replace the selected character with another random character. There is also an option to make the characters appear as stick figures (which don't render), normal mesh or high-res mesh. One drawback to the Populate feature is that you cannot edit or load your own custom people into the system, but you can change the materials applied to the people.

This feature has a coolness factor to is, but the amazing thing about it is just how simple it is to use. Just drag to define the areas and click Simulate and the characters appear and start moving about the scene. This is fantastic for architectural presentations that need a human element.


Attached Image: Figure 1 - Populate.jpg
Figure 1: The Populate feature lets you easily add walking and idle crowds to the current scene.


Search Command Field


Sometimes, it is the smallest new feature that makes the biggest impact. 3ds Max 2014 includes a simple new feature called Search Command. You can access this simple text field from the Help menu or directly by pressing the X key. The text field box appears directly under the cursor and lets you begin typing text immediately. Beneath the text field, the software presents a list of commands that match the text that you've typed. You can continue to type or select a command from the list and the selected command is executed.

This simple feature provides another innovative way to work with the interface. It also puts all the commands at your fingertips without having to comb through the help files to find the location of a specific command, which makes it ideal for users that are familiar with another 3d program, but you are new to 3ds Max.

Eliminating File Errors with Mesh Inspector


Once of the most frustrating aspects of working with 3ds Max happens when the file you are saving gets an error and crashes the system. These files can behave problematic. Sometimes they crash your system and sometimes they can't be recovered making it so you need to redo all the work again.

To address this potential problem, 3ds Max 2014 includes a new Mesh Inspector that automatically checks and fixes problems with texture channels and geometry inconsistencies in the file. The Mesh Inspector can be turned off if needed. If enabled, the Mesh Inspector works in the background as the file is saved automatically detecting and fixing errors. The Mesh Inspector only works with Editable Poly and Editable Mesh objects.

Perspective Matching with Background Images


Another major new feature in 3ds Max 2014 is the new Perspective Match feature. This feature lets you interactively match the current scene to a loaded background image. The feature adds two guides for each axis to the scene. By manipulating these guides, the scene is changed to match the background.

This makes it easy to orient the scene to the background image. It is best to use a background image that has clearly identifiable axis lines like a cityscape. This new matching feature is so much easier to use than the old Camera Match utility. The difference is the interactive lines that make the manipulation easy.


Attached Image: Figure 2 - Perspective Match.jpg
Figure 2: The Perspective Match feature lets you easily orient the scene to match the loaded background image.


Enhancing Particle Flow with mParticles


Particle Flow is a great interface for manipulating particles and now it can use mParticles, which are particles that can be used with the MassFx physics engine. This lets you add physics simulations to Particle Flow resulting in realistic effects.

New to the Particle Flow depot are nodes to create an mParticles Flow. Dropping this node into the Particle Flow interface creates a new flow of mParticles. The shape of these new mParticles can be either 2D or 3D with numerous options including circles, stars, letters, digits, spheres, musical notes, and hearts.

mParticles can be set to interact with existing scene geometry by applying a PFlow Collision Shape (WSM) modifier. This lets the scene geometry detect and react to collisions and other forces between the mParticles and the scene geometry. Other operators include Force, Drag, Buoyancy, Switch, InterCollision, Glue and Solvent.

Scene geometry can be also endowed with the Particle Skinner modifier, which allows the geometry to be deformed by the impacting mParticles.

The Particle Flow interface also includes new operators to Cache the simulation to the disc or to save out cache data for a specific selection. These operators let you save the simulation data separate from the 3ds Max file.


Attached Image: Figure 3 - mParticles.jpg
Figure 3: The Particle Flow interface now works with mParticles that can interact with scene geometry objects using the MassFx physics simulation engine.


Vector Maps


3ds Max has long been able to use bitmaps as textures on objects, but finally, they can also load and use vector-based images. These images have the distinct advantage of being resolution independent, which keeps them from pixelating when the object is zoomed in. The supported file types include AutoCAD PAT files, as well as, Adobe Illustrator (AI), SVG and SVGZ.


Attached Image: Figure 4 - Vector map.jpg
Figure 4: Vector-based images can now be used as maps to texture objects.


iray Renderer Improvements


The iray renderer continues to improve with each new version of 3ds Max. New with this version is support for many of the procedural maps that weren't previously supported including Checker, Dent, Marble, Tiles, Waves and Wood.

Nitrous Viewport Improvements


The big improvement with the Nitrous viewports is support for Direct3D 11 including support for DirectX 11 shaders.

Another small improvement is that a message appears on the prompt line when autosave is saving a backup file and, more importantly, it gives you a chance to cancel the autosave with the Escape key. I've run into this before. It seems that autosave always decides to save the current scene at precisely the worst time, like when you're trying to match up two large assets. Now if that happens, you can quickly cancel the autosave with a single key press.

The Isolate tool has also been improved including an option to Isolate Unselected. There is also an option to disable the Zoom Extents on Isolate, which keeps the objects the same size when engaging Isolate mode.

Summary


When I first heard that 3ds Max 2014 had enhanced menus, I got really nervous. As a long-time user, I was terrified that the overhaul of the interface would have me floundering around looking for commands that I was already familiar with, but happily the enhanced menus weren't forced on me and the old menus are still there. I found that the new menus made a lot of sense and with the new Search Command field, I didn't miss a step in my work.

I was also delighted to find the new Populate and Perspective Match features. Both were immediately accessible and solved specific needs that I had without a huge overblown interface.

Many of the other improvements were like gems to find like support for vector maps. They made working with the software easier in ways I hadn't even thought of.

3ds Max 2014 and 3ds Max Design 2014 are both available as a stand-alone products. 3ds Max 2014 is also available as part of the Entertainment Creation Suite, bundled with Autodesk Maya, Mudbox, MotionBuilder, Softimage, and Sketchbook Designer. For more information on 3ds Max 2014, visit the Max product pages on Autodesk’s web site at http://usa.autodesk.com. A free trial version of 3ds Max is also available at www.autodesk.com/3dsmaxtrial.

General Tips on the Process of Solo Game Development

$
0
0
As a solo game developer, it's important to learn as much as you can about everything you can, including the process you follow when making games. The process I will discuss consists of the following steps:

  1. Idea
  2. Prototype
  3. Iteration
  4. Playtest
  5. Finish

I hope to provide you with helpful tips on each stage of this process so that you can improve the speed of development and the quality of your games.

The Process, Step By Step


First of all, it's important to note that this isn't a linear "waterfall" process. It is very organic. While you will always start with an Idea, the following steps of Prototyping, Iteration, and Playtesting will happen in an unorderly fashion. This is good because it encourages experimentation and goes against the rigidity of a game's design, which allows positive evolution of the game. Over time, you'll get a feel for how you usually follow the process and how you can improve.

Step One: Idea


Beginner game developers sometimes think this is the most important step in the process, and that all development should halt while an idea is being thoroughly explored. As a result of this, they tend to overdesign (ex. huge game design documents), scale it too large for their skills, and overestimate the value of ideas. This also makes a design much more rigid, which ends up affecting the game negatively in the long run.

While the idea is important, it is most important to understand how little you can learn without making a prototype and playtesting that idea. You'll find that ideas in your head sound really fun, but actually playing a prototype of that idea ends up being really dull. You'll also find that a prototyped "dull" idea can be really fun!

During this stage, make sure that you have your goal and constraints clearly defined. For example, "make a fun game" is not a very clear goal, while "make a game in a month that uses the 'Flowers' theme and preferably utilizes the new COM system" is. The second example also shows good constraints. Some very immediate constraints that apply to most everyone are making the game in a certain amount of time ("before death" even) and staying within (and very slightly exceeding) your skill range. Constraints can help you think of more ideas, but it is often good to fluctuate the amount of free movement you have on each game project.

Inspiration is a word that often comes up when talking about ideas. Just remember that inspiration comes from anywhere, such as your life, other media, game design theory, or even random generation! The most important thing you need to know about "inspiration" is to not jump on an idea! When you are "inspired", your brain is pumping in those reward chemicals for your successful idea generation, which makes it very hard to see the bad in the idea! All you have to do is give it time. No matter how good it sounds at the moment, give it at least three or four days before doing anything more than thinking about it and writing it down.

At a lower level, it is important to focus on the mechanic(s) of your idea. Remember, you're writing a game, not a story, so you need gameplay! If you cannot think of good gameplay for that story, maybe that story is best expressed using another form of media.

When you get an idea and have waited a sufficient amount of time (and it still sounds good), the next thing you need to do is scale it down to its most basic form. Look for what makes you really like that idea. You need to find the most primal, core element(s) that makes that idea so appealling to you. This is the first thing you need to prototype as your entire game will be relying on these elements!

On a more somber note, remember that you will die. Keep that in mind, and make the game as if it was your last.

Step Two: Prototype


If the Idea section hasn't nailed it into you enough, I will repeat: you cannot see the potential (positive or negative) of an idea until you prototype it! Thankfully, this is often the most fun step in game development because you get to see your idea come to life! However, it is still critical to prototype correctly in order to use it most effectively.

When prototyping, you must be open to failure. If you fear failure, you will never release that game. I like to keep my prototypes outside my source-controlled Development folder and in my Desktop folder, just because it emphasizes the temporary playfulness that prototyping needs to succeed. If you mess up, it's important to tell yourself that that is expected and completely OK!

With this openness to failure, you also need to develop your prototypes faster, because chances are you will experience that failure many times before you create a winning prototype. A ton of speed can be obtained by optimizing your code. I don't mean performance optimizing, but life optimizing. You want to engineer your code for maximum reuse and simplicity, which means building a game library full of useful reusable code, and making a game template that you can copy and reuse. This game template should include an open window with rendered sprite, input, and a working makefile etc. You are going for convenience!

Also, you should avoid using "easier" tools for prototyping; rather, stick with what you know and have the most experience with, because that will make the fastest (and most useful) prototype. I write most of my prototypes in C++ because I have four years of experience with the language and the library.

Before you touch a line of code, make sure you have a well-defined question that you want the prototype to answer. This question should be very simple and involve the most important and/or "risky" parts of your game idea first. Optimally, you should write down the question somewhere clearly visible so you do not get off track. As soon as the prototype answers this question and explores a bit, you should move on to another.

Once you've made the prototype(s) that affirms the power of the idea, find the core element that affirms it and cut everything else away. Simplicity is the key to an elegant design!

Step Three: Iteration


At this point, you've made enough prototypes to confirm the positive potential of your idea. You've also confirmed the technical feasibility of the idea and how you might approach implementing it in production code. You've also found what you most enjoy about the idea and trimmed away what wasn't necessary to the experience. If you haven't, go back and make more prototypes!

When I say iteration, I mean the development and refinement of the final product. You should avoid using a prototype for your production code because that prototype was tailor-made to answer one question, not to become an entire game. At this point in development, you are using your most sustainable coding practices to build the production game. There are many articles that teach these practices, so now is the time to use what you've learned about high quality coding. The prototype was not the time to use these practices because they are going to be thrown away anyways!

Iterations should go from most important elements to least important (like prototypes). If you code a main menu or fancy graphics before you have your main mechanic running, you are doing something very, very wrong! No one cares about your graphics or your menues, they care about the gameplay! Make the "toy" first, and save the menues and polish for last.

Though you are now working on the "final" code, you should still experiment. Be very careful with such experiments as they can make you lose your vision of that core element that you focused on before. Make sure that you know why your game is engaging. Nine times out of ten it's not because you have realistic ragdoll physics or you can mine metals underground! Your experiments should be made seperate from production (if you're using source control, this is where "branching" becomes handy) and easily undoable. You should focus such experiments on improving the core element only! If the experiment is too divergent from the core element, you should write it down and prototype it later.

Do NOT, under ANY circumstances, prematurely optimize! As a programmer, you've probably heard of KISS, which stands for "Keep It Simple, Stupid!". Follow this principle as closely as you can! Jonathan Blow talks about algorithms and data structures and how they are optimized for performance or memory, but are not "life optimized". It is more important to finish a game in a month that runs at 20 FPS than finishing it in a year for 60 FPS!

When it comes to coding during the Iterative cycle, you want to recognize systems that can easily be reused in future games and add them to your library rather than making them game dependent. This will not only improve your library, but also make every subsequent game (and prototype) faster to develop. Don't try to reuse systems that are too specific though!

While furthering your game library through Iteration, focus on features that a) are useful not for their performance, but for what they accomplish and b) encourage code reuse and shorter development times. For example, collision detection and resolution is extremely useful and worth taking the time to implement, but implementing an "optimized" collision routine is not. Remember, don't make it unless you need it! As an example of (b) features, I recently implemented my own COM (or CES) system because I saw how much easier code reuse becomes with it. Sure, you take a performance hit, but the idea of dynamic, portable and reusable components makes up for it in life optimization!

If you're this far, there isn't any reason to scrap the project! Sure, you've learned a lot from coding mistakes and a complete recoding would probably do the project good, but don't waste your time! Many beginner game devs continuously scrap projects right in the middle of development. This is the main reason why they are still beginner devs! You've hit what's commonly known as "the Wall", the point at which you no longer want to continue development on a project. Just push through it, or all of the work you've done will be worth nothing (to consumers, at least)! I've released seven games, and two of them ended up being hell near the end. I pushed through it though, and learned from my mistakes. You MUST completely finish your projects! Do NOT give up!

Iteration will be heavily influenced by the next step, which is...

Step Four: Playtesting


I have to admit, I have a lot of trouble with playtesting. It's quite difficult to do it correctly, but when done correctly it will end up massively improving your game. Playtesting is quite straightforward, so I will present a few rules of thumb you can use to do it more effectively.

First, playtest early, playtest often! You can never playtest too much! Playtest as early as the prototype stage, and as late as release. You'll find that parts of your game that make perfect sense to you will completely confuse others, so do not develop in a vacuum!

Next, diversify! This includes people and technology. Run on as many devices and operating systems as you can. One of my games worked well on my two computers (with quite different specs) and virtual operating systems but had game breaking movement bugs on most other computers! When it comes to people, do not eliminate any demographic. Let anyone play your game, no matter what gender, age, or interest! The more diverse your playtesters are, the more diverse your audience will be, and the less biased your game will be.

Don't burn out playtesters. After a while, they get quite biased, especially because they've seen earlier versions of the game. You want the playtester to have a clean slate and an open mind, so make sure to switch out people often.

Observation is the most important thing you need to learn how to do during a playtesting session. Watch when they speak, when they stop speaking, when they die, when they succeed, and when they stop playing. Chances are most people will do the same, even though you effect them because you are there. Observation is also the only thing you should do during a playtesting session. Do not give them a background, or explain something, or help them out, or anything. Just watch, and take notes.

After they are finished playing, you should ask them the following three questions:

  1. What did you enjoy?
  2. What did you hate?
  3. What confused you?

That gives you very clear feedback on what you need to emphasize and what you need to change. Also, when the player gives you a suggestion for improving the game, focus not on the suggestion but what made them give you that suggestion.

Step Five: Finish!


If you've made it this far, you're ready to finish your game! The last 10% of development often feels like 90%, but you must not give up! Remember that even a "90% completed" game is worthless! Finishing is all about pushing yourself through the Wall and getting all of the loose ends wrapped up. Look back at how far you've come already, and look how far you have to go. It's not that far away!

While you put the finishing touches on and zip it up for the first time, test on as many platforms as possible. You need to test everything, even your "readme" files (one time my newlines were Unix-style and didn't work on Windows computers :) )!

You want the path a player takes from discovering your game to playing it as short and smooth as possible. Lower all of the barriers you can, and make it extremely easy for them to check out your game. Convenience is key at this point.

If you're wondering why your game still looks and feels like a "hobbyist" project, it's because it isn't polished enough. Very small things, like menues or loading screens, can make a big difference on the player's feelings towards your game. Remember, the very first thing the player will see will be one of those things, so it is very important to get it right. Study "professional" games and why they look more "professional". Polish is largely by feel, so you will just need to practice to get it right, but it is well worth the extra time.

If you're extremely embarassed by the "quality" of your game or know that it has serious flaws, fix as much as you can and push it out there. Finish your project, learn from your mistakes, and do better on the next game.

Conclusion


You've now made a game! Congratulations! Now do it again, only avoid the mistakes that hurt the quality of the last game.

I've covered a lot, so I'm just going to do a quick outline on each step:

  1. Idea
    1. Prototyping is still essential
    2. Set a well-defined goal and constraints
    3. Give it some time
    4. Find the core element that makes the idea shine
    5. You will die, so make it count!

  2. Prototype
    1. Be open to failure
    2. Do it as quickly as possible, and make the code as convenient as possible
    3. Use the tools you already know
    4. Have a well-defined question to answer

  3. Iteration
    1. Make high-quality code
    2. Go from most important elements to least important
    3. Experiment, but don't lose vision of the core element
    4. KISS/life optimize/NO premature optimization
    5. Add reusable code to your game library
    6. Only add features to your game library that make development faster or add capabilities (do not code something for speed unless you need it)
    7. Do not abandon/give up on a project at this point!

  4. Playtesting
    1. Playtest early, playtest often
    2. Diversify your playtester demographics
    3. Swap out overused playtesters
    4. Ask questions

  5. Finish
    1. Push through it
    2. Test, test, test!
    3. Make it easy to start playing
    4. Add polish
    5. Release it and learn from your mistakes

Thanks for taking the time to read this! I hope I helped you!

Note:  This article was written with the help of many others. There are so many that I cannot list (or remember, for that matter) all of them. If you know what article I was talking about or want me to add links, just PM me or add a comment.


Article Update Log


10 Jul 2013: First Release

Advanced Python Part Two: Utilizing Recursion

$
0
0
Python has a surprisingly great ability to perform large recursive operations, allowing for algorithms to be faster and easier to implement. In this article, I discuss the basics of recursion in python and applying divide and conquer to the sorting problem.

The Basics of Recursion


Before I start on advanced recurion topics, I'm going to discuss the basics of programming recursive functions.

First, there is the base case. A base case is how your function stops recursively. An example of a recursive function (which lists all the numbers between one and one-hundred) with a base case:

def count(basenumber):
	if basenumber >= 100:
		return
    print int(basenumber)
    count(basenumber + 1)

count(1)    

and if it didn't have a base case:

def count(basenumber):
	print basenumber
    count(basenumber)

count(1)

The first piece of code prints out all the integers between the number you specify and one-hundred. The second doesn't have the Base Case which terminates the function eventually, making it run forever. To be more clear, this was the base case:

if basenumber >= 100:
	return

This makes sure that the function stops at one-hundred. It is important to establish your base case when you are programming a recursive application.

This section should (hopefully) explain some recursion basics.

Using "Divide and Conquer" in Recursion


Divide and Conquer is an algorithm design paradigm which easily reduces problems into O(n log n) or O(log n) running time. If you don't know what these are, just know that they are very fast and often better than how fast a "brute-force" algorithm would be.

Divide and Conquer means to divide your problem into a large amount of smaller sub-problems, and then solve the original problem using the solutions to these smaller problems. This often relies on recursion. As an example of Divide and Conquer, there is Merge Sort.

If you were to try the "brute-force" solution to sorting numbers (called selection-sort), it would be an O(n^2) algorithm (or quadratic). This is considered slow compared to how fast we can get with divide and conquer (merge sort). Using merge sort, we can get O(n log n).

Selection Sort must loop over the list of numbers to sort for every number in the list...

def selectionSort(a):
    # Go through all positions except the last one 
    # (that one will automatically be correct)
    for index in range(len(a)-1):
        value = a[index]
        # enumerate all (index, value) pairs from the rest of the list 
        # and take the pair with the smallest value
        min_subindex, min_value = min(enumerate(a[index+1:]), key=lambda x: x[1])
        if min_value < value:
            a[index] = min_value
            a[min_subindex + index + 1] = value

(That was fairly advanced selection sort code, and it can be confusing)

...while merge sort recursively calls itself, dividing the problem until there is a bunch of lists with only two numbers, and then sorting those lists. It then merges these lists in linear time.

def mergesort(numbers):
    if len(numbers) == 1:
        return numbers
    elif len(numbers) == 2:
        if numbers[0] < numbers[1]:
            return numbers
        else:
            return [numbers[1], numbers[0]]
    else:
        length = len(numbers)
        if length % 2 == 0:
            list_one = mergesort(numbers[:(length / 2)])
            list_two = mergesort(numbers[(length / 2):])
        else:
            list_one = mergesort(numbers[:(length - 1)])
            list_two = mergesort(numbers[(length - 1):])
        result = []
        one_traversal = 0
        two_traversal = 0
        for traversal in range(len(numbers)):
            if one_traversal == len(list_one) and two_traversal == len(list_two):
                break
            elif one_traversal == len(list_one):
                result.append(list_two[two_traversal])
                two_traversal += 1
            elif two_traversal == len(list_two):
                result.append(list_one[one_traversal])
                one_traversal += 1
            else:
                if list_one[one_traversal] > list_two[two_traversal]:
                    result.append(list_two[two_traversal])
                    two_traversal += 1
                else:
                    result.append(list_one[one_traversal])
                    one_traversal += 1
        return result

Look at the two base cases at the top. Then look at the merging routine. Merge sort can be confusing, and I recommend you read up on it more.

Interesting Points


Hopefully you understand Python recursion with Divide and Conquer more. I tried to keep the article brief and it may be confusing, so I recommend you ask questions below.

Conclusion


This was a brief introduction to Python algorithms with Divide and Conquer. Try implementing Strassens Matrix Multiplication algorithm (look it up) or an algorithm for counting inversions if you would like to learn more about the divide and conquer paradigm.

Article Update Log


July 14 2013: Released Article

Advanced Terrain Texture Splatting

$
0
0
In this article I will explain a texture splatting algorithm which allows you to create more natural terrain. This algorithm may be used in shaders of 3D games as well as in 2D games.

One of the most common ways of terrain texturing is blending multiple tiled layers. Each layer has an opacity map which defines the extent of texture presence on the terrain. The method works by applying an opacity map to the higher levels, revealing the layers underneath where the opacity map is partially or completely transparent. Opacity map is measured in percentage. Of course on each point of a terrain the sum of opacities of all layers makes one-hundred percent as the terrain can't be transparent. Instead of tile textures, the opacity map stretches entirely on all terrain and therefore has quite a low level of detail.

Now we will pass to the most interesting part — algorithms of blending of textures. For simplicity and obviousness our terrain will consist of sand and large cobble-stones.


Attached Image: 1.jpg


Simplest way of blending is to multiply texture color with opacity and then sum results.

float3 blend(float4 texture1, float a1, float4 texture2, float a2)
{
    return texture1.rgb * a1 + texture2.rgb * a2;
}

Such a technique is used in Unity3D in the standard terrain editor. As you can see, the transition is smooth but unnatural. Stones look evenly soiled by sand, but in the real world it doesn't happen like that. Sand doesn't stick to stones, instead it falls down and fills cracks between them, leaving the tops of stones pure.

Let's try to simulate this behavior in Excel plots. As we want sand to be "fallen down" between cobble-stones, for each texture we need the depth map. In this example we consider the depth map is generated from grayscaled image and stored in the alpha channel of a texture. In Unity3D it can be done in the texture inspector by setting the flag "Alpha From Grayscale".

First of all we will consider the simplified model of depth map of sand and stones.


Attached Image: 2.png


The blue line on the plot symbolizes the depth map of sand and red is cobble-stones. Notice that tops of stones lie higher than sand level. Considering this fact, we will try to draw pixels of that texture which is above.

float3 blend(float4 texture1, float a1, float4 texture2, float a2)
{
    return texture1.a > texture2.a ? texture1.rgb : texture2.rgb;
}


Attached Image: 3.jpg


Excellent! Tops of cobble-stones remain pure whereas sand lies in cracks between them. But we didn't consider layer opacity yet. To use it we just sum depth map and opacity map.

float3 blend(float4 texture1, float a1, float4 texture2, float a2)
{
    return texture1.a + a1 > texture2.a + a2 ? texture1.rgb : texture2.rgb;
}

At the expense of summation less transparent texture will be higher than usual.


Attached Image: 4.png

Attached Image: 5.jpg


So we have a more natural transition from sand to stones. As you can see, grains of sand start filling cracks between cobble-stones, gradually hiding them. But as calculations happens pixel-by-pixel, artifacts begin to appear on the border between textures. To get a smooth result we will take several pixels in depth instead of one and blend them.

float3 blend(float4 texture1, float a1, float4 texture2, float a2)
{
    float depth = 0.2;
    float ma = max(texture1.a + a1, texture2.a + a2) - depth;

    float b1 = max(texture1.a + a1 - ma, 0);
    float b2 = max(texture2.a + a2 - ma, 0);

    return (texture1.rgb * b1 + texture2.rgb * b2) / (b1 + b2);
}

In the code above we at first get part of a ground seen at a certain depth.


Attached Image: 6.png


And then we normalize it to get new opacities.


Attached Image: 7.png

Attached Image: 8.jpg


As a result we found the algorithm of textures blending, which allows us to reach close to a natural terrain image.

In summary I want to explain what this algorithm was developed for and how we use it.

The shader was developed for turn-based strategic indie game Steam Squad. As engine and developing platform we use Unity3D. And as the Unity IDE is extremely flexible, we made our own level designer extension. In general the level designer is a simplified Unity3D terrain editor with some features of Titan Quest Editor.

When you paint the texture on a terrain there is a recalculation of the opacity map corresponding to this texture. And as the sum of all opacities has to make 100%, the editor automatically normalizes opacity maps of other layers.

Here is short YouTube video showing how it works.

How to Make Accurate Time Estimates

$
0
0
Many people are really bad at estimating how much time a task will take. Perhaps you estimate you’ll need about an hour, and it really takes you 3-4 hours to finish. Or maybe you allocate 30 minutes for a task, and you’re done in 5 minutes. What can you do to get better at making accurate estimates?

Here are several techniques you can use to make better time estimates:

Calculate Your Fudge Ratio


The best place to start is to measure your current estimation accuracy.

Make a to-do list of upcoming tasks to complete, and jot down an off-the-cuff estimate for how long you expect each task to take. As you complete each task, record the time you actually spend on each one. Then add up your total time spent, and divide it by your total time estimate for the collection of tasks. That’s your fudge ratio.

For example, if you estimate that a certain list of tasks will take 12 hours to complete, but they really take 15 hours, then your fudge ratio is 15/12 = 1.25. This means you it took you 25% longer than expected to complete the tasks.

If you measure your fudge ratio for a variety of tasks, you’ll probably find that for individual tasks, your fudge ratio varies tremendously, perhaps ranging as widely as 0.1 to 10.0. However, for groups of tasks that collectively require a few days to complete, you may notice that your fudge ratio settles into a fairly narrow range. When you average enough tasks, your fudge ratio converges on a consistent figure.

My average fudge ratio is about 1.5. This means that whenever I make an off-the-cuff estimate for how long a task will take, on average I’m too optimistic; the task ends up taking about 50% longer than my initial guess. For any particular individual task, my estimates may be much more inaccurate. However, if I estimate that a collection of tasks will require about 2 days to complete, it’s a safe bet they’ll really require about 3 days.

Once you know your fudge ratio, you can use it to generate more accurate estimates for groups of tasks. Just add up your off-the-cuff estimates, and multiple the total by your known fudge ratio. This will tend to be a fairly accurate estimate.

I tend to be consistently optimistic when estimating the time required for certain tasks. Knowing my fudge ratio has NOT made my initial estimates more accurate. My off-the-cuff estimates are just as inaccurate as they’ve always been. However, when I multiply my estimates by the fudge ratio, the estimates come pretty close to the time required. This helps me budget my time better.

Based on my fudge ratio, I know that if I want to complete about 8 hours of actual work in a day, I should only list about 5 hours and 20 minutes worth of tasks based on my off-the-cuff time estimates (5:20 = 8 hours / 1.5). While it might seem silly to make this kind of compensation every day, in practice it works quite well — far better than the alternative of listing 8 hours of tasks and then either pushing myself to work a 12-hour day or feeling bad that I only completely 2/3 of my tasks. Self-sabotage can make things even worse when I subconsciously know I’m trying to do the impossible.

It’s better to make a reasonable task list that I can actually complete by the end of the day instead of beating myself up for being bad at estimating. Even if my daily task list seems too short at first glance, it feels good to cross off the final task at the end of the day. Due to daily variations, this isn’t perfectly accurate, but overall it’s better than anything else I’ve tried, and it encourages a sustainable daily rhythm without overworking or under-working.

I recommend using at least 10-20 hours of tasks for your initial fudge ratio calculation. If you based your calculation on only a few hours of tasks, your fudge ratio may not be accurate enough.

Of course it’s a good idea to recalculate your fudge ratio every once in a while. Once a quarter should be fine. It’s also wise to update it whenever the nature of your work changes, such as when you begin a new project or switch companies or careers.

If you want to get a little more detailed, you can calculate different fudge ratios for different kinds of work. Personally I don’t do this, but if you think it’s likely that different tasks will yield significantly different fudge ratios, it may be a good idea. For example, if you’re a student who finds that math homework has a fudge ratio of 0.9, but term papers have a fudge ratio of 1.7, you’ll probably want to maintain separate fudge ratios to create better estimates.

If you manage a team of people, you can calculate a fudge ratio for each member of your team (with or without their knowledge). Ask for time estimates from each team member for a collection of tasks, measure the actual time required, and calculate the fudge ratio for each team member. Whenever you get new time estimates from those team members for upcoming tasks, you can multiply their estimates by their individual fudge ratios. This will help you create a more accurate schedule for team projects. I think you’ll find that people tend to err in their estimates in a fairly consistent manner.

Achieve Reasonable Granularity


In order to make accurate estimates, it’s important that you break your tasks down to the right level of granularity. If your chunks are too big, you’ll overlook too many details. If your chunks are too small, you’ll get buried in low-level details, and you could spend more time estimating a task than it would take to just complete it; this is too much overhead.

For example, “Overhaul my website” sounds like a complex, multi-task project. This isn’t granular enough to make a reliable estimate. You’ll need to list the individual tasks needed to complete this project.

On the other hand, “Write mailing address on envelope” is way too granular. You could have completed this task in as much time as it took to list it and estimate it. This much detail only wastes your time instead of making you more efficient.

You should experiment to find the right chunk size where you can make the most accurate estimates. I’ll offer a couple pointers based on what works well for me:

The One-Sitting Rule. My estimates tend to be best for tasks I can complete in a single sitting. In practice this means about 2-4 hours per chunk. When I go less granular than that (bigger chunks), I miss too many details and grossly underestimate the time required. When I go more granular (smaller chunks), I list out too many details, I overestimate how long things will take, and I waste too much time creating and managing my to-do list instead of getting things done.

Compensate for Experience. If I’ve completed similar tasks many time before, my estimates will tend to be fairly accurate, so I might drop my fudge factor down to 1.2 or even 1.0. For example, I’ve written 700+ articles, so I’m pretty good at estimating how long an average article will take to write (3 hours is typical). But if I have to do something I’ve never done before, a fudge ratio of 2.0 or higher may be more accurate. The less experience I have with a task, the higher my fudge ratio needs to be.

Define Clear Task Boundaries


Make sure your tasks are clearly defined. Vague or nebulous tasks are hard to estimate.

If one of my tasks is “Update accounting,” I can’t be certain of what that includes. Does that mean balancing my checking account? Doing payroll? Filling out tax forms? Recording receipts? If I want to make a reliable estimate, I need a clear picture of what I’ll be doing.

You may find it helpful to list a few keywords for the components of an otherwise unclear task. You don’t necessarily need to estimate the time for each segment. You just need to be able to visualize what you’ll be doing. The keywords can help trigger the right imagery, so you can make a better estimate.

You should be able to quickly verbalize the first and last steps of each task. For example, when I see a task labeled “Write new blog entry,” I know that the first step is to pick a topic. The last step is to click the “Publish” button. If you can’t name the first and last steps of a task on your list, then your task doesn’t have clear boundaries. In that case you’ll need to take a moment to define those steps, or you’ll need to define your task a little more clearly, possibly by breaking it into smaller chunks. Good estimates require clear start/finish boundaries.

Be especially careful to consider what will be required to bring a task to 100% completion. If your task is to “Pay your bills,” does that end when you write the checks, when you deposit the payments in the mail (or complete an online payment process), when you file the paid bills in your filing cabinet, or when you balance your checkbook? Don’t forget to consider how long it takes to clean up and put away your materials. Even if you’re just making dinner, there will be dishes to attend to afterwards.

Reuse Estimates for Recurring Tasks


Once you’ve completed a recurring task, make a record of the time required for completion, so you can reuse that estimate in the future. When that task reappears on your to-do list, you can simply look up your old estimate. These estimates will be fairly accurate because they’re based on previous results, not previous estimates.

I recommend that you create an estimation list for your common recurring tasks. Here are two methods for doing that:

Method 1 (simple version). For a very basic estimation list, you only need to record a single figure for each task. Just note how long the task took to complete the last time you did it.

Your simple estimation list might look something like this:

Grocery shopping – 55 minutes

Make and eat dinner – 42 minutes

Vacuum house – 83 minutes

Wash, fold, and put away laundry – 75 minutes

And so on…

Once you build a good list of time estimates for recurring tasks, you can create a very reasonable plan for your day by adding tasks to your schedule.

Method 2 (detailed version). For a more complex version, you can record four figures for each task: (1) the number of times you’ve completed the task since you started keeping records, (2) your best (minimum) time to complete the task, (3) your worst (maximum) time to complete the task, and (4) your average time to complete the task. You can use these figures for making reliable estimates in the future; the min-max range tells you how reliable your estimates are likely to be. Whenever you complete each task again, take a moment to update your figures. In practice this won’t take much time at all, but you’ll end up with a fairly accurate list of estimates.

To update your average task time using this method, multiply (1) by (4), add the time required to complete the most recent repetition, and then divide the result by (1)+1. For example, if you previously completed a task 10 times, averaging 30 minutes per repetition, and the 11th repetition takes 35 minutes, then your new average is (10×30+35)/(10+1)=30.45 minutes. This method allows you to keep updating your average without having to record all of your previous task completion times.

If you record your best (minimum time) to complete a task, you can also use that to challenge yourself. Beating your previous record can motivate you to maintain a faster tempo. At the very least, try to beat your average time. Putting the clock on yourself can push you to work a little faster, especially for repetitive tasks that might otherwise seem a bit dull.

For most people I recommend Method 1. Method 2 is probably overkill unless you’re really committed to optimizing your time usage.

Conclusion


Learning to make better time estimates is a useful skill to develop, one that will serve you well for life. The methods above are actually quite easy to implement.

Becoming a better estimator may improve your life at the tactical level of daily time management, but be careful not to lose sight of the strategic level. Have you taken the time to define your life purpose? Are you setting the right goals? Are you working in the right career? Mastering low-level tactics won’t provide much value when your overall life strategy is nonsensical or nonexistent.

Even so, accurate estimation can benefit you across a variety of fields, so it’s a good skill to develop early in life. It’s still okay to develop this skill before you’ve achieved clarity at the higher levels of life purpose and long-term goals. Just be sure that at some point, you remember to attend to those higher levels, so you don’t merely become a faster rat on a treadmill.

What are your personal tips for generating good estimates? I invite you to share them in the comments. And remember, this is for posterity, so please… be honest. :)

The One: A Singleton Discussion

$
0
0

Abstract


The singleton is a recurring pattern used by programmers. The singleton pattern is easy to understand but has subtle implementation problems in C++. This article discusses those problems.

The reader is assumed to have a basic understanding of the singleton pattern. There are many articles on the subject available on the net. Please read at least one as I assume some knowledge of singletons.

Note:  
This article was originally published to GameDev.net back in 2002. It was revised by the original author in 2008 and published in the book Beginning Game Programming: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.


Why singletons?


For starters, let's examine why we need singletons. Take a look at this program.

// main.cpp
#include <iostream>
using namespace   std;
extern int x;
class C
{
public:
  C()
  {
    // may not print 3
    cout << x << "\n";
  }
};
C c;
int   main()
{
  return   0;
}
// another.cpp
int F()
{
  return 3;
}
int x = F();

Whether the above program prints 3 or 0 becomes dependent on the order in which you link main.cpp and another.cpp. Switch them around and you get different results.

The initialization of static objects is well defined with each translation unit (.cpp file), but not across multiple files. This is why we need singletons.

Meyers' Singleton


This solution was first proposed by Scott Meyers (author of Effective C++ books, good read!). It often goes in the form below.

class MyClass
{
  MyClass()   {}
  ~MyClass()  {}
  // undefined
  MyClass( const MyClass & );
  MyClass& operator=( MyClass );
public:
  static   MyClass& Get_Instance()
  {
    static   MyClass  instance;
    return   instance;
  }
};

The ctor (constructor), dtor (destructor), copy ctor and assignment operator are defined privately to avoid unauthorized creation.

This simple solution works by having a local static variable inside a function. The variable is initialized the first time Get_Instance() is called and deallocated when main exits.

It is an elegant solution that fulfils what many programmers need from a singleton.

But Meyers' singleton is not without problems. Some immediate problems are

  1. By storing an auto object (one that is allocated on the stack), we cannot store derived classes.
  2. The order of destruction is undefined.

No problem, we store a local pointer instead of local instance. This would allow Get_Instance() to return a reference to its parent interface (perhaps an abstract interface).

// class MyChild : public Parent
static   Parent&  MyChild::Get_Instance()
{
  static   Parent *p = new MyChild();
  return   *p;
}

Problem 1 is solved, except that the singleton now leaks because the destructor is never called. The obvious solution here then would be to store a local static smart pointer (like std::auto_ptr) instead of the raw pointer.

Now that the first problem is taken care of, let's look at the second.

Meyer's singleton is initialized when Get_Instance() is first called. We solved the initialization order problem but another problem remains - the order of destruction.

A static variable inside a function is registered by at_exit(), and is deallocated when main exits. We have no control over when it gets called or when the destructors of global objects call the singleton - it may be already deallocated! To prove my point, try the sample breakmeyers.zip (in the attached file) that is designed to break Meyer's singleton.

In the breakmeyers code, notice the order of calls produced. After main exits, the singleton is deallocated and yet another call to Get_Instance() is made afterwards. For this simple class it probably wouldn't crash, but if your class has data members, it means those members won't be available anymore.

This occurs because we are relying on the compiler to automatically register the deallocation using at_exit(). If we store a pointer instance, we could avoid this issue, but then no one would be deallocating it. We either leak or risk crashing.

It is clear now that Meyer's singleton is insufficient for singletons that need to be called within destructors of global objects. Granted, such singletons are rare, but what would you do if you faced one?

Note:  In case you are wondering about what situations you might need a singleton that has defined order of construction/destruction, think about a global logging system.


Nifty Counter


The nifty counter singleton has a defined order of construction and destruction. This solution to the above problem is described in the book C++ Faqs (not the free online version). We observe that local variables used in functions have an undefined destruction order. Only pointers can choose their destruction time.

In essence, this is how the nifty counter works. Look at the niftycounter.zip sample in the attached file before I explain.

A nifty counter class is created to hold a pointer to MyClass. The Get_Instance is removed from MyClass into the nifty counter instead.

Note at the end of MyClass.h, there is a static instance of the nifty counter. Before you scream at the atrocity of declaring variables in headers, calm down and look at how it works.

Every file that includes MyClass.h will have a local instance of the nifty counter. The Nifty counter class stores a reference count of how many instances it was created. Within each translation unit, the order of initialization/destruction is well defined (I said this before). The order of construction/destruction is first in/last out (FILO).

Since we have to include MyClass.h before using the singleton, we are unknowingly creating a local instance of the counter in every translation unit. Since this local counter is always destructed last in the translation unit, the order of destruction is well defined (when no one needs to use it anymore).

Does this come for free? Well almost. Notice we have to create a local counter per translation unit. This means that starting/shutting down the program will take more time because we have to page those sections into memory. If you can live with that, this singleton should serve you well.

Sample Singleton


This is a bonus for those who have read the book Modern C++ Design by Andrei Alexandrescu.

This is a singleton holder inspired by the book. In the book's implementation of singleton holder, it keeps track of the order of destruction using a list. I did not particularly like that way of implementation, so I created the singleton holder available as zerobsingleton.zip in the attached file instead. It follows the book's implementation closely. However, for the singleton lifetime, I provide a nifty counter and Meyer's singleton policy instead. The book explains in more detail how the template policy works.

Take note the singleton requires a very compliant C++ compiler due to the C++ templates. While it should work on a recent C++ compiler, it does fail on some ancient, but still in-use compilers like Visual Studio 6.

More problems


We are not done yet. We have only solved the basic lifetime requirements for singletons. There are other problems.

Exception safety

Exceptions are normally caught in a try/catch block within main. If the ctor of the singleton throws, chances are you have not entered main yet. This imposes a serious design issue for the singleton. You normally have to design the singleton so that the ctor doesn't throw, as you can’t catch the exception to report it properly.

However, there is a way to throw exceptions from the ctor. Try/catch blocks can be used at function scope (known as function try blocks). You can design an exception trapping mechanism for global objects using function try blocks. There are archived discussions on USENET comp.lang.c++.moderated regarding this.

I shall not go further because there is enough material to warrant another long discussion. There are many good articles online that explains function try blocks.

Thread safety

The future of the CPU is multi core and threading is a subject we all have to face. How should we do it in a singleton?

The easiest solution would be to protect Get_Instance with a synchronization object (critical section). That would work but that would mean additional overhead when accessing the singleton. Synchronizing locks/unlocks are usually not cheap and if the singleton is called often, performance can be adversely affected.

A better design would be to let the client decide when to lock when accessing the singleton. Immediately a problem pops up - there is a possibility that multiple threads are trying to create the singleton on their first call to Get_Instance.

An ingenious idea is the "Double Checked Locking Pattern". Alas, it doesn't solve the problem thoroughly as one would hope.

In the absence of threading support from the current C++ standard library, the only guaranteed solution is to use some kind of platform specific synchronization object. In that case, the DCLP can be used with the sync object to protect the creation code of the singleton.

The sample singleton is not thread safe at the moment. With the platform-independent threading available with Boost (www.boost.org), making the sample thread-safe should be do-able.

Abuse of singletons


Implementing a singleton is fraught with dangers. Singletons provide more safety than using global variables but most of the time they are used more often than they should be.

Practical experience suggests than it is often possible to redesign so the singleton becomes a member variable or local static variable. Indeed, this should be a worthy goal to strive for. Do we really need the graphics renderer to be global? Can it be a member variable of an Application object instead? Does the factory for creating objects need to be a singleton? Wouldn't it be better if we localize all the creation code in a single module?

Think about the functionality of the class. Is it necessary for it to be a singleton or a mere convenience? For convenience sake, take note of the problems you are generating by having a singleton. A redesign would almost always benefit in the long run.

Glocal


This last section I’m adding on after having years of experience with writing engines. Yes, globals are evil, and there are times where a singleton is really handy. We want to have our cake and eat it, right?

Glocal is a term I vaguely remember hearing somewhere from school days, I don’t think it’s the same thing I’m saying but I thought it is a cool term which fits perfectly. Simply, you assign a global pointer to a local variable.

// Global.h, included by all other files
extern MyClass  * globalInstance;
// main. Cpp
int   main()
{
  MyClass localInstance;
  globalInstance = & localInstance;
  return   0;
}

This simple technique is exception safe and with a defined order of destruction yet. There are many systems which be exposed with glocals, and the order of creation and destruction is implicitly controlled by the variable definition. While most programmers cringe at exceptions due to increased size and speed, and I would agree somewhat, but I think in development, exceptions are very useful and can be turned off for shipping. Anyway exception safety is for another article, you can see my article “An Exceptional Quest”.

Of course, this means the “singleton” can only be used after it is created, and in most of the engine system, it can be. There are few cases where you need the nifty counter singleton (cout comes to mind) to have global destruction order.

Wade Not In Unknown Waters: Part One

$
0
0

We decided to write several small posts on how C/C++ programmers play with fire without knowing it. The first post will be devoted to an attempt to explicitly call a constructor.


Programmers are lazy creatures. That's why they tend to solve a task using minimal amounts of code. This aim is praiseworthy and good. But the main point is to not get too involved in the process and stop at the right time.


For example, programmers are too lazy to create a single initialization function in a class so that it could be called from various constructors later. They think: "What do I need an extra function for? I'd rather call one constructor from the other". Unfortunately, sometimes programmers can't solve even such a simple task. It is to detect such unsuccessful attempts that I'm implementing a new rule in PVS-Studio. Here is, for instance, a code sample I have found in the eMule project:


class CSlideBarGroup
{
public:
  CSlideBarGroup(CString strName,
    INT iIconIndex, CListBoxST* pListBox);
  CSlideBarGroup(CSlideBarGroup& Group);
  ...
}

CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group)
{
  CSlideBarGroup(
    Group.GetName(), Group.GetIconIndex(), Group.GetListBox());
}

Let's examine more attentively how the last constructor is implemented. The programmer decided that the code


CSlideBarGroup(
  Group.GetName(), Group.GetIconIndex(), Group.GetListBox());

simply calls the other constructor. Nothing of the kind. A new unnamed object of the CslideBarGroup type is created and destroyed right afterwards.


It appears that the programmer has actually called the other constructor. But he/she has not quite done the same thing he/she intended: the class fields remain uninitialized.


Such errors are just half the trouble. Some people do know how to call the other constructor really. And they do it. I wish they didn't know :)


For instance, the above given code could be rewritten in this way:


CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group)
{
  this->CSlideBarGroup::CSlideBarGroup(
    Group.GetName(), Group.GetIconIndex(), Group.GetListBox());
}

or in this way:


CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group)
{
  new (this) CSlideBarGroup(
    Group.GetName(), Group.GetIconIndex(),
    Group.GetListBox());
}

Now one data initialization constructor is really calling the other constructor.


If you see a programmer doing this, deal him/her one flick on his/her forehead for yourself and one more flick on my behalf. The cited examples contain very dangerous code and you should understand well how they work! Being written for the purpose of petty optimization (programmers are too lazy to write a separate function), this code might do more harm than good. Let's see more closely why such constructs sometimes work but most often don't.


class SomeClass
{
  int x,y;
public:
  SomeClass() { new (this) SomeClass(0,0); }
  SomeClass(int xx, int yy) : x(xx), y(yy) {}
};

This code will work correctly. It is safe and works well, since the class contains primary data types and is not a descendant of other classes. In this case, a double constructor call is harmless.


Let's consider another example where an explicit constructor call causes an error (the sample is taken from this discussion on the StackOverflow website):


class Base 
{ 
public: 
 char *ptr; 
 std::vector vect; 
 Base() { ptr = new char[1000]; } 
 ~Base() { delete [] ptr; } 
}; 
 
class Derived : Base 
{ 
  Derived(Foo foo) { } 
  Derived(Bar bar) { 
     new (this) Derived(bar.foo); 
  } 
}

When we call the new (this) Derived(bar.foo); constructor, the Base object is already created and fields initialized. The repeated constructor call will cause a double initialization. A pointer to the newly allocated memory area will be written into ptr. As a result, we get a memory leak. The result of double initialization of an object of the std::vector type cannot be predicted at all. But one thing is obvious: such code is inadmissible.


Conclusion


An explicit constructor call is needed only in very rare cases. In common programming practice, an explicit constructor call usually appears due to a programmer's wish to reduce the code's size. Don't do that! Create an ordinary initialization function.


This is how the correct code should look:


class CSlideBarGroup
{
  void Init(CString strName, INT iIconIndex,
            CListBoxST* pListBox);
public:
  CSlideBarGroup(CString strName, INT iIconIndex,
                 CListBoxST* pListBox)
  {
    Init(strName, iIconIndex, pListBox);
  }
  CSlideBarGroup(CSlideBarGroup& Group)
  {
    Init(Group.GetName(), Group.GetIconIndex(),
         Group.GetListBox());
  }
  ...
};

P.S. Explicit call of one constructor from the other in C++11 (delegation)


The new C++11 standard allows you to perform the call of constructors from other constructors (known as delegation). It enables you to create constructors that use the behavior of other constructors without added code. This is an example of correct code:


class MyClass {
  std::string m_s;
public:
    MyClass(std::string s) : m_s(s) {}
    MyClass() : MyClass("default") {}
};

Advertising Your Games

$
0
0
Advertising has always been a rough market. Not only is it extremely competitive, expensive, and risky, but it also changes as fast as any market. If I had written this article a few years ago, let’s say 2005, it would have been all about how to use Google Adwords. Today, while Google can still provide a good outlet, it’s not your one stop shop for advertising glory any longer.

So where do you turn? Well, the sad reality may be nowhere. Games are a low margin, low conversion business and when you are competing with spyware and big business it’s pretty clear who gets the most revenue per user. There are, however, some terms and descriptions we can go over that apply to all advertising models and methods. If your game falls into the category where you could benefit from one of these approaches, by all means use it and try to get some positive return on investment (ROI) on your ad spend.

Note:  
This article was meant to revise and conclude a marketing series published on GameDev.net back in 2004. In 2008 it was included in the book Business and Production: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.


In order of importance these terms are:
1) Bulls-Eye Targeting
2) Competitor Targeting
3) Horizontal Targeting
4) Vertical Targeting

Bulls-Eye: The key to successful advertising is targeting the ads to the users most likely to purchase your product. Seems like a no-brainer, right? Unfortunately the trouble typically comes from figuring out who the heck your target market really is and once you do figure it out, how do you go about reaching them? In order to determine the most targeted user we start at the very highest level of user interest and awareness. This is, in fact, people who are interested in your game specifically. Once again, the sad fact is unless you have a pre-existing audience then this group of people is pretty small. Still, advertising to your own audience tends to be dirt cheap and super-effective. Do not fret and worry about your pre-existing users not liking a sudden e-mail from the developer. I find that while you will get a few complaints, the impact of such a thing is tremendously positive for sales and growth.

Competitor Targeting: This leads us to the next group, people who are interested in similar games to yours. I’m not talking about people who like Risk also liking some board-strategy game; I am talking about the people who have cloned another game in part or in whole. This is where Google and other contextual/search ads come in most handy. Advertising on the keywords of your competitor’s products gives you an opportunity to piggyback on their success. Nothing in advertising is easy though, and what you’re going to find is that most of the people looking for a specific product are not looking for an alternative. This means you’ll be bidding less than everyone else and won’t be able to pull big volume even on a popular game. Still, if you have a hidden object game and put an ad on every other hidden object game’s name you’d probably do enough volume to make it worthwhile.

Horizontal Vs. Vertical Targeting: Before we dive into the next two an explanation is necessary about the difference between them. This applies to all industries, so it’s a rather useful concept you are certain to run into again some day. Vertical and horizontal are both all about getting into the mind of the consumer, which makes it a rather tricky thing to define. Vertical refers to drilling “down” into the sub-divisions of human interest. Creepy as it sounds, it is best described by example. A vertical chain looks like this: I like games. I like sports games. I like strategy games. I like sports games. I like sports strategy games. You can move up or down this chain to increase or decrease the targeting. Each level of vertical “down” is a better target, but as you may suspect – a smaller one as well. So far we’ve been very far “down” the vertical chain, so the next area we’ll go up the vertical chain a little, making things less targeted but still within the vertical area.

Vertical markets are all the rage these days, probably because they clearly make sense. Horizontal is in reference to the “breadth” of the interest level of a particular group. This makes a whole lot less sense, and you’ll find tons of different definitions of a horizontal “market” out there. One person I know defines a horizontal only as a product with near-universal appeal. Spreadsheet software or games as a whole could also qualify. An ad targeting to anyone who enjoys games would be a horizontal ad, this ad could appear in any website and because of its universal appeal, attract attention no matter what. I say that’s a load of bunk and I simply call these things universal products/ads.

So how do I define it horizontals. Visualize a point-down triangle. Draw 3 horizontal lines in that triangle. The top and broadest area in our example above is “I like games” while the bottom is “I like sports strategy games.” Where we said “I like sports games” we’d draw a vertical line and also write “I like strategy games.” This is an example of two different horizontals occupying very close proximity. Not mentioned above is the very high likelihood of this person saying “I like sports.” Here we have a completely separate triangle (vertical). It would appear horizontally across from “I like games.” In that triangle “I like soccer” would be below it, horizontally across from “I like soccer games.” This is where I derive the idea of a “horizontal” target. Because games are a universally enjoyed commodity (Yeah ok, let’s just say it is), then you can clearly advertise in a vertical market connected to your horizontal chain! Time for a shoddy picture!


Attached Image: Untitled.jpg


Horizontal Targeting: So here is the big error people make. The next recommendation I would have is aiming at the horizontal niche of your game, assuming you have one Let’s say you made a World War 2 clone of Risk. You’ve exhausted aiming ads at Risk players. The horizontal target after that would be to aim at people interested in World War 2. This is where most people make the most common mistake of thinking that their next target is a vertical niche, for instance targeting people interested in board games. The horizontal niche often has so much less competition that it is cheaper and more effective to focus on horizontal positions. So in this example, my next place to go would be to World War 2 websites. Not all games have a clear horizontal niche, however. If your game is about WW2 you’re in good shape. If your game is about cute balls of fluff you’re outta luck. If your game is about fairies you may be able to find some fairy websites, but odds are you’re not going to see much volume.

Vertical Targeting: The place most people want to put their ads for some reason is the vertical niche. If I made a board game, I would want to advertise on board game sites. The fact is I have almost never found this to be an effective method (ROI-wise) for advertising a game. You can still try it - focus on running ads on game sites or sites related to the exact genre of your game. The closer you can get to your game’s look, feel, and genre the better. I wouldn’t bet much money on it though, the simple fact is competition is too high and that drives prices up.

So that is the overall strategy of where to run ads. There are a few items that are very common questions and concerns left though, so we’ll focus on a few corner case questions.

Probably the most common question is “How does X company turn a profit when they are spending Y dollars on ads!?” The answer may be that they don’t. Not all ads are run for a positive ROI. Sometimes it is smart to employ a loss leader strategy. If you can spend 21 dollars to make 20 dollars but attract 1000 people per dollar “lost” then it may be to your advantage to lose a dollar simply to gain the attention, e-mail addresses, and name recognition of those thousand people. However, in the realm of an indie or small time development studio a loss leader strategy is rarely an option. So let's focus on getting a positive ROI.

Another good one is “What about print advertising” (or radio, or TV). The short answer is: Forget it. Print, TV, and Radio all are brand awareness focused media. You could build up your company reputation with a slick jingle and on a national campaign that could mean big bucks. For a TV ad, if you are a studio who’s made only one game and it is available only online there is a huge barrier to get people off their lazy asses, walk over to the computer, download that one game, and then make a purchase. It’s a heck of a lot easier when their lazy asses are already parked in front of the PC and your banner ad is slapping them in the face. The only exception to this is if you can somehow negotiate some huge perk to your ads, such as being the featured game on a demo CD, or getting a 3 day, 3 article front page feature on a game magazine’s website. Basically in this case you are getting something large enough that the print ad is no longer really what you’re buying, merely the ends to a means. Web-based ads will always have a better ROI, I can’t think of a single case where a single product site has seen otherwise.

Here’s another interesting one. “What is the most effective form of online ads?” The answer to this varies widely with how good you are at creating ad copy (good text, good graphics, interesting sales pitch, etc.). However, if I had to pick one I would actually say e-mail newsletters have the best bang for your buck, just be careful that you aren’t working with a spam company. If, for instance, your WW2 Risk clone found a WW2 website with a newsletter that offered an ad spot (advertorial preferably) I would be on that like white on rice. Don’t be put off by the fact the CPM is going to be high ($5-$25 is not unheard of for a newsletter).

Finally let’s define some terms/acronyms you’re bound to run into – in case you’re a total noob at this.

CTR- Click Through Rate – What % of people click on an ad. A standard banner ad should range between .5 and 2% depending on position, quality, targeting, etc.

CPM- Cost Per Mille – How much 1,000 displays of your ad costs. An ad that costs $1.00 CPM and has a 1% CTR means you are paying 10 cents per click.

CPC- Cost Per Click – How much you are paying per click.

CPA- Cost Per Action – Payment is made only when the user performs an action, such as signing up for an account, newsletter, or making a purchase. Also known as PPA, PFP, and CPL.

ROI- Return on Investment – Mentioned above, but also keep in mind ROI factors in future value of visitors as well. If the average purchasing user buys 1.2 games all of that value should be factored into the ad buy. Keep in mind, however, ad based visitors are not equal to the average visitor and should be tracked separately. An ad based visitor may convert half as likely and make only 1.1 sales per purchase.

RPM- Revenue per Mille- RPM typically refers to a single page view and all the ads on that page. So if you had 3 ads paying 1.00 CPM you’d say your page RPM is 3.00.

IAB Ad Sizes- IAB is the Interactive Advertising Bureau – They defined the ‘standard’ ad sizes that we see. If someone asks for IAB sized ads they probably mean 728x90 (superbanner / leaderboard), 160 or 120x600 (Sky and Wide Sky), and/or 300x250 (Box) sized ads. Other very common IAB sizes include 468x60 (Banner), 88x31 (Button), and 125x125 (Square). In games we also see a fair number of 100x100 ads, but those are not IAB standard.

RON/ROS– Run of Network/Site – This refers to buying or selling an ad across an entire network or site with no special targeting.

And so what can you take away from this? Hopefully your takeaway is this: Advertising is really freakin’ hard! If you’ve planned your success around an ad budget rather than a stellar product you’re in deep brahmin waste. Also other than giving you a good dose of pessimism, there are a lot of terms and concepts flung around the ad world. The reality is people will be impressed if you can fling terms and concepts back at them. Finally, if you do decide to advertise using the targeting model I went over will give you the best fighting chance for success.

Elements Of Video Game Style

$
0
0
The design of video games has rarely been a subject of formal study. All too often today games are designed simply by looking at whatever is popular at the moment and copying it. The recent interest in classic video games has a basis in something more than mere nostalgia. There is a feeling that something has been lost in game design, even while graphics and sound are reaching new heights.

This document is an attempt to explore certain concepts of videogame design. It is not a formula for designing hit video games. I firmly believe designing games by formula leads to stagnation. However, I just as firmly believe that a good videogame must be built on a solid foundation. Thus I have divided the subject into several elements and written an essay on each one. I've gathered them together here for the benefit of anyone interested in game design.

Note:  
This article was originally published to GameDev.net back in 2000. It was revised by the original author in 2008 and published in the book Design and Content Creation: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.


The Element of Presence


One of the magical aspects of video games is the ability to simulate worlds in the machine. These worlds may be tiny or huge, simple or complex. This is an element I call "presence" which approaches the sense of virtual reality. I believe the word "immersiveness" has also been used by some to describe this phenomenon. A good videogame creates a feeling of being there, not just watching a video or a slideshow.

Now I will list the components that can aid in making a successful simulation, beginning with the most basic and important, then working up toward more sophisticated frills that can enhance the experience. Take this as a recipe for creating that element of presence in the game:

1. Movement and Collision

Among the first things infants learn are simple rules of movement and collision. Things bump things. These rules are closely tied to our sense of reality. It's why smooth motion and accurate collision detection are so important. The earliest video games, such as Space War and Pong, were built around movement and collision. One might even say that this is the defining character of video games. Although movement and collision are also seen in movies, notably in action movies, it's the idea of a movement directly controlled by the player, not merely observed, that makes a videogame what it is. The connection -- the more direct, the better -- between the players hand on the controls and an object moving on the screen is the basic magic that video games are founded upon.

Recently movement has become a special concern when working with networked games, where net lag is a factor. You need to make sure, as much as possible, that movement is smooth and each player sees the same the same collision events, despite varying time lag between servers and clients. Some MMOGs have dealt with this problem pretty well. Some other environments, such as Second Life, continue to have serious problems with movement and net lag. The game needs to be designed with this concern in mind from day one. It can't be fixed as an afterthought.

2. Internally Consistent Rules

Objects in the game should interact with each other according to certain rules. These rules should be simple and consistent enough for the player to figure them out and use them. He should be able to say, "If object A does this to object B, then it should do that to object C." These rules don't necessarily have to be the same as rules in the real world. The rules may be fantastic or abstract, but they must make sense within the context of the game. These are like the laws of physics in your game world.

The worst offenders in this respect tend to be scripted games, where the player is moved from one pre-determined event to another. The events are arbitrarily created by a writer, they don't emerge from the game mechanisms.

3. Freedom of Movement

The player should generally be able to move where he wishes, when he wishes, for his own purposes. Games that force the player along a fixed path at a predetermined speed lose some sense of reality. Likewise games that force the player to follow pre-written scripts also come up short. Instead, it is best to provide the player an environment and allow him to explore it. It's okay for the environment to be small, or for there to be obstacles for the player to overcome before reaching some parts of it. However, free movement within the bounds and rules of the environment is critical. Asteroids is a good early example: a game which allowed the little spaceship to fly all over the screen, as opposed to games like Space Invaders with mere side-to-side movement.

4. Real-time Action and Background Activity

The real world does not stop and wait for us to make decisions. Some games, especially adventure and strategy games, freeze in their tracks much like a cheap choose-your-own-adventure book. Real-time action does not mean you keep your player racing against the clock (unless it's that kind of game), but the game should not freeze while he decides his move. Things should constantly be happening in the background, regardless of what the player does. This background activity, even if only partly seen by the player, adds a tremendous sense of vitality and presence to the game.

The classic Atari VCS (Atari 2600) Adventure game stands out as an early and groundbreaking example of this technique. Regardless of the player's actions, the dragons and the black bat continued roaming about and doing their own things. The classic arcade game Defender was another early example, because many events occurred outside the main view. The player had to keep tabs on these events using a small radar display and respond to them.

5. Three-Dimensional Space

Every game example I've listed so far has been two-dimensional in nature. However, people have a natural ability to think in three dimensions. Adding three-dimensional space to a game makes it more stimulating and can enhance the feeling of simulating a world in the machine. At the time when I first advocated the move to 3D environments, console hardware was just beginning to become adequate for this. (It was shortly before the introduction of the Sony Playstation.) Today a more pertinent concern is whether mobile devices can support a 3D display.

In these situations one may find that polygon-based games can work surprisingly well without any surface texturing. One of the first game consoles to break into 3D game design was the Atari Jaguar. Due to some annoying hardware bugs, its ability to perform texture mapping was very limited. Games created using Gouraud shading, such as Battlemorph, were fairly successful and created a stylized visual experience that I found reminiscent of the old 2D vector graphics monitors. At the time texture mapping was a new, exciting and fashionable technique, and the Jaguar games were soon viewed as technological laggards. In today's environment texture mapping is ubiquitous, and it's the Gouraud-shaded games which I suspect would appear novel and distinctive.

3D space is not merely a question of graphics display. It's possible to create a game on a two-dimensional, flat playfield that is populated with 3D objects. Some early 3D games such as Doom followed this approach. Such games fail to live up to the full potential of presence, or immersiveness. Allowing the player to climb into holes or onto platforms enhances the sense of space. Allowing objects to rotate on multiple axes and move naturally in three dimensions is also helpful.

6. Continuous and Contiguous Space

When placing the player into a virtual world, breaks for loading or cut scenes are best avoided. Adventure and RPG games often have very expansive worlds, and they were long plagued by these kinds of breaks due to technical limitations. This has proven especially pertinent in the MMOG space, where such breaks also interfere with players trying to interact with one another.

The groundbreaking Everquest game is a noteworthy example. When it was new it presented one of the most immersive environments ever seen, with a true 3D environment, vast world, day and night cycles and weather. I remember well getting lost in the maze-like city of Freeport. Although it was easy to get lost, it was also tremendously immersive and compelling. A big flaw, however, was the division of the world into "zones", and the need to pause the game for loading when moving from one zone to another. Later games, such as World of Warcraft, have done away with the loading delay, making the boundaries between different regions much more transparent to the player.

A related problem is that of teleportation. Teleportation refers to any situation where the game removes the player from one location and drops him into another. This trick has been used in games for a long time, for convenience or as a way of working around technical limitations. However, teleportation breaks the metaphor of space and time that we're all accustomed to in the real world. It is anti-immersive. Once again MMOGs with their typically huge worlds provide a good basis for comparison.

Everquest had very little teleportation in the beginning. It was mostly limited to death effects: killed players respawned back at their home point. Traveling long distances required either using a vehicle (such as a ship) or running long distances, often through dangerous and unfamiliar territory. This made exploration a hugely challenging and engaging activity, and it reinforced the perception of a vast and coherent world. As the game was later expanded, teleportation points were introduced which made moving from region to region safe and trivially easy. Although this was certainly convenient, it destroyed the exercise of exploration which had been such an important part of the game. It also severely damaged the player's sense of time and distance, which are important to his sense of being in a cohesive world.

The designers of World of Warcraft notably went out of their way to avoid teleportation. They provided a system of convenient transportation around their world (via griffin rides), but designed it to avoid breaking the player's sense of distance and time. The griffins fly over the entire landscape between their origin and destination points, and the player gets a view of everything and everyone along the way. The griffins also will not take the player to any place he hasn't already reached previously by other means, thereby preserving the value of exploration. This is the best of both worlds.

As a counter-example, one may look at Second Life, where teleportation is the primary means of travel around the world. The character of Second Life is less like that of a landscape to be explored, and more like that of the WWW. Locations are bookmarked and called up like web pages. This metaphor works for SL because it is, at heart, not a game at all. It's a social environment, it's a communication tool. Convenience is far more important than trying to create a game-like experience.

Having worked my way through these six ingredients of presence, or immersiveness, I have to mention virtual reality in passing. VR is an idea that once seemed hugely promising, and then later was dismissed as a fad, a failed promise. I think it was perhaps written off too quickly, and that VR's time may come again. The recent success of Nintendo's Wiimote controller is a baby step back in the direction of VR. VR headsets and controllers would be the last element needed to create a fully immersive environment.

With true VR remaining somewhere in the future, we shouldn't fall into the trap of imagining that killer, state-of-the-art hardware is required to make a compelling game. The first four techniques I listed were all used to great effect in VCS Adventure, a game that ran from 4K ROM on a console with 128 bytes of RAM and minimal graphics and processing power. VCS Adventure became one of the most popular games ever done on the Atari VCS. Needless to say, these same ideas can be used far more effectively with even the lower end of today's hardware.

The Element of Chaos


A disturbing trend has been toward games where everything is pre-scripted. Each level is the same every time: same powerups in the same places, enemies in the same places following the same patterns. The game becomes a contest of pure memorization. Once all the secrets are found and the levels memorized, it's time to buy another game. This is something that particularly came to annoy me in one of my favorite series, the Call of Duty games. The game tries to give players a realistic taste of World War 2 combat, and it succeeds pretty well in many respects. Yet, ultimately, progressing in the game requires learning where the enemy soldiers are stationed and the best order in which to attack them. It's an exercise fundamentally unlike the chaotic and confusing flow of battle that the game otherwise recreates so well.

That's not to say there's no place for scripted games, or that it's not fun to explore and learn their secrets. That learning and exploration is the hub of their appeal. All the same, we are missing an opportunity for something even better. The missing element is chaos.

Chaos means that surprising things can happen at almost any time in the game. It means that simple objects can interact in unexpectedly complex ways. The player is absorbed not in learning preset patterns, but in learning how the objects interact. This is not only great fun for the player, but also a more efficient way to program the game. Instead of working to create level after level of pre-scripted secrets, the developer can just create interesting objects and turn them loose in the game environment. He doesn't have to create all the many (possibly hundreds or thousands of) interactions that can occur in the game. These interactions arise naturally from the objects themselves, including interactions the game designers may not have thought about.

At this point a couple of examples are in order. First, let's consider Cosmic Avenger. This was a coin-op game of the early 1980s, which was also ported to the ColecoVision. At first glance it didn't seem too different from the other pre-scripted, side-scrolling shooters of the day, such as Scramble and Super Cobra. But, Cosmic Avenger had a small chaos element those games lacked: The enemy forces could accidentally shoot each other. There were also enemy missiles that would home toward the player's ship and could follow it for quite some distance. This led to great possibilities. The player could lead homing missiles into hitting enemy ships or installations. Enemy objects could collide or shoot each other at unexpected moments. These reactions made the game richer.

Now let's also consider VCS Adventure once again. It appears superficially similar to lots of graphic adventures that followed, such as the Zelda series. However, there is a big difference: Games like Zelda were based upon pre-scripted encounters. They presented a series of obstacles which were always the same, located in the same places, sitting dormant until the player arrived to trigger them. Typically a certain screen in the Zelda adventures would have X number of type Y monsters which would appear and attack when the player character entered that screen.

By contrast, VCS Adventure had four creatures in the whole game: three dragons and a black bat. These creatures got a lot of mileage because they constantly wandered about the game world doing their thing no matter where the player was. There were some limits to their movement, such as entering or leaving castles. Still, there was no way to be sure when a dragon or the bat might pounce on you. They could attack when you were unarmed, and they could attack singly, or they could gang up on you. There was no way to know. Although the dragons provided the combat aspect of the game, the black bat was the most important chaos element. It followed a simple rule: it picked things up and carried them around. After a few minutes it would get tired of whatever it was carrying; then it would look for something new to pick up, and leave the old object.

For example, I might be hunting down the dragon with sword in hand. Finally I find the beast, but about that time the bat suddenly swoops in and takes my sword and leaves me holding something useless like a key. It's time to run for my life! Or, for another example, I might be returning to the gold castle with the chalice, which is the final goal of the game. Victory is almost in sight, but suddenly the bat swoops in and grabs the chalice away from me. The chase is on, and it might be another half hour of one thing after another before I get the chalice back where I wanted it.

And, perhaps the ultimate example of bat oddities was the Grand Tour. This happened when the player simultaneously killed a dragon and was eaten by it. The player was trapped in the dragon's belly, but sometimes the bat would come and fly off with the dead dragon, thus carrying the player along for a fast tour of the game playing area. I think it's unlikely that this was ever designed or intended by the game's creator; it was just something that emerged out of the game's rules.

In addition to all the above, there was an extra playing mode in Adventure which allowed the game to start with all objects in random positions. Under these conditions there was no way to know if the game could even be won. But, it added a tremendous amount of extra play value after the standard scenario had been mastered.

So, this is what I would like to see in more games: more randomness and more simple objects that can interact in complex ways. These chaos ideas can apply to a wide range of games, from arcade action games to adventure or RPG games, to strategy games.

The Element of Challenge


A videogame, like almost any game, is a contest. The player tests his abilities against the machine or against a friend or against both at once. So, it is critically important to create the right level of challenge and difficulty. A game should not be too easy, lest it become boring. Nor should it be too difficult, lest it become frustrating. And, it's also helpful if the game is accessible.

When Nolan Bushnell created his first coin-operated videogame, Computer Space, it did very poorly. People didn't understand how to play. His next attempt, Pong, was a giant success because it was more accessible. Anyone could look at the game and immediately see how it was played. Many of the great hit video games have had this trait of accessibility, such as Space Invaders and Pac Man. More modern examples would include basic scrolling shooters like Raiden, most racing games, and any pinball machine. That is one reason pinball is still around. Everyone knows how to play.

That's not to say complicated games can't be successful. There is a feeling of accomplishment that comes from learning a complex game. This is found among the players of fighting games with their rather abstruse system of control and "combos", and among players of computer role-playing games, and flight simulators. Even so, such games are naturally limited in the width of their appeal. These kinds of games can become very popular within certain circles, but they rarely reach widespread acceptance with the general public. This is the difference between hard-core game players and the average guy. And, it's a big part of why the rather simple game of Tetris was such a huge hit. It was too small and simple to intimidate people who are not really into video games.

It is interesting to observe that console games have trended toward being quite easy. Computer games tend to be more complicated and somewhat harder. The most brutally competitive environment for games has generally been the coin-operated space. These games have to be highly accessible, because they depend on luring random passerby into a game or two, but they also have to be hard enough to get the player to either leave the machine or insert more coins in a reasonable time frame. Economic incentive drove the design decisions about how hard the game should be.

Making a game simple to learn doesn't mean making it simple to master. I note again that complex interactions in the game are desirable: they give a game depth and staying power. But, these complex interactions should arise from simple objects and rules in the game. VCS Adventure, which I have already described as an example of chaos, also had very simple controls and was as easy to play as anything. The joystick moved your little hero, and the action button picked up or dropped objects. There were no statistics to track or inventories to manage, unlike more modern adventures. Anyone could pick up the stick and start playing.

Pac Man was probably the ultimate in accessible gaming. It couldn't be more obvious how to play. Children, grandparents, people who had zero experience with games could look at Pac Man and figure out in seconds how it worked. In today's market it would probably be pigeonholed as a mere "casual game". Yet, like almost all coin-op games from that era, it was pretty darn difficult to play and keep a game going for very long. The first maze wasn't too bad, but the difficulty ramped up quite steeply.

As for the more general issue of challenge and difficulty, the easiest way to address it is by adding difficulty settings or some form of adjustment the player can make to suit themselves. This is especially important when you look at the wide range of ages the video game market caters to these days. There is no reason to artificially limit your game to a narrow age group when you can widen its appeal by providing the difficulty adjustments. However, a danger of user-settable difficulty settings is that many players are lazy and will tend to play on easy settings, even to the point of boredom, rather than realize they would actually enjoy the game more on a harder setting. Giving them some incentive to overcome their laziness and use the harder setting is a good idea. By luring players onto the harder difficulty as soon as they are ready for it, you can extend the life of the game.

Some companies don't want to extend the life of the game. They feel they can make games to be solved quickly and cast aside, so the consumer can buy another. This policy is shortsighted and ultimately counterproductive. Game players eventually realize they are being ripped off. They become bored with easy, quickly solved games and migrate away from video games toward other forms of entertainment.

Multi-player options were, for a long time, badly overlooked. The thrill that comes from playing against a human opponent is wonderful. Even games that were otherwise rather bland somehow gain new life. I have long been a big advocate of game machines with built-in support for more than two controllers. In recent years, however, most of the multi-player game activity has moved onto the internet. This has opened up great new horizons, with MMOGs being the ultimate expression.

So, why is multi-player so great? One reason is that it brings a large chaos element into the game. I've already written about the benefits of unpredictability, and there's nothing much more unpredictable than a human being. The other player doesn't follow any preset pattern. The interactions between multiple players and other game objects can be far more complex and exciting than you would normally see in a one-player game. Also, there's a greater sense of competition against a human rather than a machine. At the same time, it's good to provide a handicap option in the multi-player games where the players are competing directly against each other (as opposed to the cooperative type), so that players of different skill can still enjoy themselves.

MMOGs have raised some new problems in terms of challenge level. Not only do they have players from around the world mixing together with different skill levels, but most of them also use a level-based game mechanic such that time spent playing the game is rewarded with increased abilities. Unfortunately, players vary a lot in the amount of time they can invest in the game, and therefore they advance at different rates. If only players of similar level can effectively play together, that hampers their ability to form long-term associations. Games such as World of Warcraft encourage players to band together into cooperative groups, or parties, but the groups are short-lived. Players have compensated somewhat by forming larger communities called guilds or clans. These can provide a ready pool of players to draw party members from. Still, it remains difficult to play shoulder-to-shoulder with your two or three best pals over the long run.

This is not optimal, and I regard it as an unsolved problem of the MMOG genre. Much of MMOG design is predicated on keeping players hooked for as long as possible and rewarding the investment of time into the game is one of the main strategies used toward that goal. However, if players can more easily maintain long-term friendships with other players in the game that would probably exert at least as strong an incentive to stay, if not stronger.

One possible option might be to make player abilities tied more to the amount of time the player has been subscribed to the game, or the amount of time his character has existed, rather than the amount of time spent online and actually playing. Thus characters created around the same time would tend to progress at the same rate and maintain similar abilities and be able to continue playing cooperatively over the long term. There could and should still be a skill element involved, so there would be something to be gained from practice and familiarization with the game. The difference in skills would then exist more in the players than in their online avatars.

The Element of Atmosphere


So far I have described only game mechanics. Some abstract games have succeeded on the strength of game mechanics alone, such as Qix or Tetris. However, it's much easier to engage the player's interest and enrich his experience by creating an atmosphere.

The first rule of atmosphere is that it must not sabotage the underlying gameplay in any way. That would be self-defeating. Instead, take the game and build a mood around it.

There are two general ways to create atmosphere: It can be done in the game itself, using the game engine, or it can be done with "incidentals" in the title and intermission screens. An advantage of moving some atmosphere elements (especially the more intrusive ones like music and video clips) out into the intermission is not to distract from the focus of the game itself. In the past this approach has also allowed the game designer to introduce a video experience that the game mechanisms and game graphics were not sophisticated enough to produce. With advancing technology, however, this has become less of a concern.

In-game graphics has always been the most important component of atmosphere. Indeed, much of the history of videogame design has been driven by advancing graphics technology. The debate over the importance of graphics and how hard to push hardware requirements (particularly on PCs) has been ongoing for years. Historically there has always been an argument that gameplay is more important than graphics; yet the actions of game designers seemed to say otherwise as they pushed hardware to its limits, seeking novel visual effects to "wow" the players. Some recent events suggest that we are hitting a point of diminishing returns in graphics, and that pushing the limits of graphic hardware may no longer be the best strategy for a successful game. Two examples will illustrate this trend.

First, Sony and Microsoft engaged in a technological arms race, both coming out with new game consoles at high price points: the Playstation 3 and the X-Box 360. Nintendo entered the market with their less powerful and less expensive Wii console and experienced huge success. An innovative controller, affordable system and quality games proved more compelling to gamers than polygon fill rates.

Secondly, Sony and Blizzard both began MMORPG games at about the same time: Everquest 2 and World of Warcraft. There was no doubt that Everquest 2 was more graphically advanced. It had higher system requirements and more detailed models. Yet, there were problems with the graphical content. Some models in the game were simply unappealing. Many game elements seemed uneven in quality or mismatched in style. By comparison, hugely successful World of Warcraft had graphics with fewer polygons, lower system requirements and a stylized appearance that some called cartoony (though to my eyes it looks more like an oil painting style).

When looking at World of Warcraft, we have to ponder the vast amount of content in the game. It took an army of artists to create all the models, environments and textures used in this game. Despite that, the style is so focused and coherent that you can examine objects all around the game world and easily imagine that they were created by the same artist. It's a triumph not of technology but of art direction. This will be the trend in future games: art direction will take center stage, and graphics display technology will be merely the canvas.

However, I do want to talk more about "incidental" graphics, music and other components. These would be title screens, between-level menus, FMV introductions and intermissions, etc. These must be made so the player can get past them quickly and get into the game without delay if he wants to. Nonetheless, these components can be used to tremendous effect for creating atmosphere. A good example of this was Road Rash 3DO. Still pictures are used in the menus and intermissions, and FMV clips with fine music for the title screens, start and end of races, and so forth. All these components combine to add tremendously to the excitement surrounding the game, but they don't interfere with the actual gameplay. It's a great example of how to take a game, which in this case already had solid gameplay and excellent in-game graphics, and then surround it with incidentals that raise it over the top and make it an extra special experience. This was tough prior to the advent of CD-ROM storage technology, because such incidentals would have squeezed out actual gameplay elements. Now that we have CD-ROM and DVD-ROM, storage space should no longer be much of an obstacle.

Another good example would be FMV clips embedded in the game at certain appropriate points, such as when the player attains a goal in an adventure game. Again, it's important to let the player quickly abort out of the video clip if he's tired of seeing it. A good example of how FMV could be useful is the Atari Jaguar game Alien Vs Predator. In the course of the game, the player will sometimes find computer terminals in which crewmen have recorded log entries. These are plain text. FMV clips could have been far more effective in creating an impression of the desperate last days on board the space station. It was impractical in AvP because of the limitation of the Jaguar ROM cartridge, but the advent of CD-ROM gaming allows these kinds of uses. Some games of the pre-CD-ROM era have used various kinds of animation to create these embedded sequences. CD-ROM and FMV allow us to do it better.

The use of FMV cut scenes has always come at a price, in terms of disrupting the player's sense of continuity and presence. The advent of MMOGs raises an additional problem: a cut scene would interrupt interactions with other players and break the sense of time and space that they all share in the game. As a result, more of the elements that would normally appear in cut-scene videos are being pushed into the game engine itself. This is a healthy development and has worked its way back into single-player games over time.

Sound effects are an important atmosphere component, just like graphics. Great sound effects can sometimes contribute more to the mood of the game than graphics do. For example, Alien Vs Predator on the Jaguar had fine graphics by the standards of its time, but the excellent digitized sound effects went perhaps even farther toward setting the mood of the game. In many cases, background sound effects are superior to music for setting a mood in the game. For that matter, even eerie silence can be useful sometimes.

Voice elements can cut both ways. Long gone are the days when any kind of voice in a game was a novelty. Now its value largely comes down to the quality of the voice acting. Miserable voice acting can easily do more harm than good. Good voice acting, on the other hand, can be compelling. In a game where scripted events are minimized, the challenge is to incorporate and trigger voice sequences in a way that makes sense. I could point to Command & Conquer Generals as an example of a game that did this pretty well, with heavy use of short "catch phrases" when various in-game events occurred.

Music can be used to set mood, but only with caution. Music is a powerful, and in some ways heavy-handed, tool. It is an arrogant way to set a mood. Rather than create an environment with graphics and sound effects and let the player react to it as he wishes, music grabs the player by the nose and leads him around, saying "THIS is what you should feel!" That's fine for introductions and intermissions. It's not so bad either for shooters or other heavy action games because there's really little doubt about how the player should feel. He should feel excited, and you can use the music soundtrack to pump him up.

For other kinds of games, such as adventure games, a constant background track during the gameplay is probably bad. It is better to let the player explore the game in his own way, and save the music for brief moments when something special happens in the game. Myst used music very effectively in this way. Most of the time it used sound effects to set the mood (and rather effectively, I should note) but used music when the player found some very special places in the game.

The element of atmosphere has always existed in video games. It's possible to pick out fine examples from the early 1980s, such as the beautiful opening screen of Phoenix or the intimidating aura of Defender. Artistic sensibility and attention to detail have always been more important than technology. Advancing technology sometimes opened new possibilities, but now it seems this happens less and less often as graphics have hit a point of diminishing returns. So, let's put this in perspective! As a designer you may be tempted to pour all your efforts into the glitz and glamour of FMV introductions, intermissions, high-detail models and the like while overlooking gameplay. The elements of presence, chaos and challenge should come first. When they are in place and working well, then it's fine to go back and tart up the game with atmosphere components.

In Closing


It's been eight years since I posted the original of this document, and most of my revisions have been prompted by the rise of 3D graphics and the emergence of internet-based games. A vexing trend for at least a decade has been the Hollywood Effect intruding onto the world of video games. Games have become far more expensive to create. To make something commercially competitive on DVD-ROM requires a lot more resources than the old days of 8K ROM carts on the VCS. MMOGs increase the resources required by yet another order of magnitude. Because of the great expense, game companies become less and less willing to take risks. Increasingly they focus on well-worn genres and near copies of already successful games.

A counter trend, if one is to be found, may grow out of the shareware space and the proliferation of so-called "casual games" on mobile devices, as well as on computers and consoles. This phenomenon is not as new as it seems. Pac Man, I am sure, would be pigeonholed in today's world as a mere casual game, and today it fits well onto a cell phone (aside from the lack of a decent controller). Many game designers would be well advised to take a fresh look at those old coin-operated games, in which game play was often innovative and almost always finely tuned and balanced. They have much to teach.

I hope there will always be hackers making games on a shoestring in their basements, garages, and attics. Their product may not be as slick and glamorous, but we can count on them to turn out something different from time to time. This document is dedicated to them.

Introduction to GameMonkey Script

$
0
0
This article will introduce you to the GameMonkey Script (abbreviated to GM Script or simply GM) language and API and how it can be used in your own games. It begins with a short introduction to the language and how it compares to Lua, the most similar language to GM Script and a popular choice in game development. The article will then show you the basic features of the script language itself and teach you how to bind the virtual machine with your game applications. This article does not intend to be a comprehensive tutorial on using GameMonkey Script, nor will it cover the advanced topics and techniques that can be employed when scripting, however it will be enough to whet your appetite for GM Script and provide you with enough information to explore the language on your own. You can follow along with the examples in the code by experimenting with the gme.exe program that ships with standard GameMonkey distributions and running the example scripts that are supplied with this article. You can find official GameMonkey Script distributions at the official site.

Note:  
This article was originally published to GameDev.net back in 2006 as a two-part series. It was combined and revised by the original author in 2008 and published in the book Beginning Game Programming: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.


Prerequisites


In order to get the most of this article and indeed GameMonkey script, it is assumed that:-

a) You have a working C++ compiler environment (an IDE or command-line, it doesn’t matter)
b) You are able to program to an average level in C++
c) You are able to understand languages with a similar syntax to C/C++
d) You have an interest or requirement in embedding scripting languages

If you fail any of these prerequisites, then by all means correct them before you continue reading as you will probably not get much from this article.

What Is GameMonkey Script?


Scripting in games has long been an important technique for developers to employ. Scripting allows you to separate your game logic from your hardcore engine code and gives you the ability to tweak your content without time-consuming rebuilds of the entire engine. Some games, for example Unreal, have vastly complicated scripting environments in place which allow you to create graphical effects and perform almost every action imaginable in your game. In the past it was popular for game developers to ‘roll their own’ scripting language that was tied directly into their game systems, however language parsing and virtual machine design are complex subjects meaning that many developers are choosing to adopt a third party scripting language.

Influenced by the popular embedded scripting language, Matthew Riek and Greg Douglas set about creating their very own language for a game project whilst employment at Auran Software; and thus GameMonkey Script was born. Created specifically for game scripting, GM Script is written with speed and simplicity in mind. The original version of GM Script has been used in several commercial ventures, from PC games to console games, and is growing in popularity amongst hobby game developers.

Comparison to Lua


As GameMonkey script was originally inspired by Lua, it is useful to highlight the differences between the two environments.


Attached Image: table1.jpg


Lua has been used successfully in many commercial and amateur game projects, however it can be difficult to use without a third party binding library as its stack-access method can be confusing for people new to scripting. GameMonkey Script aims to address the complexities of Lua whilst still maintaining its inherent power through flexibility.

Introduction to the Language


In the following section I will introduce you to the GameMonkey Script language. You will learn how to manipulate variables, how to create and call functions and how to use the powerful GM Script ‘table’ object. This section is intended to be a primer to the language and it is assumed that you have some programming knowledge of C, C++, Java or other similar syntax styles. You can follow the simple examples I present here and experiment yourself using the gme executable that comes with standard GameMonkey Script distributions.

Syntax Overview


This following section will give you an overview of the syntax used by the GameMonkey Script language. You will have noticed from the previous examples that the basic syntax of GM Script is very much akin to that of C. GM Script features the common elements of most programming languages:-
  • Variables
  • Comments
  • Expressions
  • Comparison Statements (if / else)
  • Loops (for, while, foreach)
  • Functions
GameMonkey Script also features an in-built threading and state system, allowing you to create multiple threads within the virtual machine to run several parallel tasks at once. However, I will not venture into that territory in this introductory article.

GameMonkey Scripts: Defined


A 'script' in GameMonkey contains the functions, variable declarations and expressions that form the basis of the behaviours you wish to implement in your application. Scripts are primarily stored as plain-text ASCII files that are loaded and compiled into bytecode, a form of machine language that is interpreted and executed by the GM environment. It is possible to load and execute precompiled bytecode but this not often done due to it being much harder for people to understand and work with.

GameMonkey Variables


Unlike C/C++, GM Script is a dynamically typed language; variables do not have to be declared before use and can hold any type of value, even functions. The in-built basic types of GameMonkey Script are that of the number (integers or floats), the string, the function and the ‘table’. Here are some simple variables in use:

myNull = null;
myNum = 50;
myString = "Hello!";
myFloat = 3.14;

print( myNum );
print( myString );
print( myFloat );

As you will see, a variable declaration is as simple as assigning a value. You will notice that every line is terminated by a semi-colon. This denotes the end of a statement and unlike Lua, is non-optional in GameMonkey Script. To C/C++ developers this will come naturally and so shouldn’t cause too many problems.

Comments


Comments in GM Script are exactly like those in C++; you can choose between C++ style line-comments or C-style block comments:-

x = 5;		// This is a line comment

/*
		This is a block comment
*/

Expression and Conditional Syntax


GameMonkey Script can evaluate common expressions using the standard C-style expression syntax and as such, anyone familiar with C or C++ will feel comfortable using GameMonkey Script expressions.

x = 5;			// Assign a value to x
print( x );

y = (x * 10) + 56;	// Assign a value to y
print( y );

if (x == 10)		// Comparison (equality)
{
	print( "x is equal to 10" );
}
if (x != 10)		// Comparison (inequality)
{
	print( "x is not equal to 10" );
}
if (x < 10)			// Comparison (less than)
{
	print( "x is less than 10" );
}
if (x > 10)			// Comparison (greater than)
{
	print( "x is greater than 10" );
}

GM Script also features bitwise operations, such as or (‘|’), and (‘&’) and not (‘!’) which can be used in assignment or comparison expressions.

One can employ conditional logic in GameMonkey script by using the ‘if’ comparison.

if ( <condition expression> )
{
	// Do something if true
}
else if ( <second condition> )
{
	// Do if 2nd condition passes
}
else
{
	// Do if all are false
}

Unlike C/C++, GM Script does not contain a ‘switch’ or ‘case’ statement so one must emulate the functionality by using blocks of if / else if tests.

Loops and iterations


GameMonkey script has several methods of executing a loop. The first is the familiar ‘for’ statement:-

for (<statement>; <condition>; <statement>)
{
	// repeated statement
}

A simple example to iterate a variable and print the number it contains would be:-

for (it = 0; it <= 10; it = it + 1)
{
	print( "it = ", it );
}

The output will be the numbers from 0 to 10 printed on separate lines in the output console.

The ‘while’ statement is used in situations where the conditions around the loop aren’t as certain; the most common use of the while loop is to loop until a particular flag is set.

while ( <condition> )
{
  // repeated statement
}

For example, to repeat until the user has pressed the ‘quit’ button:-

while ( !quitButtonPressed )
{
  // do something in the game

  quitButtonPressed = host_check_for_quit();		
}

Note that the host_check_for_quit function is a hypothetical application-bound function. Similarly, an approximation of the ‘for’ loop you saw previously would be:-

it = 0;

while (it <= 10)
{
	print( "it = ", it );
	it = it + 1;
}

The ‘foreach’ loop allows you to iterate over the contents and keys of a table. I will cover this in more detail in the table section of this article.

That was a brief overview of the various statements, expressions and symbols used in the GM Script language; however it is far from exhaustive. For a full list of expressions and iterators along with their syntax you are advised to consult the ‘GameMonkeyScriptReference’ which comes with the official GameMonkey Script releases.

Scripted Functions


The GameMonkey Script machine has two forms of functions. The first is the scripted function, a function that is created and coded in script. A scripted function is specified in the same way as a normal variable:-

myMultiply = function( x, y )	{ return x * y; };

As functions are created and specified as variables, it is extremely important to remember the terminating semi-colon at the end of the function declaration. Calling a function is as you’d expect from a C-style syntax:-

a = myMultiply( 100, 2 );
print( a );

The second type of function is that of the host-declared function. An example of a native function is the print command which is contained within gmMachineLib and bound to every gmMachine you create. Only ‘C’ style functions or static class methods can be bound to the GameMonkey machine, but there are workarounds to this. I will cover the binding of native functions further on in this article.

GameMonkey Script Tables


The table is an important and powerful structure within GameMonkey script. At its most basic, it allows you to specify arrays of data, it its most complex, you can begin to create organised structures of data and functions for use in your games.

Tables as Arrays


A table can be used as a simple array which contains any type of data.

Initialisation Example:

myArray = table( 1, 2, 3.14, 4, 5, "dog", "cat", 100 );

Much like C/C++ arrays, you need to use indices to access the data within the table. All indices are zero-based when a table is initialised in this manner. Lua programmers should note this difference, as in Lua initialised tables begin at index 1.

Accessing the data:

myArray[0] = 50;
print( myArray[1] );
myArray[100] = "dog_100";	// new item added
print( myArray[100] );

Tables as Associative Arrays


A table can also be used as an associative array, much like the map structure in the C++ STL. An associative array can be indexed using a non-numeric key, allowing for ‘named’ data lookups. Again, the data items in an associative array can be of any type.

Initialisation:

myData = table( Name = "Test", Weight = 60 );

Accessing the data is as simple as the first example:

print( myData[ "Name" ] );
print( myData[ "Weight" ] );
myData["Name"] = "Albert";
myData["Test"] = "Some Text Here";
print( myData[ "Name" ] );
print( myData[ "Test" ] );

You will have noticed that we can assign new keys and indexes at any time as the tables have no defined bounds.

Tables as Mixed Arrays


You can use the table as a ‘mixed array’, an array that contains both indexed data and keyed data (as in an associative array). This makes the table structure very flexible:-

myTest = table( 1, 4, Test = "Text!", 7, 8 );
print( myTest[0] );
print( myTest[3] );
print( myTest["Test"] );

In the example above, the second print statement prints the number 7 and not as you may expect, the word “Text!”. The reason for this is because GM Script keeps indexes and keys separate from each other within the table.

Iterating Table Data – ‘foreach’


The ‘foreach’ statement allows you to iterate over the contents of a table in a loop. The most basic form of the foreach statement is to examine just the values within the table:

foreach ( <value> in <table> ) { // statements }

An example of this follows:

fruits = table ( "apple", "pear", "orange" );
foreach ( frt in fruits )
{
  print(frt);
}

The above code will print the contents of the table to the console in no particular order. However, you may have noticed that the table key is often as important as the value it references and may wish to capture that data too:-

foreach ( <key> and <value> in <table> { // statements }

An example:-

fruits = table ( "apple", "pear", Juicy = "orange" );
foreach ( k and f in fruits )
{
  print( "The value at key "", k, "" is "", f, """ );
}

Will print something similar to:-

The value at key ‘0’ is ‘apple’
The value at key ‘Juicy’ is ‘orange’
The value at key ‘1’ is ‘pear’


Simulation of ‘structs’ and simple classes with Tables


The final use of the table structure is to simulate C/C++ structs and classes. If you recall what I mentioned before, the GM Script table object can store any type of data, including functions. Because of this, you can assign a scripted function to an index or key within a table.

myStruct = table(
  SayHello = function() { print( "Hello, world!" ); }	
);

myStruct.SayHello();	// Call table-bound function

As you see in the example, you can access keyed table data using the period (dot) operator. This allows us to treat the table as a simple class structure, accessing the named elements in a familiar fashion.

myAlien = table(
  Name = "Alien",
  AttackPower = 20,
  Strength = 50,
  OnAttack = function( entity )
    {
      entity.Damage( this.AttackPower );
    }
);

The slightly more complex example shows how simply a generic alien scripted object can be created using the in-built GameMonkey Script types and how it is centred primarily around the use of the table object.

Unlike C++ classes, it is important to note that the GM Script table object has no constructor/destructor, cannot be inherited from and does not allow for custom operator overriding. However, you can achieve such behaviour through creating your own bound types. It should also be noted that GM tables have no concept of public, private and protected scoping as C++ presents for structs and classes. All table members are declared as being in the public scope and so can be accessed from anywhere. I will continue the scoping discussion in the next section.

Scoping


GameMonkey script has a range of scopes for variables (and hence functions). If you wish your functions or methods to be accessible from outside of the script (for example, to be read directly by the host application) you must declare them as being in the global scope. The global scope is accessible everywhere in the script; even within other functions. Without this declaration, the objects are implicitly within local scope, which means they’re only accessible to within the current scope or lower.

// Create a variable in the global scope
global myvar = 100;

// parameter "a_param" is in function local scope
myfunc = function( a_param )
{
	// variable myvar is in local scope
	myvar = a_param;
	print( myvar );
};

print( myvar );
myfunc( 50 );
print( myvar );

Hold up a minute; you will notice that I’ve created 2 variables called myvar, one in the function and the other in global scope. If you run this script you will notice that the value of the global myvar is unchanged, even though you set the value of myvar in the function. The reason for this is simple; they exist in different scopes! GameMonkey allows you to set global variables from within functions by explicitly specifying the scope of the variable. In this case, I add the global keyword to the myvar declaration in myfunc.

// Create a variable in the global scope
global myvar = 100;

// parameter "a_param" is in function local scope
myfunc = function( a_param )
{
	// Access variable myvar in global scope
	global myvar = a_param;
	print( myvar );
};

print( myvar );
myfunc( 50 );
print( myvar );

Things can begin to become tricky, however, when using tables and the this operator. Whenever a variable is part of a table or user-defined object, it exists in the member scope of the parent object, or this. This concept will be familiar to you if you’ve done any work in C++, so I will not dwell on it. Let’s have a look at the member scoping in use:-

global mytable = table(

    myMember = 50,
    setMember = function( a_value )
    {
        myMember = a_value;
    }
);

print( mytable.myMember );
mytable.setMember( 100 );
print( mytable.myMember );

The script above behaves similarly to the local scoping example; the myMember method isn’t altered. However, when you include the member scoping keyword you will see a different result.

global mytable = table(

	myMember = 50,
	setMember = function( a_value )
	{
		member myMember = a_value;
	}
);

print( mytable.myMember );
mytable.setMember( 100 );
print( mytable.myMember );

The this scoping is fairly complicated, but at the same time is very powerful. Using this scoping you can create generic delegates that can access the data of the object that is passed as this. Confused? Take a look at the following example:-

myTable = table(
	myMember = 50
);

setMember = function( a_param )
{
	this.myMember = a_param;
};

print( myTable.myMember );
myTable:setMember( 100 );
print( myTable.myMember );

In this example the function setMember is completely external to the myTable object but is able to access its data and methods. The reason it is able to do this is though use of passing the myTable object as this when calling the setMember function. The body of setMember explicitly states that it will alter the data belonging to this without actually belonging to this at compile time. This allows you to create very powerful scripted functions which can exist in the global scope and be called from objects as if they were a member of that object itself. An abbreviation for typing ‘this’ is to simply type a single period ‘.’. For a more complex example of this in action, please refer to scoping_6.gm which is available in the attached files.

Embedding GameMonkey Script


Now that you have an understanding of the language syntax and have experimented with its use, you will want to want to do something with it in your games or applications. To do this, the GameMonkey Script library needs to be embedded in your own code. In order to do this, I will first introduce you to the concept of embedding and then provide some information in order to guide you in doing it yourself.

GameMonkey Script runs as a Virtual Machine environment that requires binding manually to your host application. A virtual machine is, as its name implies, a piece of software that behaves as if it were a computer itself (Wikipedia, 2005). Typically, a virtual machine will execute programs by interpreting its own 'bytecode' which is similar to the machine code used by the CPU on the machine I'm using to write this document. The game (or application) that uses the virtual machine must interact with its interface. You cannot simply call a scripted function or access data directly from your compiler environment - you must access the correct API functions to do so. Typically, the virtual machine has a very basic set of functions and data types; it is up to your game to export an interface of its own to the script environment. As you do this, the game and scripting environment become bound in a symbiotic relationship; in this state, the game is often referred to as the 'native' (as in non-interpreted), or 'host application' in which the scripted machine is 'embedded' (Varanese, 2003).

Setting up your Compiler Environment


Compiler environments can vary quite wildly and so this section is by no means a comprehensive tutorial on setting up GameMonkey Script for your compiler. At this stage you will have downloaded the latest GameMonkey Script release zip file, unpacked it and be presented with the C++ source code for the library. To embed GameMonkey script, you can either compile a static library from the source code or compile the source along with your host application. For the sake of convenience, I will go the static-library route and describe the basic method of compiling a static library of the GM Script sources using a typical IDE.

  1. Create an empty 'static library' C++ project
  2. Locate your relevant platform configuration file(*) and copy it to your gm/src directory of the standard GM Script distribution
  3. Import the *.cpp and *.h source files from the gm/src folder excluding the gmDebugger.cpp and gmDebugger.h files
  4. Set your compiler settings, altering the optimisation, runtime library and debug settings appropriately
  5. Compile the static library and store in an accessible location

Once you have the static library, you can link it to your application like a normal library and compile programs using the GM Script API (assuming, of course, that you tell your compiler where to find the headers).

You will notice that step 2 has an asterisk next to it. The configuration file gmConfig_p.h is where all platform-specific defines and settings are kept. You can use it to specify the endian-ness of native CPU, the size of common types or even set specific compiler settings. Choosing your platform configuration is relatively simple for common Windows and Linux platforms (both MSVC and GCC), but you may need to edit your configuration file if you're using an exotic compiler and/or platform.

Creating the GM Virtual Machine


Before you can use GameMonkey Script in any way you must embed it in your application. To do so, you must create an instance of the virtual machine, which is embodied by the gmMachine class provided by the GameMonkey API.

#include "gmThread.h"

int main()
{
	// Create the GM Machine object on the stack
	gmMachine gm;

	return 0;
} 
Example: basic_1.cpp

Compiling and linking should proceed without a problem provided that you a) include the directory containing your GM headers in your project or global search path and b) link to the static library you created earlier. If you run this program you will see nothing special; but it's there. The GameMonkey VM is embedded in your application!

Executing a String as a Script


An embedded virtual machine is next to useless without a script to run on it, so this next section will cover the most basic way you can execute a script on the GameMonkey VM. The gmMachine object you created earlier has a member function called ExecuteString which provides you with a method to execute a text string as a script on the GM Machine. Let's see a simple example of this in action:-

#include "gmThread.h"

int main()
{
	gmMachine gm;

	// Execute a simple script
	gm.ExecuteString( "print( \"Hello, world!\" );" );

	return 0;
} 
Example: basic_2.cpp

Compiling and running this program should display the immortal words "Hello, world!" in your console window. There are two ways of using string literals within GameMonkey Script; the first is to use the familiar double quotes to denote the beginning and end of the string. For example:-

gm.ExecuteString( "print( \"Hello, world!\" );" );

The second way is to use the back quote character:-

gm.ExecuteString( "print( `Hello, world!` );" );

These both perform the same function but it should be noted that the two styles cannot be mixed within the same string:-

gm.ExecuteString( "print( \"Hello, world!` );" );  //ERROR

Providing GameMonkey Script with a script string containing double-quoted string literals poses us a problem as we’re using C++ and we will need to escape these elements in order to compile our program.

In the following code snippet, I will store my script in a const char* string and pass the data to GM for execution:-

#include "gmThread.h"

int main()
{
const char *myScript = "fruits = table ( \"apple\", \"pear\", \"orange\" );	foreach ( frt in fruits ) { print(frt);	} ";

	gmMachine gm;

	// Execute a simple script
	gm.ExecuteString( myScript );

	return 0;
} 
Example: basic_3.cpp

As you can see, hard coding scripts as text strings can quickly become messy and it actually defeats the need for a scripting language in the first place; your strings are still compiled along with the application and any changes to them forces a recompile! In the following section I will talk about loading scripts from a file.

Executing a Script from File


In order to overcome the need for hard-coded scripts, it is beneficial to load the scripted data from a file. However, you may notice that the gmMachine object has no member function for loading scripts from a file resource. To some, this omission may seem to be a startling oversight, but it becomes understandable when you appreciate the context that the GameMonkey Script environment is intended for - games! Games often have many weird and wonderful methods of loading file resources; some choose straight file IO, others use a zip-like format, others have encryption and some even use a network layer to receive the game content. In this context, a single LoadScript member function is fairly pointless as the majority of games embedding GM would have little use for it.

The basic method of loading a script from file is as follows:-

  1. Locate the file resource
  2. Open the file resource
  3. Obtain the size of the data to load
  4. Allocate enough memory to load the data plus one byte for the zero terminator
  5. Load the data from file into the allocated memory
  6. Close the file
  7. Compile the script from the loaded data
  8. Free up the allocated memory containing the script

It is worth bearing in mind that steps 3, 4 and 5 can be combined into a single step because we can use ifstream and string from the C++ standard library, however I have chosen to separate out the steps should you be using a non-C++ library such as PhysicsFS or the legacy C FILE functions. With this in mind, it is simple to write our own simple implementation based on the C++ standard library.

#include <fstream>
#include <string>
#include <iterator>

int gmLoadAndExecuteScript( gmMachine &a_machine, const char *a_filename )
{
    std::ifstream file(a_filename);
    if (!file)
        return GM_EXCEPTION;
	
	std::string fileString = std::string(std::istreambuf_iterator<char>(file),
                                         std::istreambuf_iterator<char>());
	file.close();
	return a_machine.ExecuteString(fileString.c_str());
}
Example: loading_scripts_1.cpp

If you now integrate this code with the simple example code shown previously, you should now be able to load and execute scripts from a text file. GameMonkey scripts usually have a *.gm, extension, but this is obviously not required.

#include "gmThread.h"

int main()
{
	gmMachine gm;

	// Execute a simple script file
	gmLoadAndExecuteScript ( gm, "scripts/hello.gm" );

	return 0;
} 

Basic Error Checking


Now that your scripts are getting more complex you may begin to experience accidental syntax errors which would cause the script compilation to fail. It is extremely important that you handle at least the basic errors that come from bad script compilation. The gmMachine object maintains an internal log of the last errors in the compilation process which will help you catch these errors should they occur.

The basic way of checking an error is to catch the return from the ExecuteString member function. If the value is zero, there is no error and you can proceed as normal. However if the value is non-zero you have an error and should handle it somehow. Simply knowing that there's an error is useless in helping you track it down, so GameMonkey gives you access to the error log for your own use. You can access the compilation log using the GetLog member function of the gmMachine object. From the retrieved gmLog object you can access the GetEntry function, which allows you to iterate through the errors the log contains, popping them from the list when you're done.

#include "gmThread.h"

void handleErrors( gmMachine &a_machine )
{
	gmLog &log = a_machine.GetLog();
  // Get the first error
  bool firstError = true;
	const char *err = log.GetEntry( firstError );
	while ( err )
	{
		std::cout << "Compilation error: -" << err << std::endl;
		err = log.GetEntry( firstError );
	}
}
Example: error_handling.cpp

In the code above you will notice the use of the firstError variable; this is used internally by GM Script to control the one-way iteration over the log entries. You must be set the variable to true for the first call to the function; it will be set to false on subsequent calls.

int main()
{
const char *myScript = "fruits = table ( \"apple\", \"pear\", \"orange\" );	foreach ( frt in fruits ) { print(frt);	} ";

	gmMachine gm;

	// Execute a simple script
	int ret = gm.ExecuteString( myScript );
  if ( ret != 0 )
  {
    // There were errors in the script, exit gracefully
    handleErrors( gm );
    return 1;
  } 

	return 0;
} 
Example: error_handling.cpp

Now that you know how to handle script compilation errors I will set you the task of updating the gmLoadAndExecuteScript function and adding better error handling and reporting compilation errors to the user.

More on Script Execution


All of the examples we've looked at so far have assumed that you want to execute a script immediately; this can often not be the case in games where data and scripts need to be loaded and compiled at a specific time, usually while a loading screen is presented. GameMonkey has two ways of preventing a compiled script from immediate execution; the first is to set the optional a_now parameter of the gmMachine::ExecuteString member function to false when you call it. This will compile the string but not execute it until you call the Execute member function on the gmMachine object. An example of this follows:

#include "gmThread.h"

int main()
{
  gmMachine gm;
    
  // Compile a simple script without running it
  gm.ExecuteString( "print( \"Hello, world!\" );", 0, false );
    
  // The text wasn't printed!
    
  // Wait for a key press
  getchar();
   
  // The script will now execute!
  gm.Execute( 0 );
    
  return 0;
} 
Example: execution_1.cpp

A task for you, the reader, is to create an enhanced version of the gmLoadAndExecuteScript function which allows you to specify when the code is actually executed by specifying the a_now parameter appropriately.

The second method of controlling when a script is executed involves compiling it down to raw bytecode and executing the stream as a library. This method is useful in games as the script is converted into a form that isn’t as easily readable as a script stored in a text file. One common use for scripting in games is to execute a script within a game level when a certain event is triggered; by using a bytecode version of your scripts you can easily include compiled script code within your game level file. This is beyond the scope of this article and will not be explored further, however if you wish to explore this on your own, you should refer to the functions CompileStringToLib and ExecuteLib which are members of gmMachine. Like using text files, you are responsible for writing your own loading and saving routines for the gmStream objects that contain the compiled scripts.

In the example code execution_1.cpp you were introduced to a new member function of the gmMachine object, namely Execute. This function effectively tells the machine to execute all active threads in turn. As you can appreciate, it is extremely powerful when it comes to running scripted threads, as you can now update the GameMonkey virtual machine along with the rest of your game logic. Using scripted threads, or co-routines, is out of this article’s scope but I will introduce you to the basic concept of how they work.

Everything within the GameMonkey virtual machine runs in a gmThread object, which has its own executable bytecode, stack and scope. The virtual machine can hold many of these threads in memory at any one time, preserving the states of them until the threads are next 'ticked' by the machine. What happens behind the scenes of the ExecuteString function is that GameMonkey will spawn a new GM Script thread, compile the script into the thread and populate its internal structures with references to the functions and variables contained within this script. If a_now is true, the thread will execute immediately on the gmMachine, causing the bytecode to be interpreted until there is no more code or the thread yields control back to the virtual machine. If a_now is false, the thread will be pushed into the queue along with the other threads and will only be updated on an Execute cycle, again executing until there is no more code or the thread yields.

The GM Script Objects


If you look back you will recall that I said GameMonkey Script has several built-in types, namely integers, floats, strings, functions, tables and null. All variables within GMScript are held and accessed initially by gmVariable objects, which is a ‘catch-all’ type and will be explained fully in the next section. Because GameMonkey Script objects are ‘reference types’ we must allocate them reference through the virtual machine; this rule applies to all but the three basic types – integers, floats and the null type. Therefore, if you need to create a function, string, Table or a user-data type you must use request them from the gmMachine in the form of the gmFunctionObject, gmStringObject, gmTableObject and gmUserObject C++ objects.

I am not going to detail exactly how to use these types because you can glean this information from the GM API docs, but I will say that every time you need GM to handle any of these types you must allocate them via the gmMachine object. For example, if you need to pass a string as a function parameter you cannot just pass the literal or a pointer to the null-terminated string data. Instead, you must allocate a new gmStringObject using gmMachine::AllocStringObject and populate the data accordingly. Likewise, if you are creating a bound type which needs to reference a native structure you must allocate a gmUserObject and set its data as your native pointer. You will see examples of these operations in the following section.

gmVariable Object


The gmVariable type is important in many areas when accessing the VM API. Instead of having to provide multiple API functions to handle every conceivable object type GM has provided the gmVariable. The gmVariable is used in everything from function parameters and return data to table data objects; this object is a 'catch-all' class which holds the variable data that can pass to and from the VM as it holds information about the type of data as well as the data itself. The GM machine will read off the variable type before deciding how to proceed with handling the data it contains so it is vitally important that you set the appropriate type when manipulating your variables. Let's have a look at how you can create and manipulate a new variable:

int main()
{
	gmMachine gm;

  // Allocate a string from the machine and set it as a variable
  gmVariable stringVar( gm.AllocStringObject("Hello, World") );
  // Allocate a variable containing an int and a float
  // note it doesn't need to be created from the machine
  gmVariable intVar( 100 );
  gmVariable floatVar( 1.5f );
  
  // Create a variable with a newly created table
  gmVariable tableVar( gm.AllocTableObject() );
  
  // Reset table var as an int, losing the table data it contained
  tableVar.SetInt( 200 );
  
  // Variable copying
  intVar = floatVar;
  
  // Make 'null'
  stringVar.Nullify();
  
  return 0;
}
Example: variables_1.cpp

As mentioned previously, you need to allocate String and Table objects from the gmMachine object, but because floats and integers are native types, you are not required to perform any extra allocation to hold the data they contain. The variables created in the example don't actually do anything, but they are all ready to be pushed as a function parameter or set in a table. As you can see above, like in GMScript itself, you are free to assign variables, reset variables with new data or even completely nullify them.

Sometimes you will be presented with a gmVariable from the machine; it may be come as a function parameter or as the result of a 'Get' on a gmTableObject. In these situations you will need to know how to retrieve the data. If you look in the gmVariable.h header you will see the following code in the gmVariable class.

gmType m_type;
union
{
  int m_int;
  float m_float;
  gmptr m_ref;
} m_value;

This should give you a hint in how to retrieve the data. Before you can do anything else you must check the m_type member. This holds the appropriate enumeration and will be either GM_NULL, GM_INT, GM_FLOAT, GM_STRING, GM_FUNCTION, GM_TABLE or an integer for your own types which increment from GM_USER. Once you have ascertained the type you can retrieve the value from the m_value union; in the case of integers and floats this is as simple as accessing m_int and m_float respectively, but what happens for other types? The answer should be obvious – you need to cast the m_ref pointer to your appropriate type. It should be noted, however, that there are no runtime checks to ensure you're casting to the correct type; it is for this precise reason that you should always check the type before you attempt to cast. In a dynamically typed environment such as GameMonkey Script there are no guarantees about the type of data you will receive, so you must place it upon yourself to check before you access the data in the variable. The following example will highlight how to check for the type and access the data accordingly. As an exercise, try experimenting by changing the value you place in the var gmVariable and examine the results you get back.

using namespace std;

int main()
{
	gmMachine gm;

    // Try setting your own variable here
    gmVariable var( gm.AllocStringObject("Hello, World") );

    switch (var.m_type)
    {
    case GM_NULL:
        cout << "Variable is NULL type" << endl;
        break;
    case GM_INT:
        cout << "Variable is INT type" << endl;
        cout << "Value:" << var.m_value.m_int << endl;
        break;
    case GM_FLOAT:
        cout << "Variable is FLOAT type" << endl;
        cout << "Value:" << var.m_value.m_float << endl;
        break;
    case GM_STRING:
        cout << "Variable is STRING type" << endl;
        cout << "Value:" << reinterpret_cast<gmStringObject *>(var.m_value.m_ref)->GetString() << endl;
        break;
    case GM_TABLE:
        cout << "Variable is TABLE type" << endl;
        cout << "Items:" << reinterpret_cast<gmTableObject *>(var.m_value.m_ref)->Count() << endl;
        break;
    case GM_FUNCTION:
        cout << "Variable is FUNCTION type" << endl;
        break;
    default:
        cout << "Variable is USER type" << endl;
        // retrieve native pointer from user object
        void *ptr = reinterpret_cast<gmUserObject *>(var.m_value.m_ref)->m_user;
        
    };
    
	return 0;
}
Example: variables_2.cpp

A final word of caution about gmVariables; numeric data can be stored with either the GM_INT or the GM_FLOAT type depending on whether it has a decimal or not. For example, imagine a script which uses a loop to count from 0 to 10 in increments of 0.5f. The variable will initially hold data of int type (zero is an int), and alternating between float and int on every second iteration. Because of this, it is important that any function that expects numeric float data performs a check to see if the variable is either a GM_INT or a GM_FLOAT.

Calling a Scripted Function


There will often be times that you would like to call a known scripted function from your application; perhaps you specify that your game will need to call an InitialiseGame scripted function, or perhaps the Attack function in scoping_6.gm that was originally presented in previous script examples. In GameMonkey Script there are currently two ways to call a scripted function from native code; the 'manual' way which uses the raw API to push variables and prepare the stack, and the 'simple' way which uses a utility call gmCall. I will take you through calling a scripted function using gmCall, since that is the simplest and most common way you'll need to call a function. For purists, I have included some example source that doesn't use gmCall with this article, but for most purposes I advise using gmCall.

Using gmCall


The authors of GameMonkey Script realised that the 'manual' way of calling functions can be long-winded and prone to error and so have thoughtfully provided gmCall to vastly simplify the calling of scripted functions.

  1. Create the gmCall object
  2. Call the BeginGlobalFunction / BeginTableFunction member function to initialise
  3. Push the parameters
  4. 'End' the call
  5. Get return values, if required

Here is an example that calls a scripted function myMultiply which takes two numbers as parameters and returns their sum.

int main()
{
    gmMachine gm;

    // Execute a simple script file
    gmLoadAndExecuteScript ( gm, "scripts/function_test.gm" );

    gmCall  call;
    if ( call.BeginGlobalFunction( &gm, "myMultiply" ) )
    {
      // Push the parameters
      call.AddParamInt( 10 );
      // Example showing a variable can be used
      gmVariable var( 2 );
      call.AddParam( var );
      // Execute the call
      call.End();
      // Handle the return value
      int myReturn = 0;
      if (call.GetReturnedInt( myReturn ) )
      {
    	std::cout << "myMultiply returned " << myReturn << std::endl;
      }
      else
      {
    	std::cout << "myMultiply returned an incorrect value" << std::endl;
      }
    }
    else
    {
        std::cout << "Error calling 'myMultiply'" << std::endl;
    }
    
	return 0;
}

The gmCall AddParam* functions allow you add parameters without having to deal with gmVariable objects (although it is entirely possible to pass your own gmVariables). Any required allocations and validations are performed automatically for you, allowing you to quickly access the scripted functions you need.

Creating a host-bound function


Now that you know how to call a scripted function from native code, you may be wondering how you export your own functions to GameMonkey Script. The reasons for wanting to do so are understandable; some people may want the speed of native code (although GM is fast, it's still slower than native code), others may want to expose aspects of their game or engine to the script. Whatever your reasons, you need to know how to export your own functions.

Unlike some scripting languages (for example AngelScript), functions must be 'wrapped' for use by the VM. The GM Script VM doesn't have the ability to translate the parameters from the script into a form understandable by your native compiled code – one reason being that function calls and their parameters aren't known until runtime, unlike in C++ where everything is checked during compilation. The process within the wrapped a function usually involves the following steps:

  1. Parsing the parameters passed from the scripting VM
  2. Converting them into a usable form for C++ code
  3. Performing the main action of the function
  4. Converting any return values to a suitable format for the VM
  5. Returning control back to script

Here's the code I will be examining for a simple 'native' version of myMultiply:-

int GM_CDECL gm_myMultiply( gmThread *a_thread )
{
	// Check the number of parameters passed is correct
	if (a_thread->GetNumParams() != 2 )
    return GM_EXCEPTION;
        
    // Local vars to hold data from params
	int a_x = 0;
	int a_y = 0;
	
	// Check params are valid types
	if (a_thread->ParamType(0) != GM_INT)
    return GM_EXCEPTION;
    if (a_thread->ParamType(1) != GM_INT)
    return GM_EXCEPTION;
	
	// Get data from params
	a_x = a_thread->Param(0).m_value.m_int;
	a_x = a_thread->Param(1).m_value.m_int;
	
	// perform calculation
	int ret = a_x * a_y;
	
	// return value
	a_thread->PushInt( ret );
	
	return GM_OK;
}

Like its scripted counterpart, this function takes two parameters and returns the sum of their values. The 'wrapped' function first checks that the number of parameters it has been passed is correct; if not we return a scripted exception, next it checks to see that the parameters are the correct type (in this case, int). It then reads the values of these parameters from the thread - you will notice that the parameters are held again in the gmVariable structure. After the parameter data has been collected and converted you can now perform the main body of the function; in this example it is a simple multiplication but in your games it is likely that you will call another function in your engine.

You will notice that a lot of the code to check parameters is repetitive and can be cumbersome to write. Fortunately for us, the GameMonkey authors have provided several macros to aid our task.

int GM_CDECL gm_myMultiply( gmThread *a_thread )
{
	GM_CHECK_NUM_PARAMS(2);
	GM_CHECK_INT_PARAM( a_x, 0 );
	GM_CHECK_INT_PARAM( a_y, 1 );
	
	int ret = a_x * a_y;
	
	a_thread->PushInt( ret );
	
	return GM_OK;
}

The GM_CHECK_NUM_PARAMS macro is fairly self-explanatory; it checks the number of parameters from the thread. The GM_CHECK_*_PARAM macros are similar, but they also declare the variables and fill them for you. These macros make the code much simpler to read while still performing the same function.

Creating a simple type


Many game developers will want to expose their own types to a scripting system, and GameMonkey Script is no exception. Imagine a simple class that is in most 3D applications, the Vector class. This can be as simple as a structure composed of 3 floats for x, y and z or can be as complex as a class which holds member functions to perform operations and calculations on the vector data. Sometimes it is possible to use the integral table object for custom-bound objects, but in some situations you require the application to have more control over the object – perhaps by providing extra validation, data conversion or updating another object in your game when the data is changed on the object. With a simple table this sort of control is impossible, hence the need for user-defined types.

Choosing what to bind


When choosing to bind your type you must consider how it will be used in script. For example, the simple Vector type could be easily represented as a fixed array of 3 floats, it could have the constituent items stored in named members (such as x, y and z), or it could be comprised of both. For example:

v = Vector( 10, 20, 50 );	// Create vector

// Vector could be represented using 3 floats
v[0] = 20;
v[2] = v[1];

// Access vector using named members
v.X = 20;				// Set X member to 20
v.Z = v.Y;

You could choose to provide the exact same interface as your engine's vector class, you could simplify it somewhat, or you could even provide an entirely different interface. For example, some people may wish to access vector maths functions using the dot operator, whereas others may wish to keep the vector type as data-only and provide specialist functions to manipulate the data. For example:

v = Vector( 10, 20, 50 );	// Create vector

v.Normalise();			// Call normalise using dot operator
NormaliseVector( v );		// Normalise using a specialist function

Many game developers seem to think they need to expose their entire C++ class to the scripting environment. While this is possible, it is often unnecessary; there could be many member functions that are useless in a scripting context so it makes little sense to include them. Even more so, there may be member functions and data that you do not want your script to be able to access so exposing the full C++ class would be undesirable. Unfortunately, the decisions about what to bind are entirely context dependant and will change for every game project, game system or even game class you wish to bind to a scripting language. In my personal experience, I have found it useful to provide a simplified and refactored interface to the scripting environment. The more complicated it is to use the scripted interface the more likely it is to confuse people and create problems for them whilst using it. If the interface is simple, you can also spend less time documenting it and writing code to debug it.

Binding a simple Vector type


The discussion around type binding will continue by expanding the Vector example examined previously. I will outline a simple 3D vector class and create a type binding for the GameMonkey Script environment.

Imagine a typical game engine's vector type (simplified):

class Vector
{
public:
  Vector() : x(0), y(0), z(0) { }
  Vector( float a_x, float a_y, float a_z ) : x(a_x), y(a_y), z(a_z) { }

  float x, y, z;
    
};

There are three constructors; one default constructor, one taking the values to initialise the vector with and a copy constructor. The first thing to do to bind this vector type to GM Script is to specify the basic type library. The 'library' is nothing more than a global function declared within the machine global context and an instruction to the gmMachine to register this function with a specified type name. This global function effectively becomes the type's constructor and is the ideal place to begin creating our bound type.

The constructor entry point is specified in a gmFunctionEntry table as such:

namespace gmVector
{
  // Declare a type ID
  gmType   Type   = GM_NULL;

  int GM_CDECL libentry( gmThread *a_thread )
  {
    Vector *p = new Vector();
    a_thread->PushNewUser( p, Type );
    return GM_EXCEPTION;
  }
    
  gmFunctionEntry lib[] =
  {
    { "Vector", libentry }		// type name, entry point
  };

}; // end of namespace gmVector
Example: vector_1.cpp

You will notice the global declaration of a gmType variable named Type; this is to hold the GM type Id once the type is registered with the gmMachine. You are free to name the constructor whatever you want, but for organisational reasons I have named it libentry and placed it within the gmVector namespace. The libentry function is where the first step of the binding is done; we create an instance of the native class object and return it to GM by pushing it onto the thread as a new gmUserObject but with the type id stored within our library.

Now that the initial structure of the library is laid out we can register it with the gmMachine. Initialisation is done with 2 simple function calls:

namespace gmVector
{
  void BindLib( gmMachine *a_machine )
  {
    // Register one function (the entry point)
    a_machine->RegisterLibrary( lib, 1 );
    Type = a_machine->CreateUserType( lib[0].m_name );
  }

}; // end of namespace gmVector
Example: vector_1.cpp

Just like function binding, you need to register the library with the machine. After doing so you create your type and store it away for use in the rest of the library. The CreateUserType member function simply takes the name of a type to register; in this case I've retrieved it directly from the library constructor's name as you should ensure that it corresponds with the function you just registered as the constructor.

Constructor with Parameters


At present, this type is pretty useless as there is no way of accessing the data the Vector contains. The decision about how to access the data is an important one; shall we allow the user to access it via named properties (e.g.: x, y, z members) or via an array of floats. I shall answer this question shortly, but first we need a way of initialising the data on construction of the type. As in the C++ class, we want to allow the user to specify a constructor with several parameters to initialise the data in the vector as well as providing a default constructor.

namespace gmVector
{
  int GM_CDECL default_constructor( gmThread *a_thread )
  {	
    // Create a native object with default params
    Vector *p = new Vector();
    a_thread->PushNewUser( p, Type );
    return GM_OK;	
  }

  /// This is the constructor for passed data items
  int GM_CDECL data_constructor( gmThread *a_thread )
  {    
    // Check for a valid number of parameters
    if (a_thread->GetNumParams() != 3 )
      return GM_EXCEPTION;

    // Loop through and grab the params, checking their types
    float v[3];
    for (int i = 0; i < 3; ++i)
    {
      switch (a_thread->Param(i).m_type)
      {
        case GM_INT:
        v[i] = a_thread->Param(i).m_value.m_int;
        break;
      case GM_FLOAT:
        v[i] = a_thread->Param(i).m_value.m_float;
        break;
        default:
          a_thread->GetMachine()->GetLog().LogEntry( "Vector: Param %d error - expected int or float", i );
          return GM_EXCEPTION;
      }
    }

    // Create a native object with default params
    Vector *p = new Vector( v[0], v[1], v[2] );
    // Return to GM
    a_thread->PushNewUser( p, Type );
    return GM_OK;	
  }
        
  /// Entry point for the library; this is effectively the constructor 
  int GM_CDECL libentry( gmThread *a_thread )
  {
    // Delegate the appropriate call based on the arg count
    switch (a_thread->GetNumParams())
    {
    case 0:
      return default_constructor( a_thread );
    case 3:
      return data_constructor( a_thread );
    };
    
    // Not handled, log an error and return an exception
    a_thread->GetMachine()->GetLog().LogEntry( "Vector: Bad number of parameters passed" );
    return GM_EXCEPTION;
  }
};	// end namespace gmVector
Example: vector_2.cpp

I have adjusted the libentry function to delegate the actual constructor operation based on the number of parameters passed. The default_constructor function is the same as the previous libentry function, but in data_constructor you can see that I engage in an operation to retrieve the parameters from the thread. This is exactly the same as if you were binding a regular function, so I won't dwell on how it works. The new constructor validates that the parameters are either integers or floats and stores them for passing to the native Vector class' constructor.

The data constructor could be extended to allow for copy construction of objects, allowing you to pass a variable of your Vector type and use it to create a copy of itself. This is important to remember as GameMonkey Script user-bound variables are reference types; if you assign a variable with another they will reference each other and any changes made to one will be visible in the other. As a learning exercise, I will leave the addition of the copy constructor as an extension for you to pursue on your own - you should be able to code it yourself with the information I’ve given you so far. If you get stuck you may refer to example vector_2a.cpp for my implementation.

Now that there's some actual data in the vector object you need to allow scripts to retrieve it for use in other operations. It is at this point that we must decide the data access method; I have chosen to provide the members x, y z as this is familiar to many people and I feel best represents how I will use the vector in script. As I said previously, the decision is entirely yours and the methods I will describe can be adapted to use the indexed access method if you choose.

Operator Overrides


If you have ever overridden a class operator in C++ you will know how powerful this feature is – operator overriding allows you to specify a function to control the behaviour of a type when certain script operators are used. The operators you can override for a type in GM Script are:


Attached Image: table2.jpg


In this simple example I will only override the dot operators to provide access to the simulated 'member' access method of the Vector data. Operator overriding follows the same structure for each operator so it is simple to adapt my example to use the index operators.

Operator functions all have the same signature:-

void GM_CDECL operator_func(gmThread * a_thread, gmVariable * a_operands);

GetDot Operator


The operands passed to the operators vary on the type of operator. For the GetDot operator the operands are as follows:


Attached Image: table3.jpg


The following code details the GetDot function we will be using:

void GM_CDECL OpGetDot(gmThread * a_thread, gmVariable * a_operands)
{
	GM_ASSERT(a_operands[0].m_type == Type);
  Vector* thisVec = reinterpret_cast<Vector*>(reinterpret_cast<gmUserObject*>(GM_OBJECT(a_operands[0].m_value.m_ref))->m_user);

  GM_ASSERT(a_operands[1].m_type == GM_STRING);
  gmStringObject* stringObj = reinterpret_cast<gmStringObject*>(GM_OBJECT(a_operands[1].m_value.m_ref));    const char* propName = stringObj->GetString();

  // Resolve the member name
  if(::stricmp(propName, "x") == 0)
  {
    a_operands[0].SetFloat(thisVec->x);
  }
  else if(::stricmp(propName, "y") == 0)
  {
    a_operands[0].SetFloat(thisVec->y);
  }
  else if(::stricmp(propName, "z") == 0)
  {
    a_operands[0].SetFloat(thisVec->z);
  }
  else
  {
    a_operands[0].Nullify();
  }
}
Example: vector_3.cpp

The process is simple; first we check that the type of the variable being operated on matches that of the newly bound Vector type. Afterwards, the string name of the member is retrieved and checked against the members we wish to 'export'. Confusingly operand zero is also the return variable, so it must be set with the relevant value from the bound class. This is as simple as copying the value from the 'real' member in our class to the variable represented by operand zero. If there is a problem, you should nullify the return variable, which returns null to GameMonkey Script.

With the GetDot operator function created, it's time to add its registration to the BindLib function we looked at earlier. This is as simple as calling the RegisterTypeOperator on the newly bound type, passing the relevant operator and function handler as parameters:

a_machine->RegisterTypeOperator(Type, O_GETDOT, NULL, OpGetDot);

SetDot Operator


If you experiment with some scripts on the new Vector type, you will notice that you can read data from it but not alter the data in any way outside of actual construction of the variable. We will now provide a SetDot operator override to handle the setting of the member data. The operands are similar to that of GetDot:


Attached Image: table4.jpg


A function to handle the SetDot operator for our vector class is as follows:

void GM_CDECL OpSetDot(gmThread * a_thread, gmVariable * a_operands)
{
  GM_ASSERT(a_operands[0].m_type == Type);
  Vector* thisVec = reinterpret_cast<Vector*>(reinterpret_cast<gmUserObject*>(GM_OBJECT(a_operands[0].m_value.m_ref))->m_user);
  
  GM_ASSERT(a_operands[2].m_type == GM_STRING);
  gmStringObject* stringObj = reinterpret_cast<gmStringObject*>(GM_OBJECT(a_operands[2].m_value.m_ref));
  const char* propname = stringObj->GetString();
  
  // Create a variable to hold the data to assign
  // handle both ints and floats
  float newFloat = 0.0f;
  if(a_operands[1].m_type == GM_FLOAT)
  {
    newFloat = a_operands[1].m_value.m_float;
  }
  else if(a_operands[1].m_type == GM_INT)
  {
    newFloat = (float)a_operands[1].m_value.m_int;
  }
  
  // Assign the data
  if(::stricmp( propname, "x") == 0)
  {
    thisVec->x = newFloat;
  }
  else if(::stricmp( propname, "y") == 0)
  {
    thisVec->y = newFloat;
  }
  else if(::stricmp( propname, "z") == 0)
  {
    thisVec->z = newFloat;
  }
}
Example: vector_3.cpp

As you can see, it follows a similar progression from the previous example, except this time we need to retrieve the value passed to the function by the script (operand #1). As before, the member is compared to the exported members and the assignment made accordingly. Adding the following line to the BindLib function will complete the simple type.

a_machine->RegisterTypeOperator(Type, O_SETDOT, NULL, OpSetDot);

I mentioned previously that GameMonkey Script user variables are reference types; this becomes evident now that you have working dot operators. For example, if one were to write a script such as:

a = Vector( 20, 20, 30 );
b = a;
b.x = 10;
print(a.x);
print(b.x);

You would see that updating the value of b.x would also update the value of a.x as they both reference the same data. The solution to this would be to implement a copy constructor for the Vector type (or add a Copy function to the type) which would be used as such:

a = Vector( 20, 20, 30 );
b = Vector( a );
b.x = 10;
print(a.x);
print(b.x);

In this case you would notice that the data is copied and the two variables retain their own data.

And there we have it; an almost complete simple vector type for you to use in your scripts. It is almost complete because we have not yet touched on the subject of Garbage Collection, which plays an important role in any bound type you create.

Garbage Collection


GameMonkey Script, like many other scripting languages, has an in-built garbage collector (GC). A garbage collector will periodically examine the data within the machine and determine whether an object is being referenced or not – if an object is deemed to have no referencing objects, the machine can assume it is no longer in use and will delete it to free memory. The GameMonkey machine uses an incremental mark & sweep garbage collector which has two main phases of operation. The first stage will examine all of the objects in the machine and 'mark' the unreferenced objects for deletion. Once this phase is complete, the 'sweep' stage is executed which calls the destructors (or more correctly finalizers) on all the marked objects and removes them from the machine. The GC is said to be incremental as it performs a little work on each execution cycle to reduce the amount of time the script execution cycle of the machine is 'paused' for a full collection sweep.

Garbage collection is important when binding your own types as you can often create a memory leak if an object is garbage collected without cleaning up any native objects it creates. Even more dangerously, if your object references other objects internally the GM machine may delete them and leave you with a bad pointer which could cause your game to crash. Both of these scenarios are undesirable so you need to make the garbage collector aware of your new object and tell it how to handle it.

In order to handle the garbage collection on your bound type you must provide two things to the GameMonkey machine; a 'trace' callback and a 'destruct' callback. The trace callback is used during the mark phase of the GC and is used to tell the garbage collector which objects your native type is referencing. If you examine the code for the gmTableObject you will notice that its trace method will trace each of the objects the table holds; without doing this, the GC may remove some of the items from the machine erroneously. The destruct callback is used when the sweep stage is operational; it is used to free up any memory used by your bound type.

I will complete my simple Vector type example by providing it with full garbage collection callbacks:

// Effectively the destructor for the object
void GM_CDECL gc_destruct(gmMachine * a_machine, gmUserObject* a_object)
{
	std::cout << "Vector: destructing\n";
	GM_ASSERT(a_object->m_userType == Type);
  Vector* object = reinterpret_cast<Vector*>(a_object->m_user);
  delete object;
}
// Trace the object in the mark phase
bool gc_trace( gmMachine * a_machine, gmUserObject* a_object, gmGarbageCollector* a_gc, const int a_workLeftToGo, int& a_workDone )
{
  // Make sure this is the correct type
  GM_ASSERT(a_object->m_userType == m_gmType);

  // If your object immediately references other gmObjects, you should call GetNextObject for each member here

  // mark 'me' as done
  a_workDone++;
  return true;
}
Example: vector_4.cpp

The gc_destruct destructor is simple; it merely calls delete on the memory we allocated in the constructor. The trace callback, gc_trace, is also simple for this particular type, but can get complicated as your type begins to reference more objects. Each immediate member object your type references should be pointed to using the GC method GetNextObject, which will allow GM to build the tree of referenced objects. You should also update the a_workDone parameter reference to indicate how many objects you are currently holding a reference to; this allows the incremental GC to effectively govern its own workload. Finally, we must register these callbacks in the BindLib function:

a_machine->RegisterUserCallbacks(Type, gc_trace, gc_destruct, NULL);

In this call we tell the gmMachine to register both a trace and destruct callback for this type. If you don't need them you can always pass NULL, but in the majority of times you will need to specify these callbacks.

Further Exploration


This article has provided you with enough information to begin exploring GameMonkey Script on your own. I have tried to cover as much of the important information as possible without getting too complicated or in depth and so there will invariably be areas that some people wish to expand upon. The topic of binding, for example, can fill many more pages and is a subject that many people will wish to explore. If you are such a person, I will set you the task of building on the simple vector example I provided here. Some example experiments may be:
  • Adding copy constructor behaviour to types (Simple)
  • Adding Indexed access operators (Simple)
  • Handling additional operators on the vector class (Simple)
  • Adding extra methods to the class, such as Normalise (Moderate)
  • Create a function which takes Vector as a parameter (Moderate)
  • Create a simple library and type to bind simple C File I/O to GameMonkey (Moderate)
  • Create more complex types which are bound to your game engine (Advanced)
You are urged to experiment further with the examples I have provided for you and begin looking further into what GameMonkey has to offer. Your game may wish to take advantage of scripted threads to remove many asynchronous operations, a subject which is useful but has not been covered here in this introductory article.

References


Code::Blocks IDE.

Douglas, G. & Riek, M. (2003). GameMonkey Script Reference. Included with all official GameMonkey Script distributions.

Lua Website.

Varanese, A., (2003). Game Scripting Mastery. Premier Press, Ohio, USA.

Wikipedia (2005). Wikipedia: Virtual Machines.

Wade Not In Unknown Waters: Part Two

$
0
0

This time I want to speak on the 'printf' function. Everybody has heard of software vulnerabilities and that functions like 'printf' are outlaw. But it's one thing to know that you'd better not use these functions, and quite the other to understand why. In this article, I will describe two classic software vulnerabilities related to 'printf'. You won't become a hacker after that but perhaps you will have a fresh look at your code. You might create similar vulnerable functions in your project without knowing that.


STOP. Reader, please stop, don't pass by. You have seen the word "printf", I know. And you're sure that you will now be told a banal story that the function cannot check types of passed arguments. No! It's vulnerabilities themselves that the article deals with, not the things you have thought. Please come and read it.


The previous post can be found here: Part one.


Introduction


Have a look at this line:


printf(name);

It seems simple and safe. But actually it hides at least two methods to attack the program.


Let's start our article with a demo sample containing this line. The code might look a bit odd. It is, really. We found it quite difficult to write a program so that it could be attacked then. The reason is optimization performed by the compiler. It appears that if you write a too simple program, the compiler creates a code where nothing can be hacked. It uses registers, not the stack, to store data, creates intrinsic functions and so on. We could write a code with extra actions and loops so that the compiler lacked free registers and started putting data into the stack. Unfortunately, the code would be too large and complicated in this case. We could write a whole detective story about all this, but we won't.


The cited sample is a compromise between complexity and the necessity to create a code that would not be too simple for the compiler to get it "collapsed into nothing". I have to confess that I still have helped myself a bit: I have disabled some optimization options in Visual Studio 2010. First, I have turned off the /GL (Whole Program Optimization) switch. Second, I have used the __declspec(noinline) attribute.


Sorry for such a long introduction: I just wanted to explain why my code is such a crock and prevent beforehand any debates on how we could write it in a better way. I know that we could. But we didn't manage to make the code short and show you the vulnerability inside it at the same time.


Demo sample


The complete code and project for Visual Studio 2010 can be found here: Attached File  printf_demo.zip   3.47KB   5 downloads
.

const size_t MAX_NAME_LEN = 60;
enum ErrorStatus {
  E_ToShortName, E_ToShortPass, E_BigName, E_OK
};

void PrintNormalizedName(const char *raw_name)
{
  char name[MAX_NAME_LEN + 1];
  strcpy(name, raw_name);

  for (size_t i = 0; name[i] != '\0'; ++i)
    name[i] = tolower(name[i]);
  name[0] = toupper(name[0]);

  printf(name);
}

ErrorStatus IsCorrectPassword(
  const char *universalPassword,
  BOOL &retIsOkPass)
{
  string name, password;
  printf("Name: "); cin >> name;
  printf("Password: "); cin >> password;
  if (name.length() < 1) return E_ToShortName;
  if (name.length() > MAX_NAME_LEN) return E_BigName;
  if (password.length() < 1) return E_ToShortPass;

  retIsOkPass = 
    universalPassword != NULL &&
    strcmp(password.c_str(), universalPassword) == 0;
  if (!retIsOkPass)
    retIsOkPass = name[0] == password[0];

  printf("Hello, ");
  PrintNormalizedName(name.c_str());

  return E_OK;
}

int _tmain(int, char *[])
{
  _set_printf_count_output(1);
  char universal[] = "_Universal_Pass_!";
  BOOL isOkPassword = FALSE;
  ErrorStatus status =
    IsCorrectPassword(universal, isOkPassword);
  if (status == E_OK && isOkPassword)
    printf("\nPassword: OK\n");
  else
    printf("\nPassword: ERROR\n");
  return 0;
}

The _tmain() function calls the IsCorrectPassword() function. If the password is correct or if it coincides with the magic word "_Universal_Pass_!", then the program prints the line "Password: OK". The purpose of our attacks will be to have the program print this very line.


The IsCorrectPassword() function asks the user to specify name and password. The password is considered correct if it coincides with the magic word passed into the function. It is also considered correct if the password's first letter coincides with the name's first letter.


Regardless of whether the correct password is entered or not, the application shows a welcome window. The PrintNormalizedName() function is called for this purpose.


The PrintNormalizedName() function is of the most interest. It is this function where the "printf(name);" we're discussing is stored. Think of the way we can exploit this line to cheat the program. If you know how to do it, you don't have to read further.


What does the PrintNormalizedName() function do? It prints the name making the first letter capital and the rest letters small. For instance, if you enter the name "andREy2008", it will be printed as "Andrey2008".


The first attack


Suppose we don't know the correct password. But we know that there is some magic password somewhere. Let's try to find it using printf(). If this password's address is stored somewhere in the stack, we have certain chances to succeed. Any ideas how to get this password printed on the screen?


Here is a tip. The printf() function refers to the family of variable-argument functions. These functions work in the following way. Some amount of data is written into the stack. The printf() function doesn't know how many data is pushed and what type they have. It follows only the format string. If it reads "%d%s", then the function should extract one value of the int type and one pointer from the stack. Since the printf() function doesn't know how many arguments it has been passed, it can look deeper into the stack and print data that have nothing to do with it. It usually causes access violation or printing trash. And we may exploit this trash.


Let's see how the stack might look at the moment when calling the printf() function:


image1.png


Figure 1. Schematic arrangement of data in the stack.


The "printf(name);" function's call has only one argument which is the format string. It means that if we type in "%d" instead of the name, the program will print the data that lie in the stack before the PrintNormalizedName() function's return address. Let's try:


Name: %d

Password: 1

Hello, 37

Password: ERROR


This action has little sense in it for now. First of all, we have at least to print the return addresses and all the contents of the char name[MAX_NAME_LEN + 1] buffer which is located in the stack too. And only then we may get to something really interesting.


If an attacker cannot disassemble or debug the program, he/she cannot know for sure if there is something interesting in the stack to be found. He/she still can go the following way.


First we can enter: "%s". Then "%x%s". Then "%x%x%s" and so on. Doing so, the hacker will search through the data in the stack in turn and try to print them as a line. It helps the intruder that all the data in the stack are aligned at least on a 4-byte boundary.


To be honest, we won't succeed if we go this way. We will exceed the limit of 60 characters and have nothing useful printed. "%f" will help us - it is intended to print values of the double type. So, we can use it to move along the stack with an 8-byte step.


Here it is, our dear line:


%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%x(%s)


This is the result:


image2.png

</p>

Figure 2. Printing the password. Click on the picture to enlarge it.


Let's try this line as the magic password:


Name: Aaa

Password: _Universal_Pass_!

Hello, Aaa

Password: OK


Hurrah! We have managed to find and print the private data which the program didn't intend to give us access to. Note also that you don't have to get access to the application's binary code itself. Diligence and persistence are enough.


Conclusions on the first attack


You should give a wider consideration to this method of getting private data. When developing software containing variable-argument functions, think it over if there are cases when they may be the source of data leakage. It can be a log-file, a batch passed on the network and the like.


In the case we have considered, the attack is possible because the printf() function receives a string that may contain control commands. To avoid this, you just need to write it in this way:


printf("%s", name);

The second attack


Do you know that the printf() function can modify memory? You must have read about it but forgotten. We mean the "%n" specifier. It allows you to write a number of characters, already printed by the printf() function, by a certain address.


To be honest, an attack based on the "%n" specifier is just of a historical character. Starting with Visual Studio 2005, the capability of using "%n" is off by default. To perform this attack, I had to explicitly allow this specifier. Here is this magic trick:


_set_printf_count_output(1);

To make it clearer, let me give you an example of using "%n":

int i;

printf("12345%n6789\n", &i);

printf( "i = %d\n", i );

The program's output:

123456789

i = 5


We have already found out how to get to the needed pointer in the stack. And now we have a tool that allows us to modify memory by this pointer.


Of course, it's not very much convenient to use it. To start with, we can write only 4 bytes at a time (int type's size). If we need a larger number, the printf() function will have to print very many characters first. To avoid this we may use the "%00u" specifier: it affects the value of the current number of output bytes. Let's not go deep into the detail.


Our case is simpler: we just have to write any value not equal to 0 into the isOkPassword variable. This variable's address is passed into the IsCorrectPassword() function, which means that it is stored somewhere in the stack. Do not be confused by the fact that the variable is passed as a reference: a reference is an ordinary pointer at the low level.


Here is the line that will allow us to modify the IsCorrectPassword variable:


%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f %n


The "%n" specifier does not take into account the number of characters printed by specifiers like "%f". That's why we make one space before "%n" to write value 1 into isOkPassword.


Let's try:


image4.png

</p>

Figure 3. Writing into memory. Click on the picture to enlarge it.


Are you impressed? But that's not all. We may perform writing by virtually any address. If the printed line is stored in the stack, we may get the needed characters and use them as an address.


For example, we may write a string containing characters with codes 'xF8', 'x32', 'x01', 'x7F' in a row. It turns out that the string contains a hard-coded number equivalent to value 0x7F0132F8. We add the "%n" specifier at the end. Using "%x" or other specifiers we can get to the coded number 0x7F0132F8 and write the number of printed characters by this address. This method has some limitations, but it is still very interesting.


Conclusions on the second attack


We may say that an attack of the second type is hardly possible nowadays. As you see, support of the "%n" specifier is off in contemporary libraries by default. But you may create a self-made mechanism subject to this kind of vulnerabilities. Be careful when external data input into your program manage what and where is written into memory.


Particularly in our case, we may avoid the problem by writing the code in this way:


printf("%s", name);

General conclusions


We have considered only two simple examples of vulnerabilities here. Surely, there are much more of them. We don't make an attempt to describe or at least enumerate them in this article; we wanted to show you that even such a simple construct like "printf(name)" can be dangerous.


There is an important conclusion to draw from all this: if you are not a security expert, you'd better follow all the recommendations to be found. Their point might be too subtle for you to understand the whole range of dangers on yourself. You must have read that the printf() function is dangerous. But I'm sure that many of you reading this article have learned only now how deep the rabbit hole is.


If you create an application that is potentially an attack object, be very careful. What is quite safe code from your viewpoint might contain a vulnerability. If you don't see a catch in your code, it doesn't mean there isn't any.


Follow all the compiler's recommendations on using updated versions of string functions. We mean using sprintf_s instead of sprintf and so on.


It's even better if you refuse low-level string handling. These functions are a heritage of the C language. Now we have std::string and we have safe methods of string formatting such as boost::format or std::stringstream.


P.S. Some of you, having read the conclusions, may say: "well, it's as clear as day". But be honest to yourself. Did you know and remember that printf() can perform writing into memory before you read this article? Well, and this is a great vulnerability. At least, it used to. Now there are others, as much insidious.

Wade Not In Unknown Waters: Part Three

$
0
0

I'm going on to tell you about how programmers walk on thin ice without even noticing it. Let's speak on shift operators <<, >>. The working principles of the shift operators are evident and many programmers even don't know that using them according to the C/C++ standard might cause undefined or unspecified behavior.


You can read the previous articles here: [1], [2].


Excursus to the history


A bit of history first. The necessity of bit shifting operations is evident to any programmer. Anyone sooner or later faces the need to handle individual bits and bit masks. However, shift operators are much more popular among programmers than they should be. The reason is that you can multiply and divide numbers by powers of two. For example, the "X << 3" operation will multiply X by 8. In the past, the advantage of this number multiplication/division method lied in the speed of its work.


I've just got a book from the dusty shelf with a description of assembler commands for processors from 8086 to 80486. I've found a table with the number of clock cycles necessary to perform various instructions.


Multiplying a 16-bit register by a memory cell using the MUL instruction takes about 124-139 clock cycles on the 8086 processor!


A shift of a 16-bit register by N digits using the SHL instruction takes 8+4*N clock cycles on the 8086 processor. That is, it will take 72 clock cycles at worst.


You could get a noticeable speed gain by using various tricks handling bitwise operations when calculating arithmetic expressions. This is what became the reason for massively using shifts - first in assembler, and then in C and C++. The first C/C++ compilers were simple. You could get a performance gain by explicitly prompting the compiler to use a shift instead of multiplication or division instructions in certain places.


As processors were developing, shift operators were of use for a long time. On the 80486 processor, multiplication now took about 26 clock cycles. Seems like it became much better, doesn't it? But a shift operator took just 3 clock cycles at that time and again appeared to be better than multiplication.


Fortunately, most of these forced optimizations have been forgotten by now. First, compilers have become smarter and now use an optimal instruction set to calculate arithmetic expressions. Second, processors have undergone great changes too. Pipelines, branch predictions, register renaming and many other things have appeared. That's why an ordinary programmer of nowadays cannot tell for sure how much time will take the execution of a certain instruction. But it's clear that if some fragments of code are not ideal, you may not even notice it. The processor will split instructions into micro-instructions and start executing them in parallel. To be honest, I don't make out now how it all goes on there. I've come to the understanding that it's no longer reasonable to know all the subtleties starting with the Intel Pentium processor. So, I've concluded that one should not think that one knows better how to write optimized code and use shifts and bitwise operations wherever possible. It's not necessarily true that you can make the code faster than the compiler's optimizer can, but you can tell for sure that the program will become complicated and difficult to understand in that case.


Note:  Everything said above doesn't mean that you cannot benefit from bitwise operations anymore. There are many interesting and useful tricks [3]; just don't get too fond of them.


Undefined behavior


It all began when I decided to create more diagnostics related to undefined behavior [4] and unspecified behavior [5] in PVS-Studio. It took me rather little time and effort to create a rule to detect incorrect use of shift operators. And after that I had to stop and think it over.


It turned out that programmers are very fond of shifts. They use them in every way they can, which often leads to undefined behavior from the viewpoint of the coding standard. But theory is one thing and practice is another. Is there sense in persecuting code that has been faithfully serving you for many decades and gone through many compilers? That's a difficult question. Despite the code being incorrect, compilers adhere to some secret agreement and process it uniformly.


After pondering over it for a long time, I finally decided to leave this diagnostic rule in PVS-Studio without making any exceptions to it. If there are too many complaints from users, maybe I will change my mind. However, perhaps users will be satisfied by the capability of disabling this diagnostic or use other methods of warning suppression.


By the way, it is these painful thoughts that made me write the article. I hope that you will find the information I'm going to show you interesting and useful.


So, let's see what the C++11 standard has to say about shift operators:


The shift operators << and >> group left-to-right.


shift-expression << additive-expression


shift-expression >> additive-expression


The operands shall be of integral or unscoped enumeration type and integral promotions are performed.


1. The type of the result is that of the promoted left operand. The behavior is undefined if the right operand is negative, or greater than or equal to the length in bits of the promoted left operand.


2. The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned type, the value of the result is E1 * 2^E2, reduced modulo one more than the maximum value representable in the result type. Otherwise, if E1 has a signed type and non-negative value, and E1*2^E2 is representable in the result type, then that is the resulting value; otherwise, the behavior is undefined.


3. The value of E1 >> E2 is E1 right-shifted E2 bit positions. If E1 has an unsigned type or if E1 has a signed type and a non-negative value, the value of the result is the integral part of the quotient of E1/2^E2. If E1 has a signed type and a negative value, the resulting value is implementation-defined.


It is sad to read such texts. But don't worry - now we will study various issues by examples.


The simplest case leading to undefined behavior is the situation when the right operand has a negative value. For example:


int A = 10;
int B = A << -5;

Thank God, nobody does it that way. Well, at least we haven't seen such errors after analyzing more than 70 open-source projects.


The next case is much more interesting. This is a shift by N bits where N is larger than the number of bits in the left operand. Here is a simple example:


int A = 10;

int B = A << 100;

Let's see what such an error looks like in practice. The next code fragment was found in the Lib7z library:


SZ_RESULT
SafeReadDirectUInt64(ISzInStream *inStream, UInt64 *value)
{
  int i;
  *value = 0;
  for (i = 0; i < 8; i++)
  {
    Byte b;
    RINOK(SafeReadDirectByte(inStream, &b));
    *value |= ((UInt32)b << (8 * i));
  }
  return SZ_OK;
}

PVS-Studio's diagnostic message: V610 Undefined behavior. Check the shift operator '<<'. The right operand ('(8 * i)' = [0..56]) is greater than or equal to the length in bits of the promoted left operand. lib7z 7zin.c 233


The function tries to read the 64-bit value byte-by-byte. Unfortunately, it will fail if the number was larger than 0x00000000FFFFFFFF. Note the (UInt32)b << (8 * i) shift. The size of the left operand is 32 bits. The shift takes from 0 to 56 bits. In practice, it will cause the high-order part of the 64-bit value to remain filled with zeroes. Theoretically, it is undefined behavior here and the result cannot be predicted.


This is the correct code:


*value |= ((UInt64)b << (8 * i));

Readers may ask if the code below is correct:


char A = 1;
int B = A << 20;

Yes, it is. To the left of the << operator is the A variable consisting of only 8 bits. But the left part will be extended to the int type before the shift. Therefore, a value of the 'int' type can be shifted by 20 bits.


And now for the most interesting thing - shifting of negative values. Here is a simple example:


int A = (-1) << 5; // undefined behavior
int B = (-1) >> 5; // unspecified behavior

We can see undefined or unspecified behavior in this code. There's no difference between them from a practical point of view. Only one conclusion is to be drawn from this case - you should not write such code.


We could finish at this point and cite a couple of examples. But unfortunately, there are two peculiarities that spoil this idealistic picture.


The peculiarities that spoil the idealistic picture


Peculiarity N1. In the old C++ language standard of 1998, cases with undefined behavior are avoided. It says only how the << operator behaves when unsigned values are shifted, but it doesn't say anything about signed values. So, it is that very case when reading the standard doesn't make the point any clearer to you: this case is simply not considered, and that's it.


So, from the viewpoint of C++ of 1998, the (-1) << 5 construct doesn't cause undefined behavior. However, it doesn't describe how it should work either.


Peculiarity N2. Programmers feel safe to shift negative values in many programs. It's hard to argue with them, as the code does work.


Let's try to find out if we should refuse implementing the new diagnostic because of the above mentioned peculiarities. We believe that we shouldn't.


The old C++ standard doesn't say anything about undefined behavior. But the new one does. It turns out that the old standard simply was not precise enough. By the way, the new C language standard (I checked the rough copy of 25th June, 2010) also says that shifts of negative values cause undefined behavior. The conclusion is you should eliminate incorrect code.


Now to the subject of a widespread use of dangerous shifts. They are really numerous. For example, in the JPEG library you need to fill an array with the following values:


11...11111111111111b
11...11111111111101b
11...11111111111001b
11...11111111110001b
....

This is how it is written:


/* entry n is (-1 [lessthan][lessthan] n) + 1 */
static const int extend_offset[16] = { 0,
  ((-1)<<1) + 1, ((-1)<<2) + 1, ((-1)<<3) + 1,
  ((-1)<<4) + 1, ((-1)<<5) + 1, ((-1)<<6) + 1,
  ((-1)<<7) + 1, ((-1)<<8) + 1, ((-1)<<9) + 1,
  ((-1)<<10) + 1, ((-1)<<11) + 1, ((-1)<<12) + 1,
  ((-1)<<13) + 1, ((-1)<<14) + 1, ((-1)<<15) + 1
};

We cannot tell that the JPEG library is a bad one. This code is time-proven and has gone through various compilers.


From the standard's viewpoint, it should be rewritten in the following way:


static const int extend_offset[16] =
{ 0,
  ((~0u)<<1) | 1, ((~0u)<<2) | 1, ((~0u)<<3) | 1,
  ((~0u)<<4) | 1, ((~0u)<<5) | 1, ((~0u)<<6) | 1,
  ((~0u)<<7) | 1, ((~0u)<<8) | 1, ((~0u)<<9) | 1,
  ((~0u)<<10) | 1, ((~0u)<<11) | 1, ((~0u)<<12) | 1,
  ((~0u)<<13) | 1, ((~0u)<<14) | 1, ((~0u)<<15) | 1
};

But it's up to you to decide whether or not you need such corrections. I can only advise that you should do this: you don't know when and to what consequences it may lead.


We could give you other examples of negative value shifts, but they are all alike and won't be interesting to read about.


Conclusions


  1. Using bitwise operations and shifts was earlier considered as a token of programmer's skill and allowed you to write fast code. Now it has almost no relevance. It's much more important that the code is understandable. I advise that you play with bits only when it is really necessary.
  2. Expressions of the "(-1) << N" kind are now declared as incorrect and leading to an undefined behavior.
  3. Expressions of the "(-1) << N" kind have been used for a long time and quite often. That's why we cannot give strong arguments against using such constructs. The only arguments are the new C and C++ language standards.
  4. It is up to you to decide if you should fix negative value shifts. But I do recommend doing this. Just in case, at least.
  5. Diagnostic messages covering dangerous shifts will be available in PVS-Studio starting with version 4.60 which is to be released soon.

References


  1. Wade Not In Unknown Waters: Part One
  2. Wade Not In Unknown Waters: Part Two
  3. Sean Eron Anderson. Bit Twiddling Hacks
  4. Wikipedia. Undefined behavior
  5. Wikipedia. Unspecified behavior

BVH File Loading and Displaying

$
0
0
In this article we will look at the most common motion capture format: BVH. BVH is an acronym that stands for BioVision Hierarchical data and is used for storing motion capture data. It is simple and easy to understand. We will write a simple class that can load, display and play data from the file.

BVH format


Much about format can be found at these two links:
http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html
http://www.dcs.shef.ac.uk/intranet/research/resmes/CS0111.pdf

Basically, it has two parts HIERARCHY and MOTION. Like the names suggest those two parts contain just that: hierarchies of skeletons and motion data. Inside the hierarchy part we have a description of skeletons. Even if the format permits having multiple skeleton definitions, rarely it will contain more then one. Skeletons are defined by defining bones which themselves are defined with joints; meaning we define a skeleton by defining joints. But if an elbow joint is the child of a shoulder joint how do we know the length of the upper arm? By defining an offset.

Lets look at an example:

HIERARCHY
ROOT Hips
{
	OFFSET 0.00 0.00 0.00
	CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation
	JOINT Chest
	{
		OFFSET 5.00 0.00 0.00
		CHANNELS 3 Zrotation Xrotation Yrotation
		End Site
		{
			OFFSET 0.00 5.00 0.00
		}
	}
	JOINT Leg
	{
		OFFSET -5.0 0.0 0.0
		CHANNELS 3 Zrotation Xrotation Yrotation
		End Site
		{
			OFFSET 0.0 5.0 0.0
		}
	}
}
MOTION
Frames:		2
Frame Time: 0.033333
 0.00 0.00 0.00 0.00 0.00 0.00  0.00 0.00 0.00  0.00 0.00 0.00
 0.00 0.00 0.00 0.00 0.00 0.00  0.00 45.00 0.00  0.00 0.00 0.00

First joint of the hierarchy is a root joint so it is defined by using the keyword ROOT. Every other joint that is a descendant is defined using the JOINT keyword followed by the joint name. Special joints are End Site joints which are joints without any children or name.

Contents of a joint are OFFSET and CHANNELS. We use an offset to know the length (or offset from) of bones between joints of a joint's parent and itself. Most commonly, a ROOT joint will have an offset of (0, 0, 0) (note these are, of course: x, y, z components). CHANNELS line defines the number of channels following which channels that MOTION parts contain animation data for. Again, the most common use is a ROOT joint that has 6 channels (xyz position and zxy rotation) while other joints will have 3. End Site joints don't have animation data so they do not need to have CHANNELS data. They only have an OFFSET so we know it's length.

The MOTION part contains two lines (frames defining number of frames ... and frame time which is frame rate; bvh motion FPS = 1. / frame_time) followed by lines for each frame that has float data of each joint/channel(specified) combination beginning from parent to children nodes, just in same order they were specified in hierarchy part, from top to bottom. The example is dull and has all zeroes but you get the point. When we make the loader you can change values and play with it.

Interpreting MOTION and actually changing joint positions is described later on. First, we'll do the loading.

Code


We will define a few structures we'll need for storing data:

#define Xposition 0x01
#define Yposition 0x02
#define Zposition 0x04
#define Zrotation 0x10
#define Xrotation 0x20
#define Yrotation 0x40
    
typedef struct
{
    float x, y, z;
} OFFSET;

typedef struct JOINT JOINT;

struct JOINT
{
    const char* name = NULL;        // joint name
    JOINT* parent = NULL;           // joint parent
    OFFSET offset;                  // offset data
    unsigned int num_channels = 0;  // num of channels joint has
    short* channels_order = NULL;   // ordered list of channels
    std::vector<JOINT*> children;   // joint's children
    glm::mat4 matrix;               // local transofrmation matrix (premultiplied with parents'
    unsigned int channel_start = 0; // index of joint's channel data in motion array
};

typedef struct
{
    JOINT* rootJoint;
    int num_channels;
} HIERARCHY;

typedef struct
{
    unsigned int num_frames;              // number of frames
    unsigned int num_motion_channels = 0; // number of motion channels 
    float* data = NULL;                   // motion float data array
    unsigned* joint_channel_offsets;      // number of channels from beggining of hierarchy for i-th joint
} MOTION;

Most of these parameters are self-explanatory. For each joint we need a list of children, local transformation matrix and channel order at least.

Bvh class


class Bvh
{
    JOINT* loadJoint(std::istream& stream, JOINT* parent = NULL);
    void loadHierarchy(std::istream& stream);
    void loadMotion(std::istream& stream);
public:
    Bvh();
    ~Bvh();

    // loading 
    void load(const std::string& filename);

    /** Loads motion data from a frame into local matrices */
    void moveTo(unsigned frame);

    const JOINT* getRootJoint() const { return rootJoint; }
    unsigned getNumFrames() const { return motionData.num_frames; }
private:
    JOINT* rootJoint;
    MOTION motionData;
};

This is a simple class and below are functions for loading:

void Bvh::load(const std::string& filename)
{
    std::fstream file;
    file.open(filename.c_str(), std::ios_base::in);

    if( file.is_open() )
    {
        std::string line;

        while( file.good() )
        {
            file >> line;
            if( trim(line) == "HIERARCHY" )
                loadHierarchy(file);
            break;
        }

        file.close();
    }
}

void Bvh::loadHierarchy(std::istream& stream)
{
    std::string tmp;

    while( stream.good() )
    {
        stream >> tmp;

        if( trim(tmp) == "ROOT" )
            rootJoint = loadJoint(stream);
        else if( trim(tmp) == "MOTION" )
            loadMotion(stream);
    }
}

JOINT* Bvh::loadJoint(std::istream& stream, JOINT* parent)
{
    JOINT* joint = new JOINT;
    joint->parent = parent;
    
    // load joint name
    std::string* name = new std::string;
    stream >> *name;
    joint->name = name->c_str();

    std::string tmp;
    
    // setting local matrix to identity
    joint->matrix = glm::mat4(1.0);

    static int _channel_start = 0;
    unsigned channel_order_index = 0;
    
    while( stream.good() )
    {
        stream >> tmp;
        tmp = trim(tmp);

        // loading channels
        char c = tmp.at(0);
        if( c == 'X' || c == 'Y' || c == 'Z' )
        {
            if( tmp == "Xposition" )
            {
                joint->channels_order[channel_order_index++] = Xposition;
            }
            if( tmp == "Yposition" )
            {
                joint->channels_order[channel_order_index++] = Yposition;
            }
            if( tmp == "Zposition" )
            {
                joint->channels_order[channel_order_index++] = Zposition;
            }

            if( tmp == "Xrotation" )
            {
                joint->channels_order[channel_order_index++] = Xrotation;
            }
            if( tmp == "Yrotation" )
            {
                joint->channels_order[channel_order_index++] = Yrotation;
            }
            if( tmp == "Zrotation" )
            {
                joint->channels_order[channel_order_index++] = Zrotation;
            }
        }

        if( tmp == "OFFSET" )
        {
            // reading an offset values
            stream  >> joint->offset.x
                    >> joint->offset.y
                    >> joint->offset.z;
        }
        else if( tmp == "CHANNELS" )
        {
            // loading num of channels
            stream >> joint->num_channels;

            // adding to motiondata
            motionData.num_motion_channels += joint->num_channels;

            // increasing static counter of channel index starting motion section
            joint->channel_start = _channel_start;
            _channel_start += joint->num_channels;

            // creating array for channel order specification
            joint->channels_order = new short[joint->num_channels];

        }
        else if( tmp == "JOINT" )
        {
            // loading child joint and setting this as a parent
            JOINT* tmp_joint = loadJoint(stream, joint);

            tmp_joint->parent = joint;
            joint->children.push_back(tmp_joint);
        }
        else if( tmp == "End" )
        {
            // loading End Site joint
            stream >> tmp >> tmp; // Site {

            JOINT* tmp_joint = new JOINT;

            tmp_joint->parent = joint;
            tmp_joint->num_channels = 0;
            tmp_joint->name = "EndSite";
            joint->children.push_back(tmp_joint);

            stream >> tmp;
            if( tmp == "OFFSET" )
                stream >> tmp_joint->offset.x
                       >> tmp_joint->offset.y
                       >> tmp_joint->offset.z;

            stream >> tmp;
        }
        else if( tmp == "}" )
            return joint;

    }
}

void Bvh::loadMotion(std::istream& stream)
{
    std::string tmp;

    while( stream.good() )
    {
        stream >> tmp;

        if( trim(tmp) == "Frames:" )
        {
            // loading frame number
            stream >> motionData.num_frames;
        }
        else if( trim(tmp) == "Frame" )
        {
            // loading frame time
            float frame_time;
            stream >> tmp >> frame_time;

            int num_frames   = motionData.num_frames;
            int num_channels = motionData.num_motion_channels;

            // creating motion data array
            motionData.data = new float[num_frames * num_channels];

            // foreach frame read and store floats
            for( int frame = 0; frame < num_frames; frame++ )
            {
                for( int channel = 0; channel < num_channels; channel++)
                {
                    // reading float
                    float x;
                    std::stringstream ss;
                    stream >> tmp;
                    ss << tmp;
                    ss >> x;

                    // calculating index for storage
                    int index = frame * num_channels + channel;
                    motionData.data[index] = x;
                }
            }
        }
    }
}

The loading code should be easy to read. load() calls loadHierarchy() which calls loadRoot() for root joint and loadMotion() when the time comes. loadJoint() loads joint and all those ifs just try to take care of channel ordering.

loadMotion() just loads frame number and frame time, and then iterates through all channels, reads float, calculates where to store a float and stores it.

This version does not support multiple hierarchies, which can be easily added.

JOINT transformations


If we imagine a simplified human skeleton, hand would be child of an arm and itself child of a shoulder etc... We can go all the way up to the root joint which can be, for example, hips (which it actually is in most files). In order to find out the absolute position of all of a root joint's descendents we'll have to apply the parent's transformation onto them. You probably know that this can be achieved using matrices. That's why we have a joint's "local transformation matrix".

Basically, the transformation matrix is composed of rotation and translation parameters (BVH does not support bone scaling so we dont have one). This can be represented using a standard 4x4 matrix where translation parameters are present in the 4-th column. Note that OpenGL uses column-major ordering which looks just like the transponse of a row-major ordered matrix. Since OpenGL uses it GLSL uses it and also GLM which is based on GLSL which we use here. This is said because we need to know it and we'll need it later.

The function that does the positioning is moveTo() and uses a static helper function defined inside the .cpp file (it cannot be used outside, and does not need to):

/** 
	Calculates JOINT's local transformation matrix for 
	specified frame starting index 
*/
static void moveJoint(JOINT* joint, MOTION* motionData, int frame_starts_index)
{
    // we'll need index of motion data's array with start of this specific joint
    int start_index = frame_starts_index + joint->channel_start;

    // translate indetity matrix to this joint's offset parameters
    joint->matrix = glm::translate(glm::mat4(1.0),
                                   glm::vec3(joint->offset.x,
                                             joint->offset.y,
                                             joint->offset.z));

    // here we transform joint's local matrix with each specified channel's values
    // which are read from motion data
    for(int i = 0; i < joint->num_channels; i++)
    {
        // channel alias
        const short& channel = joint->channels_order[i];

        // extract value from motion data
        float value = motionData->data[start_index + i];
        
        if( channel & Xposition )
        {
            joint->matrix = glm::translate(joint->matrix, glm::vec3(value, 0, 0));
        }
        if( channel & Yposition )
        {
            joint->matrix = glm::translate(joint->matrix, glm::vec3(0, value, 0));
        }
        if( channel & Zposition )
        {
            joint->matrix = glm::translate(joint->matrix, glm::vec3(0, 0, value));
        }

        if( channel & Xrotation )
        {
            joint->matrix = glm::rotate(joint->matrix, value, glm::vec3(1, 0, 0));
        }
        if( channel & Yrotation )
        {
            joint->matrix = glm::rotate(joint->matrix, value, glm::vec3(0, 1, 0));
        }
        if( channel & Zrotation )
        {
            joint->matrix = glm::rotate(joint->matrix, value, glm::vec3(0, 0, 1));
        }
    }

    // then we apply parent's local transfomation matrix to this joint's LTM (local tr. mtx. :)
    if( joint->parent != NULL )
        joint->matrix = joint->parent->matrix * joint->matrix;

    // when we have calculated parent's matrix do the same to all children
    for(auto& child : joint->children)
        moveJoint(child, motionData, frame_starts_index);
}

void Bvh::moveTo(unsigned frame)
{
    // we calculate motion data's array start index for a frame
    unsigned start_index = frame * motionData.num_motion_channels;

    // recursively transform skeleton
    moveJoint(rootJoint, &motionData, start_index);
}

What we do (for each joint, starting from root) is take the value from the motion data and apply it in the order it was loaded / defined in the file with both the glm::translate() and glm::rotate() functions. We use static helper function moveJoint() to help us with transforming joints using recursion.

What we yet need to do is display it.

Using the class and displaying a skeleton


Constructing vertices array from skeleton's joint data is not BVH class' job. We'll do that where we need it. Using recursion and std::vector() we can easily construct the vertices array:

std::vector<glm::vec4> vertices;
std::vector<GLshort>   indices;

GLuint bvhVAO;
GLuint bvhVBO;
Bvh* bvh = NULL;

/** put translated joint vertices into array */
void bvh_to_vertices(JOINT*                  joint,
                     std::vector<glm::vec4>& vertices,
                     std::vector<GLshort>&   indices,
                     GLshort                 parentIndex = 0)
{
    // vertex from current joint is in 4-th ROW (column-major ordering)
    glm::vec4 translatedVertex = joint->matrix[3];

    // pushing current 
    vertices.push_back(translatedVertex);

    // avoid putting root twice
    GLshort myindex = vertices.size() - 1;
    if( parentIndex != myindex )
    {
        indices.push_back(parentIndex);
        indices.push_back(myindex);
    }

    // foreach child same thing
    for(auto& child : joint->children)
        tmpProcess(child, vertices, indices, myindex);
}

void bvh_load_upload(int frame = 1)
{    
    // using Bvh class
    if( bvh == NULL )
    {
        bvh = new Bvh;
        bvh->load("file.bvh");
    }
    
    bvh->moveTo(frame);
    
    JOINT* rootJoint = (JOINT*) bvh->getRootJoint();
    bvh_to_vertices(rootJoint, vertices, indices);
    
    // here goes OpenGL stuff with gen/bind buffer and sending data
    // basically you want to use GL_DYNAMIC_DRAW so you can update same VBO
}

Note there are some C++11 features. If you use GCC you should add few C++11 switches like -std=c++11 and -std=gnu++11 to make it compile.

The bvh_tovertices() function helps us to reconstruct vertices using skeleton info.

Outro


So we've looked at the BVH format and how to load and display it. This is just a basic loader, which can be the stepping stone for some more advanced things like animation blending and mixing.

That's it. Hope you like it.

Writing Endian Independent Code in C++

$
0
0

What does "endian" mean?


Endians are a confusing topic for many people. Hopefully, by reading this article you will understand both what endian means and how to write code to deal with it. So I might as well start with the obvious question—"What does endian mean?"

The actual origin of the term "endian" is a funny story. In Gulliver's Travels by Jonathan Swift, one of the places Gulliver visits has two groups of people who are constantly fighting. One group believes the a hardboiled egg should be eaten from the big, round end (the "big endians") and the other group believes that it should be eaten from the small, pointy end (the "little endians"). Endian no longer has anything to do with hard boiled eggs, but in many ways, the essence of the story (two groups fighting over a completely pointless subject) still remains.

Note:  
This article was originally published to GameDev.net in 2004. It was revised by the original author in 2008 and published in the book Advanced Game Programming: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.


Suppose we start with an unsigned 2 byte (16 bit) long number; we'll use 43707. If we look at the hexadecimal version of 43707, it's 0xAABB. Now, hexadecimal notation is convenient because it neatly splits up the number into its component bytes. One byte is 'AA' and the other byte is 'BB'. But how would this number look in the computer's memory (not hard drive space, just regular memory)? Well, the most obvious way to keep this number in memory would be like this:


| AA | BB |


The first byte is the "high" byte of the number, and the second one is the "low" byte of the number. High and low refers to the value of the byte in the number; here, AA is high because represents the higher digits of the number. This order of keeping things is called MSB (most significant byte) or big endian. The most popular processors that use big endian are the PPC family, used by Macs. This family includes the G3, G4, and G5 that you see in most Macs nowadays.

So what is little endian, then? Well, a little endian version of 0xAABB looks like this in memory:


| BB | AA |


Notice that it's is backwards of the other one. This is called LSB (least significant byte) or little endian. There are a lot of processors that use little endian, but the most well known are the x86 family, which includes the entire Pentium and Athlon lines of chips, as well as most other Intel and AMD chips. The actual reason for using little endian is a question of CPU architecture and outside the scope of this article; suffice to say that little endian made compatibility with earlier 8 bit processors easier when 16-bit processors came out, and 32-bit processors kept the trend.

It gets a little more complicated than that. If you have a 4 byte (32 bit) long number, it's completely backwards, not switched every two bytes. Also, floating-point numbers are handled a little differently as well. This means that you can't arbitrarily change the endian of a file; you have to know what data is in it and what order it's in. The only good news is that if you're reading raw bytes that are only 8 bits at a time, you don't have to worry about endians.

When does the endian affect code?


The short answer is, endians come into play whenever your code assumes that a certain endian is being used. The most frequent time this happens is when you're reading from a file. If you are on a big endian system and write, say, 10 long integers to a file, they will all be written in big endian. If you read them back, the system will assume you're reading big endian numbers. The same is true of little endian systems; everything will work with little endian. This causes problems when you have to use the same file on both little and big endian systems. If you write the file in little endian, big endian systems will get screwed up. If you write it in big endian, little endian systems get screwed up. Sure, you could keep two different versions of the file, one big, one small, but that's going to get confusing very quick, and annoying as well.

The other major time that endians matter is when you use a type cast that depends on a certain endian being in use. I'll show you one example right now (keep in mind that there are many different type casts that can cause problems):

unsigned char EndianTest[2] = { 1, 0 };
short x;

x = *(short *) EndianTest;

So the question is, what's x? Let's look at what this code is doing. We're creating an array of two bytes, and then casting that array of two bytes into a single short. What we've done by using an array is basically forced a certain byte order, and we're going to see how the system treats those two bytes. If this is a little endian system, the 0 and 1 will be interpreted backwards, and will be seen as if it is 0,1. Since the high byte is 0, it doesn't matter and the low byte is 1—x will be equal to 1. On the other hand, if it's a big endian system, the high byte will be 1 and the value of x will be 256. We'll use this trick later to determine what endian our system is using without having to set any conditional compilation flags.

Writing Endian Independent Code


So we finally get down to the most important part; how does one go about writing code that isn't bound to a certain endian? There are many different ways of doing this; the one I'm going to present here was used in Quake 2, and most of the code you'll see here is somewhat modified code out of the Quake 2 source code. It's mostly geared towards fixing files that are written in a certain endian, since the type casting problem is much harder to deal with. The best thing to do is to avoid casts that assume a certain byte order.

So the basic idea is this. Our files will be written in a certain endian without fail, regardless of what endian the system is. We need to ensure that the file data is in the correct endian when read from or written to file. It would also be nice to avoid having to specify conditional compilation flags; we'll let the code automatically determine the system endian.

Step 1: Switching Endians

The first step is to write functions that will automatically switch the endian of a given parameter. First, ShortSwap:

short ShortSwap( short s )
{
  unsigned char b1, b2;

  b1 = s & 255;
  b2 = (s >> 8) & 255;

  return (b1 << 8) + b2;
}

This function is fairly straightforward once you wrap your head around the bit math. We take apart the two bytes of the short parameter s with some simple bit math and then glue them back together in reverse order. If you understand bit shifts and bit ANDs, this should make perfect sense. As a companion to ShortSwap, we'll have ShortNoSwap, which is very simple:

short ShortNoSwap( short s )
{
  return s;
}

This seems utterly pointless at the moment, but you'll see why we need this function in a moment.

Next, we want to swap longs:

int LongSwap (int i)
{
  unsigned char b1, b2, b3, b4;

  b1 = i & 255;
  b2 = ( i >> 8 ) & 255;
  b3 = ( i>>16 ) & 255;
  b4 = ( i>>24 ) & 255;

  return ((int)b1 << 24) + ((int)b2 << 16) + ((int)b3 << 8) + b4;
}

int LongNoSwap( int i )
{
  return i;
}

LongSwap is more or less the same idea as ShortSwap, but it switches around 4 bytes instead of 2. Again, this is straightforward bit math.

Lastly, we need to be able to swap floats:

float FloatSwap( float f )
{
  union
  {
    float f;
    unsigned char b[4];
  } dat1, dat2;
    
  dat1.f = f;
  dat2.b[0] = dat1.b[3];
  dat2.b[1] = dat1.b[2];
  dat2.b[2] = dat1.b[1];
  dat2.b[3] = dat1.b[0];
  return dat2.f;
}

float FloatNoSwap( float f )
{
  return f;
}

This looks a little different than the previous two. There are three major steps here. First, we set one of our unions, dat1, equal to f. The union automatically allows us to split the float into 4 bytes, because of the way unions work. Second, we set each of the bytes in dat2 to be backwards of the bytes in dat1. Lastly, we return the floating-point component of dat2. This union trick is necessary because of the slightly more complex representation of floats in memory (see the IEEE documentation). The same thing can be done for doubles, but I'm not going to show the code here, as it simply involves adding more bytes, changing to double, and doing the same thing.

Step 2: Set function pointers to use the correct Swap function

The next part of our implementation is the clever twist that defines this method of endian independence. We're going to use function pointers to automatically select the correct endian for us. We'll put their declarations in a header file

extern short (*BigShort) ( short s );
extern short (*LittleShort) ( short s );
extern int (*BigLong) ( int i );
extern int (*LittleLong) ( int i );
extern float (*BigFloat) ( float f );
extern float (*LittleFloat) ( float f );

Remember to put them in a C or C++ file without extern so that they are actually defined, or you'll get link errors. Each one of these functions is going to point to the correct Swap or NoSwap function it needs to invoke. For example, if we are on a little endian system, LittleShort will use ShortNoSwap, since nothing needs to change. But if we are on a big endian system, LittleShort will use ShortSwap. The opposite is true of BigShort.

Step 3: Initialization

In order to initialize all of these function pointers, we'll need a function to detect the endian and set them. This function will make use of the byte array cast I showed you earlier as an example of a byte cast that assumes a certain endian.

bool BigEndianSystem;  //you might want to extern this

void InitEndian( void )
{
  byte SwapTest[2] = { 1, 0 };

  if( *(short *) SwapTest == 1 )
  {
    //little endian
    BigEndianSystem = false;

    //set func pointers to correct funcs
    BigShort = ShortSwap;
    LittleShort = ShortNoSwap;
    BigLong = LongSwap;
    LittleLong = LongNoSwap;
    BigFloat = FloatSwap;
    LittleFloat = FloatNoSwap;
  }
  else
  {
    //big endian
    BigEndianSystem = true;

    BigShort = ShortNoSwap;
    LittleShort = ShortSwap;
    BigLong = LongNoSwap;
    LittleLong = LongSwap;
    BigFloat = FloatSwap;
    LittleFloat = FloatNoSwap;
  }
}

Let's examine what's going on here. First we use the cast to check our endian. If we get a 1, the system is little endian. All of the Little* functions will point to *NoSwap functions, and all of the Big* functions will point to *Swap functions. The reverse is true if the system is big endian. This way, we don't need to know the endian of the system we're on, only the endian of the file we're reading or writing.

A Practical Demonstration


Ok, let's suppose we have some sort of structure we need to read from file and write to file, maybe a vertex. Our structure looks like this:

struct Vertex
{
  float Pos[3];
  float Normal[3];
  long Color;
  float TexCoords[2];
};

Nothing special here, just a typical vertex structure. Now we're going to decide that vertices will always be stored to file in little endian. This is an arbitrary choice, but it doesn't matter. What we're going to do is add a function to this struct that will fix the endian after loading or before saving it:

struct Vertex
{
  float Pos[3];
  float Normal[3];
  long Color;
  float TexCoords[2];

  void Endian()
  {
    for(int i = 0; i < 3; ++i) //our compiler will unroll this
    {
      Pos[i]= LittleFloat( Pos[i] );
      Normal[i] = LittleFloat( Pos[i] );
    }
    Color = LittleLong( Color );
    TexCoords[0] = LittleFloat( TexCoords[0] );
    TexCoords[1] = LittleFloat( TexCoords[1] );
  }
};

Let's be honest here; it's not exactly the easiest thing ever. You're going to have to write one of those painfully boring functions for each and every structure that goes in and out of files. After those functions are written, though, writing endian independent code elsewhere is going to be a breeze. Notice that we used Little* functions here because our files are all little endian. If we had decided to use big endian, we could have simply used the Big* functions.

Now what will the actual functions for working with the vertices be like? Well, to read a vertex:

void ReadVertex( Vertex* v, FILE* f )
{
  if( v == NULL || f == NULL )
    return;
    
  fread( v, sizeof(Vertex), 1, f );
    
  //now, our not quite magical endian fix
  v->Endian();
}

Notice the simplicity of correcting the endian; although our structure definitions are slightly messy, the loading code has become very simple. The code to save is equally simple:

void WriteVertex( Vertex* v, FILE* f )
{
  if( v == NULL || f == NULL )
    return;

  v->Endian();
  fwrite( v, sizeof(Vertex), 1, f );
}

And that's all there is to it. I'm including a sample C++ header/source file pair that you're free to use in your own programs; it's GPL code though (since it's actually part of a GPL project I'm working on) so you'll have to write your own if you don't want to release your code.

Wade Not in Unknown Waters: Part Four

$
0
0

This time we will discuss virtual inheritance in C++ and find out why one should be very careful using it. See other articles of this series: N1, N2, N3.


Initialization of Virtual Base Classes


First let's find out how classes are allocated in memory without virtual inheritance. Have a look at this code fragment:


class Base { ... };
class X : public Base { ... };
class Y : public Base { ... };
class XY : public X, public Y { ... };

It's pretty clear: members of the non-virtual base class Base are allocated as common data members of a derived class. It results in the XY object containing two independent Base sub-objects. Here is a scheme to illustrate that:


image1.png
Figure 1. Multiple non-virtual inheritance.


When we deal with virtual inheritance, an object of a virtual base class is included into the object of a derived class only once. Figure 2 shows the structure of the XY object in the code fragment below.


class Base { ... };
class X : public virtual Base { ... };
class Y : public virtual Base { ... };
class XY : public X, public Y { ... };


image2.png
Figure 2. Multiple virtual inheritance.


It is at the end of the XY object that memory for the shared sub-object Base is most probable to be allocated. The exact implementation of the class depends on the compiler. For example, the classes X and Y may store pointers to the shared object Base. But as far as I understand, this practice is out of use nowadays. A reference to a shared sub-object is rather implemented through offset or as information stored in the virtual function table.


The "most derived" class XY alone knows where exactly a sub-object of the virtual base class Base is to be allocated. That's why it is the most derived class which is responsible for initializing all the sub-objects of virtual base classes.


XY constructors initialize the Base sub-object and pointers to it in X and Y. After that, all the remaining members of the classes X, Y and XY are initialized.


Once the XY constructor has initialized the Base sub-object, the X and Y constructors are not allowed to re-initialize it. The particular way it will be done depends on the compiler. For example, it can pass a special additional argument into the X and Y constructors to tell them not to initialize the Base class.


Now the most interesting thing which causes much confusion and a lot of mistakes. Have a look at the following constructors:


X::X(int A) : Base(A) {}
Y::Y(int A) : Base(A) {}
XY::XY() : X(3), Y(6) {}

What number will the base class's constructor take as an argument - 3 or 6? None!


The constructor XY initializes the virtual sub-object Base yet does that implicitly. It is the Base constructor which is called by default.


As the XY constructor calls the X or Y constructor, it doesn't re-initialize Base. That's why Base is not being called with an argument passed into it.


Troubles with virtual base classes don't end here. Besides constructors, there are also assignment operators. If I'm not mistaken, the standard tells us that an assignment operator generated by the compiler may assign values to a sub-object of a virtual base class multiple times or once. So, you just don't know how many times the Base object will be copied.


If you implement your own assignment operator, make sure you have prevented multiple copying of the Base object. The following code fragment is incorrect:


XY &XY::operator =(const XY &src)
{
  if (this != &src)
  {
    X::operator =(*this);
    Y::operator =(*this);
    ....
  }
  return *this;
}

This code leads to double copying of the Base object. To avoid this, we should add special functions into the X and Y classes to prevent copying of the Base class's members. The contents of the Base class are copied just once, in the same code fragment. This is the fixed code:


XY &XY::operator =(const XY &src)
{
  if (this != &src)
  {
    Base::operator =(*this);
    X::PartialAssign(*this);
    Y::PartialAssign(*this);
    ....
  }
  return *this;
}

This code will work well, but it still doesn't look nice and clear. That's the reason why programmers are recommended to avoid multiple virtual inheritance.


Virtual Base Classes and Type Conversion


Because of the specifics of how virtual base classes are allocated in memory, you can't perform type conversions like this one:


Base *b = Get();
XY *q = static_cast<xy *="*">(b); // Compilation error
XY *w = (XY *)(b); // Compilation error

A persistent programmer, though, will achieve that by employing the operator reinterpret_cast:


XY *e = reinterpret_cast<xy *="*">(b);

However, the result will hardly be of any use. The address of the beginning of the Base object will be interpreted as a beginning of the XY object, which is quite a different thing. See Figure 3 for details.


The only way to perform a type conversion is to use the operator dynamic_cast. But using dynamic_cast too often makes the code smell.


image3.png
Figure 3. Type conversion.


Should We Abandon Virtual Inheritance?


I agree with many authors that one should avoid virtual inheritance by all means, as well as common multiple inheritance.


Virtual inheritance causes troubles with object initialization and copying. Since it is the "most derived" class which is responsible for these operations, it has to be familiar with all the intimate details of the structure of base classes. Due to this, a more complex dependency appears between the classes, which complicates the project structure and forces you to make some additional revisions in all those classes during refactoring. All this leads to new bugs and makes the code less readable.


Troubles with type conversions may also be a source of bugs. You can partly solve the issues by using the dynamic_cast operator, but it is too slow and if you have to use it too often in your code it means that your project's architecture is probably very poor. Project structure can be almost always implemented without multiple inheritance. After all, there are no such exotica in many other languages, and it doesn't prevent programmers writing code in these languages from developing large and complex projects.


We cannot insist on total refusal of virtual inheritance: it may be useful and convenient at times. But always think twice before making a heap of complex classes. Growing a forest of small classes with shallow hierarchy is better than handling a few huge trees. For example, multiple inheritance can be in most cases replaced by object composition.


Good Sides of Multiple Inheritance


OK, we now understand and agree with the criticism of multiple virtual inheritance and multiple inheritance as such. But are there cases when it can be safe and convenient to use?


Yes, I can name at least one: Mix-ins. If you don't know what it is, see the book "Enough Rope to Shoot Yourself in the Foot" [3]


A mix-in class doesn't contain any data. All its functions are usually pure virtual. It has no constructor, and even when it has, it doesn't do anything. It means that no troubles will occur when creating or copying these classes.


If a base class is a mix-in class, assignment is harmless. Even if an object is copied many times, it doesn't matter: the program will be free of it after compilation.


References


  1. Stephen C. Dewhurst. "C++ Gotchas: Avoiding Common Problems in Coding and Design". - Addison-Wesley Professional. - 352 pages; illustrations. ISBN-13: 978-0321125187. (See gotchas 45 and 53).
  2. Wikipedia. Object composition.
  3. Allen I. Holub. "Enough Rope to Shoot Yourself in the Foot". (You can easily find it on the Internet. Start reading at section 101 and further on).
Viewing all 17825 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>