QA, Localisations and Age Ratings

Authored by: Richard Hill-Whittall

The Indie Game Developer Handbook

Print publication date:  February  2015
Online publication date:  February  2015

Print ISBN: 9781138828421
eBook ISBN: 9781315738413
Adobe ISBN:

10.4324/9781315738413-4

 

Abstract

Traditionally the domain of the publisher QA department, one of the major time-sinks with self-funded development, is testing. If you have limited resources, try to test as you develop—the more ongoing gameplay testing and tweaking you do, the better the end game will be. Test, balance, tweak, polish and test some more!

 Add to shortlist  Cite

QA, Localisations and Age Ratings

 

QA Bug Tracking

Traditionally the domain of the publisher QA department, one of the major time-sinks with self-funded development, is testing. If you have limited resources, try to test as you develop—the more ongoing gameplay testing and tweaking you do, the better the end game will be. Test, balance, tweak, polish and test some more!

Added a new feature? Test it thoroughly before moving on; make sure it works and works well. The worse thing to do is to leave testing until the end when you are getting tired of the project, perhaps a little burnt out—how could you possibly give the game the play testing it deserves in that scenario? Your aim should be to focus on how a game plays from the early gameplay stages.

There are a few other reasons why it’s important to fix bugs now rather than putting them off until after Alpha:

  • Bugs are often quicker to deal with the sooner they’re dealt with, as the code is fresh in the mind.
  • If you implement a feature and then test it until it is bug free and playing right, you know you can rely on that part of the game to perform well.
  • Crash bugs can slow down adding new features because if the game keeps crashing, it can slow down implementing all other features of the game. There is nothing more frustrating than trying to design levels in a game that often crashes.
  • It’s harder to implement features that work and play well if the underlying implementation is bugged. Aspects such as player control, movement and interaction can never be properly balanced if the code is not solid.
Key to your entire bug fixing efforts is the process of tracking those pesky bugs. In the initial stages notes in a pad may well be sufficient, but don’t delay setting up a solid bug tracking system. So what is bug tracking?

Bug Tracking Software

A bug tracking system is a software application that is designed to help keep track of reported bugs. You can add issues, mark them as fixed or send them back to the developer if they are not properly fixed.

Ideally this should be an online database (some systems require you to run your own servers), and look out for extra functionality such as companion iPhone apps, so you can test and note down bugs on the move.

A typical bug tracking process is as follows:

Add an Issue

Here the developer adds a new issue. The issue does not have to be programming related, it can be a problem found in the levels, graphics, audio etc. The circumstances of the bug and steps to reproduce it are included in the report. Often including a screen-grab of the issue in action can be a big help, or even better a video reproducing the bug.

Assign the Issue

After adding an issue, it must then be assigned to the team member responsible for that aspect of the project. Once the bug is assigned, the individual is sent an email with the bug information, along with a link to the bug on the tracker.

Set the Priority

Each bug is given a priority ranking, from urgent must-fix bugs down to suggestions that can be waived if it isn’t deemed important or is a ‘feature.’ This ensures critical bugs are fixed first.

Verification

After the assigned developer fixes the issue, the tester then verifies that the bug no longer occurs. In some cases bugs may be marked as features (i.e. not a bug) and closed.

So let’s take a look at a few of the most popular bug tracking solutions out there. I have only included the free or cheaper options, as there really isn’t a lot of point in a smaller indie studio paying upwards of $20 per month for bug tracking.

Bugzilla is a ‘Defect Tracking System’ or ‘Bug-Tracking System.’ Defect Tracking Systems allow individuals or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licencing fees. Despite being ‘free,’ Bugzilla has many features its expensive counterparts lack.

MantisBT is a free, popular web-based bug tracking system. It is written in the PHP scripting language and works with MySQL, MS SQL and PostgreSQL databases and a webserver. MantisBT has been installed on Windows, Linux, Mac OS, OS/2 and others. Almost any web browser should be able to function as a client.

There is also a mobile app for iPhone, Android and Windows Phone which allows you to track and log bugs on the move. The price for the mobile app is $50.

Trac is an enhanced wiki and issue tracking system for software development projects. Trac uses a minimalistic approach to web-based software project management.

It provides an interface to Subversion and Git (or other version control systems), an integrated Wiki and convenient reporting facilities.

I have used Trac for all of our bug tracking for the past few years. It is very reliable and does the job well. It can be a little fiddly to set up, but once you are up and running it is easy to maintain.

GoPlan lets you keep track of your projects and collaborate with your colleagues securely through an intuitive user interface.

JIRA is the tracker for teams planning and building great products. Thousands of teams choose JIRA to capture and organize issues, assign work and follow team activity. Many game studios use JIRA, most likely because there is an additional Agile package, which includes Scrum and Kanban project management tools. The ‘Pro’ choice!

Working With External QA

If your budget stretches to it, there are QA companies out there who can assist with the bug tracking process, but they can quickly become very expensive. Shop around and find a good one to work with that won’t break the bank—often a single test round can cost in the thousands, and to make it worthwhile you will need more than one round of testing.

You will have to manage any external QA carefully, so first and foremost make sure to only get the QA Company involved when the game is ready (i.e. feature complete and internally tested). Keep a close eye on how they are spending their time; I actually heard that one QA company tried to charge for their testers sitting around waiting for a build! Also bear in mind that the QA companies are running a business to make money, so they will push you for extra test rounds that may not actually be necessary.

Always make sure when you supply builds that you send along clear Build Notes. Detail what is open for testing, list any known issues to avoid and be very specific.

There are various different types of testing that are offered, including:

Functionality Testing

Generally, it focuses on improving the quality of video games from the player’s viewpoint. Functionality testers hunt for all issues pertinent to gameplay including but not limited to:

  • Crashes
  • Game mechanic issues
  • Level flow issues
  • User interface functionality
  • Testing the game’s stability
  • Ensuring correct scoring
  • Game performance
  • Game asset integrity

Compliance Testing

Compliance testing is the process of testing a game to ensure that it meets the rules and requirements set out by the console manufacturer, i.e. Sony publishes a Technical Requirements Checklist (TRC), Microsoft publishes Technical Certification Requirements (TCR), and Nintendo publishes a set of ‘guidelines’ (LotCheck). More often than not you will have a build failed on one or more of those issues, so the more you address before platform holder QA, the smoother and quicker the submission process will be.

Example of compliance issues include:

  • Making sure the game handles things properly if the controller is disconnected
  • Save devices used properly
  • Network packets aren’t encrypted
  • All standardized text (error messages, info messages etc.) are correct
  • Correct usage of proprietary logos
  • Ensure saves can’t be copied between profiles
  • Make sure achievements can’t be won by cheating

Compatibility Testing

Compatibility testing is normally required for PC titles, near the end of development. They check whether or not the game runs on different configurations of hardware. The hardware could include various brands of CPUs, video cards, sound cards, and input peripherals such as gamepads, joysticks and other components. It will also evaluate performance for the game’s minimum system requirements.

Usually two rounds of compatibility tests are performed—the first during the Beta test phase, and the second just before the release candidate is ready.

Testing Companies

I have only included companies with a good reputation within the industry, and again, be aware that costs can run into the many thousands:

Language Translations

There are various degrees of localization. If you are self-publishing on an open format such as PC you have the easy option of no translation of any sort, while releasing games on consoles often requires compulsory text translation for things like store text and electronic manuals. There is never any compulsory requirement to translate in-game text, although you should consider the benefits that this may bring. It can certainly help when trying to promote your game in foreign speaking regions, but if you have a lot of text and a number of languages this can be very expensive.

One thing is for sure, if you intend to release on consoles you will need to arrange for language translations. Make sure you find a good, low-cost translation company that you know provides a fast turnaround and a reliable service. I’ve had a few occasions where I have needed translations done for a promotion within a couple of days; fortunately, we use a great translation company (Rangeela) and they always deliver on time.

The first thing you’ll learn is to write everything fairly concisely. When you are paying for translations, wordiness will cost you! Make your manual text lean and efficient, although leave enough info in there to still make it useful. If you are careful with word count you can bring an EFIGS&D e-manual in at under $500.

So, where to go? There are a fair few translation houses out there, but to make life easier the list below includes only the ones I have received personal recommendations for. It is worth noting that it really is best to go to a company specializing in games translations, as they will be familiar with the TRC/LotCheck terminology requirements.

Translation Companies

Age Ratings

Depending on your release platform(s), you may be required to get ESRB (US), PEGI (Europe), USK (Germany) and OFLC (Australia) ratings.

For the home console downloadable services you will need to obtain age ratings for each of your games. It is a very straightforward process, the only hassle being that it (in some cases) costs money. Boo!

An interesting recent development is that PEGI has announced the formation of an International Age Rating Coalition. This new system will be deployed as a single online form, which developers and publishers complete to generate a rating for their product. Instant product classification at no cost, as the various platform holders will cover all costs. These global ratings are operational now, but the platform holders haven’t yet finalised their storefronts to support them.

Keep track of updates on their official site:

The participating territories include the UK, USA, South America, Australia, New Zealand and Brazil amongst others. Once fully implemented by the various platform holders, this should cut out the European and Australian age ratings costs completely—which is very good news indeed!

For now though, you will need to contact each ratings board in turn. First off, you should register and set up an account for your company:

To do this just drop them an email and they will send the necessary forms. Once that is done, you will have access to the publisher areas on their sites, to which you can submit a product for rating. Using a downloadable game as an example, I have detailed the process for each of the ratings boards below.

The Ratings Process

PEGI

Complete the online submission form. You must then send them a copy of the game along with the fee (in this case the casual game fee of €250). You should also include a 10-minute gameplay video, a selection of screenshots along with the game manual to cover all bases. Within 10 working days, you should receive your rating.

USK

Download and print the submission form. You must then send a copy of the game along with the fee of €1200. You should also include a 10-minute gameplay video and a selection of screenshots along with the game manual. The examination will be completed within 15 working days.

AGCB

Download and print the submission form. You must then send a copy of the game along with the fee (in this instance, Level 2, AUS $890). You should also include a detailed written description of gameplay, a separate recording of any contentious material in the game and typical gameplay footage. A classification decision will be made within 20 working days after the application has been recommended to the Board.

ESRB

A lot has changed over at the ESRB recently, and they are now the most progressive ratings board for small studios. For a download only title (e.g. Xbox Live, Nintendo eShop, PlayStation Store, PlayStation Mobile, Windows 8, PC download, websites) you complete the online form, submit it and receive an instant rating for no charge. If only all ratings were this straightforward!

Search for more...
Back to top

Use of cookies on this website

We are using cookies to provide statistics that help us give you the best experience of our site. You can find out more in our Privacy Policy. By continuing to use the site you are agreeing to our use of cookies.