View Single Post
Lt. Commander
Join Date: Dec 2007
Posts: 120
# 1 Insight into the QA process
03-11-2010, 06:54 PM
Hello Everyone,

This response was originally part of this Thread, after significant request to have it stickied I decided it would be best to create its own thread in the Federation News Network.


First off, I want to make the following explcitly clear, Cryptic Studios does not in any way "expect" our users to submit bug reports. This is an entirely voluntary process provided so that those users interested can submit both Feedback and any Issues they encounter.


Now that we have that out the way, let me take some time to provide some insight into how QA (Quality Assurance) functions in general within the gaming industry and here in-house.
  • QA process differs from company to company, especially between a Publishing and Development house.
  • There are many types of QA: Game QA, Infrastructure QA, LIVE QA, Dev Support QA, Standards QA, and many more depending on where you work.
  • Each QA team generally has a specific area they focus on and do not detract from this unless there is specific direction or special circumstances to do so.
  • Resources are always limited: Staff, Time, Budget are just a few of the bigger examples.
  • QA most often, will not get the final say as to when something is released and in what state.

Those are rough basics you will find at most companies which maintain QA. That of course does not even delve into companies which utilize off-site (external) QA sometimes referred to as blackbox testers.


Alrighty, with that, let us talk a bit about how we handle tickets from A to Z.

We encourage all users to participate in the voluntary process of submitting either feedback or issues (bugs) you may find on your travels through the Alpha Quadrant.

Once an issue goes to our C.S team, they collate all similiar issues and take as much information as they can in order to formulate an ideal bug to then forward off to DEV directly or in most cases QA to verify.
Tip - The more an issue is reported the more attention that issue gets
Once the ticket is in QA's hands it enters a queue and is processed in turn. The issue is checked for validity and that enough information is present to accurately reproduce the issue for DEV. If there is not enough information it is sent back to C.S and they are informed we need more information before continuing.

The issue is sent off to DEV which is given a severity of how bad the issue is from the QA perspective. A Team Lead and or Producer will triage issues submitted and prioritize them for their respective teams to address. The DEV will attempt to fix his assigned highest priority issue, upon doing so will send the ticket back to us.

QA waits for the fix to enter an internal build, at this point the issue is regressed and confirmed whether or not the fix has taken place. If the fix is confirmed the issue is closed out and we wait for the fix to make it into a build destined for either Tribble the PTS then Holodeck (LIVE) or straight to Holodeck depending on the issue.

If the fix fails, QA sends the issue back to the DEV and the process repeats until fixed.

Now please bear in mind that the above described process is a gross over simplification and many details and further processes were left out as this is internal information not available to the public. However the overall information indicates clearly the stages of issue submission to being fixed on LIVE.


Now, onto the issue of bugs making it into release.

As previously stated QA and the company overall have many limitations (Time, Budget, Staff, Etc.). That means regardless of how Chuck Norris we are, there are going to be regular cases where an issue finds its way into LIVE.

This is no way means QA didn't find it, or that DEV hasn't been actively trying to fix it. All that means is that when the decision is made to move forward we like the Borg will adapt.

Many factors come into play, for example:
  • Would you rather a DEV artist fix those windows on the Galaxy or keep modelling his sweet looking new ship variant being built.
  • Would you rather a DEV programmer fix that super rare crash that affects 10 people or continue optimizing code so that the game runs smoother, or add a new tech feature for Territory Control.
  • Would you rather a DEV designer fix that minor typo is his mission text or keep putting together entertaining and challenging End Game content.

Some of you will be on one side of the coin because that issue is very important to you, others will be on the opposite because they just want new stuff. No company on the planet in the whole of existence has ever been able to make everyone happy, it is a theoretical impossibility.

However I will say this, we are human just like you (unless you are a certain programmer whom I still claim is a terminator re-programmed to program). We respond to the full range of emotions that you all do, when we get told our customers don't like something we feel bad. When we hear about a change we like/dislike we stand up nerdrage and go talk and debate the issue just like you would.

If you talk to us as adults, speak politely and with respect, be concise and clear in what you are saying, have some empathy and treat us just how you want to be treated. We are all the more enticed to interact and provide insight where we are able.

This should clear up a great deal for many of you.

Always remember if you have questions, concerns, feedback or issues to submit, a good place to get some perspective and DEV interaction is IRC.

I and a few others maintain a regular presence in #sto and #stoqa. You can always PM me or participate in discussion, I will most likely be busy helping others but if you are patient I will respond to you as I am able. This also helps to expedite issues as I can walk over to the DEV and tell them personally about user reported issues.

Q speed all.