Lieutenant
Join Date: Sep 2012
Posts: 52
This is mostly a question for the PWE staff, but I'm sure the community will weigh in as well.

Now that the 5+ hour long outage on 5/2/2013 has finally been resolved, could we get some comments about what happened, and some reassurances about what's being done to prevent it in the future?

Was it a piece of networking gear that failed? A server? A connectivity issue that was more or less outside of your control? Something that caused multiple cascading failures?

Also, as a general question, what is your uptime goal for STO? It's clearly not the "five nines" (99.999% uptime) standard for major online services, as that only allows about 5 minutes of downtime per year, and the weekly maintenance alone is way more than that.

I understand that you run a business, and while you could theoretically have 5 independent data centers all over the globe, any one of which is capable of running the entire system, that would be prohibitively expensive. But neither are you just running the server on a single box plugged into a cable modem. There's a balance point somewhere in between, and it's different for every company and situation.

My question is: how much downtime (planned and unplanned) do you consider "acceptable?" I really hope that the 5 hour unexpected downtime is as unacceptable to you as it is to the players, as reflected by the 200 posts/hour on the two different threads about the outage.
Rihannsu
Join Date: Jun 2012
Posts: 308
# 2
05-03-2013, 12:03 AM
Havent played many mmos have you no mmo is 99% uptime they all go down alot and for lots of reasons as dumb as a missing image.

I for one am HAPPY they stayed for overtime to fix the issue at hand.

Thank you Cryptic.
-Spells

|| Open Door Policy ||
| Dues Ex Mechina |
Fleet Leader

Captain
Join Date: Apr 2013
Posts: 1,204
# 3
05-03-2013, 03:51 AM
Quote:
Originally Posted by srspells View Post
I for one am HAPPY they stayed for overtime to fix the issue at hand.
Stayed for overtime? you do realize that when the crash happened any 9-5 employee was in the middle of dinner already. Servers are in California and the crash happened around 6:30 PM easter time.

I give kudos to the employees who fixed the server catastrophy but the company itself none. there is no excuse for server crashes this day and age when you release a new IP and it was the release of the Neverwinter open beta that pushed the system past it's limits. It is well known that server overload happens on release day. they could rent servers or a whole farm to prevent such things and still make a huge profit.
Rihannsu
Join Date: Jun 2012
Posts: 308
# 4
05-03-2013, 03:54 AM
neverwinter may of overloaded the client server but not stos, essentially it was the client that made cryptics games out of order.
-Spells

|| Open Door Policy ||
| Dues Ex Mechina |
Fleet Leader

Starfleet Veteran
Join Date: Jun 2012
Posts: 2,735
# 5
05-03-2013, 06:26 AM
Neverwinter very much CAN overload STO. They're hosted together, they share a chat server (custom channels go to all Cryptic games and can even be reached outside the games), patch server, and account server. That's more integrated than, say, two different domains in the same data center, which itself is a situation where one site can cause an outage on the other.


99.999% is rare for server hosting, generally attained by multicast redundancy which is not feasible for real time applications and not cost feasible for most others. Three nines is considered beyond outstanding for real time applications, including MMOs and other game servers. Counterintuitively, the few MMOs capable of that kind of uptime are not the biggest names, but the smallest - largely unmaintained by their dwindling development teams but not yet abandoned by their players, going months or years without patches and neglecting regular maintenance until the server crashes or dies.


I mean, an extended surprise downtime is frustrating to all involved, especially those affected during peak hours, but let's not use silly and unrealistic standards to judge it.

Last edited by hevach; 05-03-2013 at 06:33 AM.
Lieutenant
Join Date: Jul 2012
Posts: 51
# 6 Really?
05-03-2013, 10:03 AM
So this came off the Neverwinter Site...its one of the first things you see on thier board..

http://nw.perfectworld.com/news/?p=880781

Has the STO team offered and explanation?
Career Officer
Join Date: Jun 2012
Posts: 886
# 7
05-03-2013, 04:34 PM
Quote:
Originally Posted by srspells View Post
Havent played many mmos have you no mmo is 99% uptime they all go down alot and for lots of reasons as dumb as a missing image.
Yeah, BUT, this recent crash, was just one incident, among an ever increasing list of incidents: like constant server disconnects, lag with bank items, extreme lag in general, along with launcher connectivity issues.

Is it possible that Never Winter might make things worse?, some concern is understandable.
Quote:
Originally Posted by wudwaen View Post
Wasn't there supposed to be a game one played for entertainment in here someplace?
Captain
Join Date: Jun 2012
Posts: 1,489
# 8
05-03-2013, 01:19 AM
Who wants to bet that recent shenanigans in STO are related to the infrastructure for Neverwinter getting some hardcore stress testing with the opening of that game's beta?
Rihannsu
Join Date: Jun 2012
Posts: 308
# 9
05-03-2013, 01:44 AM
not likely as each game is hosted through different servers, but what the client was the issue as its the launcher, if launcher goes down so does all the games as its just a clients acess to the game.
-Spells

|| Open Door Policy ||
| Dues Ex Mechina |
Fleet Leader

Lt. Commander
Join Date: Jun 2012
Posts: 207
# 10
05-03-2013, 06:18 AM
Quote:
Originally Posted by srspells View Post
not likely as each game is hosted through different servers, but what the client was the issue as its the launcher, if launcher goes down so does all the games as its just a clients acess to the game.
-Spells

1) They are the same data centers
2) They have shared resources
3) They share a common networking set(Login authentication)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


All times are GMT -7. The time now is 10:53 PM.