Server Issues
I\'m sure everyone has noticed a few bugs around the site the past few days. Things aren\'t working properly in relation to time (like auto training, or gold accounts expiring, or quests resetting) or any other number of things.
What happened is our development server crashed. This is the server where all of our tools are kept (including all the files that run the cron which makes time tick on Subeta) as well as gives us (the programmers) the ability to edit the site while it\'s live.
The server essentially crashed and burned, one of the hard drives is completely unreachable and had to be replaced. Unfortunately, it took awhile to be replaced, and it meant that we have to do an entire OS reload on that server. We\'ve lost months of work (because our top secret projects are kept on that server, and off the other webservers) between all of the developers, as well as access to those tools I was talking about above.
We\'re going to be spending the weekend trying to get things working. There are two things that we are going to be doing.
1.
Priming our new hosting solution at Rackspace for a move soon. The type of hosting we have at Rackspace should prevent something like this from happening in the future. We\'re also now taking preventive measures to keep our code in multiple (safe!) places to ensure that we don\'t lose tons of work in the future.
We anticipate moving to Rackspace sometime in the next two weeks, which will mean (at most, hopefully!) a day of downtime. Once we are closer to that, I will fill you in on the exact date/time that we intend to switch.
2.
Getting our development server up and running on our existing hosting. Once we\'ve got this all worked out, we should have the cron running again, and be able to make changes to the site. Right now we can, it\'s just complicated and annoying.
SO FOR THE GOOD NEWS, RIGHT?!
We are going to be extending the Masquerade dancing until Monday. At that point, we should have firm control over the situation. And it gives you a few more days to collect those trinkets ;).
As always, thank you guys so much for being patient with us. This is just as frustrating (if not more) for us as it is for you. We :heart: you!
Posted by Keith
KimC
hugs I am so very very sorry to hear that you lost so much hard work. The work you put into this sight is awe-inspiring and I can only imagine your frustration at losing projects you had put so much effort into. The only upside to this is I think I can make it to 2500 dances now...
Thats just awful to hear about all of your work lost. You guys work so hard on making this site so much fun. To think I was getting a little frustrated cause I had a pet stuck in training. This is way worse than the just site clock getting stuck. I hope you get every thing as fixed as can be soon. The extra dances are nice but I wish we weren't getting them for this reason.
Erf, crashes are never fun - here's hoping to a speedier go-round when recoding stuff.
Appreciated the update, now I know why Zhuge's been at a more nonstop AT than normal. Lol My pet the gym rat.
Cheers for the dancing extension, I might just get 2500 trinkets by monday ...hahaha right. Lol
~Kai
Appreciated the update, now I know why Zhuge's been at a more nonstop AT than normal. Lol My pet the gym rat.
Cheers for the dancing extension, I might just get 2500 trinkets by monday ...hahaha right. Lol
~Kai
I'm so sorry you guys lost all that work. =( That's awful. Thanks for explaining things, though, and extending dancing time!
Awwww huggles I hate it when things like that happen. Good luck and thank you so much for working so hard for us. The extended Masquerade more than makes up for the loss of autopricing and interest, as I am still prowling for trinkets anyway. :D Don't burn yourself out and remember to take breaks during your hard work! You love us, but we love you too!
Thanks for the update. I'm sorry to hear about the work that you lost, but hope that you can eventually retrieve it. Keep up the good work and we do appreciate all you do for us members.
Keith
STAFF
@Rocketlauncher
Unfortunately we didn't really think to backup this deep. All of the "live" files are always running on the site. We've got multiple backups of it (as it's deployed across multiple servers) and it's really impossible to lose that information.
Our private developer "branches" of the code on the other hand weren't pushed across all the webservers, so it never really clicked to really put it anywhere else. Our mistake, and we're going to pay for it :(
Unfortunately we didn't really think to backup this deep. All of the "live" files are always running on the site. We've got multiple backups of it (as it's deployed across multiple servers) and it's really impossible to lose that information.
Our private developer "branches" of the code on the other hand weren't pushed across all the webservers, so it never really clicked to really put it anywhere else. Our mistake, and we're going to pay for it :(
Oh gosh, you guys... That's terrible, I'm so sorry to hear that you've lost all that hard work. I know how much time and effort you dedicate to the site, and it really is just heartbreaking to hear about your problems. I'm willing to wait as long as it takes for you to get things back into place. You've made Subeta a wonderful place to be; the least I can do is sit patiently while I wait for you to fix the site. Good luck, all, and thank you for the update~!
Best of luck fixing it over the weekend! I'm shocked about the amount of work lost, aren't they regularly backed up? :( Thank you also for letting us know about the move.
Good luck, Keith/Team... We love you! This is an amazing place and thanks for keeping us in the loop.
Sorry to hear that things malfunctioned like that... Just out of curiosity, does this mean that the User Interaction Interface is going to be postponed again? Is it one of the projects that was wiped out? D: I sure hope not :/
Best of luck to you guys in fixing it all and getting the kinks worked out though. :heart:
Best of luck to you guys in fixing it all and getting the kinks worked out though. :heart:
Oh man; I know what it's like to lose all your hard work. I lost over 1200 pictures because my computer crashed and I hadn't had the chance to back them up. 3 years of work lost.
Good luck you guys! Just know we appreciate you!
Good luck you guys! Just know we appreciate you!
I'm really sorry about the crash. Best wishes for getting things back together. And I say, the more Masquerade, the better!
aw man sorry about all the work you guys lost :(
Thanks so much though for getting everyone updated about the situation! We all know how much work is put into this site, so a little down time is no biggie if it means that you can keep all your hard work safe.
We all love you guys for everything that you do for us and put up with!
Big thanks for the extension for dancing :)
Thanks so much though for getting everyone updated about the situation! We all know how much work is put into this site, so a little down time is no biggie if it means that you can keep all your hard work safe.
We all love you guys for everything that you do for us and put up with!
Big thanks for the extension for dancing :)
Thanks for keeping us updated! I hope you are able to recover the lost work :( sounds so complicated, good luck getting things back to normal!
Thank you for the update on the situation! I feel sorry for you guys... All that time and effort... Gone.
I appreciate everything you all are doing to get it back up and running. I will be waiting, patiently, for things to return to normal. You rock, Subeta staff!
I appreciate everything you all are doing to get it back up and running. I will be waiting, patiently, for things to return to normal. You rock, Subeta staff!
Thanks so much for the updates, and for extending the masquerade. :D Collecting more trinkets is always exciting.
Thank you so much for giving us an update! So SO sorry you guys lost all that work though. :( That's never ever fun!
BEST WISHES IN FIXING STUFF!
We :heart: you too!
BEST WISHES IN FIXING STUFF!
We :heart: you too!
I'm so sorry you lost months of work.
Thank you for the update and your hard work.
We love you too!
Thank you for the update and your hard work.
We love you too!
There wont be a rollback will there? :( :(
I'm sorry for your lost Keith and the other programmers. There is no worst feeling then working on something for a long time and then losing it to a crash.
I'm sorry for your lost Keith and the other programmers. There is no worst feeling then working on something for a long time and then losing it to a crash.
Oh no!! I feel so bad you guys have lost so much hard work that is awful :(
Thankyou for the update and everything you all do to make us happy :)
Thankyou for the update and everything you all do to make us happy :)
I was wondering when you'd make a public announcement about this, and I'm glad you have. Thanks for the update and especially for the masquerade extension. I'm sorry you guys lost so much work, but at least it's a lesson well learned and hopefully this won't happen again.
We love you guys too, and we're all thankful for your hard work <3
We love you guys too, and we're all thankful for your hard work <3
Good luck! You won't hear me complaining about it at all, I know you're doing what you have to. Thanks for letting us know.
Just as I went to the bugs forums to see what this was.
Thank you for informing us, I was terribly confused.
Thank you for informing us, I was terribly confused.
While I'm happy to see the Masq extending, I'm so sorry to hear about all the work that's been lost. I know how much goes into this site and that's just heartbreaking! Thanks for letting us know though, and good luck getting things together! We love you guys!
Aww, it sucks about losing all that work you guys did! D; I hope it was backed up somewhere at least...
You all work hard to keep this site up and running as smoothly as you can. I for one appreciate all the effort, work and time you all put in, to make this such a great site. Thank you
thank ypou so much for working on this site so hard!! im really greateful you guy do this! keep up the good work!
Oh, I'm so sorry about all your lost work! Hang in there. We'll keep dancing and hopefully raise your spirits.
Thanks for the update! Sorry for the stress and loss of work tho, thats not good :( But Once again Thank you ALL for what you do!!
I'm so sorry about losing all your hard work! thanks for the extended dance time, and good luck with the move and all that other stuff!
We :heart: you too, guys.
Thanks for letting us know what is going on and that you are all actively working on it. It makes the little annoyances less annoying :D
Thanks for letting us know what is going on and that you are all actively working on it. It makes the little annoyances less annoying :D
No problem ... we all love being here and support your efforts 100% .. do what you have to do we will all wait until you get it squared away ... have a great weekend Keith and staff!
That's horrible! It's all good though. You're working on it and that's all that matters. Thanks for informing us about it though!
Thanks for the update and dance extention. More art for me to gank with dancing. :D
I must say I'm so sorry about you guys losing so much stuff... i can't even imagine how much of a headache and just heart breaking that would be for those that put the effort into it. Hopefully all the new safeguards will work out!
I must say I'm so sorry about you guys losing so much stuff... i can't even imagine how much of a headache and just heart breaking that would be for those that put the effort into it. Hopefully all the new safeguards will work out!
Oh man month's of work lost? That really sucks :( Thank you for the updates, and for letting us know what is going on!