From March 23rd 7:15 AM to March 24th 10:50 PM Subeta time, there was an error where every time an item was removed through the new wardrobe, it would be deleted from other users' wardrobes as well. We fixed this error as soon as we could, but many items have been lost. Our daily backup service had also stopped working, so we were unable to roll back the database by a day to restore the items.

We own up to this; this was an egregious, unacceptable error, and there is no excuse for not better monitoring our backups.

This error has now been fixed, and we are taking steps both to resolve this in the immediate future and to prevent this from ever happening again. Here's what you need to know.

Immediate Actions

We understand many items are irreplaceable, and we're going to do everything we can to restore all individual items. While our daily backup service had failed, we've booted up an older backup from about 4 months ago which will help a lot. We will be using this backup, various logs, and existing avatar data to further determine which items need to be restored.

We've set up a report form here that should streamline the process of reporting items you know you're missing. From now on, please go ahead and use that report form instead of the ticket system to make your reports.

We've also anticipated some questions which we've put in the FAQ spoiler below. If you have further questions, please leave them in this thread; we want to make sure your concerns are heard and taken care of.

Finally, we are giving 1000 CSC to every user as additional compensation. Please click here to receive your CSC. This link will also be available indefinitely for all accounts potentially affected by the glitch.

Future Preventative Measures

While coding errors will sometimes slip through, the full scale and impact of this error could have been prevented. And while it is too late for this situation, we are taking actions to ensure this never happens again in the future.

1. More reliable backups - We've improved our backup service and have added monitoring so that we will be alerted immediately if a problem with it occurs.

2. Changing the way wardrobe data is deleted - We've changed the wardrobe to no longer delete item data immediately when you remove items; we'll now have recent delete data in our database itself as an extra layer of backup. This gives us more ability to respond to all wardrobe-related incidents moving forward, not just incidents like this one, and makes our system more robust in general.

3. Further database reinforcements - Finally, we're investigating other changes we can make to our database infrastructure to further ensure resilient data and backups.

We want to make it clear that we understand how truly upsetting this is for everyone. We are so, so sorry. We are working hard to make this right, and are putting every effort in to restore your wardrobes and your faith in us.