Entertainment at it's peak. The news is by your side.

I deleted the production database by accident


We deleted the production database by likelihood 💥

by Caspar

As of late at around 10: 45pm CET, after a pair of glasses of purple wine, we deleted the production database by likelihood 😨.

Fortunately our database is a managed database from DigitalOcean, which manner that DigitalOcean automatically enact backups once a day. After 5 minutes of hand-wringing and dismay, we went into maintenance mode and had been ready to restore a backup. At around 11: 15pm CET, 30 minutes after the catastrophe, we went back online, however 7 hours of scoreboard data change into gone forever 😵.

To be precise, any scoreboards created or ratings added between 15: 47 CET and 23: 21 CET were lost. We’re extraordinarily sorry about this.

Blue green tennis

What took web site?

It’s tempting in payment the catastrophe on the couple of glasses of purple wine. On the opposite hand, the feature that wiped the database change into written whilst sober. It’s a feature that deletes the local database and creates the entire required tables from scratch. This evening, whilst doing some slack evening coding, the feature connected to the production database and wiped it. Why? Here’s one thing we’re quiet looking out for to establish out.

Here is the code that triggered the catastrophe:

def database_model_create():
    """Handiest works on localhost to forestall catastrophe"""
    database = config.DevelopmentConfig.DB_DATABASE
    person = config.DevelopmentConfig.DB_USERNAME
    password = config.DevelopmentConfig.DB_PASSWORD
    port = config.DevelopmentConfig.DB_PORT
    local_db = PostgresqlDatabase(database=database, person=person, password=password, host='localhost', port=port)
    local_db.drop_tables([Game, Player, Round, Score, Order])
    local_db.create_tables([Game, Player, Round, Score, Order])
    print('Initialized the local database.')

Demonstrate that host is hardcoded to localhost. This suggests it would perchance perhaps quiet never join to any machine rather than the developer machine. We’re too tired to establish it out exact now. The gremlins won this time.

What enjoy we realized? Why won’t this happen all but again?

We’ve realized that having a feature that deletes your database is just too terrible to enjoy mendacity around. The venture is, you might perchance well also never if truth be told test the safety mechanisms neatly, on fable of testing it would perchance perhaps suggest pointing a gun on the production database.

In fact, we can never be 100% definite that one thing admire this won’t happen all but again. Computer systems are exact too advanced and there are days when the complexity gremlins secure. On the opposite hand, we can establish out what went gross and be certain that that that particular error doesn’t happen all but again.

All but again, we’re very sorry. Correct night.

Picture by Crack of morning time Armfield on Unsplash

Read More

Leave A Reply

Your email address will not be published.