DejaCloud Server Down

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • DJOCTO
    Senior Member
    • Apr 2010
    • 2386
    #1

    DejaCloud Server Down

    A table crashed in our server. We are rebuilding that table. While it rebuilds, the server has locked all other tables.

    We will have to wait until the rebuild completes.

    Our best guess is it will be complete by 5pm Pacific Time 10-6-16. At that time, DejaCloud services should be back running.

    If the rebuild does not complete, we will begin restore from backup at 8pm, and we anticipate the system will be running by 6am Pacific Time October 7, 2016.

    We are sorry for the inconvenience.
    Tags: None
    • DJOCTO
      Senior Member
      • Apr 2010
      • 2386
      #2
      DejaCloud is back up at 4:30pm Pacific Time October 6, 2016.

        Comment

        • DJOCTO
          Senior Member
          • Apr 2010
          • 2386
          #3
          Here's a report and summary for Friday October 7.

          Yesterday, in the morning, the DejaCloud server was experiencing slow service. We derived that one account had overloaded the server and the queries were slowing it. On rebooting the server the problem resolved for a time and then returned about 2pm. Also around 2pm, the Calendar table became corrupt (was marked corrupt by MySQL). The Calendar is distributed between three tables, so a given user has a 1/3 chance of being in this one. We did a QUICK rebuild which is indexes only. The rebuild brought the whole system down (locked) for 90 minutes. It was successful and services were restored. The Server appeared to be slow for another 60 minutes or so as accounts reconnected. After that it has been fast.

          We will continue to monitor the speed. We will be clearing old accounts and we may divide the calendar further to distribute the load. None of these activities should affect downtime. DejaCloud has been running continuously with only minor outages for five years. The 90 minute outage yesterday was a reminder to keep our tables clean and spiffy.

            Comment

            • comptuter
              Senior Member
              • Dec 2012
              • 116
              #4
              ny changes I have made on my phone since yesterday do not show up on my tablet or 2nd phone. They are not in DJC either. I made two calendar changes on Wed and made a change today. However phone 2 and tablet ARE syncing. On phone 1 I have tried re-reading DJC no change, I tried re-rereading handheld no change. The problem I guess is with phone 1 not with DJC. I made no changes to phone 1 before Wed when it last synced successfully. Any help would be appreaciated
              Galaxy Note 8 GT-N5110 3.1.2 Android 4.4.2
              Galaxy Tab S2 Android 6.0.1 DJO 4.3.1
              Galaxy Note 10+ SM-N975U Android 9 DUO 4.4.29 (1186)

                Comment

                • comptuter
                  Senior Member
                  • Dec 2012
                  • 116
                  #5
                  Apparently on my new phone 1, one way sync somehow got enabled on Wed, with two way sync the problem seems to be resolved.
                  Galaxy Note 8 GT-N5110 3.1.2 Android 4.4.2
                  Galaxy Tab S2 Android 6.0.1 DJO 4.3.1
                  Galaxy Note 10+ SM-N975U Android 9 DUO 4.4.29 (1186)

                    Comment

                    Previous template Next
                    Working...