cover photo

Disroot State

disroot_state@hub.disroot.org

  

Midnight outage

As some of you might have noticed some of the services have been offline for about three hours. This was due to emergency maintenance work at the datacenter where work has been done to power system thus it was required to put affected servers off. However, once the power was back on, one of our servers did not come back online. Reason? Very simple. Our BIOS battery went flat causing the system to stop during post on silly “no keyboard” error and wrong date :( After very refreshing 70km trip to the datacenter in the middle of the night, server got battery replacement and nicely came back online.
 from ActivityPub
  

Nextcloud - End-to-End Encryption - Disabled

Due to the issue in end to end encryption app, causing latest nextcloud client to break (version 2.5) we have to disable it until the issue is resolved. (issue in question) https://github.com/nextcloud/server/issues/12365 We will look into it deeper starting from tomorrow and try to provide more information to nextcloud devs to help them solve the issue.
  

CPU upgrade

Tomorrow evening (Sunday) we are planning to replace Processors (CPUs) in our main server with a bit more powerful ones \o/ This means we need to put the entire server down. Since we are also going to move the old processor to database server, it means things will be offline for the duration of the operation. We will do our best to be as efficient as possible but the downtime is inevitable.
  

Service updates - routine maintenance

Tonight (Amsterdam Local time), starting from 22:30 we will be doing routine monthly maintenance (operating system updates etc). Additionally we will be updating services like Nextcloud, Diaspora, Lufi, Framadate, Etherpad and Ethercalc, and Taiga. So expect short downtimes during which we will be working on upgrades. Note! Nextcloud update, since we want to make a proper backup (just in case) might take about an hour.
  

Service updates - routine maintenance

Tonight (Amsterdam Local time), starting from 22:30 we will be doing routine monthly maintenance (operating system updates etc). Additionally we will be updating services like Nextcloud, Diaspora, Lufi, Framadate, Etherpad and Ethercalc, and Taiga. So expect short downtimes during which we will be working on upgrades. Note! Nextcloud update, since we want to make a proper backup (just in case) might take about an hour.
 from ActivityPub
@disroot_state
Is discourse included in the update?
  

Cloud - Login with email address disabled.

For those of you trying to or using email to login to the cloud. This is now disabled also for all new users. If you happen to use this method to login, you will have to change it to just username
  

Everything back up and running

Looks like we have managed to get the gitchy services back online and fully working. \o/
  

System reboot

In about 30 minutes (10:30 Amsterdam Local Time) we will restart one of the servers. Affected services: - nextcloud - email - xmpp - matrix Downtime shouldnt be longer then 15-20 minutes.
  

Nextcloud upgrade postponed

Due to issues with JABBER/XMPP and Matrix we focused on today, and which are not fully resolved yet, we are postponing the nextcloud upgrade operation.
  

XMPP/Jabber server to server connectivity issues

Seems like we have some connectivity issues with other servers. We still dont know what causes it (maybe the network glitch form the morning), but we are looking into it. So if you experience connection issues with your buddies or rooms on other servers, this is the reason.
  

Nextcloud upgrade

Tonight (Amsterdam Local Time CET) at about midnight, we will commence nextcloud upgrade procedure. This means https://cloud.disroot.org will be down for about an hour. During that time, we will create a backup (just in case things go wrong) and upgrade nextcloud. Of course we will try to be as fast as possible and we hope things will go rather smooth. Nonetheless prepare for short downtime.
  

Nextcloud upgrade

Tonight (Amsterdam Local Time CET) at about midnight, we will commence nextcloud upgrade procedure. This means https://cloud.disroot.org will be down for about an hour. During that time, we will create a backup (just in case things go wrong) and upgrade nextcloud. Of course we will try to be as fast as possible and we hope things will go rather smooth. Nonetheless prepare for short downtime.
  

Nextcloud upgrade

Tonight (Amsterdam Local Time CET) at about midnight, we will commence nextcloud upgrade procedure. This means https://cloud.disroot.org will be down for about an hour. During that time, we will create a backup (just in case things go wrong) and upgrade nextcloud. Of course we will try to be as fast as possible and we hope things will go rather smooth. Nonetheless prepare for short downtime.
  

Nextcloud upgrade

Tonight (Amsterdam Local Time CET) at about midnight, we will commence nextcloud upgrade procedure. This means https://cloud.disroot.org will be down for about an hour. During that time, we will create a backup (just in case things go wrong) and upgrade nextcloud. Of course we will try to be as fast as possible and we hope things will go rather smooth. Nonetheless prepare for short downtime.
  

Outage caused by Emergency maintenance work at the DataCenter

Today due to emergency work in the data-center where our servers are hosted, some of our servers were unreachable. This was caused by faulty data-center network equipment. After few hours of what we believe a "Sunday's engineer nightmare", the issues seem to be solved and everything is back online \o/ We hope engineers at the data-center will have a relaxed Sunday form now on. They for sure deserved it.
  

Outage caused by Emergency maintenance work at the DataCenter

Today due to emergency work in the data-center where our servers are hosted, some of our servers were unreachable. This was caused by faulty data-center network equipment. After few hours of what we believe a "Sunday's engineer nightmare", the issues seem to be solved and everything is back online \o/ We hope engineers at the data-center will have a relaxed Sunday form now on. They for sure deserved it.
  

Outage caused by Emergency maintenance work at the DataCenter

Today due to emergency work in the data-center where our servers are hosted, some of our servers were unreachable. This was caused by faulty data-center network equipment. After few hours of what we believe a "Sunday's engineer nightmare", the issues seem to be solved and everything is back online \o/ We hope engineers at the data-center will have a relaxed Sunday form now on. They for sure deserved it.
  

Outage caused by Emergency maintenance work at the DataCenter

Today due to emergency work in the data-center where our servers are hosted, some of our servers were unreachable. This was caused by faulty data-center network equipment. After few hours of what we believe a "Sunday's engineer nightmare", the issues seem to be solved and everything is back online \o/ We hope engineers at the data-center will have a relaxed Sunday form now on. They for sure deserved it.
  

Hubzilla hiccup

For last few days our hubzilla instance is not very happy. We did not have much time to investigate the root cause yet. Now that the issue of hubzilla being unavailable becomes more frequent (few times a day) we will have a look at it and try to findout whats going on. Sorry for turbulence and thank you for flying disroot.