Re: Filesystem becomes readonly.

From: "Daniel Jitnah" <djitnah@greenwareit.com.au>
1.Linux (Debian Wheezy) running on a cloud server 2.Everything working fine for weeks/months 3.Last nite, for some reason all disk writes to server stops 4.Even root cannot write to anywhere on the disk, and plenty of space left on disk (~80%) All file permissions look OK.
5.After reboot all return to normal.(ie problem is fixed)
My questions:
Is it fair to assume that the problem was caused by something that the cloud provider has done? Or could it be something on server OS side?
What can cause this?
(I am thinking the virtual disk hosting the VM has become readonly somehow, but how? )
I had this in the past, with a VirtualBox VM (Ubuntu Server) on FreeBSD ZFS. The performance of the ZFS became very bad - resulting in disk write timeouts in the Ubuntu VM. The Ubuntu server assumed a faulty disk and remounted the disk as read-only to prevent further damage. Although you are very unlikely to have FreeBSD/ZFS underneath there are other ways to "produce" high I/O latency. If so you should find these disk I/O related errors in your kernel logs. I am afraid there is not much you can do (if you do not run the host system). It only needs a short period with this problem and your system becomes unresponsive. Monitoring.. I did not do that for the read/only mount but all services running on that VM turned red on the Icinga webpage. Regards Peter
participants (1)
-
Peter Ross