Thursday, April 12th, 2007
always check the disk free!
Came up against the strangest problem the other day, which in the end made it blatantly clear that the most simple 1st checks should always be done; that is things like disk space etc.
A server I have access too uses LDAP for user info and Kerberos5 for realm authentication. It was reported that this server wasn’t letting anyone login via ssh, and the only way that I was able to login, was via the console connection for the box (so essentially the only way to connect was locally).
I was able to prove that LDAP lookups were working, by simply id’ing on user accounts I knew to not exist locally which were stored in LDAP. I was also able to init a kerberos ticket when logged in, and login as ldap/krb5 users “locally”.
After a while of faffing about, enabling debug logging on sshd and so on, it dawned on me to check the disk space, thanks to an odd I/O moan in the sshd debug log. Low and behold! the partition where the kerberos key cache for ssh was completely full!
It goes to show that even simple checks like that which sometimes seem noddy, should always be done!
Tags: disk, geek