I recently needed to reboot a server with a huge uptime (unfortunately I didn't write it down) where an old CMS system (OpenCMS version 7.0.x) was running on an old Tomcat (5) on an old OS (Solaris 10) running on old hardware (HP DL 380 G5).
The good news first: The hardware didn't have any issues and booted as it should have.
The bad news: The web-application running on Tomcat didn't start up correctly anymore.
catalina.out showed the following entries in the last lines of the log:
Starting OpenCms, version 7.0.5 in web application "ROOT"
Copyright (c) 2007 Alkacon Software GmbH
OpenCms comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to
redistribute it under certain conditions.
Please see the GNU Lesser General Public Licence for
further details.
--------------------
The following critical error occurred:
Critical error while loading driver manager.
Giving up, unable to start OpenCms.
--------------------
Sep 11, 2013 12:40:38 PM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Sep 11, 2013 12:40:38 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8089
Sep 11, 2013 12:40:38 PM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/16 config=null
Sep 11, 2013 12:40:38 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 5348 ms
According to a mailing list post (for OpenCms 8.0.3 though), such an error message usually has something to do with a database problem. But unfortunately that was all the information I could find - at least in catalina.out.
But after a "find . -mtype 0" showed me that another log file was written: /export/home/tomcat/webapps/ROOT/WEB-INF/logs/opencms.log.
In this log file, the following error was shown, when I restarted Tomcat:
Caused by: java.sql.SQLException: Table './cmsdb/CMS_USERDATA' is marked as crashed and should be repaired
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2985)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3256)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1313)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1448)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:93)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:93)
at org.opencms.db.generic.CmsUserDriver.readUserInfos(CmsUserDriver.java:1449)
... 34 more
Here we have our database error! That makes much more sense now. So I manually checked this table:
mysql> use cmsdb;
mysql> check table CMS_USERDATA;
+--------------------+-------+----------+-----------------------------+
| Table | Op | Msg_type | Msg_text |
+--------------------+-------+----------+-----------------------------+
| cmsdb.CMS_USERDATA | check | warning | Table is marked as crashed |
| cmsdb.CMS_USERDATA | check | warning | 3 clients are using(...) |
| cmsdb.CMS_USERDATA | check | error | Found 142 keys of 143 |
| cmsdb.CMS_USERDATA | check | error | Corrupt |
+--------------------+-------+----------+-----------------------------+
Indeed. Time to repair:
mysql> repair table CMS_USERDATA;
+--------------------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------------+--------+----------+----------+
| cmsdb.CMS_USERDATA | repair | status | OK |
+--------------------+--------+----------+----------+
After that, Tomcat was restarted and OpenCms (and the web-application) was working again.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder