Virtualization, Cloud, Infrastructure and all that stuff in-between
My ramblings on the stuff that holds it all together
No Response from vCD Web Interface
Posted by on September 21, 2010
I encountered a problem recently in my vCD lab environment where the cell server wasn’t responding to any HTTP requests following some re-configuration work.
After some investigation I found my Oracle back-end DB server had fallen over (this was because it’s a VM and I un-presented its storage which BSOD’d the OS (caveat:Lab setup!) so I rebooted it and not being an Oracle DBA, it looked like the Oracle services had all started correctly but my cell still wouldn’t initialize.
For reference the /opt/vmware/cloud-director/logs/cell.log file looks like this when it isn’t happy (IP’s changed to protect the innocent – me :));
|
[root@cloud ~]# tail /opt/vmware/cloud-director/logs/cell.log *DEBUG* Running task Update: pid=org.apache.servicemix.features *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.apache.servicemix.features *DEBUG* Running task Fire ConfigurationEvent: pid=org.apache.servicemix.features *DEBUG* Scheduling task Update: pid=org.ops4j.pax.url.mvn *DEBUG* Running task Update: pid=org.ops4j.pax.url.mvn *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.ops4j.pax.url.mvn *DEBUG* Running task Fire ConfigurationEvent: pid=org.ops4j.pax.url.mvn Application startup begins: 9/21/10 9:54 AM Successfully bound network port: 80 on host address: 192.168.xx.241 Successfully bound network port: 443 on host address: 192.168.xx.241 [root@cloud ~]# service vmware-vcd restart |
The basic test is to check that the cell server can talk to the Oracle DB where the configuration is stored (the cell server is essentially a stateless web-app in the vCD architecture), this goes over port 1521/tcp – so a quick telnet check from the cell server to the back-end DB proved that this wasn’t working
|
[root@cloud bin]# telnet mgt-db01.v0id.ads 1521 |
When looking at my Oracle server, (which is on Windows in my lab (sorry!)) the OracleOraDB11g_home1TNSListener service didn’t start up correctly and wasn’t running.
I did a manual start of this service, then restarted the vmware-vcd service on my cell server
|
[root@cloud bin]# service vmware-vcd start |
and then checked the cell.log file, this time I saw more progress until it started correctly (successful initialization shown below)
|
[root@cloud bin]# cd /opt/vmware/cloud-director/logs/ [root@cloud logs]# cat cell.log *DEBUG* Scheduling task ManagedService Update: pid=org.ops4j.pax.url.mvn *DEBUG* Scheduling task ManagedService Update: pid=org.ops4j.pax.url.wrap *DEBUG* Running task ManagedService Update: pid=org.ops4j.pax.url.mvn *DEBUG* Running task ManagedService Update: pid=org.ops4j.pax.url.wrap *DEBUG* Scheduling task ManagedServiceFactory Update: factoryPid=org.apache.servicemix.kernel.filemonitor.FileMonitor *DEBUG* Running task ManagedServiceFactory Update: factoryPid=org.apache.servicemix.kernel.filemonitor.FileMonitor *DEBUG* Scheduling task Update: pid=org.apache.servicemix.management *DEBUG* Running task Update: pid=org.apache.servicemix.management *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.apache.servicemix.management *DEBUG* Running task Fire ConfigurationEvent: pid=org.apache.servicemix.management *DEBUG* Scheduling task Update: pid=org.apache.servicemix.transaction *DEBUG* Running task Update: pid=org.apache.servicemix.transaction *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.apache.servicemix.transaction *DEBUG* Running task Fire ConfigurationEvent: pid=org.apache.servicemix.transaction *DEBUG* Scheduling task Update: pid=org.apache.servicemix.shell *DEBUG* Running task Update: pid=org.apache.servicemix.shell *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.apache.servicemix.shell *DEBUG* Running task Fire ConfigurationEvent: pid=org.apache.servicemix.shell *DEBUG* Scheduling task Update: pid=org.apache.servicemix.features *DEBUG* Running task Update: pid=org.apache.servicemix.features *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.apache.servicemix.features *DEBUG* Running task Fire ConfigurationEvent: pid=org.apache.servicemix.features *DEBUG* Scheduling task Update: pid=org.ops4j.pax.url.mvn *DEBUG* Running task Update: pid=org.ops4j.pax.url.mvn *DEBUG* Scheduling task Fire ConfigurationEvent: pid=org.ops4j.pax.url.mvn *DEBUG* Running task Fire ConfigurationEvent: pid=org.ops4j.pax.url.mvn Application startup begins: 9/21/10 2:33 PM Successfully bound network port: 80 on host address: 192.168.xx.241 Successfully bound network port: 443 on host address: 192.168.xx.241 Application Initialization: 9% complete. Subsystem ‘com.vmware.vcloud.common.core’ started Successfully connected to database: jdbc:oracle:thin:@mgt-db01.v0id.ads:1521/cloud Successfully bound network port: 443 on host address: 192.168.xx.242 Successfully bound network port: 61616 on host address: 192.168.xx.241 Successfully bound network port: 61613 on host address: 192.168.xx.241 Application Initialization: 18% complete. Subsystem ‘com.vmware.vcloud.common-util’ started Application Initialization: 27% complete. Subsystem ‘com.vmware.vcloud.consoleproxy’ started Application Initialization: 36% complete. Subsystem ‘com.vmware.vcloud.vlsi-core’ started Application Initialization: 45% complete. Subsystem ‘com.vmware.vcloud.vim-proxy’ started Successfully verified transfer spooling area: /opt/vmware/cloud-director/data/transfer Application Initialization: 54% complete. Subsystem ‘com.vmware.vcloud.backend-core’ started Application Initialization: 63% complete. Subsystem ‘com.vmware.vcloud.ui.configuration’ started Application Initialization: 72% complete. Subsystem ‘com.vmware.vcloud.imagetransfer-server’ started Application Initialization: 81% complete. Subsystem ‘com.vmware.vcloud.rest-api-handlers’ started Application Initialization: 90% complete. Subsystem ‘com.vmware.vcloud.jax-rs-servlet’ started Application initialization detailed status report: 90% complete com.vmware.vcloud.backend-core Subsystem Status: [COMPLETE] com.vmware.vcloud.ui.configuration Subsystem Status: [COMPLETE] com.vmware.vcloud.consoleproxy Subsystem Status: [COMPLETE] com.vmware.vcloud.vim-proxy Subsystem Status: [COMPLETE] com.vmware.vcloud.common-util Subsystem Status: [COMPLETE] com.vmware.vcloud.ui-vcloud-webapp Subsystem Status: [WAITING] com.vmware.vcloud.rest-api-handlers Subsystem Status: [COMPLETE] com.vmware.vcloud.common.core Subsystem Status: [COMPLETE] com.vmware.vcloud.vlsi-core Subsystem Status: [COMPLETE] com.vmware.vcloud.jax-rs-servlet Subsystem Status: [COMPLETE] com.vmware.vcloud.imagetransfer-server Subsystem Status: [COMPLETE] Application Initialization: 100% complete. Subsystem ‘com.vmware.vcloud.ui-vcloud-webapp’ started Application Initialization: Complete. Server is ready in 2:35 (minutes:seconds) Successfully initialized ConfigurationService session factory Successfully started scheduler Successfully started remote JMX connector on port 8999 [root@cloud logs]# |
And I could now log in to the web UI of my vCD cell.
Virtualization, Cloud, Infrastructure and all that stuff in-between 