September 11th, 2013 by Lincoln Baxter III

OpenShift Pro-tip – scaling: tail server logs on all gears of your app at once

UPDATE: I’ve filed an issue for this feature request. Until it is completed, you will need to use the command below.

So i’ve recently started working on making http://redoculous.io function better in a cluster, to support scaling out for large-scale use, and I’ve been working with http://openshift.com/ as my PaaS hosting provider. I started to run into some problems where I wasn’t sure if the cluster was responding how I wanted to.

Being a good developer, my application has logging, so I know that if I can just watch the logs as they occur (or sift through them after the fact), I can probably figure out what is wrong, and fix it.

Unfortunately, running rhc tail will tail the gear running HA-proxy, but not the gears running the actual application, or any of the duplicate gears if HA-proxy is sharing a gear with an app! You’ll just get HA-proxy logs or maybe one of your application gears if you are lucky. What to do? I ran crying for help to the #openshift channel on IRC, where the openshift gurus quickly set me straight. This is all you need to do (for the ‘jbosseap’ cartridge, at least):

rhc ssh <app> --gears 'tail -f jbosseap/logs/*';

The rhc ssh command will execute a command on each of your gears simultaneously, and stream the output to your termainal, which is just an absolute perfect match for tail -f. Note, though, that you’ll need to pass in your application name, and the path to your logs on each gear (which should be fairly easy to find out if you ssh in and take a look around. The log location is even documented for your cartridge on the openshift site: ) I retrieved it using the same rhc ssh command:

sharktop:gems lb3$ rhc ssh redoculous --gears 'echo $OPENSHIFT_JBOSSEAP_LOG_DIR';
[gear1name jbosseap-6+haproxy-1.4] /var/lib/openshift/gear1name/jbosseap/logs/
[gear2name jbosseap-6+haproxy-1.4] /var/lib/openshift/gear2name/jbosseap/logs/
[gear3name jbosseap-6+haproxy-1.4] /var/lib/openshift/gear3name/jbosseap/logs/

So in this case, the log-dir relative to the home directory is just, jbosseap/logs. Good to go!


Now, I do think that this would be much better suited as rhc tail-all (DOES NOT CURRENTLY WORK), but I’ll just have to file an issue request, or maybe take a stab at the code here if nobody beats me to it:

https://github.com/openshift/rhc/blob/master/lib/rhc/commands/tail.rb#L18
https://github.com/openshift/rhc/blob/master/lib/rhc/commands/ssh.rb#L37

I hope this helps. Good luck, openshifters!