For those that don’t already use Last Pass! Check out this great site!

For those that don’t already use Last Pass! Check out this great site!

troll

logwatch not sending emails via cron.d script

I ran into an issue recently when trying to configure logwatch to send daily emails for my postfix installs via a script in the /etc/cron.d directory. The same applies for user crontabs.

Here is a nice howto for installing logwatch itself and creating the crontab if you don’t know how to do that.

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-logwatch-log-analyzer-and-reporter-on-a-vps

What this tutorial doesn’t say though (and I should really have known this :P) was that the default mailer for logwatch is /usr/sbin/sendmail and /usr/sbin is not accessible by the cron process unless you extend the PATH variable at the top of your script.

Here is the example. The important bit is in bold.

PATH=/usr/sbin:/usr/bin:/sbin:/bin

# +---------------- minute (0 - 59)
# |  +------------- hour (0 - 23)
# |  |  +---------- day of month (1 - 31)
# |  |  |  +------- month (1 - 12)
# |  |  |  |  +---- day of week (0 - 6) (Sunday=0 or 7)
# |  |  |  |  |
# *  *  *  *  *   

55 11,23 * * * root logwatch --detail High --service postfix --range Today --mailto someone@email.com

Install Vertica 7.1.x with Management Console on Ubuntu 14.04 LTS

Follow this tutorial to install HP’s Vertica version 7.1.x on your Ubuntu 14.04 LTS server / desktop.

Visit HP’s my.vertica.com site and download the .deb package for the server

  1. Visit this URL : https://my.vertica.com/download-community-edition/#server
  2. Click on the link for “Debian Linux 6 and Ubuntu 14.04 LTS”.
  3. Agree with the Terms and Conditions and download the .deb file to your Ubuntu machine

Install the package with dpkg

On the command line of the new server type the following..

root@ubuntu:~# dpkg -i vertica_7.1.1-0_amd64.deb

Configure the OS

Install Some Required Packages

root@ubuntu:~# apt-get install mcelog pstack sysstat ntp

Edit the /home/dbadmin/.bashrc

I live in Brisbane Australia so I will make this my default time zone for the dbadmin user created during install. Just add the following line to the end of /home/dbadmin/.bashrc to set the time zone..

TZ='Australia/Brisbane'; export TZ;

Run the blockdev command

We need to increase the hard-disk read buffer size higher than the default for Ubuntu. On the virtual machine I am installing this on there is only one ext4 partition (/dev/sda1) we need to modify the read buffer for. Run the following command substituting the correct hard disk for the appropriate one in your setup.

root@ubuntu:~# /sbin/blockdev --setra 2048 /dev/sda1

Edit the /etc/rc.local and add blockdev option for reboot

Add the following to the line above the line that reads exit 0 like so…

#!/bin/sh -e
 #
 # rc.local
 #
 # This script is executed at the end of each multiuser runlevel.
 # Make sure that the script will "exit 0" on success or any other
 # value on error.
 #
 # In order to enable or disable this script just change the execution
 # bits.
 #
 # By default this script does nothing.
 /sbin/blockdev –setra 2048 /dev/sda
 exit 0

Install sysv-rc-conf to check NTP client runlevel

The NTP client is needed for Vertica and we should confirm that this client starts at runlevels > 1 by installing the sysv-rc-conf package

root@ubuntu:~# apt-get install sysv-rc-conf

Run the following command to start the sysv-rc-conf interface.

root@ubuntu:~# sysv-rc-conf

You should see a grid interface showing you which runlevels the various daemons and client software installed on the machine are set to start on.  We just need to make sure NTP client starts at runlevel 2.

Complete the Vertica Installation

root@ubuntu:/opt/vertica/sbin# ./install_vertica --host localhost

Create the first testdb

dbadmin@ubuntu:/opt/vertica/bin$ ./adminTools -t create_db -d testdb --hosts localhost

You should get something similar to the following output..

Info: no password specified, using none
 Database with 1 or 2 nodes cannot be k-safe and it may lose data if it crashes
 Distributing changes to cluster.
 127.0.0.1 OK [vertica][(7, 1, 1)][000][all]
 Creating database testdb
 Starting bootstrap node v_testdb_node0001 (127.0.0.1)
 Starting nodes: 
 v_testdb_node0001 (127.0.0.1)
 Starting Vertica on all nodes. Please wait, databases with large catalogs may take a while to initialize.
 Node Status: v_testdb_node0001: (INITIALIZING) 
 Node Status: v_testdb_node0001: (INITIALIZING) 
 Node Status: v_testdb_node0001: (INITIALIZING) 
 Node Status: v_testdb_node0001: (UP) 
 Database testdb created successfully.

Visit HP’s my.vertica.com site and download the .deb package for the Management Console

  1. Visit this URL : https://my.vertica.com/download-community-edition/#mc
  2. Click on the link for “Debian Linux 6 and Ubuntu 14.04 LTS”.
  3. Agree with the Terms and Conditions and download the .deb file to your Ubuntu machine (tutorial assumes you are putting this in the “root” directory).

Install the package with dpkg

On the command line of the new server type the following..

root@ubuntu:~# dpkg -i vertica-console_7.1.1-0_amd64.deb

The following should be the output of the above command…

Selecting previously unselected package vertica-console.
(Reading database ... 64059 files and directories currently installed.)
Preparing to unpack vertica-console_7.1.1-0_amd64.deb ...
Starting installation....
Unpacking vertica-console (7.1.1-0) ...
Setting up vertica-console (7.1.1-0) ...
[postinstall] Removing jre 1.6 ...
 Adding system startup for /etc/init.d/vertica-consoled ...
 /etc/rc0.d/K20vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc1.d/K20vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc6.d/K20vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc2.d/S50vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc3.d/S50vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc4.d/S50vertica-consoled -> ../init.d/vertica-consoled
 /etc/rc5.d/S50vertica-consoled -> ../init.d/vertica-consoled
Cleaning up temp folder...
Starting the vertica management console....
root@ubuntu:~# Attempting to load properties from /opt/vconsole/config/console.properties

This will be followed by a bunch of output from the Java spring application that has launched.

Visit the website and configure the console.

https://<server-ip>:5450/webui

Follow the steps in the wizard shown to complete the installation. Once installed you will need to add the localhost cluster in order to see any analytical information for the cluster.

During this process it will ask you at some point to discover the GUID of the dbadmin user created during installation. Typing this on the command line will give you this GUID..

root@ubuntu:~# id -g dbadmin

This tutorial used this blog post as a starting point. Some minor changes were made to keep up with some of the latest packages and directory structure for Ubuntu 14.04 LTS

Postfix Stats using AWStats on an ISPConfig 3, Ubuntu 14.04 Server running Nginx

So I was looking for a quick way to get SMTP statistics for the Postfix install on an ISPConfig 3 server and thought I would share the configuration for anyone else that would want to do that.

After following one of the “Perfect Server” tutorials like this one below for Ubuntu 14.04 you can configure awstats and the cron to generate some SMTP stats from your postfix mail logs.

http://www.howtoforge.com/the-perfect-server-ubuntu-14.04-nginx-bind-mysql-php-postfix-dovecot-and-ispconfig3

1. Copy maillogconvert.pl to the aws directory..

root@server.com:/root/# cp /usr/share/doc/awstats/examples/maillogconvert.pl /usr/share/awstats

2. Create awstats postfix database directory:

root@server.com:/root/# mkdir /var/lib/awstats/postfix

3. Copy the AWStats Config as a template /etc/awstats/awstats.conf to /etc/awstats/awstats.postfix.conf

root@server.com:/root/# cp /etc/awstats/awstats.conf /etc/awstats/awstats.postfix.conf

4. Edit the /etc/awstats/awstats.postfix.conf file and modify the options specified.

root@server.com:/root/# nano /etc/awstats/awstats.postfix.conf


# line 51: change
LogFile="/usr/share/awstats/maillogconvert.pl standard < /var/log/mail.log |" # line 63: change LogType=M # line 122 make it comment and add the line below # LogFormat=1 LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" # line 153 specify hostname SiteDomain="server.com" # line 222: change DirIcons="./icon" # line 831: change like below LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForSearchEnginesDetection=0 LevelForKeywordsDetection=0 LevelForFileTypesDetection=0 LevelForWormsDetection=0 # line 905: change your language Lang="en" # line 949: change like below ShowMonthStats=UHB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowOSStats=0 ShowBrowsersStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1

5. Comment out the include at the bottom of the /etc/awstats/awstats.postfix.conf file.


#-----------------------------------------------------------------------------
# INCLUDES
#-----------------------------------------------------------------------------

# You can include other config files using the directive with the name of the
# config file.
# This is particularly useful for users who have a lot of virtual servers, so
# a lot of config files and want to maintain common values in only one file.
# Note that when a variable is defined both in a config file and in an
# included file, AWStats will use the last value read for parameters that
# contains one value and AWStats will concat all values from both files for
# parameters that are lists of values.
#

# Include "/etc/awstats/awstats.conf.local"

6. Install apache2-utils with apt-get if not already installed.

root@server.com:/root/# apt-get install apache2-utils

7. Create the .htpasswd password db to protect the cgi with basic auth.

Remember the username and password you setup here.

root@server.com:/root/# htpasswd -c /etc/nginx/.htpasswd_postfix

8. Add the location to the ISPConfig NginX configuration : /etc/nginx/sites-enabled/000-ispconfig.vhost

Add this code just before the example for phpmyadmin that should be commented out...


location = /awstats.pl {
root /usr/lib/cgi-bin;
gzip off;
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
# Not needed if already in fastcgi_params.
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd_postfix;
}

9. Do the first run of awstats to generate the database .

root@server.com:/root/# /usr/lib/cgi-bin/awstats.pl -config=postfix -update

10. Add this line to /etc/cron.d/awstats to keep the database up to date every hour.

00 * * * * root /usr/lib/cgi-bin/awstats.pl -config=postfix -update

11. Symlink the "icon" directory to the ISPConfig web root so we can see the icons..

root@server.com:/root/# ln -s /usr/share/awstats/icon/ /usr/local/ispconfig/interface/web/

12. Restart NginX to get the new configuration

root@server.com:/root/# service nginx restart

13. Visit the ISPConfig 3 Web UI

You should now be able to visit your ISPConfig 3 web interface and after authenticating view the parsed log statistics.

An Example URL : http://server.com:8080/awstats.pl?config=postfix

You will be prompted for the username and password you set up earlier to view the statistics.

Installing Redmine 2.1.x on Ubuntu 12.04 for an ISPConfig 3 Client

This tutorial explains how to install Redmine 2.1.x for one of your clients in an ISPConfig 3 server setup running on Ubuntu 12.04 Precise Pangolin. It does not explain how to install ISPConfig 3 on Ubuntu 12.04 Precise Pangolin. An excellent tutorial on the steps involved to do that can be found here.

1. Create the Client account in ISPConfig and a new Site

Once you get ISPConfig running correctly you should log in and setup your “Client” account in ISPConfig and then create a new “Site” to run the Redmine installation for this client.

2. Install the .deb packages to support Redmine 2.1.x

Next we start by installing the required ruby / rails and apache2 modules needed to run Redmine 2.1.x. These instructions assume you have root ssh access to the server but they could easily be run by a user with the correct sudo privileges.

# apt-get install ruby-full libmysqlclient-dev libmagickwand-dev libapache2-mod-passenger

3. Download Redmine 2.1.x

Navigate to the directory for the new “Site” that was setup. You should of course change client1 and web1 for the appropriate client number in your ISPConfig 3 setup.

# cd /var/www/clients/client1/web1/

At the the time of this writing the latest Redmine 2.1.x release was at 2.1.2 so we download the tar/gz from the ruby forge website, unzip it and make it the “web” directory of your new Site.

# wget http://rubyforge.org/frs/download.php/76495/redmine-2.1.2.tar.gz
# gunzip redmine-2.1.2.tar.gz
# tar -xf redmine-2.1.2.tar
# mv web web_orig
# mv redmine-2.1.2 web

4. Create the Database for the Redmine installation

Now you should create a new mysql database for your client’s Redmine installation in ISPConfig 3. Remember the username and password you setup for this database as you will need this for the next step when you change the Redmine database configuration.

5. Copy the database configuration example file to a new configuration file

# cp ./web/config/database.yml.example config/database.yml

6. Edit the new database configuration file and change the following values for production: to the database you have setup in ISPConfig 3.

production:
adapter: mysql
database: c1_redmine
host: localhost
username: c1_redmine
password: <enter-the-password>

6. Now we need to ensure that we have installed rake and the ruby bundler package.

# apt-get install rake ruby-bundler

7. Now go to the config directory and run the following..

# cd /var/www/clients/client1/web1/web/config/
# bundle install --without development test postgresql sqlite

This should out put something similar to the following…

Fetching source index for http://rubygems.org/
Installing rake (0.9.2.2)
Installing i18n (0.6.1)
Installing multi_json (1.3.6)
Installing activesupport (3.2.8)
Installing builder (3.0.0)
Installing activemodel (3.2.8)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.1)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.1.3)
Installing actionpack (3.2.8)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.10)
Installing mail (2.4.4)
Installing actionmailer (3.2.8)
Installing arel (3.0.2)
Installing tzinfo (0.3.33)
Installing activerecord (3.2.8)
Installing activeresource (3.2.8)
Using bundler (1.0.15)
Installing coderay (1.0.7)
Installing fastercsv (1.5.5)
Installing rack-ssl (1.3.2)
Installing json (1.7.5) with native extensions
Installing rdoc (3.12)
Installing thor (0.16.0)
Installing railties (3.2.8)
Installing jquery-rails (2.0.3)
Installing mysql (2.8.1) with native extensions
Installing net-ldap (0.3.1)
Installing ruby-openid (2.1.8)
Installing rack-openid (1.3.1)
Installing rails (3.2.8)
Installing rmagick (2.13.1) with native extensions
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.

Make sure the above line is show before continuing…

8. Create the secret token.

# rake generate_secret_token

9. Initialise the Redmine Installation

# RAILS_ENV=production rake db:migrate
# RAILS_ENV=production rake redmine:load_default_data

You should see the output from creating the database and also a message stating that the default data has been loaded.

10. Make sure that passenger is enabled for apache2

# a2enmod passenger

11. Create the appropriate permissions for some directories in the Redmine setup.

# chown web1:client /var/www/clients/client1/web1/web/ -R
# chmod 755 /var/www/clients/client1/web1/web/
# mkdir /var/www/clients/client1/web1/web/public/plugin_assets
# chmod -R 777 /var/www/clients/client1/web1/web/public/plugin_assets
# chmod -R 777 redmine/files

12. Override the default DocumentRooT for your Site

You do this by adding the following to the Apache Directives section of your ISPConfig Site configuration.

DocumentRoot /var/www/clients/client1/web1/web/public/

When ISPConfig’s cron update runs and apache2 restarts you should be able to go to the URL for the Site you have setup and you should see your Redmine installation.

Hope this helps those wanting to run Redmine for their ISPConfig clients 😀 Of course keeping this up to date is another story 😉


This tutorial uses the following articles as as a references…

Moving an app from CakePHP 1.3 to CakePHP 2.0 … quite painless actually

Over the past year and a bit I have been working with Monash University to develop a web based learning tool for their School of Pharmacy. The project is called “MyDispense” and after some initial twists and turns working out which tools we were going to use to deliver the job, our team eventually settled on the CakePHP Framework (version 1.3) 😀

Eventually the project hit a third development cycle and this coincided with the CakePHP team’s stable release of version 2.0. We were planning some much needed refactoring and D.R.Ying up anyway and naturally wondered if we should move the app to the new version of the framework while we were at it? There were of course some nice new things in CakePHP 2.0 that looked good, not to mention it was considered faster due to the lazy loading of models and removal of php4 support. So we thought what the hell and made the choice to move to CakePHP 2.0. A couple of small gotcha road bumps did crop up due to some differences in Cake 2.0 and 1.3 but they were quite easy to solve as you will see.

Okay so we made the decision but where do we start? Of course we started with some google searches 😛 We looked for tutorials on the web. Some advice from brave peepz that have gone before us and might save us some time and headaches. There wasn’t much out there saying people had trouble so that was a good sign.

Our first port of call was the CakePHP 2.0 Migration Guide, for obvious reasons 😛

I read this page and of course got to the “file names” section and my first thought was … “f#@k we were going to have to modify quite a few folders and file names for this migration”. My second thought.. “erm.. wait a tick… doh! Why don’t we just use the ‘bake’ tool from the Cake Console and bake it all out?” So my colleagues and I stripped the database of unwanted tables and redundant stuff and set to the task of baking out the models, views, admin views and the controllers using the console in Cake 2.0.

One little “gotcha” for me using the bake command in 2.0 was “which directory I should execute the “cake bake” command from?”. After a little bit of head scratching I realised you should do this from the “app” directory as your working directory. I would ssh to my “app” directory on my development setup and type “Console/cake bake”. This produced the familiar interface for selecting what I wanted to “bake” without throwing any errors and of course would put the new files in the correct Model, View and Controller directories in my application.

The next “gotcha” was in regards to the way MyDispense’s dynamic PNG images were displayed in the browser. Some changes to the way that the CakeResponse api works required us to add the “$this->response->send();” call after after setting the PNG image header. In version 1.3 of Cake this wasn’t a requirement.