DisasterMan's [pre]posterous

Plesk/Postfix Forwarders: Some mail does not get forwarded

Symptom:

Emails are not forwarded and postmaster receives messages like:

Subject: Postfix SMTP server: errors from unknown[xxx.xxx.xxx.xxx]

Transcript of session follows.

 Out: 220 {mail server} ESMTP Postfix
 In:  EHLO {senders address}
 Out: 250-{mail server}
 Out: 250-PIPELINING
 Out: 250-SIZE 10240000
 Out: 250-VRFY
 Out: 250-ETRN
 Out: 250-STARTTLS
 Out: 250-AUTH DIGEST-MD5 PLAIN LOGIN CRAM-MD5
 Out: 250-ENHANCEDSTATUSCODES
 Out: 250-8BITMIME
 Out: 250 DSN
 In:  MAIL FROM: <{senders email}> 
 Out: 250 2.1.0 Ok
 In:  RCPT TO: <{recipient address}>
 Out: 250 2.1.5 Ok
 In:  DATA
 Out: 354 End data with <CR><LF>.<CR><LF>
 Out: 451 4.3.0 Error: queue file write error
 In:  QUIT
 Out: 221 2.0.0 Bye

Cause:

If you create a mail account without a mailbox and only a forwarder, it seems unnecessary to create a password for the account. Sadly this is not the case!

Solution:

Set a password for the account under Plesk > Domain > {domain} > Mail Accounts > {mail account} > Preferences

*nix - Read .db files

Incredibly simple tip.

Convert the file with makemap then view with your preferred text editor:

# makemap hash access.db < access
# vi access

To rehash the file after editing:

# postmap virtual

If you are making major changes to the file and aren't 100% sure of what you are doing, BACK UP THE ORIGINAL FIRST!

Resolving Canonical domains - Death to www!

Many moons ago, one of my webmastery guru's told me that he considered www to be antiquated bullshit in a domain name - a waste of time and space. I agreed with his logic, and have supported the cause ever since.

The issue arose today when a client noted an issue with his Joomla sites - if you login on http://domain.tld, then click on a link that takes you to http://www.domain.tld, you will no longer be logged in, as the cookies are set for different domain names. To the casual user with or without www is the same thing, but as far as t'internet is concerned, www.domain.tld is actually a SUBDOMAIN of domain.tld.

It is also an issue for SEO and indexing, because as far as search engines are concerned, the 'two' sites are counted differently, meaning that links to one or t'other do not count towards a single total for page rank. It also means that the 'two' are considered duplicate content, reducing the perceived value of your data, especially as they both share an IP address so it looks to the search engines as if you are engaging in blackhat SEO techniques. This was certainly true in the past, and while I would have thought that search engines would be quite beyond such blatant foolishness, it's best to play safe.

Luckily this is very easily cured.

If you ARE using Joomla and are not very technically minded, there is an SEO Canonicalisation Plugin plugin that will sort you out.

Wait, canonicalisation? REALLY? What kind of etymological rape have you people committed there? Can I suggest, I dunno, 'canonifcation' instead? But wait, it's actually a real, technical term! It even has US and Anglicised versions, like a proper grown-up word and everything. Still, correct or not, it's damn ugly, and blatantly coined by an American. However, it does not mean 'to adjust the topography of an object in such a way as to cause it to resemble a big gun'.

Tragic that.

Haaaanyway, back to the point:
I personally would rather have less crap installed in Joomla, and want a solution that is not dependant on it.

.htaccess to the rescue! Feel the power of the rewrite rule!

What fun.

In your .htaccess file, ensure that you have 'RewriteEngine On' and add the necessary RewriteCond and RewriteRule. The rule tells the browser (and search engines) that the change is a 301 redirect, a very healthy way to go about things - html redirects, by contrast, being potentially indicative of blackhat behaviour.

Redirect www.domain to domain in .htaccess

RewriteEngine On
# Redirect http://www.domain.tld requests to http://domain.tld
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ http://%1/$1 [R=301,L]

Bang. And the dirt is gone...

So, whagwandere? Well, RewriteEngine uses Regular Expressions (RegEx) to define the condition and the rule.

The Condition:

The condition says ONLY apply the following rule IF these conditions are met.

First we define variables:
%{HTTP_HOST} - A predifined variable meaning this domain name and tld without www or trailing slash

Then we define the conditions to match:
^www\. -  An http request (which is all .htaccess will process) that starts (^ = start of a match) with the string www followed by a period (\. - .= any character - the period is preceded by a backslash to 'escape' it, meaning ignore any special meaning of the following character)
(.*)$ - Any number (* = any number) of any characters (. = any character). The parentheses creates a group ensuring the asterisk only applies to the preceding period, not a larger string, and also creates a backreference. $ indicates the end of the match.
[NC] -  A flag telling Apache the rule is not case-sensitive

(.*) is an incredibly greedy regular expression, and should normally be avoided, but it is the right thing in this case, as we DO want to match absolutely ANYTHING after the www. and it is safe to use in this situation because we know precisely the nature of the input.

The Rule:

The rule consists of two parts, the match and the replacement. The match is pretty simple:
^(.*)$ - The greediest Regex EVER! It matches anything and everything. As we have already defined our condition, we know we want to replace EVERYTHING. It says start of match(^) followed by any number of any characters ((.*)) before the end of the match ($).
The replacement is slightly more complex:
http:// - string literal.
%1 - the first (in our case only) varable.
/ - string literal.
$1 - Backreference 1 defined by the parentheses in the condition regex.
[R=301,L] - Apache flags - R indicates which http status code to return, in this case 301 (Redirect: Permanently moved). L indicates that Apache should apply no more rules once this rule has bee applied.

What's that? You want to direct domain to www.domain? Nob off. That's not helping the cause. GIYF.

*nix - Format and mount a drive - ext4

Great little guide here:

Skullbox.net | Adding a Hard Drive in Linux

But before you format a drive with the ext3 file system, be aware that ext4 is now available, and if it's good enough for Google...

Computerworld |  The best Linux file system of all?

Ext4 may not be available for mkfs on your system. I don't know about your system, but for CentOS 5, I did the following:

# yum provides "*/mkfs.ext4"

This asks yum to check which available packages provides ext4 support for mkfs:

e4fsprogs-1.41.9-3.el5.x86_64: Utilities for managing the fourth extended (ext4) filesystem
Repo: base
Matched from:
Filename: /sbin/mkfs.ext4

I'll have that then!

# yum install e4fsprogs-1.41.9-3.el5.x86_64

DO NOT rely on this being the best and most up to date package - check yum. If you are concerned about the validity of some repositories, check out the priorities plugin for yum

But now you can format your disk with ext4

mkfs -t ext4 /dev/{drive}

Bonza. Time for a beer...

Plesk backup file location

/var/lib/psa/dumps

More detail:
Parallels Forums: [Info] Server backup repository [new window]

osTicket - Plesk 9 - Postfix - Pipe mail to a program or script

Another day, another dent in the wall. And yet my head remains intact...

I'm currently installing osTicket for my most complex server. With 3 sites (and more in the pipeline), thousands of users and what is normally a complex task list of internal support requests, it's been on my mind that I need to have as support desk system running. Primarily for us to manage the internal support and development tasks, but also so the site administrators can have a more efficient way of dealing with user support requests rather than the somewhat slapdash forum that's in use currently.

So I researched the various open source options out there.

There are a bunch of thing to consider when choosing a support system that I won't go into here, but if you are looking into it, here's a good place to start:
TechRepublic.com | 10 things to consider when choosing a help desk system [new window]

With these considerations in mind, my key requirements were:
Open Source
PHP/MySQL driven
Email & Web interface

The system that I chose then was osTicket as it seems to fulfil these requirements. And their website didn't look like shit. Call me shallow, but I never trust software supplied by someone who has an ugly website with bad UI. There really should be a symbiotic relationship between your ability to create functional software, and present it. If you can't design and code a good site, it's less likely that your software will be any good - especially when it's a web application!

Installation is straight forward and painless - typical web based configuration script. Quite happy with that. Then it gets a little more complex.

[Update] I later notice that osTicket is available for installation via the Plesk Application Vault. How well this works, I don't know. Can't say I trust the Plesk interface do do much at the moment![/Update]

In order for emails to be processed by the software, you need to be able to pipe it to the osTicket script. To do that you need one of two things - cPanel or root access.
cPanel, while limited in certain ways, has some really good features - like the ability to set up email pipes without needing root access. osTicket's wiki even tells you how to do this:
osTicket wiki | Email Piping [new window]

Plesk by contrast can be incredibly powerful, but omits such simple functionality, so a little more knowledge is required. Luckily the process isn't too painful, if you don't balls it up. Plesk uses postfix mail server by default - if you use something different DO NOT follow these instructions. They won't work!
To check if you have Postfix, look for a /etc/postfix folder.
File locations may vary dependant on OS flavour and if you don't use Plesk.
The instructions are provided as is and without warranty of any sort. Y'know - be careful! I knocked out the mail server for an entire night without realising, which is why I'm trying to share the benefit of my errors, but don't blame me if you do worse.
Read the instructions through fully before continuing, and proceed with caution.

You will be creating a pipe command with a unique name. You can name it what you want. I called mine pipeSupportEmail. In the following instructions I use the placeholder {pipeName}. Be sure to use your unique name to replace all instances.

Step 1

SSH in to your server.
Back up main.cf for postfix and then edit it. I use VI.
There should be a copy of main.cf.default, but this probably won't match your working version.

# cp /etc/postfix/main.cf /etc/postfix/main.cf.bak
# vi /etc/postfix/main.cf

Find 'transport_maps'. To do this in vi, hit [esc] to enter command mode and type

/transport_maps

and hit [return].
Note: There is only one incidence of this in the file, but if you ever need to repeat a search in VI, just hit /[return]

You need to add to the line an option 'hash:/var/spool/postfix/plesk/{pipeName}', so after editing it should read:

transport_maps = hash:/var/spool/postfix/plesk/transport, hash:/var/spool/postfix/plesk/{pipeName}

Note: This MUST be all on ONE line
If you have multiple pipes you will enter them all consecutively, so if you had three line might look like this:

transport_maps = hash:/var/spool/postfix/plesk/transport, hash:/var/spool/postfix/plesk/{pipeName}, hash:/var/spool/postfix/plesk/{pipeNameB}, hash:/var/spool/postfix/plesk/{pipeNameC}

Again, this would be all on ONE line.

Once done, save and quit - (in VI [esc]:wq[return])

Step 2

Create a new file /var/spool/postfix/plesk/{pipeName}
This is a transport_map file and will direct all mail sent to your chosen addresses into the pipe (see, the internet really is a series of tubes...)

# vi /var/spool/postfix/plesk/{pipeName}

Add all the email addresses that should be piped to your software in the following format:

{address}@{domain.tld} {pipeName}:{logDescription}
{address}@{domain.tld} {pipeName}:{logDescription}

Where {logDescription} is a note for the mail log. Make this meaningful and unique - then you can easily search the log for it, and distinguish the messages.

So a file might read:

support@company.com pipeSuportEmailt:osTicket support piped
internalsupport@company.com pipeSuportEmail:osTicket internalsupport piped
sales@company.com pipeSuportEmail:osTicket sales piped

Save and close the file.

With these rules in place I can search my mail log later for osTicket to check whether mail is being piped.

Step 3

Hash the file we just created as postfix wants a database file to work with

postmap /var/spool/postfix/plesk/{pipeName}

If you run 'ls /var/spool/postfix/plesk/' you will see there is now a file called {pipeName}.db

Step 4

The last step of coding is to tell postfix which script or program all the email will be piped to, by editing master.cf. Kinda important I suppose...
Oh yeah - BACKUP FIRST!

# cp /etc/postfix/master.cf /etc/postfix/master.cf.bak
# vi /etc/postfix/master.cf

Search for 'plesk_saslauthd', and on the NEXT line, enter the following ALL ON ONE LINE:

{pipeName} unix - n n - - pipe flags=Fq user=apache:apache argv={pathToPipe} ${sender}

{pathToPipe} is the path to the script you want to pipe to. In the case of osTicket this will probably be /var/www/vhosts/{domain}/httpdocs/{osTicketLocation}/api/pipe.php
Again, {domain} and {osTicketLocation} will depend on your installation. 
IMPORTANT: {sender} is NOT to be replaced - it is the script variable ${sender}, being the email address the message was sent from
IMPORTANT: Ensure the are NO spaces after ${sender} or postfix will throw it's toys out of the pram.

If you have multiple pipes to different scripts or programs, enter each one on a new line.

Step 5

Restarting postfix seems like the easiest part of the process, but if you have made a mistake, it doesn't complain much, so first, open a new SSH terminal, and tail your mail logfile so you can see if any errors are thrown:

# tail -f /usr/local/psa/var/log/maillog

This will watch the tail of the file, and you will see activity logged as it happens.
The name and location of your maillog may vary if you are using a different version of Plesk, or not using Plesk at all.

Now you can restart postfix

# postfix reload

If it complains that postfix is not running, you can try 'postfix start' instead, but if postfix isn't running, you have another problem with you mail server, and it might not start - look to the mail log for information on what's going wrong.

These instructions based on Nexology Community | Creating Email Pipe with Plesk 9.2.2 [new window]

[Update]Oh dear, it doesn't seem to work yet.

From the maillog:

Oct  1 14:10:39 serverXXX-XX pipe[9594]: fatal: pipe_command: execvp /var/www/vhosts/{domain}.com/subdomains/support/httpdocs/api/pipe.php: Permission denied
Oct  1 14:10:39 serverXXX-XX postfix/pipe[9088]: EF2541117B5: to=<support@{domain}.com>, relay=pipeSupportEmails, delay=3.5, delays=3.4/0/0/0.02, dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: pipe_command: execvp /var/www/vhosts/{domain}.com/subdomains/support/httpdocs/api/pipe.php: Permission denied )

This is not an uncommon error, but none of the suggested solutions I have found online yet resolve it. I will update when I have more idea.
It would seem (perhaps obviously) to just be a matter of setting the permissions correctly, but I've clearly not worked out just how they should be set.[/Update]

Piping with qmail

If your setup uses qmail rather than postfix, these instructions may help. BE WARNED - I have not tested this:

Do this for each address that you wish to be piped to osTicket

# vi /var/qmail/mailnames/yourdomain.com/{address@domain.com}/.qmail

Where {address@domain.com} is the address you are receiving mail to - e.g. support@somecompany.com

Add the following as the FIRST line of the file:

| php -q /var/www/vhosts/{domain}/httpdocs/{osTicketLocation}/api/pipe.php

Save and quit.
Restart qmail

# service qmail restart

Hmm. I'm jealous now... That looks really simple.

[Update 4-10-2010]I've also found this advice on Plesk with qmail:

Plesk, Email Piping/Email Pipe (osTicket / Trellis) | Steve Jameson [new window]

After Plesk update Apache fails - can't get fastcgi file info: dynamic, errno: 2

It seems the Plesk updater when used from the interface can be a bit of a shitter.

Updating from 9.2.2 to 9.2.3 all my sites on the server went down. No biggie, I'll just restart Apache.

Starting http: Syntax error on line 59 of /var/www/vhosts/{domain}/conf/httpd.include:


can't get fastcgi file info: dynamic, errno: 2


[FAILED]

Oh awesome...

Luckily some research revealed that the problem lies in /etc/psa/psa.conf where the php-cgi location gets changed to 'dynamic'

So, to fix it edit /etc/psa/psa.conf and search for the line

CGI_PHP_BIN dynamic

and replace 'dynamic' with the location of php-cgi for your OS flavour.

Examples (CHECK before using!)
CentOS 5: /usr/bin/php-cgi
Suse 10.3: /usr/bin/php-cgi5
Debian: /usr/bin/php5-cgi

So, for my server which is CentOS 5:

CGI_PHP_BIN /usr/bin/php-cgi

# /usr/local/psa/admin/sbin/websrvmng -v -a
# service httpd restart
Starting httpd:                                       [ OK ]
#

Source: http://forum.parallels.com/showthread.php?t=89939

mod_suphp kills your website on Plesk 9

This doesn't only happen in Plesk apparently as I have seen it reported with cPanel.

Essentially, after installing mod_suphp, if your website starts throwing error 404s when calling a file and shows the Plesk default page when visiting the site root, it's probably because the www prefix wasn't properly configured during domain setup. Essentially this is the same problem and solution as my last post, which was handy, cos I already had the solution once I worked out what the hell was going on, so see that post for the answer:

http://blog.absolutedisaster.co.uk/www-prefix-not-working-centos-5-plesk-922

'www' prefix not working - CentOS 5 & Plesk 9.2.2

If you don't tick the 'www' checkbox when creating a domain in Plesk, the CNAME record for www.{domain}.{tld} is not created.

However, simply changing the setting or creating the CNAME record is not enough to get it working - Apache config needs rebuilding.

So to fix the situation, follow these steps:

STEP 1

EITHER:

Change domain settings:
Plesk > Domain > {domain}.{tld} > Domain Administrator Access
Domain name: Check the 'WWW' box
[OK]

OR:

Create CNAME record:

Plesk > Domain > {domain}.{tld} > DNS Settings > Add Record
Record Type: CNAME
Enter domain name: www
Enter IP address: {Main IP address for domain}

[OK] 

STEP 2

Rebuild Apache config:

Login to server via SSH as 'root'

# /usr/local/psa/admin/sbin/websrvmng -v -a

OR

# /usr/local/psa/admin/bin/websrvmng -v -a

DONE

If you had 'www' enabled or the CNAME record in place already the change will be reflected instantly, otherwise you will have to wait for the DNS changes to propogate.

Thanks to GeeksPal for pointing out the config rebuild!
http://www.geekspal.com/plesk/‘www’-prefix-not-working-centos-5-plesk-9-2/

Checking Disk Usage (*nix)

Brazenly lifted from 

http://kb.mediatemple.net/questions/916/Managing+your+disk+usage

View Disk Space from SSH

To check your Total Disk Usage for your entire server via SSH you can type the following:

df -h

The first column displayed is the device. The next three columns show the total size, the amount used, and the amount available. 

Filesystem            Size  Used Avail Use% Mounted on
/dev/vzfs              20G  658M   19G   4% /
simfs                  20G  658M   19G   4% /tmp
simfs                  20G  658M   19G   4% /var/tmp

Another useful command is du. Running this command will list all directories with their filesize from your current directory.

du

A good example of using this command would be to view the sizes of your site's directories:

du -m /var/www/vhosts/mt-example.com/httpdocs

To show the total directory size of the current directory you can run the following command:

du -csh

To show directory sizes as a listing you can use the command:

du -sh *

Searching for Large Files

You can use the following command to search for files over 10MB in size:

find / -type f -size +10000k -exec ls -lh {} \; | awk '{ print $5 ": " $9 }' |sort -n