Jonathan's blog

Today I signed up for Twitter – not for personal reasons but because I needed it for work. Of course I’ve heard about it in the past as the leading micro-blogging service, but it hadn’t interested me in the slightest.

I have used Facebook since its early days (when you could only get in if you had an academic email address!), primarily to keep in touch with friends. When they brought out the status updates feature, I couldn’t see the point. Even less so having something like Twitter that’s purely status updates.

There are so many ways these days to get content out there and onto the web. But I can’t see the point in many of the newer sites.

Facebook

Like I said, I use Facebook as a way to keep up with friends whom I don’t see very often. While I was at school, MSN Messenger was all the rage, but now we’re a bit more grown up we don’t all have time to sit on MSN all night. So Facebook is a convenient way to keep in touch from time to time – given that email is only really used for work these days.

But my only friends on Facebook are friends in real life. I don’t meet people through Facebook. For me, it’s just a direct replacement for emailing friends or chatting on MSN. I don’t broadcast my life to the world. I can’t see why they’d care.

Twitter

Which brings me onto Twitter. I can’t imagine that anyone would be interested in snippets of my daily life. If they’re that interested, they can text me and ask. As I mentioned, I now have a Twitter feed and you can follow me if you want – but I don’t recommend it. I’m not intending to write anything interesting – only to use it for following boring feeds like these from the University of Bristol.

Blogs and websites

I’m more interested in personal websites, often in blog format. Maybe it’s because I used the web for years before these social, collaborative sites popped up, and the only resources available were traditional websites.

I’ve had my own website for over a decade now, in one form or another. When I was a kid, I didn’t have much of interest to say and there was nothing on the site. Nowadays I have two blogs: this one, mainly for technical articles, guides, reviews and so on; and my photo blog where I publish photos that I have taken.

Flickr

I’m a geek, and so I have my own server and I run these blogs from scratch using Wordpress. Obviously such an approach isn’t going to work for everyone, which is why I like sites like Flickr. It’s a really easy way to get your work online. I set up my own Flickr page some time ago, before I decided where I was primarily going to host my photos.

As you can see, there’s hardly anything on it and only two comments. I’ve worked a bit harder to promote my official photo blog, which also gives me the freedom to customise it exactly as I want, and here I have had thousands of views of my photos.

In summary

I’m not saying that “Web 2.0” is a bad thing – I’m just saying it only works for me in limited ways.

I want to publish my articles and photos in a more traditional format, and I only use Facebook because most of my mates don’t use MSN any more.

A few months ago I set up a website, Memories of Korea, to showcase some slides I inherited. Naturally I was keen to find out how many visitors I’d had, so I set about finding something that could draw pretty graphs.

Based on my experiences setting up website statistics with AWstats, I’ve now prepared a guide for anyone else wishing to do the same.

This guide assumes you are running either Fedora or CentOS, with Apache httpd web server. The majority of the AWstats config will apply on any distro, and with several different web servers, but paths and installation procedures may vary.

Installing AWstats

First things first, let’s install AWstats. On Fedora:

sudo yum install awstats

On CentOS, you need to jump through a hoop first by enabling the EPEL repository:

sudo rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
sudo yum install awstats

Tweaking your Apache logs

AWstats works by reading your httpd access logs. Somewhere in your httpd.conf you should have a line like this.

CustomLog logs/access_log common

or

CustomLog logs/access_log combined

If you have virtual servers, each server is likely to have its own log file and associated CustomLog directive, too. For now, just pick one set of logs to work with and do the rest later. If your CustomLog directive ends in common, change it to combined. This keeps the log format the same, but causes a couple of extra fields to be logged.

Don’t worry if you can’t change the log format for one reason or another – AWstats will still work but you won’t get quite as many juicy stats.

While you’re nosing around in httpd.conf, make a note of the path and filename of the access log – you’ll need it in a second. In my case it’s /var/log/access_log, which is the default for non-virtual Apache servers.

Go into /etc/awstats. There should be a sample config file called awstats.model.conf. This contains most of the default options you will need, so let’s make a copy of it and work on that. Give the copy the same name as your website

cp awstats.model.conf awstats.www.memoriesofkorea.com.conf
vi awstats.www.memoriesofkorea.com.conf

You don’t need to change many options to get it going, so I will outline the basics here. Find the following directives in the config file, and change their values appropriately. Leave everything else alone – for now!
Tell AWstats where your Apache log file is:

LogFile="/var/log/httpd/access_log"

Leave this as 1 if you are using combined Apache logs. Change it to 4 if you are using common Apache logs.

LogFormat=1

Set this to the main name of your website.

SiteDomain="www.memoriesofkorea.com"

If your website has other names, add them here. Usually the only “other” name is simply omitting the www. Leave in 127.0.0.1 and localhost, which may be important if you access your website from the server it is running on.

HostAliases="memoriesofkorea.com 127.0.0.1 localhost"

Save your changes and exit.

Run AWstats for the first time

When you installed AWstats, it was automatically configured to run and collect log information hourly, but you’re too impatient to wait for cron, run the first AWstats update now:

/usr/share/awstats/tools/awstats_updateall.pl now

Grant access to the AWstats page

You need to edit the file /etc/httpd/conf.d/awstats.conf. This just tells Apache who can view the statistics. Somewhere in the middle of the file there should be a block like the one below. By default only 127.0.0.1 (the web server itself) is allowed to view the page, so if your web browser isn’t running on the server, you will need to change something.

You might want to add a single IP address, an IP address range (e.g. 192.168.0.1/24 for a home network) or simply all to grant access to the world.

    Options None
    AllowOverride None
    Order allow,deny
    Allow from 127.0.0.1
    Allow from 192.168.0.0/24
    Allow from all

Save the file and exit. Restart Apache.

service httpd restart

See your statistics

Assuming all went well, you should be able to view your statistics page at http://www.mysite.com/awstats/awstats.pl

Other options

You no doubt saw in the /etc/awstats.conf file that there are many, many more configurable options for AWstats, including user authentication to name but one.

The config file is well commented and there is plenty of documentation online to help you along.

This evening I stumbled across my first web site. I put this together using Publisher 97, probably in the year 1997. It looks rather, uh, dated these days.

While browsing the tree of that site, I also found my brother Oliver’s website, and a site I set up in 1999 about my year 9 class at school.

Frankly I’m amazed it’s still there – I set this site up in webspace provided with a dialup ISP and I terminated my account once we had ADSL – probably in 1999 or so.

This guide is particularly aimed at novice owners of Linux servers at home, such as the one described in a guide on this blog.

A transparent web proxy caches web content without having to make any changes on the clients on the network. For a proxy to run transparently, it must be running on a Linux server that’s acting as your network gateway/firewall/router. If you have a standalone server on your LAN, you can still set up a non-transparent proxy, but this guide isn’t for you.

To get started, install squid:

[root@zeus ~]# yum install squid

Backup and edit the original squid config:

[root@zeus ~]# cp /etc/squid/squid.conf /etc/squid/squid.conf.old
[root@zeus ~]# vim /etc/squid/squid.conf

Erase all of the original contents and replace them with the blurb below.

The important lines are in bold, and in my case:

• I’ve asked squid to use 50MB of RAM for the cache…
• …and 4096MB (4GB) of disk space. You can change the path if you want to use a different disk or even a memory card for low seek time.
• I also increased the maximum object size from its default small size to a larger size of 40MB. (This is so it can cache updates from the Fedora repository – after the first PC on my LAN has updated, the rest can then fetch the same updates from the local cache at high speed.)

http_port 3128 transparent
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 192.168.0.0/24 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localhost
http_access allow localnet
http_access deny all
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
http_port 3128
hierarchy_stoplist cgi-bin ?
cache_mem 50 MB
maximum_object_size_in_memory 50 KB
cache_dir ufs /var/cache/squid 4096 256 256
maximum_object_size 40000 KB
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern (cgi-bin|\?) 0 0% 0
refresh_pattern . 0 20% 4320
icp_port 3130
coredump_dir /var/spool/squid


With the squid config in place, let’s start the service and set it to run on boot:

[root@zeus ~]# service squid start
[root@zeus ~]# chkconfig --level 2345 squid on

This is only half the problem though. Squid is running now, but no requests are being sent to it. So we need to tweak the firewall config to send passing web traffic through the squid server. Open /etc/sysconfig/iptables for editing and just before the line

-A POSTROUTING -o eth1 -j MASQUERADE

add this line:

-A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Restart your firewall…

[root@zeus ~]# service iptables restart

…and now all web traffic from your clients should be being sent through the transparent proxy.