This evening I stumbled across my first web site. I put this together using Publisher 97, probably in the year 1997. It looks rather, uh, dated these days.
While browsing the tree of that site, I also found my brother Oliver’s website, and a site I set up in 1999 about my year 9 class at school.
Frankly I’m amazed it’s still there – I set this site up in webspace provided with a dialup ISP and I terminated my account once we had ADSL – probably in 1999 or so.
I recently had a need to convert 3,000 scanned TIFF images to a more sensible format for distribution, such as JPEG. I’ve been using GIMP to edit the individual photos but sadly it doesn’t have a batch format conversion tool.
So I wrote my own little script in bash. You will need to install ImageMagick (a commandline image editor) on your box first, but this is bundled with most distributions and be installed from a repository.
Things to be aware of when using this script:
It does not recurse into subdirectories – you have to place the script in the same directory that the TIFFs are in
At the time of writing, ImageMagick is single-threaded and can only use one core of a multicore CPU. If you want to use N cores, you have to split the photos into N directories and run N copies of the script. However be aware that, in my case at least, each TIFF was around 45MB and took between 1 and 1.5 seconds to convert. This means the disk is reading at maybe 40MB/s, so running two threads you will quickly hit the limit of your disk.
You can change the compression of the JPG by altering the value 85 in the script below.
You can also convert your TIFFs to other formats. See man convert for details.
Finally, the code you’ve all been waiting for. Feel free to use, edit, distribute this anywhere you please.
#!/bin/bash
total=`ls *.tif | wc -l`
count=0
ls *.tif | while read i
do
file=`basename "$i" .tif`
echo $file
convert "$file.tif" -quality 85 "$file.jpg"
count=`expr $count + 1`
percent=$(($count / $total))
echo $percent% completed, done $count of $total
done
echo All done!
Note: This article is under development. More info and screenshots will be added later. Keep checking back!
I decided to buy a mobile broadband USB modem yesterday. For those of you who don’t know, I work as a wireless network & VPN specialist for the University of Bristol. At the time of writing, there are 507 access points around campus, giving pretty good coverage. However I often have to visit locations with dead wireless to fix it – and it’s times like these that mobile broadband would come in really handy.
After a bit of market research, I found that Three was the cheapest network; however their standard issue USB modem is made by ZTE and some Googling shows that this isn’t well supported by Linux. That’s a showstopper for me. However, O2 offer Huawei modems which, according to the Internet, work out of the box with recent versions of NetworkManager, which is included with distros such as Fedora and Ubuntu.
After a quick word with an O2 sales adviser, I was told that Windows and OS X are supported, and Linux is not only unsupported, but “won’t work”. Of course, I took this with a pinch of salt. I prefer to believe articles on FedoraForum, for example.
I run a number of operating systems on my various computers. It’s not essential to have it working on all of them, but it would be nice. Here’s what I’ve got:
Home desktop PC: Dual boot Fedora 11 with Windows Vista for occasional gaming or audio work
Main laptop: EeePC 901 with Ubuntu Netbook Remix 9.04
Secondary laptop: HP nx7400 dual boot Fedora 11 with Windows XP for occasional stuff (such as using my slide scanner)
Work PC: Fedora 10
Work laptop: Macbook with OS X 10.5 Leopard
So how did O2 Mobile Broadband really fare on these various systems?
Ubuntu Netbook Remix
First I tried it with the computer I am likely to use most on the move – my trusty EeePC. I connected the modem and was immediately greeted with a screen saying it had detected a USB modem.
I was asked to pick a network provider from a short list.
Setup was complete!
I now connect by using the NetworkManager list, the same way as for regular wireless networks. However, even if I connect to O2 Mobile Broadband, it still uses wireless networks in preference if possible. Very sensible!
Using NetworkManager to select a 3G connection
Fedora
Fedora runs a very similar version of NetworkManager to Ubuntu. The main difference is that you don’t get a magic wizard to put the settings in for you. But don’t worry, it’s quite simple.
Right click the NetworkManager icon, click Edit Connections
Under the Mobile Broadband tab, click Add
Do the same on your Ubuntu PC, except click Edit.
Copy the settings over (or look them up from O2).
Done!
Windows XP
Getting it to work on Windows was easy, although I was not impressed by the software. When you plug in the USB modem, it automatically mounts as a virtual CD-ROM which then autoruns the installer. It’s all themed in the O2 colours and looks smart, even if horrible and blue. Installing it is a simple case of clicking Next repeatedly.
But what I didn’t like was the way that the software automatically takes control of your wired and wireless connections too, without consent.
The main screen
In its defence, it does work and it does do the job. It even has nice features, such as being able to prioritise various connections or networks. You can say you want the wired to always take precedence, followed by your home wireless network, then the 3G network, and finally your neighbour’s wireless.
Listing wireless and 3G networks
However, the application is ugly, blue and blobby. It looks like a UFO. It installed yet another tray icon I don’t want, and it’s changed the way I connect to wireless networks. Of course I’m capable of adapting to the change, but I liked the way I connected to networks before. Why force me to change?
One thing I haven’t tested yet is its ability to connect to WPA2-Enterprise (802.1x) networks, which I use daily as part of my job. This is where lots of other third-party wireless applications fall down.
Tonight I will look into ways to remove the app but keep the driver – and perhaps have the 3G connection available to me as an on-demand connection, the same way that Windows handles VPN connections.
Overall, I’m glad I will almost never be using this laptop with my 3G connection.
Mac OS X Leopard
Despite Apple Macs having a reputation for being intuitive and easy to use, installing the USB modem was a nightmare and I still haven’t got it to work.
While Windows and OS X are the two officially supported platforms, the OS X software is just the default Huawei software without any O2 theming at all. O2 haven’t even bothered to bap in the config in advance. Plugging in the USB modem causes a directory to be mounted, containing an installer and a PDF document. The PDF briefly explains how to install the application and fill in the settings. It has a series of screenshots and simple instructions, although still daunting enough compared with the ease by which most Mac applications are installed.
The software itself is ugly and a bit clunky. It doesn’t have anything sensible like a “tray” icon. You have to load the application manually before you want to use the connection. If you frequently use the software, it would be best to make a dock icon, which you also have to do manually.
Worst of all, it doesn’t actually work (for me). I followed the guide exactly and yet when I came to connect, it always fails with a useless error message of “Connection failure!”.
I think I may have stumbled across a security problem in OS X on my Macbook. To recreate it, you need to satisfy the following conditions:
Enable locking the screen after waking from sleep or screensaver
Connect to an 802.1x-authenticated wireless network. Don’t set to remember password.
Shut the lid to put it to sleep
Open the lid to wake it up. There will be a time during which the screen backlight is on, but the screen is displaying plain black before the unlocking password box appears. During some of this time, any keys you press will be sent to the 802.1x authentication window, which is “behind” the black screen, as it also appears upon waking. The timing is hard to get right though.
These screenshots show the two states of the unlock box. Apologies for the quality – can’t take screenshots while locked so had to take a photo
The unlock box - with focus
The unlock box - without focus
After a second, the unlocking password box will appear. If you got your timing right, it will appear without focus. In this case, your keystrokes are still being sent to the 802.1x password box which has focus, despite being invisible. You also have the ability to press Tab to move between fields and Return to submit. This gives you the ability to authenticate someone on a wireless network using any credentials! When they come to unlock their Mac, there will be no obvious indication that they’re authenticated on a wireless network as someone else.
This next screenshot shows what the 802.1x box looks like, after unlocking. As you can see, I’ve entered the word hello through the locked screen!
The 802.1x login box that causes the problem
I spotted this bug accidentally on my Mac when I was a bit hasty typing in my unlock password, and was shocked to see it appear in the 802.1x username box after I unlocked successfully.
I’m using Leopard, 10.5.7. I’d be very interested to hear from anyone who can recreate this bug on other versions of the OS.
I’d be ever more interested to hear from anyone who figures out how to enter keystrokes through a lock screen to an arbitrary application. I’ve tried this, and Control+Tab doesn’t work, so you can’t immediately switch to other windows. I’ve also tried successfully entering my 802.1x credentials through the lock screen to dismiss the 802.1x box, but after that my keystrokes are not accepted.
In this article I will talk you through how to set up automatic backups from your Linux computer, to another Linux/Unix computer of some description. This works for OS X, too. I have tested these instructions on Fedora and Ubuntu.
These backups will use rsync, a handy program for copying only the files that have changed. The first backup will obviously copy all of your stuff, but after then if you only create/change one text file, it only copies one text file.
Your receiving computer (let’s call it the server) needs to be running an SSH server. This is standard on almost every Linux/Unix/OS X computer. If not, consult your OS documentation for guidance on how make it go. Don’t forget you need to poke a hole in your firewall (port 22/tcp) as well as setting up the SSH server.
Setting up key authentication
Your sending computer (the client) needs to be set up with a private/public key pair. This is so it can communicate with the server without having to ask for a password. First check if you have keys by doing the following:
jonathan@hyperion:~$ ls ~/.ssh
If this command results in displaying the files id_rsa and id_rsa.pub then you are all set. Skip ahead to the step about copying the key to the server.. If those files or that directory don’t exist then you’ll need to create them, like so:
jonathan@hyperion:~$ ssh-keygen
Accept the default options and it will create a private/public key pair for you. Now we set permissions on your keys to keep them secure, and to ensure that the key authentication works properly:
Copy the public key (identified by its .pub extension) to the server. Don’t whatever you do copy the private key to any other computer. You should treat the private key as securely as your password Copying is easiest using scp, but you can use a USB memory stick, email attachment or any other method of copying data.
Now you need to log onto the server and tell it to trust your key. Be doubly sure to use a double >>, otherwise you will overwrite the authorized_keys file rather than appending to it.
Your key authentication should now be fully set up. You can test it by connection to the server from the client using ssh – if the key authentication is set up properly you will get automatically logged in without being asked for a password.
The backup script
Consider what you actually want to back up. Most likely just your home directory, i.e. /home/jonathan. Also think about where the backups will be stored on the server. If this is your home directory again, then no problem. If you wanted to store it in a different directory, you would probably need to grant write access to that directory to your user account.
In this article, we will assume that I want to back up /home/jonathan on my client to /media/private/Backup/hyperion on the server. So let’s make a script to do this. Call it backup.sh and save it in your home directory. Change the paths, usernames and server name in this example to suit your setup. You can also use an IP address instead of a server name if you don’t have DNS running on your LAN.
Note: the line starting with rsync is one long line – don’t put a linebreak in Note: this excludes hidden files (ones starting with a dot, such as .test). If this is not what you want, remove --exclude=".*" from the rsync line.
Set it as executable, and run it for the first time
Depending on the size of your home folder, the first run could take ages. I strongly recommend having a gigabit LAN in your home for copying large amounts of data. After it has completed, check on the server that your files have indeed made it across to the place you intended.
Now run it again. It should take only a few seconds to run, as no files have changed since you last ran the script.
Create a text file in your home directory with a few words on it. Run your script again. Check the new file got copied to the server.
Delete the text file from your home directory. Run your script again. Check the the file got deleted from the server.
Setting the script to run regularly
If your server and client are on the same LAN, the server is always on, and it’s a fast LAN, the best option will probably be to set this to run regularly.
For example, my desktop PC is set to sync with my server every hour. They are both on gigabit so even if I’ve got loads of new data to copy it rarely takes longer than a few minutes.
My server then syncs with an offsite server every night at 3am, when I don’t notice if my broadband is running slowly due to the traffic. (Yes, maybe you consider this to be OTT, but I have lots of irreplacable photos and recordings, and had you considered what might happen to my data if I was burgled and both PC and server were taken, or if a fire destroyed my home and its contents?)
For scheduling regular jobs, cron is your friend. The syntax can be a bit odd but if you open up /etc/crontab in an editor, you can add some comments to the start as a reminder of what each field means. When you’re done, add a new entry to the bottom, like I have here.
# +---------------- minute (0 - 59)
# | +------------- hour (0 - 23)
# | | +---------- day of month (1 - 31)
# | | | +------- month (1 - 12)
# | | | | +---- day of week (0-6) (Sun=0 or 7)
# | | | | |
# * * * * * command to be executed
This command tells my script to run at 30 minutes past the hour, every hour of every day of every month. It is vitally important that you run the script as the same user you configured the key authentication for, otherwise it won’t be able to authenticate. Finally fill in the full path to the backup.sh script we prepared earlier.
Setting up an icon in Ubuntu Netbook Remix
If you are running Ubuntu Netbook Remix (UNR), I assume you are also using a netbook. The hourly (or daily, etc) backups described above might not be what you want – you probably don’t have a permanent network. And what if you’re connected via a 3G USB modem, or the slow wireless at friend’s house? Unlikely you’d want to copy several gigabytes of data over such a connection. So the obvious choice here is to put you in charge of when the script runs.
You already know how to run the script manually from the command line, but here we will set up a pretty icon for your desktop. While these instructions are particularly aimed at UNR (because I have it on my EeePC), a similar method will also work in any GNOME environment.
Click on Preferences, and then Main Menu
Under the Favourites category, click New Item
Give it a sensible title, like Backup to server
Fill in the path to your script
Pick your favourite icon
That’s it! Now you can just click the icon to start a backup when you know you’re in a position to make a backup.
For those of you who don’t know, I’m taking part in a weekly photo challenge, namely Stu’s Tuesday Challenge.
This week the challenge is not to take a new photograph, but to submit the best photo we’ve ever taken. I’ve managed to whittle it down to 2 photos, but I can’t decide between them.
So I’ve set up a poll.
Please register your vote over at my photo blog. The poll is in the top corner of the site, and will be open until Monday 22nd June! The winning photo will be submitted on that day. Thanks!
Today at work I had to install a switch in a rack in a riser. The riser goes from top to bottom of the building in question (5 storeys + basement). Access is through an iron door on each floor.
There is a grille to stand on, but it was slightly wobbly and nowhere near as big as the size of the riser, so I had to be careful where I was putting my feet. The building was constructed in the 1920s and the iron grille and brickwork is original.
It’s a long way to the bottom of the basement. While I was working I dropped a bolt. I heard it hit every grille as it went down, and it took about four seconds to reach the bottom.
The riser grille
Looking down the riser
Looking up the riser
This is what I was sent to work on – installing a small new switch in that tiny rack. It’s a Power-over-Ethernet switch to power some new Cisco APs that are going in soon. However, the new switch needed an uplink to the network, provided by…
The patch panel I worked on
… the Cisco switch that was bolted flat to the wall above my head, out of reach. You can see the bundle of cables that join the switch to the patch panel.
In order to reach the switch, I needed to stand on a chair. In order to get a chair to not fall through the grille, I had to go and find some pieces of wood. Yikes.
I recently inherited a box of around 4,000 film slides and was asked to scan them in. I decided that my existing film/slide scanner, a VehoVFS-001, was not up to the task so I investigated alternatives, and eventually settled on a Reflecta DigitDia 5000 – which is simply a rebadged (and cheaper!) Braun Multimag 4000.
First let me show you the specs, pinched directly from the manufacturer’s page:
The consistent advancement of DigitDia 4000 resulted in a substantially reduced scanning speed. More efficient working is now ensured by the faster scanning speed of only 90 seconds at full resolution of 3600 dpi. In addition, the scanner now has an improved and more stable USB 2.0 connection. Scanning is done directly from the slide magazine with a capacity of 100 slides and Digital ICE Technology.
No large-scale and manual inserting of single slides. The easy and fast way for high quality scanning of slides format 24x36mm for archival on CD, projection with digital-projectors, viewing on PC or TV, sending the images by e-mail, printing at home or exposure the images on professional lab on high quality photo-paper. Compatible for CS/Universal/Braun and LKM Magazines.
Digital ICE™ – Hardware-based Dust-and Scratch removal with infrared sensors and automatically Software correction.
DigitalROC™ – Color Restoration; Image Quality will be restored.
DigitalGEM™ – Grain Management; minimize grain to restore the image’s sharpness.
Image Sensor: Linear Array Color CCD
Scanning mode: 48 Bit
Optical Resolution: 3600×3600 dpi
Lamp: Cold Cathode Fluorescent lamp
Scanning preview: 15 seconds
Scanning Speed (ICE off):
ColorScan 1800 dpi 60 seconds
ColorScan 3600 dpi 120 seconds
Max. Scanning Area: 37.5 x 37.5 mm
Dynamic Range: 3.8 Dmax.
Batch Scanning: directly from magazine
PC Interface: USB 2.0
Output connector: 1x USB 2.0
Viewer: Slide Viewer with backlight to examine slide
All sounds promising so far, and the Digital ICE sounds like a definite bonus, as frankly I can’t be bothered to clean 4,000 slides.
I’ve now scanned in over 2,000 of my slides so I feel I can write about the scanner with at least some experience.
Image quality
In general, no problems at all with the image quality. For a 35mm slide, you get an image that’s around 14 megapixels in resolution. Some of the slides I encountered were some other format (sorry, don’t know much about film!) and were square. With these, you get around 10 megapixels.
The colour reproduction isn’t great. The colours are not well saturated and the scans have a distinctly washed-out feel to them. This is no problem if you plan to edit the photos manually later, e.g. using Photoshop or GIMP. The scanner package also includes a feature for colour restoration, DigitalROC, which brings me on to my next point…
DigitalROC is useless. Avoid avoid avoid. In photos that are well exposed and colourful in the first instance, it does an OK job and livens the colours up a little. In photos that are anything more than slightly underexposed, DigitalROC comes crashing in and ruins the photo. It makes parts of the picture look like they are in 16 colours. Unfortunately I didn’t realise this until after I’d done a few hundred slides, and I’ve had to go back and redo them without DigitalROC. In the end I resorted to using GIMP to adjust the colours – I found that the “Auto white balance” tool usually fixed the colours.
The original sample photo below was underexposed but by no means beyond repair with a little bit of gentle tweaking. But here’s what the DigitalROC made of it…
DigitalROC on an underexposed photo
The other software-based enhancement, DigitalGEM is not a lot better. This is supposed to automatically sharpen the image, but I guess the risk with any automatic tool is that there’s no “one size fits all” solution. On some of my photos, DigitalGEM did such a “good” job of sharpening that it cut people’s faces out with hard edges that made them look like cartoon characters.
This next sample shows how DigitalGEM has oversharpened the glasses on the bridge of my great-grandmother’s nose. Perhaps you can’t see in this scaled-down version, and maybe that’s OK; maybe that means DigitalGEM has got away it. But click for the full-resolution version and see how odd it looks. I’m not really happy about this being done to my photos!
DigitalGEM oversharpens
The only enhancement I found to be any good was Digital ICE – the hardware-based technology for removing (well, working around) scratches and dust using infrared light. I haven’t directly compared the same slide with and without Digital ICE, but in the scans that I have done, there isn’t a lot of obvious dust. Sure, it misses larger items like deep scratches and eyelashes, but it seems good at minimising the effect of dust.
The scanner automatically crops the image. This usually works well, although almost all of my scans were rotated by approximately half a degree. Sometimes the auto cropper crops close enough that you can’t tell, other times you get annoying wonky black borders.
Software user interface
The supplied software is rather crude. It is unlike any scanning software I’ve seen before, in that you don’t actually get to see your photos anywhere in the software. In my case, that didn’t bother me. I simply turned off DigitalROC and DigitalGEM, increased the resolution to 3600dpi from its default of 1800dpi (this took some searching) and from then on I simply commanded it to scan trays in batch.
It does not work on Windows Vista, and for you Linux users out there, I couldn’t get it to work on wine either. Maybe with some fighting you could make it go, but it was quicker for me to dig out my old laptop with Windows XP. An Apple OS X version of the software is included, so I can’t vouch for it and I don’t know if it’s any different.
Auto-loading mechanism
For me, this was the main reason for buying such an expensive scanner, and the main advantage over the Veho. In reality, it has been a bit of a disappointment.
First off, Reflecta provide a 100-slide tray with the scanner. Great, I thought, and immediately loaded 100 slides into it. But it quickly became clear that this tray was not well designed and does not hold the slides straight (only up on edge, not along the bottom). If the slides are not straight, they can’t slide neatly into the scanner – and worse yet, as the slides are only held upright along one edge, as soon as they start to slide into the scanner, away from the support, they immediately fall over. Useless!
Luckily, I had a 50-slide tray that came bundled with the slides I inherited. This holds the slides and supports them all the way as they slide in and out. But I still had difficulty with the reliability of the loading mechanism. The first video shows how it should work, and the second shows what it does when it fails.
The most reliable slides in my experience are the fairly thick plastic-framed ones with rounded corners. The mechanism seems fairly able to capture them properly and slide them into the scanner. The round corners means they don’t snag on the “track” they slide along.
The "Goldilocks" slide - just thick enough
I also had some rather thicker slides, with the film sandwiched between two pieces of glass. These had square corners. They usually load into the scanner fine, but upon trying to unload them they often jam and have to be manually removed. This is an unrecoverable jam and so your batch job stops. Pretty annoying if this is in the first few of the tray!
A thick glass FerraniaColor slide
The vast majority of my slides are “standard” thickness and either plastic with round corners, or card with square corners. These almost never get loaded properly, as the arm that pokes the slides into the scanner misses the slide completely, and you get a blank scan. So if you’re scanning thinner slides, you do really need to sit with the scanner and help it along when it changes over. Pretty annoying, as each scan takes almost two minutes, so a tray of 50 keeps you tied to your desk for an hour and a half!
A thin card slide
I’ve managed with this scanner by saving my few thick, plastic slides for unattended overnight jobs, and scanning the thin ones whenever I have time to supervise the scanner.
Verdict
Very good image quality if you disable DigitalROC and DigitalGEM, and are prepared to play with the white balance and/or saturation yourself
The autoloader is not very good at all unless you have a specific type of slide, and defeats the object of having a batch scanner.
It is rather noisy – not only when changing slides but also when scanning slides. Comparable to an inkjet printer, perhaps.
Software is rather crude, but does its job. You’ll be editing your photos in a third-party application anyway so who cares!
It has still saved me significant labour compared with scanning 4,000 slides in the Veho!
I ran a speed test from Namesco on my PC at work. It has a gigabit Ethernet connection but more importantly, it is served by JANET – the high speed academic network.
As you see in the screenshot, it achieved 215 Mb/s download!
The best Internet connection in the world? I think so.
Loading ...