Image via Wikipedia

Maverick Meerkat is still beta software, so your mileage my vary. Plus, Ubuntu upgrades are like carrying nitroglycerin around. Most of the times if you are careful everything will be ok, then sometimes you step on something, stumble and everything blows up in your face. Why on earth would you want to upgrade then? Because you’re adventurous and you want to live on the edge, right? Upgrade worked for me on my computer. I had a minor problem with Nvidia graphics card which was solved quickly. In any case, if you are happy with Lucid Lynx and you have a stable production environment, then you should probably wait. Final release is scheduled for October 10th and most of the show stopper bugs should be gone by that time.

There are a few things that you need to take care before you upgrade. First you need to install update manager core.

$ sudo apt-get install update-manager-core

Then it would be wise to install all the updates for your current installation.

$ sudo apt-get update

$ sudo apt-get upgrade

Now you are all set and ready to go. A warning, that you should backup your important configuration files and your data, would be appropriate. Consider yourself warned.

$ sudo do-release-upgrade -d

With this command you will start the upgrade process. After upgrade manager calculates on what it has to do you will be notified and you will get a chance to abort the upgrade. This is your last warning. After your start the upgrade you should not interrupt it. If you do that you will end up with a broken Ubuntu and tears in your eyes. Relax, sit back and enjoy watching how packets are installed. Or perhaps, grab a bite.

Nvidia problems

When the upgrade was finished I rebooted and I was greeted with text mode login prompt.

Ubuntu maverick (development branch) thefish tty1

thefish login:

(Yes, my hostname is thefish.) If you do not get a nice login window as you used to then first thing you need to do is to inspect your Xorg log file.

$ less /var/log/Xorg.0.log

If you are running Nvidia and you have a dual monitor setup then you will see this error in your log file.

[    10.038] ================ WARNING WARNING WARNING WARNING ================

[    10.038] This server has a video driver ABI version of 8.0 that this driver does not officially support.  Please check http://www.nvidia.com/ for driver updates or downgrade to an X server with a supported driver ABI.

[    10.038] =================================================================

[    10.038] (EE) NVIDIA: Use the -ignoreABI option to override this check.

[    10.038] (II) UnloadModule: "nvidia"

[    10.038] (II) Unloading /usr/lib/xorg/extra-modules/nvidia_drv.so

[    10.038] (EE) Failed to load module "nvidia" (module requirement mismatch, 0)

[    10.038] (EE) No drivers available.

[    10.038] Fatal server error:

[    10.038] no screens found

Fortunately the fix for this is displayed right there on the screen, you will have to edit your Xorg configuration file and add one more option to the ServerLayout section so that it looks similar to this:

Section "ServerLayout"

    Identifier     "Layout0"

    Screen         0  "Screen0" 0 0

    Screen1        "Screen1" RightOf "Screen0"

    InputDevice    "Keyboard0" "CoreKeyboard"

    InputDevice    "Mouse0" "CorePointer"

    Option         "Xinerama" "0"

    Option         "IgnoreABI" "True"

EndSection

You only need to add IgnoreABI option! Your Xorg configuration file is most likely /etc/X11/xorg.conf. After you made the change restard gdm and Xorg will start and you will see login window. No need to reboot your computer.

$ sudo restard gdm

You are good to go, login and start exploring and trying out new stuff.

New stuff and the problems

First thing I noticed is the default background. What is the deal with those big orange blobs? Nobody knows, but they say that default background will be replaced before final release.

Default background

Windicators are slowly coming into play. When clicking on a sound icon, or better sound windicator, in the panel you will be present with more options than before. Now you can launch Rythmbox from it and when it is running you can also control it from here. One icon less in the panel, which is good. The bad part is that Rythmbox simply quits when you click close button in the main window. It will have to stay open if you want to listen to the music. To make things worse, minimizing will not get rid of the task list icon and you will have to move Rythmbox to another desktop to get rid of the clutter. On Ubuntu forums there is already a debate on this subject.

Sound windicator and Rythmbox

Gwibber got support for lists, inline pictures from TwitPic and alike and direct messages for Twitter and Identi.ca accounts. Also the infamous OAuth¹ is supported, it has to be since Twitter made it mandatory. Gwibber was also upgraded with a new database; say goodbye to the slow couch database and welcome SQLite. This means, more speed and no more syncing between different computers. A reasonable trade-off.

New Gwibber

After that I gave a shot to Shotwell, a new photo manager. I had to wait for quite some time for Shotwell to import my 30.000 pictures library, but I was pleasantly surprised when it imported my older JPEG images and raw Nikon images that were taken more recently. You will be able to adjust few image settings, but for more complex editing you will have to set external editor. Gimp by default. Most of the other software was upgraded, Gnome in this release it carries version number 2.31.92, kernel will be 2.6.35-19 Among other changes I also noticed that Evolution will display emails in threads by default.

In the end, I can say that 10.10 is a next step in the evolution of Ubuntu, few minor changes here and there and Ubuntu community proved once again that they can deliver. Ubuntu 10.10 feels faster than 10.04, but this might only be my feeling. For a proper benchmark you will have to wait for the final release. All in all the first impression is good and now the only thing left is hope that one of the future releases will be named as Horny Hippo.

1. Excellent article about OAuth by Ryan Paul.

Related posts:

1. Ubuntu 10.04 Lucid Lynx – What’s New? ...
2. Upgrade Ubuntu 8.10 to 9.04 ...
3. When you break your windows – Compiz in Ubuntu dies ...

Encoding videos can be a real pain. While ffmpeg is an excellent utility for converting videos between various formats it is also a pain to use. If you put H.264 codec on top of this, you will get a real mess with command lines that are longer than 200 characters. H.264 is a great format and linux¹ implementation is hiding in the libx264. It will come pre-installed with Ubuntu 10.04, but if you are more adventurous you should try and install the latest versions from SVN repositories as described on Ubuntu forums. The whole procedure is quite easy and very well documented so in this post I will assume that you have installed the latest versions.

The focus of this test and tutorial will be on video production, more specifically on an average YouTube Vlogger video. This means, one pass encoding, high compression and very uniform videos. For encoding movies and other videos with very dynamic content and big changes in the scenes you will have to look elsewhere of wait for another post. :)

Test setup

For the test video I used a short clip that was provided by Nixie Pixel, do check her out on YouTube. Video was recorded with Logitech QuickCam Pro 9000 with the resolution 960×720 at 25 frames per second. GStreamer‘s gst-launch utility was used for capturing and the clip was recorded as raw data – video/x-raw-yuv. Audio was captured with an external studio USB microphone and encoded with 1536 kb/s.

The initial file size was 745 MB (780252094 bytes) for a 29.8 seconds of video. All encoding was done on an old Core2Quad Q6600 running at 2.4 GHz with 4GB of RAM. I used -threads 0 parameter with ffmpeg to use all CPU cores while encoding. System load during conversion was around 5.5 and it took from just a few seconds up to two minutes to encode the whole clip.

Latest version of ffmpeg comes with a decent number of preset options for encoding with libx264. I will not go into details about each and every option because that would be a wrist slashing material and quite a long read. However, I will use the presets to simplify the encoding. I decided to use six different presets: hq, veryslow, default, fast, veryfast and lossless_slower. With this I used two different crf² parameters: 15 and 28. They are both maximum and minimum of what is usually recommend. 15 being lowest compression and highest quality/bitrate and 28 being the opposite. Time was measured with the time command and file size was reported by ls -lh command.

Encoding command that I used while working with different -vpre and -crf parameters:

$ ffmpeg -threads 0 -i bw-test.avi -vcodec libx264 -vpre default -crf 15.0 \

  -acodec libmp3lame -ab 64k -ar 48000 -ac 2 -f mp4 out_default_15.mp4

Sound was always encoded with libmp3lame, I know that mp4 files should have AAC encoded sound, but a lot of times audio and video were out of sync after YouTube processed the video. That is why I usually stick with mp3 encoding for sound. For vlog type videos, I encode sound with only 64kbs. Which should be more than enough for human speech.

The results

Results include time needed to perform encoding, file size of the encoded clip and a couple of screen shots. One of Nixie talking and the other of llama walking, so there is at least some motion blur on the video from the llama.

$ ffmpeg -i bw-test.avi -ss 00:00:6.7 -vframes 1 -vcodec png \

  -f image2 sample_original_1.png

$ ffmpeg -i bw-test.avi -ss 00:00:24.10 -vframes 1 -vcodec png \

  -f image2 sample_original_2.png

Here are the two screenshots from the original uncompressed clip, clicking on them will give you a larger version.

Results are in the table below, clicking on time and file size will open both captured frames for the respective test clip.

Complete test video is available on YouTube, I decided to upload the -crf 15 -vpre veryfast version of the video.

The conclusion

End results are a little bit surprising. After numerous tests I decided that I can only show a relevant difference between both extremes; minimum and maximum compression. If you compare the -crf 15 screenshots, they are almost identical. Even when comparing very fast and very slow or lossless presets.

When the value of crf is increased the quality will decrease and when it reaches 28 the change is very obvious. If you use -crf 20 you will get a decent balance between the file size and the quality. Just to be sure, I spent some time on YouTube watching various vlogs and other stuff that people post there. After seeing that, I am pretty sure that you can use the worse settings that I mentioned before. Veryfast preset and -crf 28. Your videos will be encoded in no time and upload will be fast and nobody will notice the loss of quality. ;)

And now the time has come to talk of many things, of shoes and ship and sealing wax, of cabbages and kings and why the sea is boiling hot and whether pigs have wings. :)

1. and other similar operating systems
2. CRF – constant rate factor

Related posts:

1. Webcam and Linux – GStreamer tutorial ...
2. Screencasting your games – Monkey Island 2 ...

Image via Wikipedia

Your Ubuntu Linux installation is getting bigger and bigger and you are running out of space on your hard disk. You got your new hard drive and now you need to clone your old drive to the new one. The dreaded cloning is actually quite easy and relatively simple to do. The only thing you will need is a little patience and some typing skills. No special software just your already installed Ubuntu Linux.

The regular obligatory disclaimer: This procedure worked for me several times, I use it almost on a daily basis and I had no problems with it. Your mileage my vary. Make sure you double check all the commands that you type and that you do not format your data drive. You will end up with two empty disks. If you have any questions feel free to ask.

The very basic scenario is assumed here. You have one hard drive installed as /dev/sda and you want to replace it with a new one. You could just add the new drive and mount it to /home directory for example, but sometimes that is not what you want. I am also assuming that you have SATA drives. This procedure can also be used if you get a new computer and you do not want to bother with new installation. In this case you will have to be more careful if you are moving from one platform to another.

Step 0 – Prerequisites

First make sure that you have all the software installed:

$ sudo apt-get install grub-pc rsync

Yes, that is it.

Step 0.5 – Disk juggling

Turn off your computer, connect your new hard drive and turn your computer back on. Make sure that you do not disconnect the old drive, you still need it. ;)

Step 1 – Partitioning, formatting and mounting

Check where your new drive is, it will usually be accessible as /dev/sdb. Use fdisk to partition the hard drive and then mkfs to format it. If you created more than one partition then you will have to mount all of them. In this example the unknown user created swap, root and home partitions. Here’s how you mount them:

$ sudo mount /dev/sdb2 /mnt

$ sudo mkdir /mnt/home

$ sudo mount /dev/sdb3 /mnt

$ sudo mkswap /dev/sdb1

Step 2 – Cloning

After your new partitions are mounted you need to copy all your data to the new location. You will do this with rsync.

$ sudo rsync --exclude="mnt" --exclude="lost+found" --exclude="sys" --exclude="proc" -avP / /mnt/

$ sudo mkdir /mnt/mnt /mnt/proc /mnt/sys

No need to copy /proc and /sys, they are virtual file systems and they will be created later on. You still need to create the directories that were not copied.

Step 3 – Rescuing the system

Data is transferred and now you need to take care of few other minor details. In /mnt/etc/fstab you will have to change the UUID string for mounting the partitions. I suggest that you replace UUID with /dev/sda2 for root disk and /dev/sda3 for /home. This will vary depending on your configuration. If you are feeling adventurous you can find the UUID’s of your new partitions and replace the old ones with them. Use the blkid command to get the UUID for each partition.

$ sudo blkid -p -o full /dev/sdb2

/dev/sdb2: UUID="1a78c1f3-d9d0-4bbc-8dc2-603c2b46f5fc" VERSION="1.0" TYPE="ext4" USAGE="filesystem"

$ sudo blkid -p -o full /dev/sdb3

/dev/sdb3: UUID="7f5a59ae-43c2-4747-a369-6fc01244ee21" VERSION="1.0" TYPE="ext4" USAGE="filesystem"

Now just copy and paste the UUID into /mnt/etc/fstab.

If you are cloning your disk to be used on a different computer then you should also take care of network interfaces. Delete the last line, or the last two, depends on how many network interfaces you have, in /etc/udev/rules.d/70-persistent-net.rules file. The lines should look roughly like this:

# PCI device 0x8086:0x294c (e1000e)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

I will not go into details here, let’s just say that Ubuntu will remember what names were used for your network interfaces and when you boot into the new system you will end up having eth1 instead of eth0.

Don’t worry, the lines will be back once you reboot your system. After this you can shutdown and disconnect the old drive. Now turn on your computer and boot into a non working system. Have no fear, you will get it fixed in no time.

Step 4 – First boot

Your boot loader will die. Don’t worry it was supposed to, because it doesn’t know what to boot from and where your new disk is. What you see now is a Grub rescue shell. Does it say on the screen that you can use help? Lovely, because you can’t, Grub still didn’t load the help. Now you have to boot your computer manually. Like this:

insmod /boot/grub/linux.mod

linux /vmlinuz root=/dev/sda2 ro

initrd /initrd.img

boot

First you load the Linux module so that you can tell Grub where to find Linux kernel and initial ram disk. Then you can boot. If you have your root partition on /dev/sda2 then you should see your computer rebooting. After you login, open a terminal so that you can fix Grub:

$ sudo update-grub

$ sudo grub-install /dev/sda

$ sudo reboot

Step 5 – Second Reboot & The End

After the reboot everything should be back to normal. If you are again stuck at the Grub rescue prompt, then repeat the above procedure with Grub and rerun update-grub and grub-install. As you can see, cloning disk isn’t not so hard and you don’t need any special tools or software, except your beloved Ubuntu Linux.

Related posts:

1. Losing your password in Ubuntu Linux ...
2. Network boot and install of Ubuntu Linux ...
3. Boot VirtualBox machine from a USB flash drive ...

Menu structure on www.twm-kd.com is not that complicated, we have only two buttons on our front page, three if you are one of the three wise men. We replicated this structure in menu editor, which has matured since the last version we tested.

Three simple buttons and we are basically done. Then we had to change our functions.php file in our theme directory, we simply added one line to the file inside the first php block:

add_theme_support('nav-menus');

Then we had to change the header.php where the menus are loading. Previously the relevant part of header file looked like this:

<div id="menu">

<ul>

	<li class="page_item"><a href="&lt;?php echo get_option('home'); ?&gt;/">Home</a></li>

	<li class="page_item"><a href="&lt;?php echo admin_url();?&gt;">Admin</a></li>

</ul>

</div>

And now, after the change, it looks like this:

<?php wp_nav_menu(); ?>

Yes, we are down to a single line, great isn’t it? Menus did not look like we wanted them, but that was a minor change to the style.css. This is how our menu used to look like with WordPress 2.9.x:

And this is how it looks like now after the change:

Both versions now look the same and the effort was minimal. Experienced developers should be able to make a transition in no time. We did notice one thing, our Admin button was shown only when a user with admin privileges was logged in, menus could also support this. It is not something that would be hard to implement or add, privilege based menus would be nice. We are still unable to add a non functional, label only, items to the menu that would serve as parent items for sub-menus. This could also be achieved if we could include other menus as sub menus.

Related posts:

1. WordPress 3.0 – What’s new ...
2. WordPress plugins you need to survive ...
3. Optimizing WordPress – Tips for Speed ...

The importance of backups? It actually differs from case to case, but it seems that in the end 95% of the people do not know how important backups are until their hard drive dies and their data is gone. In the past ten days I had three people complaining about losing their data. Two hard drive failures and one accidental Shift-Delete & Click Yes without reading the warning message. Backing up on Linux is a pain if you are not at least demigod of Linux administration. Linux really lacks a simple backup solution for regular users.

Read on if you want to know more on how to perform simple backup in Linux, what backups are and how to handle them.

DISCLAIMER: These instructions and scripts were written and tested on Ubuntu Linux Server 9.10, Ubuntu Linux Server 8.04 LTS and Ubuntu Linux Desktop 9.10, but they might not work for you. You will need to install openssh-client, openssh-server and rsync if you did not use them before. Before using these scripts for backup you should test them, we take no responsibility if you backed up something and then lost it.

No, that is not a backup

Raid arrays, software or hardware, are not backup solutions, they are just protection in case of a hardware failure. If your hard disk dies raid array will save you a lot of trouble and a lot of time that you would spend to restore the system from backup. Raid array will never save you from a human failure. If you often read mail really fast¹ then files will be deleted from all the disk drives in the array at the same time and raid array will not provide any protection from this kind of an error. Raid should be backup supplement not a replacement.

Yes, that is a backup

If you have an external hard drive and you copy your important data on it, then you have at least a partial backup. If you have another computer in the house and you copy important data on it every now and then, then you also have a backup. You need to keep in mind that external drives should be disconnected and unplugged after you make the file transfer. Lightning strike can destroy your computer and all the external devices connected to it, so your external drive has to be disconnected from your computer if you want to call it a backup drive.

Hardware failures

We all had them and we all will have them again. In my fifteen years of experience with hardware I think I saw every single brand of hard drive failing. If there are no faults in the production process², then it does not really matter which brand of hard drive you buy. They all die and they all have very similar MTBF rating and I still did not see any relevant statistics that proves one brand is better than the other. Make sure that you check your disks on a regular basis, Linux provides many tools for that.

How to backup

Creating backups can be complicated and difficult to setup. I will not bother you with incremental or differential backups, with compression, daily backups, weekly rotation and other similar stuff that is used in a corporate world where big boys play. We will be creating a couple of shell scripts that will help you backup your data to a remote server.

The scripts and the tools

As we said already, Linux backup solutions are really not worthy of their names and that is why we need to make our own solution. Let us assume that you have a web server with data in a MySQL database and you want to perform daily backups of your web server. We will use a couple of simple crontab scripts that are placed in /etc/cron.daily on your web server and on your backup server. The script below you need to put on your web server or any other remote computer that needs to perform any tasks before you start backing it up.

#!/bin/bash

mv /srv/www/database.sql.bz2 /srv/www/database.sql.bz2.old

/usr/bin/mysqldump --defaults-extra-file=/root/.my.cnf -A --all-databases -u root | /bin/bzip2 > /srv/www/database.sql.bz2

chown www-data:www-data -R /srv/www

find /srv/www -type d -exec chmod g+rwx {} \;

find /srv/www -type f -exec chmod g+rw {} \;

First we move database backup to a new file, overwriting previous old backup. This way we make sure that there is at least one database backup available if sql dump fails for some reason. Then we use mysqldump to dump all the content of all the databases, pipe it through bzip2 to compress them and finally write the output to the file database.sql.bz2 in the /srv/www directory. For a 100 megabyte dump this takes approximately 30 seconds. Password for mysql is stored in the /root/.my.cnf file and it needs to be passed to mysqldump. Password could be supplied in the command line with -p parameter but that would be very insecure. Because a lot of things can happen on the web server during the day we also make sure that all permissions and ownerships are set correctly.

We will call this script PrepBackup, place it in /etc/cron.daily directory and make it executable. In the end the script needs testing, so we execute it.

$ sudo cp PrepBackup /etc/cron.daily/

$ cd /etc/cron.daily

$ sudo chmod 755 PrepBackup

$ sudo ./PrepBackup

Now we need to take care of a timing issue. All Ubuntu servers are executing cron.daily jobs at the same time. We need to run our web server script before we run the actual backup script on the backup server. Cron entries for /etc/cron.* directories are located in /etc/crontab and we will need to edit this file. Look for a line that is similar to this:

25 5	* * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )

Notice the first two numbers on the line. First number represents minutes and the second number represents hours. Simply reduce the number of hours for one and your daily cron scripts will run an hour early. Now that we took care of the web server and we are moving on to the backup server.

Backup is performed with rsync as an unattended process and we need to take care that rsync will be able to login without user intervention. Create an ssh-key for root user on the backup server.

$ sudo ssh-keygen

Default filename should be /root/.ssh/id_rsa and no pass phrase for the key. Now we will need to add public key from the backup server to authorized keys on the web server. We need to do this as user who has access to the /srv/www directory:

$ mkdir ~/.ssh

$ vim ~/.ssh/authorized_keys

Paste the content of the id_rsa.pub file in the authorized_keys file. Pico or Nano or any other editor can be used for editing the file, just make sure that the whole key is one single line of text.

Now onto the backup server, testing if this works.

$ sudo ssh username@your.backup.server.com

If ssh did not ask for a password then we are in business.

The second script that we will need is the one that will actually take care of the backup. We will be backup /srv/www directory on the web server.

#!/bin/bash

HOME=/root

rsync -avpe ssh user@web.server.net:/srv/www/ /srv/Backup/www/

First we need to set the $HOME properly otherwise we will never know what user is running cron job and where his home is. Put this script in /etc/cron.daily on the backup server. It is wise to test everything manually before using it in the production environment.

$ sudo cp PerformBackup /etc/cron.daily/

$ sudo chmod 755 /etc/cron.daily/PerformBackup

$ sudo -s

# cd /etc/cron.daily

# ./PerformBackup

# exit

If both scripts worked then the content of /srv/www on the web server should be on the backup server in /srv/Backup/www directory together with your database dump. Try changing few files on the web server and run the second script again. It will take much less time to run and all the changed files will be backup up.

History?

The downside of this approach is that you don’t have any history of your backups and you can not restore a file that was available three days ago. PerformBackup script could be modified and a simple cp command inserted in front of the rsync.

cp /srv/Backup/www /srv/Backup/www_`date +%d-%M-%Y`

This will take care of copying backup directory to www_ with the date suffix. Crude, slow and disk space inefficient, but it works if you really need that history and do not want to invest some time and/or money in a different, more advanced solution.

1. Using command rm -rf; which removes the target and all its subdirectories.
2. Remember IBM’s DeathStar disk series?

Related posts:

1. Cloning Ubuntu Linux – The easy way ...
2. Linux Tips Aggregated – 1 ...
3. Network boot and install of Ubuntu Linux ...

Image by koka_sexton via Flickr

New major release of WordPress is coming our way. Current release is marked as WordPress 3.0 beta1. We will take a look at what is new and if the new is good or bad. And we will provide screen shots in our quick preview, we promise.

Upgrading WordPress is always something that we don’t take so lightly. A typical website based on WordPress, it could be a blog or not, will have a lot of plugins installed, some widgets and a more or less customized theme, sometimes perhaps even a custom made theme. Upgrading a minor version is a minor pain which means that upgrading a major version will be a major pain. That is why we did this just for you. So that our site will be broken first and you can see where we went wrong. Read on if you want to see how we handled the upgrade.

Of course we are not that silly¹ to play on our main site, so we did it on our test site.

Upgrade was smooth, the usual delete everything except wp-config.php and wp-content/ worked for us. More detailed upgrade step by step instructions would follow these steps:

• Login as Admin
• Disable all plugins
• Delete everything except wp-config.php and wp-content/ in your wordpress directory
• Visit http://yourblog/wp-admin/upgrade.php
• Follow the instructions

Database upgrade completed without any difficulties and after the upgrade we started to turn on our plugins. We turned on all of the plugins that we use except the WP-SuperCache and Ozh’ Admin Drop Down Menu. Heavily modified theme that we use currently on our website didn’t work as it was supposed to, but we didn’t expect it to actually work. We switched to the new default theme which is supposed to demonstrate few of the new Theme API features.

New default theme, Twenty Ten, is simple and comes with few interesting features. First we noticed that Zemanta plugin didn’t render floating descriptions on images correctly, but it isn’t clear if this is plugins fault or new theme breaks something.

Broken description

Other than that we did not encounter any other problems with rendering. Default theme is fast and light and it comes with the built in option to change header image of your blog. Support for threaded comments is also there.

Header Selection

New theme is also packed with placeholders for widgets and because sidebars are so 2009, they are now called widget areas. Which can cause some confusion since you must guess where each area is located.

New widget areas

Theme changer, editor and installer got some love and tabbed interface will make theme installing and switching a little easier. One of the new features is also the new editor-style.css. This style is applied to the built in editor and now editor in visual mode can display what really gets rendered on your screen². The whole admin interface and dashboard were changed a little bit, but not as much as you would expect in a major release. We were hoping for more color schemes, unfortunately we are still stuck with blue and gray.

Another important feature in WordPress 3.0 will be menus. Menus give you the ability to create customized menus for your blog. With beta1 each menu item can contain existing pages, custom links and existing categories. This can, and we hope it will, change in the next beta. We really did miss an option to add a label item to the menu and we managed to kill our test blog by experimenting with the various menus. Right now you will have to create a new dummy custom link if you want group options in sub menus.

Current Menu Editor

Menus will come in handy to those who want to organize their menu based structure on the front page. Now you can drag and drop and create sub menus easily. On the Twenty Ten theme the above menu will look like this:

Rendered Menus

Even in beta1 menus look promising and creating a little more complicated structure is not that difficult compared to previous WordPress versions.

More advanced writers will be happy about ability to add custom post types. Default theme does not have any support for that, but theme developers will be able to add various post types with many different parameters. Custom posts are described at wpengineer.com.

Another big thing in 3.0 release is merge of WordPress and WordPress MU. WordPress MU was used for large sites that were providing WordPress as a service for many, many users. If you host only one blog for yourself then nothing will change for you. If you have multiple users with separate installations, then things will be a bit simpler from now on.

In the end we came to the thing called Canonical Plug-ins. There is this movement that is trying to unify plugin developers and create more dedicated community so that plugins don’t die when a single developer, who was developing it, leaves to chase other things in life. We are keeping our fingers crossed. Really!

All in all WordPress 3.0 looks promising so far. Not too many noticeable bugs and we managed to kill our blog only once when we were playing with the menus. Promised changes and some more work on this feature will most likely deal with these issues. Smooth and steady sailing into the beta2 and then final release that you can jump on.

1. We aren’t?
2. WYSIWYRG – What You See Is What You Really Get

Related posts:

1. Using menus in WordPress 3.0 ...
2. WordPress plugins you need to survive ...
3. Optimizing WordPress – Tips for Speed ...

In a little bit more than a month before the release of Ubuntu 10.04 Lucid Lynx I decided that I want to try how KDE 4.4 behaves and if I am ready to switch from Gnome to KDE again. My heart was always with KDE and Qt but after KDE 3.5 everything went downhill and I switched to Gnome. I really want to try the new KDE, but going to Beta software on my primary workstation is not an option and I will stick with Karmic Koala for a while after the release of Lucid Lynx which will be shipping with KDE 4.4. The only option for me and others like me is to install KDE desktop on top of existing Ubuntu taking it from the backports repository. Here are simple step by step instructions on how to do this.

First you need to add another software source. Clink on the System menu then select Administration sub-menu and run Software Sources. Select the Other Software tab and click Add then copy the following line into the input box.

deb http://ppa.launchpad.net/kubuntu-ppa/backports/ubuntu karmic main

Now select the Updates tab and enable unsuported updates for karmic backports.

Close the Software Sources and open the terminal. It will make things easier, really. Type in the following commands:

$ sudo apt-get update

$ sudo apt-get install kubuntu-desktop

$ sudo apt-get dist-upgrade

You will first perform an update, then you will perform basic install of kubuntu-desktop. Unfortunately some things are so deeply rooted inside the distribution that in the end you will want to perform dist-upgrade. The process will take some time, depending on the speed of your Internet connection and the speed of your computer.

There is no need to reboot after the process is completed. All you need to do is to logout, change your session to KDE and log back in. You should see KDE splash screen and then whole KDE after few moments.

Related posts:

1. Kubuntu 9.10 Karmic Koala preview ...
2. Diving into KDE 4.3 – Survival guide for Ubuntu ...
3. Upgrade Ubuntu 10.04 Lucid Lynx to 10.10 Maverick Meerkat ...

Image by digitalART2 via Flickr

Lucid Lynx is the new release of Ubuntu Linux operating system. We decided that it should be us testing it for you and telling you what’s new and what’s hot. This release of Ubuntu will be marked 10.04 LTS. Ubuntu is released every six months so you are always up to date with your software. LTS versions are released every two years and Canonical will be providing support for three years for every LTS release (five for servers). Let’s see what was simmering for the past six months and is due to release at the end of the April.

Here are the biggest changes that you will notice and care about:

• GNOME 2.29.92 – desktop environment that is already showing its age was again updated, unfortunately nothing major is new in the Gnome camp and for Gnome 3.0 we will have to wait at least six more months if not a whole year. Other minor changes include new default colors and re-arrangement of the title bar buttons. All three of them are on the left side of the windows, Mac OS X style. Something you will need to get used to.
• KDE SC 4.4 – No more backports, the latest and the greatest KDE release is included in Kubuntu spin-off.  KDE Software Compilation 4.4 was released in the beginning of February, barely catching Lucid Lynx feature freeze. More about changes in the official announcement.
• Kernel 2.6.32 – Latest kernel is most of the time a good thing. New drivers added and old drivers fixed.
• Firefox default search engine is now Yahoo. You can still use Google, but Yahoo is now the default. Why? More income for Canonical. ;)
• New graphic drivers – Nvidia users be happy, Lucid Lynx will come with new open source drivers and three different closed source drivers. Ati users will have to stick with the open source driver for now because new fglrx driver does not support X server that is shipped with Lucid Lynx.
• Social networks – MeMenu panel for easy access to chat services and microblogging and new Gwibber microblogging client with support for many social networks. From now on only fire will force you to leave your house.

Installation & First Impression

Installation inside a VirtualBox was much slower compared to the installation of Kubuntu 9.10 Karmic Koala which we performed on the same hardware inside the same VirtualBox. Apart from that, there are no significant differences between Karmic and Lucid installer. Everything looks almost the same and ext4 is still the default file system. If there are any under the hood changes in the installer we did not notice them.

After the reboot, first thing you will notice is the new look. User interface evolved a little, but nothing revolutionary new. Screen locking is now enabled by default. We had to try and see how Gwibber and MeMenu work. After couple of hours we can say that both are still a little awkward to use and they both lack certain features.

MeMenu

MeMenu is an attempt to integrate social networks into the Gnome Desktop. You can add two types of accounts to MeMenu: chat accounts (AIM, Yahoo, Jabber, MSN Messenger, …) and broadcast accounts (Twitter, Flickr, Facebook, FriendFeed, …). From MeMenu you can then change online presence for chat accounts and you can send status updates to your broadcast accounts. Unfortunately the former happens with no visible feedback and you really don’t know if you actually posted anything or not, beside that, you do not know if you are changing your status on chat account or you are broadcasting something¹. Doing both at the same time is something that we will not like.

Another thing that bothered us is the fact that you cannot use MeMenu to start Empathy or Gwibber. It is even worse than that, MeMenu is inactive until you start Empathy by yourself. When you close Empathy MeMenu will still show you as online and you are still able to change your status and receive messages. The problem is that notifications about the new messages pop up as balloons and you cannot click on them² to open chat window and you cannot open chat in MeMenu, you need to run Empathy again.

If you miss the balloon notification you missed the message. There is no indicator that you have messages waiting, again you need to open Empathy to see the blinking icon beside your contact.

Gwibber

Integration of Gwibber and MeMenu could be better. As we said before, there is no indication of what you are doing when you are entering text inside the input field in MeMenu. Gwibber by itself is a completely different story altogether. It is already usable but I would be careful before calling it beta.

It lacks a lot of features, for example sending direct messages on Twitter. You can see all your direct messages, received and sent, but you cannot send new direct messages with a simple click. There is no option to send direct message. If there is, it managed to stay hidden from us.

Beside that, there are no separate settings for retrieving public tweets, @ replies and direct messages. So if you set retrieval time to 1 minute all three types of tweets will be picked up every minute which means that you will reach your API limit in less than an hour³. Preview of images uploaded on twitpic or on other picture posting sites would be nice too. Option to post images that are automatically uploaded to twitpic would be pushing it.

Gwibber is a nice start, but it still needs time to develop and we have to give it time since there are no decent and native Twitter clients for Linux. Adobe Air apps do not count.

Is that it?

Mostly yes, there are not many major changes, which is good, because we are looking at the LTS release. It appears that social networks are becoming a part of our working environments and if half a year ago cell phones started to support them on a larger scale, desktops are now following.

Among other changes it is also worth mentioning that HAL⁴ was thrown out and it was replaced by Devicekit which should result in faster boot. Gimp was removed from the default installation. You  can install it later at any time with Synaptic package manager or with command line utilities. Obligatory change in splash screen and GDM background is also here. The latest release is a step forward into the right direction, but we cannot help in feeling that Ubuntu 10.04 was rushed out at least half a year if not one whole year too soon. MeMenu and Gwibber need some more polishing and few features are still missing. Features that cannot be added after the feature freeze with beta1 release at our doorstep.

During the test we also noticed that VirtualBox’s Guest Additions cannot be installed on Lucid Lynx. Kernel modules fail to compile, but this is probably because of the new kernel and older version of VirtualBox.

1. Sending your new status update to Twitter and other services is now called broadcasting.
2. In fact if you bring your mouse to the balloon it will disappear.
3. Unless something has changed lately.
4. Hardware Abstraction Layer

Related posts:

1. Upgrade Ubuntu 10.04 Lucid Lynx to 10.10 Maverick Meerkat ...
2. Installing KDE 4.4 in Ubuntu 9.10 Karmic Koala ...
3. Diving into KDE 4.3 – Survival guide for Ubuntu ...

Image via Wikipedia

WordPress is crap, but it is the best crap we can get. When your website, with WordPress sitting in the back, starts to pick up traffic, it will slow down and become unresponsive. Adding hardware will help, but correctly implemented load balancing and load distribution across more than one computer can be tricky and expensive. Trying to optimize WordPress and the web server itself usually gives better short-term results. In the long run even this won’t help and it will just postpone the inevitable, you will have to cluster in one way or another. I am not going to talk about clustering and load balancing today, I will focus on optimizing your WordPress. Newbie friendly, yet hardcore, meaning that I will provide explanation for all the complicated stuff.

What is slowing you down?

This is essential question with all optimizations. Figure out what is causing the slowdown! I have worked in IT for all my life and I was called to help with optimizations on all sorts of things. Let it be a local network or some internal database for a service provider, the procedure is always the same. Figuring out bottle necks and getting rid of them.

With WordPress, there is no big secret, PHP is slowing you down¹. The first thing you can do, is to try to localize the bottleneck. It could be the new plug-in you just installed and it turned out that it does not scale as it should. Get rid of it or go around it. As with the most WordPress installations the slowdown will be more global. Everything will be slow, sluggish and unresponsive.

Optimize where it makes sense

Second thing that you need to be careful with. I had the pleasure of working with people who spend weeks on optimizing a function that was perhaps called once every few hours. Don’t go and optimize things that are accessed rarely, for example the ‘My Life Story’ page on your blog. No real need unless it is blocking the site and you are sure that it is the cause of your troubles.

WordPress plugins that will help you

Now, lets take a look at couple of WordPress plugins that will greatly improve responsiveness of your blog.

WP Super Cache – Properly configured Super Cache plugin is a must. Since we know that PHP is slowing us down we need to take it out of the equation. This plugin will generate static HTML files and with a little help of the web server it will serve those static files to non-logged in visitors if they didn’t post any comments on your site. This accounts for the majority of the traffic.

Getting rid of the dynamic stuff is probably the most important thing that you can do and it will also matter the most. PHP is eating a lot of your resources away and compared to this, serving static pages is easier than eating pancakes.

WP Minify – Scripts and style sheets can become quite large and you will probably end up with more than one of each. Minifiying them means erasing all the extra characters that are there for readability. Web browsers usually don’t need them.

Here is an example of ordinary CSS that comes with Sociable plugin and below it is the minified version:

div.sociable { margin: 16px 0; }

span.sociable_tagline { position: relative; }

span.sociable_tagline span { display: none; width: 14em; }

span.sociable_tagline:hover span {

	position: absolute;

	display: block;

	top: -5em;

	background: #ffe;

	border: 1px solid #ccc;

	color: black;

	line-height: 1.25em;

}

div.sociable{margin:16px 0;}

span.sociable_tagline{position:relative;}

span.sociable_tagline span{display:none;width:14em;}

span.sociable_tagline:hover span{position:absolute;display:block;top:-5em;background:#ffe;border:1px solid #ccc;color:black;line-height:1.25em;}

It might not seem much, but CSS files and JavaScript files can grow pretty big. TWM website has CSS that is larger than 12kB, properly minimified is just a bit more than 9kB. Three kilobytes less for server to send and for the client to process.

WP Minify will, before the page is displayed, catch all the style sheet and java script files, glue them together, minifty them and then output them. The good part is that Minify plugin can work together with Super Cache plugin. Minified versions of your files are then cached and sent to the clients.

Make sure you run some tests before you start using it. Minify doesn’t work with slightly hacked Zemanta plugin² and WP-Cumulus plugin³ for the pretty tag cloud. So make sure that it doesn’t make a mess with any of your Java Scripts files or it could in theory mess up your Style Sheets too.

Warning: If you are adding extra CSS or JS files in the configuration interface that need to be minified make sure that text area where you enter file names doesn’t end with an empty line. It will break the plugin.

So much for the WordPress

Not much more can be done in WordPress itself unless you are determined to hack into it. You could do that, but then you will have to do it each time you upgrade WordPress. And you will want to upgrade every now and then. Now, comes the hard part, optimizing general page performance. We did great with minifying everything and serving it as static content and this will matter the most on the server side.

The question now is where else can we gain some speed? Where are we losing precious CPU cycles and sometimes even more precious bandwidth? Maybe if we try to minimize number of requests to the web server itself?

Optimize your theme

Get rid of all the theme images if you can. That means, not getting rid of them entirely, but put them in a single static bitmap and then use it as CSS sprite. This will greatly reduce the number of requests on your web server. The whole procedure is too long to describe here, but Google will return plenty hits on CSS sprites. The theme you see here had more than twenty images and we were able to merge all of them in to a sprite. Now CSS is taking care of displaying the correct part of the image. Client has to make only one single request to download the bitmap. In our case the bitmap was also smaller than all of the images together. Win – Win!

Keep your redirects at minimum

Redirects are a convenient way to point the client in a different direction. If you change the permalink of a blog post, WordPress remembers that. Them it performs a 301 redirect to the new address if someone tries to access the old one. 301 is an HTTP response code that is followed by a new URL and tells the client to go look to the new address for content. It is also called a ‘search engine friendly’ redirect. This is because search engines will most of the time remember that the location was moved permanently and they will use the new address.

Because of this, each redirected page will generate two hits on your web server. Avoid redirects and use them only when necessary.

Use trailing slashes

Make sure that you are using trailing slashes. When you are referencing something on your blog, a category for example, the URL should have a trailing slash. If it is lacking one Apache web server will automatically create a 301 redirect and add a trailing slash. And we learned above that redirects are not good, especially not the ones that are really unnecessary.

Right:

http://www.twm-kd.com/category/computers/software/

Wrong:

http://www.twm-kd.com/category/computers/software

Enable compression

Compressing plain text content can reduce payload up to 70% or even more and this saves precious bandwidth. Keep in mind that compressing small files, smaller than couple hundred bytes, can actually make them larger! Also if you are having CPU problems and your CPU can’t cope with the load, then you need to turn off the compression, but you will need to test this.

There is an option for this in WP Super Cache plugin. If you are not using Super Cache you will have to enable compression manually. To enable this across the site add the following line to your apache configuration:

        SetOutputFilter DEFLATE

        BrowserMatch ^Mozilla/4\.0[678] no-gzip\

        BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

        # Don't compress images

        SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|swf)$ no-gzip dont-vary

If you are using Ubuntu Server or Debian GNU/Linux distribution then you will have to enable mod_deflate:

$ sudo a2enmod deflate

This will enable module deflate. Make sure that there are two symlinks in /etc/apache2/mod-enabled, deflate.conf and deflate.load. Then you will have to restart the web server.

Optimize your database

MySQL database can scale pretty well, but it has few problems here and there. One of them is not storing the data as efficiently as possible (for reading) and tables can get dirty. You will need to optimize them. If you are using phpMyAdmin for administration then you can do if from the main database view by checking the tables you want to optimize and then selecting optimize below in the operations. We here at TWM prefer the command line:

$ mysql -u user -p wordpress

mysql> OPTIMIZE TABLE wp_options;

+-----------------------+----------+----------+----------+

| TABLE                 | Op       | Msg_type | Msg_text |

+-----------------------+----------+----------+----------+

| wordpress.wp_options | OPTIMIZE | STATUS   | OK       |

+-----------------------+----------+----------+----------+

1 row IN SET (0.02 sec)

mysql> OPTIMIZE TABLE wp_posts;

mysql> OPTIMIZE TABLE wp_comments;

Which tables to optimize? You can check this with SHOW TABLE STATUS; the data_free column will tell you how much space is allocated and not used by the table. With a lot of inserts, updates and deletes these numbers can grow and you want to avoid that. Depending on the traffic you will need to run optimize once a week or once a month.

Take care of proper caching of your static files

Your blog is more or less dynamic, but there are certain parts of your blog that are static and browsers can cache them. Images and theme files are two things that you can cache properly, even WordPress itself could be cached, but we mostly took care about them with Super Cache. Once I write a blog post, add images to it and publish it, I rarely change images that were attached to the post. That is why we can safely say that all the uploaded images can be cached for at least a year. Add the following lines to the .htaccess file in /wp-content/uploads directory:

ExpiresActive On

ExpiresDefault "access plus 1 minutes"

ExpiresByType image/gif "access plus 1 years"

ExpiresByType image/png "access plus 1 yearss"

ExpiresByType image/jpeg "access plus 1 years"

ExpiresByType image/jpg "access plus 1 years"

Now you set the expiration date to one year in the future for all the images. If you decide to change an image inside the post just keep in mind that you will have to rename it, otherwise clients will not show the new image, but the old one that they still have it in the cache. Unless the client is specifically told to reload the page. For Firefox this is performed by holding shift key while clicking on the reload button.

Using lightweight web servers

You could use lighttpd and php fast cgi binary to render WordPress, this is an alternative solution but it needs more detailed description. Lighttpd is a replacement for Apache 2 web server. It is small and fast, but it lacks support for certain fetaures. WordPress can run with lighttpd, but you will need to deal with different URL rewriting rules that need to be fixed for SEO friendly permalinks and for Super Cache to work. This can be done, but it will be the subject of another post.

Client side optimization

We pretty much dealt with the server side and not much more can be done there. We can still do some optimization for the clients. Web page rendering is a process that is quite complex and web page developers can (and should) help clients with rendering. A very good tool for benchmarking load time and render time of your web page is Firebug add-on for Firefox. Combine it with Google’s Page Speed add-on and you have everything you need to help you with optimizations.

Optimize your images

Try to avoid HTML or CSS resizing of images. Bigger images will need more time to download and then browser will have to resize them. Much easier is to resize them first with some resizing and optimizing tool. You will also get better results and better quality.

Avoid third-party service providers

WordPress can be extended by a number of plugins, add-ons, themes and various other hacks. The problem with third-party service providers is that their services usually come with some cost. Clients will have to contact their servers and download their content which might not be as optimized as your web site is. This also means that the clients will have to perform at least one extra DNS lookup which takes some time and will need more requests to fetch all the content.

No complicated CSS

CSS rules are very inefficient sometimes so try to avoid too much nesting and don’t use eval in CSS. Avoid things like this:

Wrong:

#mylist ul li ul li {

  font-style: bold;

}

Right:

.inner-list {

  font-style: bold;

}

.outer-list {

  font-style: normal;

}

Use class selectors, they will make CSS a little bit longer, but client will render it much faster. If you minify it, the difference will be just a few bytes. Unfortunately you are limited with WordPress here. Writing efficient CSS for auto generated sidebar widgets is a pain since HTML returned by WordPress isn’t something they should be proud of. However, it is somewhat manageable.

Also make sure that you don’t have any leftover CSS rules in your file and you are not using it. Note that Page Speed will tell you that you have a lot of unused CSS, but keep in mind that not all of the CSS is used on the first page or on the other pages in your blog. Splitting this code in separate files is an idea that we didn’t try yet.

Speed, this is what you need

Load time for an average blog, with a decent internet connection⁴, should not be much more than a second or two. When there is no other traffic on the site. This should be a general guideline. When you start picking up traffic your server should be able to scale and if you are lucky enough the first bottleneck will be your network and that is good enough to shift the blame to your ISP, right? ;)

Next stage would be offloading static content to a third-party provider or a different server that you have dedicated for serving static content. Replacing Apache2 with lighttpd web server could also be a next step then in the end we will have to separate web server and database server. We will describe all this in one of the future posts here.

1. Author of this text firmly believes that PHP should rename itself to LOL, reasons will be provided at some other time
2. Using Zemanta loader in the blog footer.
3. Here at TWM we still can’t decide if we want to keep this plugin or not.
4. And hardware that wasn’t assembled by Moses

Related posts:

1. WordPress plugins you need to survive ...
2. Zemanta plugin and WordPress 2.7 ...
3. WordPress 3.0 – What’s new ...

Did you ever have any problems with Eclipse? With initial versions of Galileo¹ and in previously released Gynamed there was a problem with preassigned keyboard shortcuts on some locale’s². The problem that Eclipse created when using Slovene keyboard layout was that while programmer wanted to type the ] key, Eclipse run File Search action.

Slovenian users are not to blame that their keyboard has ] key mapped to AltGr+G key combination. And Eclipse developers could browse through Wikipedia to discover what obscure keyboard layouts exist.

Any competent programmer will notice that with the default key mapping Eclipse guards programmers from using too many arrays since typically array is marked with [] key combination. They are advancing object oriented design and programming. But in some cases arrays actually make your code faster and optimize memory usage and fragmentation.

To discover how to remedy the problem read on.

Fortunately Eclipse has built in keyboard mapping manager. Until Eclipse developers don not remove the mapping or replace it with something more appropriate you can do the following:

1. Open Window->Preferences->General->Keys
2. Sort table by Binding and find the one that is bothering you³. Unbind all commands that are bind to the combination. If you don’t know which combination you should be searching for use Binding text box and type in the key that is bothering you.

There you have it problem is solved. File search gone by removing the CTRL+ALT+G or AltGr+G key combination from Eclipse key mapping. If you can not live without the combination you could replace it with something more appropriate.

1. should be corrected in 3.5
2. Slovenian and Hungarian, if my memory serves me
3. In my case CTRL+ALT+G

Related posts:

1. Eclipse Galileo and SVN support ...
2. Eclipse Code Assist / Code Completion / Content Assist hang-up ...
3. XMLbeans scomp fails with “java.io.IOException: CreateProcess error=2, The system cannot find the file specified” ...