Friday 29 April 2011

Firefox Quick Tip

At some point you will have come across a website that resizes or moves your browser, sometimes it can be in the form of those annoying popups.  I recently came across this when watching a news stream from Japan covering the Fukushima nuclear plant.

Well here's how to stop those sites hijacking your browser positioning.

Click Tools, then scroll down to Options.
Select the tab Content from the top.
You will now see some options.  Look across at the one that says javascript and click Advanced.
Uncheck the first option Move or resize existing windows.

That's all there is to it.  You can test by loading this site.
http://www3.nhk.or.jp/nhkworld/r/movie/index.html

Thursday 7 April 2011

uTorrent Tuning

For most people, the default settings in uTorrent will work fine.  You will generally max out your connection before you hit the maximum number of connected peers for example.  However I have recently upgraded to 50Mb/s (6MB/s) fibre optic broadband, and found that I wasn't getting the most from my connection.

One thing I found is that I was connecting to the maximum default number of peers and still not maxing out my connection.  Consider this:  If you can download at 6000KB/s, but your software is set to connect to a maximum number of 100 peers, what happens if the fastest they can upload is 30KB/s each?  That's right, you connect to 100x30KB/s peers, and you only achieve 3000KB/s download (half your maximum throughput).  It's even more of a problem on connections such as 100Mb/s where the downstream is approximately 12MB/s.

One real life scenario was when I was downloading an episode of Nichijou.  I had already set my max connected peers to 150 from previous.  I left it to download in the background while I chatted in IRC for a little bit and then thought to myself, "Why hasn't it finished yet?".  It hadn't finished because I had connected to something like 135 seeds and 15 peers and was only managing to pull 800KB/s (new torrent).  I upped the limit to 200 just to see if it would help and although it didn't connect to 200 (maybe 180 in total), I got 3.8MB/s.  It's definitely worth adjusting your maximum connected peers if you have a good connection but rarely get your maximum seeds on well seeded torrents.

Then I ran into another problem.  When you start downloading a torrent, the default action is to write that file to disk and fill it with zeros.  Kind of a placeholder for the file if you like.  For MP3 albums and small downloads, it's not a big deal because your HDD will have written the file before you've had time to connect to all your peers and reach a high download speed.  The problem arose when I was downloading an OpenSUSE DVD ISO.  uTorrent wanted to write 4.3GB as soon as I had started the download.  It took a little while to write this, during which time I had connected to all my peers and hit 6MB/s download.  Because the HDD was busy, I got a disk overloaded 100% warning in uTorrent and my download speed dropped drastically (a few hundred KB/s) since the HDD couldn't keep up with writing the file at maximum speed, plus the stress of an extra 6MB/s.

Writing the file and zeroing it out when the download starts is annoying.  Not only will it cause your download speed to drop, but it also has the potential to cripple your system for a good few minutes.  What happens if you download a 50GB Bluray ISO?  It would take a good 15 minutes at least to write that at full speed, during which time I doubt you could load much else.  Fortunately I found two methods within the uTorrent settings that help alleviate the problem.

Another annoyance that can hinder you from reaching your full speed is traffic management.  An automated system that checks the packet headers to see what type of data it is.  If it detects torrent traffic, it throttles the speed.  You can work around this by using encryption in uTorrent.

So now let's move on to the actual tuning.  I use uTorrent 1.6, but these options are pretty much universal.  Click Options > Preferences to get started.

Downloads
Click the downloads tab, and ensure "Pre-allocate all files" is unchecked.

Connection
Bandwith Limiting
Did you know that uploading at your maximum speed can cripple your download speed?  The latest versions of uTorrent set your maximum upload speed to 80% of what you can actually achieve to prevent that happening.  If you don't know what your upload speed is, go to Speedtest, pick a local server and make a note.  Multiply your rating in Mb/s by 128 to get your upload rate in KB/s.  Then multiply that by 0.8.
Eg.  4.7Mb/s * 128 = 602KB/s * 0.8 = 481KB/s
Your result is what you should set your Global maximum upload rate to.  Check the box for Alternate upload rate when not downloading and set the KB/s to 0 (unlimited).  With these settings your downloads will not be crippled, and when they are done uTorrent will seed at maximum speed.

BitTorrent
Number of Connections
This will control how many peers you can connect to.  Take a look at the image below
Here you can see that under the Seeds heading, it says 119 (845).  What that means is that I am connected to 119 seeds out of a potential 845.  The same applies for the Peers heading (connected to 6 peers out of 126).  You will see that the setting Maximum number of connected peers per torrent controls how many seeds and peers your are able to connect to.  In this case my limit is set to 125, and uTorrent has decided to connect me to 119 seeds and 6 peers.  Fortunately for this torrent I had reached my maximum download speed, but there are occasions where you will connect to your maximum number of peers and still not reach your maximum speed.  In this case, increasing the maximum number of connected peers should help, providing the number of seeders in the brackets is larger than the number you are already connected to.

Be aware that setting this number too high could have adverse effects, so it's a case of trial and error.

Protocol Encryption
Setting encryption to Enabled and Allow incoming legacy connections will encrypt some of your traffic.  This is particularly useful if you suspect your ISP is throttling your torrent traffic as it scambles the data and makes it unreadable by bandwith management systems.  Again this is another setting that you should experiement with.  You could set encryption to Forced and uncheck Allow incoming legacy connections for full encryption, but it will also limit the amount of peers you can connect to.  It's probably better to have a little encryrption and a little throttling than full encryption and less connected peers.

Advanced
net.max_halfopen
This option specifies how many connections uTorrent should try to establish at any one time.  The default is 8 which is very low.  I have mine set to 100.  The theory behind this is that it will take less time for you to connect to all your peers and reach your highest speed.  Before tweaking this, I found that for files in the region of 200MB, that it would have completed before I had connected to all the peers and reached my full speed.

Users of XP SP2 or later should patch their tcpip.sys file to make the most of this increase, as the max number of halfopen connections was reduced.  More information can be found here.  This limitation was again removed in Windows Vista and Windows 7.

gui.update_rate
This won't affect your download speed but simply changes how often the data reported by the GUI is updated.  The default is 1000ms (1 second) and the lowest you can select is 500ms.  Any lower figures will be igored and 500ms will be used.  I prefer having it update at half second intervals, but if you find CPU usage is high, you can set it to update less frequently if you wish.

bt.connect_speed
This option specifies the number of connections uTorrent should make each second up to the limit set in net.max_halfopen.  The default is 20, and I set mine to 40.

diskio.sparse_files
This is an important option as it controls how the data is allocated on your hard drive.  If False is selected (which is the default), then uTorrent will write a dummy file to the hard drive as soon as you add the torrent, but fill it with zeroes (the place holder file as explained earlier).  This is fine for small files that can be written in a few seconds, but for large files like Bluray rips it can cause your drive to be busy for a long time, and the download speed will likely drop during this process as the HDD can't keep up with the incoming data.  It may also cause programs to become unresponsive.

Setting this option to True (as I have done) will inform the filesystem of the size of the file, but will not physically zero out the data.  Instead the only write operation uTorrent performs are writes for data actually downloaded.  This not only saves your computer becoming unresponsive when adding large torrents, but also means that the hashing process for incomplete files is much faster as it only hashes what has been downloaded, rather than a full size pre-allocated file.

In my opinion True should be set by default, however there are some limitations regarding the support of this mode.  It apparently only works on partitions formatted as NTFS (as opposed to FAT32 I suppose), however I can't speak of other filesystems such as EXT3.  Also if you are using a non-administrator account with a disk quota, sparse files won't work and it will still be allocated.

I encourage you to at least try this option if you've noticed high amounts of disk activity or unresponsiveness when adding large torrents.  This completely cured it for me.

Advanced - Disk Cache
Providing you set diskio.sparse_files to true, you don't really need to edit any settings here, but in case you couldn't (due to filesystem limitations for example), then these settings will help ease the pain of the file allocation stage.

These settings are fairly self explanatory, but most people rarely think about changing them.  I have done a little experimentation and find that the settings above give me the most preferable disk access.  Rather than writing data constantly, it stores it in RAM and writes it in chunks of about 80MB, or whatever your HDD can write in the space of a second.  If I'm downloading at my maximum speed, this means time between writes is 13 seconds plus, rather than the HDD writing 6MB every second.

You can check read and write patterns yourself in uTorrent.  Simply click on the Speed tab in the lower half of the window (the tab that gives you a upload and download speed graph) and change it to show disk statistics.  You can now see a bunch of cool stuff like how the cache is being used, how regular uTorrent is reading and writing to the HDD, and the throughput.

One setting I suggest you do experiment with is the cache size.  While the settings above should be good for everybody, the amount of RAM you have to spare for a disk cache will vary from user to user, and the amount of RAM you will need to allocate to make a decent buffer length depends directly on your download speed, ie a 1MB connection will require a lot less RAM buffer than a 6MB connection if the HDD is busy/unwritable for 2 minutes.

If you have a fast connection and download large files, I strongly suggest setting Override automatic cache size and specifiy the size manually to whatever figure you are happy to use.  This acts like a buffer in case the disk is busy and data cannot be written (a good example being the file allocation at the start of a large torrent, or if you do video editing etc).  The downloaded data is stored in RAM, instead of the download speed being dropped to a rate the HDD can handle.  Once the HDD becomes free again, the data stored in the RAM is then written to the file.  It's a lot more efficient than dropping the speed and/or disconnecting from peers and then reconnecting once the HDD becomes usable again.

You can calculate an effective cache size using the typical size of files you download and your maximum download speed.  Let's say that my system is set up to allocate disk space to new torrents and that I want to download a 4GB Linux ISO.  For the purpose of this example we'll say that my drive can write this file at 35MB/s - that means it would take 115 seconds to complete.  That's 115 seconds of the HDD thrashing and not being able to download at a decent speed (I'd usually get figures under 100KB/s when the disk was overloaded).

I know that I can download at a maximum of 6MB/s, and that I need to store this in RAM until the allocation has been written (115 seconds).  So 115s * 6MB/s would give me a value of 690MB.  In other words during the time it takes to write the pre-allocation, I could have downloaded 690MB, but without the buffer this wouldn't have happened.

And that concludes our uTorrent tuning.  If you find any better settings, please post them below

Upgrading to an "Advanced Format" HDD (Windows 7)

About a month or two back I bought a new laptop, since my old HP was literally falling to pieces (the LCD hinge broke and it the metal bracket was stuck.  Not even a hammer would get it to move).  So rather than run the risk of the LCD cable becoming severed one day and not having a laptop, I decided to buy a Vaio and went for the VPCF12M0E.  It has an i7 740QM CPU, 4GB of RAM and 500GB HDD.  I decided on this model becuase in my opinion, anything over 4GB of RAM right now is a waste.  I'm a pretty demanding user and I've yet to reach a point where I feel 4GB isn't enough.

It's pretty nice.  The CPU encodes video literally 5x as fast as my old dual core Athlon 64 2x2GHZ TL-60 in the HP, the memory scores high on the Windows Experience Index, and the GPU is decent too.  The only thing that let it down (as is usually the case with laptops or manufacturer assembled computers) was the HDD.  It shipped with a Western Digital 500GB Scorpio Blue (WD5000BEVT).  I could tell straight away that it wasn't great, and while I didn't perform any proper benchmarks, it scored 5.8 in the Windows Experience Index, compared to an old Western Digital 250GB Scorpio Black (WD2500BEKT) which scored 5.9.

At this point I should tell you that the in Windows 7, the Windows Experience Index caps the score for platter based HDDs (ie not solid state) at 5.9.  It's possible that it would have scored even higher.

The Blue didn't seem unusually slow as such, just I remember my 250GB Black feeling faster and more responsive.  The main differences between the Blue and Black series notebook HDDs are:
  • Blue - 8MB cache.  Black - 16MB cache
  • Blue - 5400 RPM.  Black - 7200 RPM
The Black drives also typically have a longer warranty (5 years compared to 3), but are generally a step behind the Blue when it comes to capacity (at the time of writing the largest Blue notebook drive is 1TB, and the Black is 750GB).

However data density also plays a part in speed, and the general rule of thumb is that the larger the HDD, the higher the write/read rates, so although the 500GB Blue lost out on cache and RPM to my old 250GB Black (which are pretty major points to lose out on), it does high a higher data density, so sequential throughput shouldn't have suffered quite as much - it's more seek operations and loading multiple files where the Black series outclasses the Blue.

So after looking at my options, I decided to go with a Western Digital 750GB Scorpio Black.  It was the right combination of capacity and speed for me (though I really wish they had made it available in 1TB.  I smell a marketing ploy...).

I had also looked at the Seagate Momentus XT 500GB.  It's an interesting drive as it's a hybrid.  It has platters like any other harddrive, but it also has 4GB of on-board SSD storage where most accessed files are stored (eg the OS files).  The performance is really impressive, but since it's only 500GB it would have been a sidegrade, so I passed on it.  Hopefully this type of drive will catch on.  Who would need SSD if we have 2TB platter HDDs with 16GB of SSD?

Advanced Format
My choice of the 750GB Black was a relatively easy one.  Right now it's probably the best performing HDD of that capacity.  The Momentus is awesome, but I wanted extra space as well as extra speed.  However I ran into a little problem.  Something called "Advanced Format".

What is this and why is it such a big deal?  Ever since HDDs were first created way back in 1956, they have used 512 byte sectors.  It was predicted that using current technology that HDD sizes were going to stagnate.  One solution to this was to use larger sector sizes which allows for more efficient usage of the recording surface which in turn allows the manufacturer to use more robust error correction which is needed at the ever increasing data densities.

The 750GB Black uses 4096 byte physical sectors, but the controller still communicates in 512 byte sectors (4k emulation).  Think of it as a kind of translation.  The decision was made to use this kind of emulation because of the many programs and operating systems that presume a HDD will have 512 byte sectors - think of it in a similar way to how your 64-bit OS and CPU still offers emulation for 32-bit programs.  Going full 64-bit would be a pain right now.

I had read about this prior to buying the 750GB Black and I got myself confused.  Due to the fact that the sectors are larger now (8 sectors combined into one if you think of it that way), it's possible for data to be unaligned.  That is instead of the 4k sectors sitting nicely like this:

Some operating systems and partition creation utilities can create paritions that are unaligned.  I do not understand fully, but I believe it's to do with the location and/or size of the Master Boot Record.  In this case, data can become unaligned like this:
That creates two specific situations.  One where data is aligned (as the first example) and one where it isn't.  To the best of my knowledge it doesn't cause any compatability problems, just decreased read and/or write performance.  If your data is unaligned, then you need to use a tool like WD Align to fix it.

Here is a chart Western Digital put together to help end users determine whether or not they need to run the WD Align tool on their disk.

As you can see, Windows XP (and presumably earlier) requires you to run the WD Align tool, since the OS is not 4k sector aware, and neither does the OS really "know" that the drive has sectors of these size since the drive's firmware still communicates in 512 byte sectors (4k emulation).  In this instance, XP creates a partition that is at a sub optimal offset for 4k drives, and the WD Align tool fixes that.

Windows Vista, Windows 7 and Mac OS use an offset that is optimal for 4k Advanced Format drives and does not require alignment under normal circumstances (for example a clean install of one of these operating systems).

However cloning utilities are a grey area.  Western Digital claim that you should align the partition after restoring the data to ensure full performance, however I believe that some disk cloning utilities (such as Macrium Reflect) support 4k drives and would not require alignment after (providing the OS supports the correct drive offset).

This is what raised questions with me.  I needed to migrate my data from a 512 byte sector HDD that I did not know the offset of to a 4096 byte sector HDD.  Luckily since the old HDD was formated with Windows 7, it already had it's data at the right offset, but I think that would probably end up to be irrelevant anyway.

I decided to use the internal Windows 7 backup/restore utility.  One of the reasons was because it's was free and readily available.  The other reason was that since this is a backup tool, rather than a dumb data cloning program, I expected that upgrading a HDD this way would be less hassle as this type of situation will have been considered when Microsoft were designing the software.  Think about what happens if you have been making regular drive images, and 5 years later the original disk dies?  You may not be able to source an identical drive, so I expect there is some amount of flexibility within how much, and the type of data is stored, to allow you to restore to other brands, sizes and types of drive.

The Migration Process for Windows 7 SP1
This process will most likely work for Windows 7 installations without the service pack, but the following guide will be based on SP1 since that's what I used and can confirm it was successful.

What you will need:
  • USB HDD with at least as much storage as the HDD you are migrating (prefereably 2x and not password protected)
  • Screwdrivers (this is obvious but you usually need a very small screwdriver also to attach your HDD to a cradle)
  • Latest chipset drivers (Intel Rapid Storage Technology must be 9.6 or higher)
  • "F6" chipset drivers if your vendor supplies them
  • Blank media (At least 1 CD/DVD/BD for the recovery disk, more if you plan to save your HDD image to disc instead of USB HDD)
1) Connect your USB HDD.  We are going to make an image of your current drive, prior to messing around installing updated drivers.  This is just a precautionary step, but is good practice if you intend to update your HDD controller driver.  You should ensure your HDD is not encrypted, hidden or requires unlocking of any sort.  You won't have access to Windows during the recovery stage, so it will not be possible to run the unlocking software for these types of drive.

2) Grab your blank media and run the program.  Once the image has been made, it will prompt you to burn a recovery disk.  This contains the bootable program that will transfer your disk image stored on the USB HDD to your new internal HDD.  To start the image creation process click Start > Control Panel > Backup and Restore.  In the left hand pane, click Create a system image.  Follow the prompts to create a system image.  How long it will take to make the system image depends entirely on how much is stored on your HDD and how fast your external is, so it may take a while.  Once the image has been stored, you will be prompted to burn a recovery disc, click yes.

3) Now that you have a system image of your current configuration and a restore disc, it's time to ensure your chipset drivers are up to date.  For Intel chipsets, the Rapid Storage Driver must be version 9.6 or higher for 4k sector support.  The Windows driver can be found here and the F6 driver (which you load during the recovery process) can be found here.  Download and install the Windows software.  In my case I had to boot into safe mode (keep pressing F8 as your system boots) to install, as it would fail under normal conditions.  Unzip and place the F6 driver on a USB stick or the HDD that you stored your drive image on.  At this point, it would also be good to download and install any Windows updates.

4) Once your chipset driver is up to date and any OS updates are installed, it's time to make another image.  Before we do this, we need to rename, or move the original image to another location (since the default action is to overwrite the old image).  Open your USB HDD in your file browser and look for a directory called WindowsImageBackup which should be in the root.  Within that directory will be another directory named the same as your computer (in my case it says RX-93).  Rename this directory so that you will be able to tell it's the previous version (eg. RX-93 Old).  Now repeat step 2 to make your new drive image.  This will appear in the same directory as the name of your computer and this will be the image that we restore onto the new HDD - that's why it's important that we mark the first image as old.

5) By now you should have 2 system images on your HDD (the original one before installing updated drivers, and the current one with the new drivers/updates), a recovery disc (CD/DVD/BD) and dependant on your chipset vendor, some F6/installation drivers on the external HDD to be loaded during the restore process.  If you have all that, power off your computer/laptop, remove any batteries if applicable and press and hold the power button for a few seconds to drain any extra power.  Before removing the HDD from it's anti-static packaging or from the laptop, you should ensure you are earthed and/or that any static electricity has gone.  You can do this by touching a radiator for example.  The enthusiastic lady at Western Digital tells us that clothing can also generate static electricity, so install it naked if you wish.

If you haven't replaced a laptop HDD before, check out this video.  The laptop used in the video is the same model HP laptop I have, and I can tell you that the process is very similar.

6) Once you have your new HDD installed, and everything is closed back up and screwed into place, it's time to attach the battery and the AC cord.  Do not try to run the restore process using the battery alone because it will take a long time, and will likely run flat during the process.  Turn on your laptop and insert the recovery disc if it isn't already inside (you may need to power off and power on again if you missed the point where the system reads from the optical drive).

7) Once the recovery program starts to load, attach your USB HDD.  Follow the prompts, but look out for the Load drivers option.  Click this if you have downloaded the F6/install drivers, browse to them and load the applicable chipset driver.  It is likely to give you a list of different drivers, so pick the AHCI driver for your chipset (which you found earlier in the device manager).  Proceed through the prompts, selecting to restore using an image.  The option to format the drive will be greyed out as this is the default action for a blank disk.  Click next and the restore process begins.

8) All being well, your new drive should boot up with all your configuration, programs and files intact.  If you used a different brand or model of HDD, you will be prompted to restart your computer.  This is normal as it detects the HDD as new hardware since it's ID was different to the original.

You may notice that once your computer boots that even if you installed a larger HDD, that the capacity appears to be the same.  The reason for this is that the restore program only formats the same amount of space that was available to the old drive.  However we can extend the partition to use the full capacity, or even format the remainder so it appears like a seperate HDD.  Click Start > Control Panel > Administrative Tools.  Double click Computer Management and then select Disk Management in the left hand pane.  This will bring up a list of drives attached to the computer.  You will notice Disk 0 will have some space unallocated.  Right click on this and choose Extend Partition.  Follow the prompts all the way through (this will extend the C:\ partition to use the whole of the unformatted space).  The process takes a matter of seconds.

Once that's done, enjoy your new HDD.



    For a more information on Advanced Format, check these links:
    http://en.wikipedia.org/wiki/Advanced_Format
    http://www.wdc.com/advformat
    http://consumer.media.seagate.com/2010/03/the-digital-den/4k-sector-hard-drive-primer/

    Wednesday 6 April 2011

    About this blog

    Well I finally cracked and decided to start a blog.  I've always said I hate these things, and that's mostly because I associate blogs with people posting random crap of no worth or relevance (having said that, I guess it's subjective).

    So I hope that this blog turns out different.

    With that in mind, this will be a place for me to post information and tips on subjects that interest me, so that means audio, cameras, computers and of course, Street Fighter.

    Initially I hope to post a few different things that I've been meaning to do but never got round to.  One of my projects for example was making my Xbox 360 wireless without buying their wireless adaptor, and I want to post details of a recent HDD upgrade method for my laptop that was so stupidly easy that anyone can do it.