« Grady Booch on SOA, and he's so right | Main | Off topic: Intel shipping 16 thread CPU in 08 »

March 28, 2007

Off topic: Using flash drives for development builds

I was getting frustrated with my T60 when doing builds so I started to look in to alternative drive solutions. USB Flash drive looked very promising. I saw articles showing them at 20MB/sec and sub 1ms access times. I figured, cool, I'm getting one of those.

I bought two 4GB models, a Sandisk cruiser and a crucial 4gb gizmo drive. The Sandisk rates with HDTune at 10MB/sec and 1.0ms access time, the gizmo at 23mb/sec and 0.7ms access times. My T60 100gb drive is 45mb/sec and 17ms access time. Should be a no brainer, right?

I used an ObjectGrid build to try them out and compare with my laptop. First problem was FAT32. I had to change the device policy to optimize for performance and then reformat it as NTFS. Once that was done, the build fits on the drive.

The laptop drive wins I'm sorry to say. The faster of the flash drives, the crucial gizmo takes 26 minutes to build ObjectGrid. My laptop drive does it in 18 minutes. I don't get why? Maybe the transfer speed is why or maybe it's the USB interface. Apparently Lenovo has a 60GB flash drive for a t60 now so I may try get one of those. But for now, the laptop is, amazingly, faster than a solid state disk that on paper should murder it.

March 28, 2007 | Permalink

Comments

"My laptop drive does it in 18 minutes. I don't get why?"

I'm not an expert at this... but as these flash disks are targeted at things such as digital cameras, maybe they are optimized for storing large files such as 6-12 MPixel photos, just to make an example, while an app build creates and modifies a lot of small files... Have you tried reformatting with different block sizes (I know it can be done with FAT32, I don't know about NTFS)?

Posted by: Fabrizio Giudici | Mar 28, 2007 1:31:39 PM

Could be but when you look at the access times with the hdtune tool, it doesn't appear to be lacking as a random access device.

I'm using NTFS, I haven't tried FAT32 as wastage was an issue and the 2.7GB of data took more than 4GB with wastage.

Posted by: Billy | Mar 28, 2007 1:48:18 PM

You can't beat SATA or even IDE interface with DMA support with lousy USB 2.0...

BTW, have you tried to enable compression on your source folders? It may help to reduce amount of data transfered to and from disk.

Posted by: Eugene Kuleshov | Mar 28, 2007 3:56:43 PM

I would like to try 2 SATA flash drives in a RAID 0 array, this should get around the lower data transfer rate.

In the meantime I will soldier on with my 15K SCSI rig :)

Posted by: jonathan | Mar 28, 2007 4:15:49 PM

Those times are probably for reads. Write time to flash is typically abysmal, as the device has to 'burn' blocks at a time on write.

What you really want is just pure ram drive. Especially if the needs for the 'drive' are completely transient.

Posted by: Patrick Mueller | Mar 28, 2007 4:53:21 PM

Compression makes it worse :( I'm giving up for now, it's slow. I'm wondering with vista ready boost how this is going to speed stuff up when it looks like paging to disk is way faster even with the fastest of usb flash disks.

Posted by: Billy | Mar 28, 2007 5:34:22 PM

Caching in the HD makes it faster that its specs, especially for small files. When flash drives gets better caching algorithms and bigger cache sizes they will be faster, but that will take a year or two.

Posted by: Mikael Grev | Mar 29, 2007 8:45:40 AM

I personally like to make my develop environment as fast as I can. So I would rather stick on HDD and SYNC my workspace and my USB storage

Posted by: Carfield Yim | Mar 30, 2007 6:13:13 AM

Post a comment