I found that the SpeedTest function in RMPrepUSB/RMPartUSB no longer seems to work on Windows 10 and gives a write error. This is now fixed in v2.1.734.
I also took the time to look into some niggly bugs in DiskDoctor (use CTRL+D inside RMPrepUSB to launch it after selecting a disk (does not work on disk 0) in RMPrepUSB).
If you want to try DD, use a disk that contains unwanted data first, until you become familiar with it!
For instance, if you start a Sequential Read of the whole disk, DiskDoctor (aka DD) will attempt to read all sectors, in blocks of a 'track' at a time, into a track buffer (63x255 sectors). This can take several hours for a 2TB HDD!
On a normal spinny-disk, the time taken to read sectors sequentially should be reasonably constant and will depend on the linear speed of the sector travelling under the head. For instance, one revolution of a 7200rpm disk takes 8.33 ms for one rotation. On an inside track nearest to the disk hub (which is not as long as an outside track), you may only have half the number of sectors per track, so sequentially reading sectors from an outside track will always be faster than reading the same amount of sectors from an inner track on the disk.
Now modern disk drives will cache the data and try to 'read forward', however, if we suck off the data faster than the disk can move, there will be no sector-read caching. Track-to-track latency on sequential reads is usually inconsequential when the sectors are correctly interleaved.
DiskDoctor times each block of reads and will report if any block operation had a delay of more than 10ms (you can adjust this threshold using 'Set Slow Xfer Limit' or set it to Auto-adjust). A delay of more than this limit indicates that the disk controller had a problem reading one or more sectors for that requested block of sectors and so possibly had to perform a retry.
Each sector (modern disks have 4K sectors) also contains 'checksum' bytes at the end of the sector. The disk controller knows if the data in the sector was good, because the checksum of the 4K block of data from the sector will match the checksum bytes that it also read.
If a disk controller could not read a sector correctly, they usually simply try to read it again. This means they have to wait for that sector to come round again under the head. If the read was successful on the 2nd attempt, we will have a delay of approx. 16ms for a 7200rpm drive. If the 2nd attempt also failed, most disk controllers will attempt several more read-again operations.
If all read-retry operations fail, the disk controller will then attempt to perform an error correction operation on the ECC-encoded data that it has read off the disk. This is only able to correct a reasonably small number of bit errors (e.g. 11 or so). Also, the error correction takes quite a while (>50ms) to perform. If ECC-correction fails, the disk controller will retry the sector read and then retry the error-correction several times before reporting back an error.
So you can see that if a disk starts to report errors, something is seriously wrong with it! However, before it gets to this state, it will be performing retries and we can detect that by measuring the time delay between each sequential read.
Windows itself will also perform a retry by repeating the same read-sector or write-sector command several times if the previous command failed. This is why, on a faulty hard disk, it can take several minutes to read or write to a single 'bad' area on a disk.
Note: In my experience, if a disk reports SMART errors (above the SMART error threshold) then that disk is really bad and should not be used.
Also note, that there can be valid reasons for a perfectly healthy hard disk to perform a retry or two on occasion, such as vibration or the disk is too hot.
If a disk controller has problems reading particular sectors, it will reallocate that sector to use one of the many spare sectors on the disk (usually there is one spare sector per track available). An access to a reallocated sector can also cause a 'delay' in the disk operation, if the reallocated sector is not on the same physical track. This is not a retry, but does indicate that the drive has many re-allocated sectors because normally each track contains a spare sector for use as a reallocated sector, which would not cause an excessive delay. So an excessive delay >10 ms can also indicate that the spare sector on that track has been already used and a sector in a different area had to be accessed.
On a brand new, factor fresh hard disk, there should be very few (if any) slow transfers (retries/delays) reported by DD when the whole disk surface is read.
On a well used disk, you may get multiple 'slow transfers' detected by DD, however there should not be too many (typically < 10). Tracks nearest the centre of disk typically produce more retries than tracks on the outside edge ('end' of the disk), but you may also get more retries and higher data densities in the middle of the disk because some disk controllers can vary the number of sectors they store in a physical track in order to maximise the usable storage area of each disk platter.
Note: A data rate shown by DD as ???MB/s indicates that the time delay was too short to be reliably measured by DD. Either the disk is very fast or that disk operation was cached.
Note: DD supports most types of block storage 'disk' devices, but not native 4K disk drives as found in some external USB drive enclosures. To read native 4K disks, you need to read the disk in multiples of 8 sectors.
DD and RMPrepUSB both display the sector data as simple text (hex+ASCII) format. This means that you can cut-and-paste the lines of text into NotePad or emails, etc,
For instance, set up DD (assuming your USB drive is Drive 5) as:
Drive 5
Start 0
No. Sectors 1
and click Sequential READ which reads data into a buffer and then Display and Edit Buffer (to display a maximum of 20 x 512 byte sectors) and then maximise the form.
You can now edit any of the hex bytes in the window (the ASCII characters on the far right are ignored and do not need to be changed, but must be present).
You can even cut-and-paste whole lines (complete with the far-right ASCII characters).
You should ensure that the 4-character left-hand address field is correct for each line, e.g.
Note that this will write 0's + byte AA to address 161h, because the red line starts with 0160 and not 0170!
DD will tell you how many bytes have been changed and prompt you before writing the sector(s) when you click on the Write using Buffer button.
If the DD window contains the line:
============= EDIT SECTOR DATA (up to first 20 sectors) ================
then DD will parse and use the hex text displayed under that line as the data, instead of the data currently held in the data buffer.
This feature had some bugs in the previous version of DD which hopefully have now been corrected.
It supports four types of search:
The search will begin at the sector set in the Start field and for the number of sectors set in the No. Sectors field. If you want to search the whole disk, select a different drive and then re-select the original drive, the Start and No. Sectors fields will be set to 0 and + the total number of sectors for that drive.
Fill Buff
Fills the 8MB buffer with this pattern. For instance, if you set No. Sectors to 1000 and perform a Sequential Read operation, the buffer will contain 1000 sectors. You can use Fill Buff to then fill the 1000 sectors with a certain data pattern (two bytes) and display the first 20 sectors using Display and Edit Buffer, or write that data to any part of the disk by setting the Start field. You can only use this for a reasonably small number of sectors (i.e. maximum of 1 track = 63x255 = 16065 sectors = 8MB).
Fill Buffer with Pattern and WRITE
This button uses the Pattern set in the field above and then performs a write operation to the Start address for No. Sectors. A track (63x255) at a time is written (or less). This can be used to wipe a whole disk.
Non-Destructive Random WRITE
This will perform a write sequence (in blocks) across the disk (area is determined by the Start and No. Sectors fields). You will be prompted for the number of passed to be performed and the number of sectors to be used in each block. The sequence is: choose a random sector - read and keep data - write random pattern - read - compare - restore original data.
Random READ
You will be prompted for the number of passes and number of sectors per operation. Blocks of data will then be requested from randomly chosen areas on the disk within the Start and No. Sectors limits.
DD also looks at the Windows Event Log before and after an operation. It is often the case that the disk driver will report log error information in the Windows Event Log (for instance a Windows driver may automatically switch to a slower IDE or SATA data transfer rate if it starts to get 'DMA errors', this will cause a sudden decrease in performance across the whole drive). DMA errors ('ULTRA_DMA_CRC_ERROR_RATE') and switching to a slower transfer protocol are indicative of a bad EIDE or SATA data cable, or possibly a badly designed/terminated disk controller PCB.
The WI_D variables are DD Disk operational errors, whereas the WI_E variables are used for Event log event differences and WI_S variables are for SMART error reporting differences:
SET WI_D_TOTERR - Total of all disk errors
SET WI_D_READERR - Total of all disk read errors
SET WI_D_WRITEERR - Total of all disk write errors
SET WI_D_CMPERR - Total of all write-read-data compare errors where the wrong data has been returned.
SET WI_E_INFO - Total Event Log informational messages that occurred during last operation
SET WI_E_WARNINGS - Total Event Log warnings
SET WI_E_ERRORS - Total Event Log errors
SET WI_S_ERRORS - SMART errors
SET WI_S_WARNINGS - SMART warnings
SET WI_S_INFO=0 - SMART info (should be examined manually)
The raw output of SMART data (before and after) is also displayed. Note that the meaning of the raw data varies from manufacturer to manufacturer and so it is tricky to analyse (e.g. READ_ERROR_RATE raw data).
To see the SMART data, start a Sequential Read operation on a SATA hard disk and after a minute or so, click Cancel to see the SMART data results.
DD analyses some SMART data differences and adds up any changes in error or warning counts after an operation has completed, for instance the following changes are taken into account:
WI_S_ERRORS:
POWER_CYCLE_COUNT
UNCORRECTABLE_SECTOR_COUNT
REALLOC_SECTOR_COUNT
SEEK_ERROR_RATE
If TEMPERATURE_Deg_C_or_F > 79 Deg C
WI_S_WARNINGS:
SPIN_RETRY_COUNT
REALLOCATION_EVENTS_COUNT
ULTRA_DMA_CRC_ERROR_RATE
SlowRateErrors
WI_S_INFO:
READ_ERROR_RATE
Please tick one of the Reactions boxes below if you found this useful.
I also took the time to look into some niggly bugs in DiskDoctor (use CTRL+D inside RMPrepUSB to launch it after selecting a disk (does not work on disk 0) in RMPrepUSB).
DiskDoctor
DiskDoctor is a disk testing and editing tool which must be run with Admin privileges. The version included in RMPrepUSB has been stripped down quite a bit since I wrote it some years ago, and some functions and command line support have been removed. It was originally used as an automated test tool in a factory environment and used to test literally millions of computers.If you want to try DD, use a disk that contains unwanted data first, until you become familiar with it!
For instance, if you start a Sequential Read of the whole disk, DiskDoctor (aka DD) will attempt to read all sectors, in blocks of a 'track' at a time, into a track buffer (63x255 sectors). This can take several hours for a 2TB HDD!
On a normal spinny-disk, the time taken to read sectors sequentially should be reasonably constant and will depend on the linear speed of the sector travelling under the head. For instance, one revolution of a 7200rpm disk takes 8.33 ms for one rotation. On an inside track nearest to the disk hub (which is not as long as an outside track), you may only have half the number of sectors per track, so sequentially reading sectors from an outside track will always be faster than reading the same amount of sectors from an inner track on the disk.
 |
| Note that the text above refers to ATA (IDE) disks and not SATA disks which are much faster! |
Now modern disk drives will cache the data and try to 'read forward', however, if we suck off the data faster than the disk can move, there will be no sector-read caching. Track-to-track latency on sequential reads is usually inconsequential when the sectors are correctly interleaved.
DiskDoctor times each block of reads and will report if any block operation had a delay of more than 10ms (you can adjust this threshold using 'Set Slow Xfer Limit' or set it to Auto-adjust). A delay of more than this limit indicates that the disk controller had a problem reading one or more sectors for that requested block of sectors and so possibly had to perform a retry.
Each sector (modern disks have 4K sectors) also contains 'checksum' bytes at the end of the sector. The disk controller knows if the data in the sector was good, because the checksum of the 4K block of data from the sector will match the checksum bytes that it also read.
If a disk controller could not read a sector correctly, they usually simply try to read it again. This means they have to wait for that sector to come round again under the head. If the read was successful on the 2nd attempt, we will have a delay of approx. 16ms for a 7200rpm drive. If the 2nd attempt also failed, most disk controllers will attempt several more read-again operations.
If all read-retry operations fail, the disk controller will then attempt to perform an error correction operation on the ECC-encoded data that it has read off the disk. This is only able to correct a reasonably small number of bit errors (e.g. 11 or so). Also, the error correction takes quite a while (>50ms) to perform. If ECC-correction fails, the disk controller will retry the sector read and then retry the error-correction several times before reporting back an error.
So you can see that if a disk starts to report errors, something is seriously wrong with it! However, before it gets to this state, it will be performing retries and we can detect that by measuring the time delay between each sequential read.
Windows itself will also perform a retry by repeating the same read-sector or write-sector command several times if the previous command failed. This is why, on a faulty hard disk, it can take several minutes to read or write to a single 'bad' area on a disk.
Note: In my experience, if a disk reports SMART errors (above the SMART error threshold) then that disk is really bad and should not be used.
Also note, that there can be valid reasons for a perfectly healthy hard disk to perform a retry or two on occasion, such as vibration or the disk is too hot.
If a disk controller has problems reading particular sectors, it will reallocate that sector to use one of the many spare sectors on the disk (usually there is one spare sector per track available). An access to a reallocated sector can also cause a 'delay' in the disk operation, if the reallocated sector is not on the same physical track. This is not a retry, but does indicate that the drive has many re-allocated sectors because normally each track contains a spare sector for use as a reallocated sector, which would not cause an excessive delay. So an excessive delay >10 ms can also indicate that the spare sector on that track has been already used and a sector in a different area had to be accessed.
On a brand new, factor fresh hard disk, there should be very few (if any) slow transfers (retries/delays) reported by DD when the whole disk surface is read.
On a well used disk, you may get multiple 'slow transfers' detected by DD, however there should not be too many (typically < 10). Tracks nearest the centre of disk typically produce more retries than tracks on the outside edge ('end' of the disk), but you may also get more retries and higher data densities in the middle of the disk because some disk controllers can vary the number of sectors they store in a physical track in order to maximise the usable storage area of each disk platter.
Note: A data rate shown by DD as ???MB/s indicates that the time delay was too short to be reliably measured by DD. Either the disk is very fast or that disk operation was cached.
Disk Editing
The main reason why DD is included with RMPrepUSB is to allow you to perform sector editing.Note: DD supports most types of block storage 'disk' devices, but not native 4K disk drives as found in some external USB drive enclosures. To read native 4K disks, you need to read the disk in multiples of 8 sectors.
DD and RMPrepUSB both display the sector data as simple text (hex+ASCII) format. This means that you can cut-and-paste the lines of text into NotePad or emails, etc,
For instance, set up DD (assuming your USB drive is Drive 5) as:
Drive 5
Start 0
No. Sectors 1
and click Sequential READ which reads data into a buffer and then Display and Edit Buffer (to display a maximum of 20 x 512 byte sectors) and then maximise the form.
 |
| You can edit any bytes (e.g. address 01BE as shown) simply by editing the data, then click WRITE using Buffer. You can also cut-and-paste whole rows into the form. The data is in the same format that is returned by RMPrepUSB - Drive Info or File Info commands. |
You can even cut-and-paste whole lines (complete with the far-right ASCII characters).
You should ensure that the 4-character left-hand address field is correct for each line, e.g.
...
0140 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 ........ ........
0150 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 ........ ........
0160 00 FF 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 .ÿ...... ........
0160 00 AA 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 .ÿ...... ........
...
DD will tell you how many bytes have been changed and prompt you before writing the sector(s) when you click on the Write using Buffer button.
If the DD window contains the line:
============= EDIT SECTOR DATA (up to first 20 sectors) ================
then DD will parse and use the hex text displayed under that line as the data, instead of the data currently held in the data buffer.
Search Disk
Another useful feature of DD is the ability to search the disk for certain bytes.This feature had some bugs in the previous version of DD which hopefully have now been corrected.
It supports four types of search:
- t - search for a string (case sensitive) - e.g. tOperating
- T - search for a string (case insensitive) - e.g. Toperating (will find Operating or OPERATING, etc.)
- H -search for a sequence of hex bytes - e.g. H0055AA (will find byte 00 followed by byte 55 followed by byte AA)
- !H - search until you find a single byte that is NOT a certain value - e.g. !H00 will search the disk until it finds a non-zero byte.
The search will begin at the sector set in the Start field and for the number of sectors set in the No. Sectors field. If you want to search the whole disk, select a different drive and then re-select the original drive, the Start and No. Sectors fields will be set to 0 and + the total number of sectors for that drive.
Other Functions of DD
The four buttons at the top-right are intended for advanced users:Fill Buff
Fills the 8MB buffer with this pattern. For instance, if you set No. Sectors to 1000 and perform a Sequential Read operation, the buffer will contain 1000 sectors. You can use Fill Buff to then fill the 1000 sectors with a certain data pattern (two bytes) and display the first 20 sectors using Display and Edit Buffer, or write that data to any part of the disk by setting the Start field. You can only use this for a reasonably small number of sectors (i.e. maximum of 1 track = 63x255 = 16065 sectors = 8MB).
Fill Buffer with Pattern and WRITE
This button uses the Pattern set in the field above and then performs a write operation to the Start address for No. Sectors. A track (63x255) at a time is written (or less). This can be used to wipe a whole disk.
Non-Destructive Random WRITE
This will perform a write sequence (in blocks) across the disk (area is determined by the Start and No. Sectors fields). You will be prompted for the number of passed to be performed and the number of sectors to be used in each block. The sequence is: choose a random sector - read and keep data - write random pattern - read - compare - restore original data.
Random READ
You will be prompted for the number of passes and number of sectors per operation. Blocks of data will then be requested from randomly chosen areas on the disk within the Start and No. Sectors limits.
Reporting
DD also notes the SMART data parameters before and after each disk operation and reports any differences. This is useful for identifying any errors which may have been logged during that operation. This obviously only works on hard disks that support SMART and not on USB Flash memory drives or SD cards or USB drives. The original version used to output the results as variables in a .cmd batch file, so that another batch file could be used to analyse the results and determine whether or not to fail the drive. This version just displays the values on the form.DD also looks at the Windows Event Log before and after an operation. It is often the case that the disk driver will report log error information in the Windows Event Log (for instance a Windows driver may automatically switch to a slower IDE or SATA data transfer rate if it starts to get 'DMA errors', this will cause a sudden decrease in performance across the whole drive). DMA errors ('ULTRA_DMA_CRC_ERROR_RATE') and switching to a slower transfer protocol are indicative of a bad EIDE or SATA data cable, or possibly a badly designed/terminated disk controller PCB.
The WI_D variables are DD Disk operational errors, whereas the WI_E variables are used for Event log event differences and WI_S variables are for SMART error reporting differences:
SET WI_D_TOTERR - Total of all disk errors
SET WI_D_READERR - Total of all disk read errors
SET WI_D_WRITEERR - Total of all disk write errors
SET WI_D_CMPERR - Total of all write-read-data compare errors where the wrong data has been returned.
SET WI_E_INFO - Total Event Log informational messages that occurred during last operation
SET WI_E_WARNINGS - Total Event Log warnings
SET WI_E_ERRORS - Total Event Log errors
SET WI_S_ERRORS - SMART errors
SET WI_S_WARNINGS - SMART warnings
SET WI_S_INFO=0 - SMART info (should be examined manually)
The raw output of SMART data (before and after) is also displayed. Note that the meaning of the raw data varies from manufacturer to manufacturer and so it is tricky to analyse (e.g. READ_ERROR_RATE raw data).
To see the SMART data, start a Sequential Read operation on a SATA hard disk and after a minute or so, click Cancel to see the SMART data results.
DD analyses some SMART data differences and adds up any changes in error or warning counts after an operation has completed, for instance the following changes are taken into account:
WI_S_ERRORS:
POWER_CYCLE_COUNT
UNCORRECTABLE_SECTOR_COUNT
REALLOC_SECTOR_COUNT
SEEK_ERROR_RATE
If TEMPERATURE_Deg_C_or_F > 79 Deg C
WI_S_WARNINGS:
SPIN_RETRY_COUNT
REALLOCATION_EVENTS_COUNT
ULTRA_DMA_CRC_ERROR_RATE
SlowRateErrors
WI_S_INFO:
READ_ERROR_RATE
No comments:
Post a Comment