Bulk duplication of USB drive images

Let us suppose that you have a 32GB Corsair Voyager GT multiboot USB Flash drive (e.g. containing Easy2Boot) which contains all of your boot files and utilities and now your 12 colleagues each want to have the same files on their USB Flash drives.

Now you obviously cannot just do an 'image copy' from one USB stick to another because they will be of slightly different makes, models and sizes. A 32GB Patriot XT USB stick may be up to 500MB smaller than a 32GB Corsair Voyager flash drive for instance. Here is what RMPrepUSB - Drive Info reports for these two 32GB drives:

"Corsair Voyager GT 3.0" (32,054,968,320 bytes)
Reported size 32,054,968,320 bytes (29.8535GiB)  Last LBA 62,607,359

"Patriot Memory" (32,019,316,736 bytes)
Reported size 32,019,316,736 bytes (29.8203GiB)  Last LBA 62,537,727

So if you tried to copy an image of the Corsair Voyager onto the Patriot XT stick, you would get an error. Even if you ignored the error, the last 300MB on the partition would not exist and the user would get a problem if he/she tried to fill up the drive.

What you may not realise is that even if all the USB sticks were of the same make, model and revision, they can vary in reported size! This is because the manufacturer will map out any bad 'pages' of memory if they fail during factory testing.

How to bulk duplicate USB drives

Step 1. Find the smallest USB drive that you will need to copy the image to - use RMPrepUSB - Drive Info to do this. e.g.

                               Reported size 32,019,316,736 bytes (29.8203GiB)  Last LBA 62,537,727

If you don't know the smallest size, then guess 'under' - e.g. if 16GB is the smallest size anyone has, choose 15250MiB (16,000,000,000 / 1024*1024).

Flash drive manufacturers always advertise drive capacity in GB not GiB. 

1GB = 1000x1000x1000 = 1000000000 bytes.

1Gib = 1024x1024x1024 = 1073741824 bytes.

1MB = 1000x1000 = 1000000 bytes

1MiB = 1024x1024 = 1048576 bytes

so:

32GB = 30517MiB   32GiB = 34359738368 bytes

16GB = 15258MiB   16GiB = 17179869184 bytes

So a '32GB drive' is 32 thousand million bytes or 29.8023GiB. That is why Windows reports it as 29.8GB (it really means 29.8GiB!).

Note that a 32GB drive will have 32GiB of memory inside it. This leaves up to 2.359 GB of 'spare' memory which the manufacturer can use for bad memory management, page swapping, etc. and still have a 32GB drive available for the user.

Step 2. Prepare your USB drive using the 'smallest' partition size - let us assume 15250MiB if the smallest drive is going to be 16GB. We can use RMPrepUSB and put 15258 in the Size field (use 30517 for 32GB drives).

If you don't want to prepare a new drive. Use Easeus Partition Master to shrink the partition so that it ends at  15258MiB.

Step 3. Use RMPrepUSB - Drive Info to check that the partition ends at the correct place.

This example shows the End of the last partition at 32,019,316,736 bytes which is 19MB larger than the smallest possible 32GB drive, so it is slightly too large and needs to be reduced by 19MB:

P1   Start=63 (32,256 bytes) End=62,537,727 (32,019,316,224 bytes)

P2   Start=62,537,728 (32,019,316,736 bytes) End=62,537,790 (32,019,348,480 bytes)

Step 4. Now make an image of all partitions using:

RMPrepUSB - Drive->File

• Name = myimage.bin
• Drive Sector Start = 0
• Length = PALL
• File Start Byte Position = 0

This will only capture up to the end of the last partition and not the whole drive.

5. Now download and install PassMark's ImageUSB utility (free). You can now plug in all your USB drives to all your USB ports and write the image to all of them at the same time.

ImageUSB is a free utility which lets you write an image concurrently to multiple USB Flash Drives. Capable of creating exact bit-level copies of USB Flash Drive (UFDs), ImageUSB is an extremely effective tool for the mass duplication of UFDs. ImageUSB also supports writing of an ISO file byte by byte directly to an USB drive

Note: ImageUSB cannot be used for image capture because it saves all physical sectors of a USB drive and so can only reliably write the image that is created by it to identically sized or larger flash drives.

Tip 1: Use a powered USB 3.0 hub - these are generally much faster than USB 2.0 hubs

Tip 2: If you just want to write to one USB Flash drive at a time, you can use the RMPrepUSB - File->Drive button instead of using ImageUSB.

6. If, for instance, you put a 32GB image on a 64GB USB Flash drive, you can expand the partition using Easeus Partition Master. This should not affect it's bootability but check it boots afterwards just to be sure!

Note: A 'free for home use' Easeus Partition Master version is also available.

P.S. Although you can duplicate most USB boot drives in this way, you cannot duplicate HitMan Pro USB Flash drives unless the 'target' USB drive has EXACTLY the same reported size as the original drive (check using RMPrepUSB - Drive Info) and you make an image of the whole drive and not just the drive partitions. HitMan Pro writes code to the very end of the drive.

Not enough room to defragment files? Try Refresher.cmd

Easy2Boot and many grub4dos bootable USB drives require that the ISO files are contiguous (i.e. the ISO file occupies sequential clusters on the drive).

If your Flash drive is almost full, you may find that you cannot defragment a large ISO that you have just copied over to it. This is because there is not enough free space left on the drive.

By the way: If there is enough free space on an NTFS drive and you still cannot make a very large file contiguous, see my previous blog post here.

For instance, if you have a 16GB USB Flash drive with 3GB of free space, you cannot defragment and make contiguous a file that is larger than 3GB.

Even if there was enough free space, the defrag of the ISO file could take a very long time.
If your USB Flash drive has reached this stage, it is best and quickest to reformat it. If you want to keep all the files on it, then you obviously need to make a backup first.

You also stand a better chance of having all files contiguus, if you copy over the large ISOs first, before the rest of the smaller files.

To simplify this process, I wrote a Windows batch file to automate this. Just drag-and-drop your USB drive icon onto the Refresher.cmd file (keep it on your Desktop for convenience) and it will guide you through process of:

1. Backing up all files
2. Formatting the USB volume (choose NTFS or FAT32)
3. Copying back to the USB drive all the large >500MB files first
4. Copying back to the USB drive the rest (<500MB) of the files
5. Copying back the empty folders
6. Comparing the file count in Step 1 with that of the USB drive now.
7. Deleting the backup folder

If a folder is chosen instead of a drive, then instead of formatting the drive volume, the Windows 'rd' command is used to delete the folder and all sub-folders, and then the original files are copied back. In practise, this rarely defragments the files however. You will be far more successful if you reformat the whole drive by choosing the Drive icon as shown above.

To use Refresher.cmd, copy the Refresher.cmd file to your Windows Desktop, then drag-and-drop the drive icon of your USB drive onto the Refresher.cmd Desktop icon (or type Refresh N: from a Windows command prompt console if you need Admin rights). Then just choose either NTFS or FAT32 to reformat the drive when prompted. At this point you can reformat using RMPrepUSB if you wish and just Skip the Windows format. Note that RMPrepUSB partitions and formats, it is not a format-only tool. If you have a multi-partition USB Hard disk, you will have to use the Windows format tool as the contents of only one drive volume letter is backed up by Refresher.cmd.

User input shown in red squares.

WARNING: As this is quite a powerful batch file, please take care when using it. If the contents of the drive being 'Refreshed' are very valuable, make a backup first (you should have one anyway!). The batch file assumes you will press CTRL+C if you see anything wrong. The temporary backup folder will be in the same folder as the Refresher.cmd file, so make sure there is enough room on the hard disk. A new backup folder is made (and deleted) each time.

Note that Windows format will remove the grub4dos boot loader from the Partition Boot Record (PBR). If you find that the USB drive no longer boots after running Refresher.cmd, use RMPrepUSB - grub4dos Install and choose 'No' for a PBR install of grub4dos. This should fix the boot problem. Then test that it boots using QEMU or VBox or test on a real system. To avoid losing the PBR code each time you use Refresher.cmd, install grub4dos to the MBR instead.

Download Refresh.zip

Notes on text 'screengrabbing' and data collection in the grub4dos console

Sometimes you may be asked for debug information using the grub4dos shell/console.

The problem is how do you get all this information to the grub4dos developer if this needs to run on a real system and not a Virtual Machine? Normally, you would take a photograph with your digital camera or cell-phone, download the pictures onto your computer and then upload them or email them to the developer.

However, you can use my grab.g4b batch file to capture the screen instead:

1. Download and extract the contents of the Grab.zip file to the root of your grub4dos drive
2. Boot to grub4dos and get to the grub4dos command console
3. Type graphicsmode 3 to get into text mode
4. Execute any debug commands so that the results are displayed on the screen (80chars x 24 lines)
5. Type /grab.g4b to capture the text into the file grab.txt.
6. If you want to capture another screen to a different file, use /grab.g4b 1  to capture the screen text to the file grab1.txt.
      grab.txt, grab1.txt, grab2.txt...grab8.txt are provided in the download. They must exist before the command is executed.
7. You files can now be sent to the developer.

Capturing memory data

If you are asked to send the results of a cat command - e.g.

cat --hex (md)0x4e+3

This will display quite a lot of data - far more than one screen full. You can redirect the output to an existing file using:

cat --hex (md)0xe4+3 > /myfile.txt

as long as myfile.txt exists and is big enough. Note: If using an NTFS filesystem, the file must be at least 1K or more or the writes won't work.

Unfortunately you cannot use more than 3 sectors as the grub4dos redirection buffer appears to be limited to just 8192 bytes - e.g. cat /file1.txt > file2.txt will only work for the first 8192 bytes.

If you need to collect more than 3 sectors of hex data ouput, you can use the mdcat.g4b batch file included in the download.

/mdcat.gb4 0x4e

The results will be in /mdcat.txt - use NotePad++ to open it (not Windows NotePad).

Easy2Boot v1.30 Trial version available

Changes:

• New file extension .isopup supported for Puppy linux ISOs (no persistence)
• E2B can now be in a Logical partition on the USB HDD boot drive
• grub4dos 0.4.6 with USB driver menu option in Utilities menu
• Option in MyE2B.cfg to use grub4dos 0.4.6 USB driver on start-up of E2B
• If CheckAccess fails on start-up due to buggy BIOS, the grub4dos 0.4.6 USB drivers are used automatically.
• Make_E2B_USB_Drive.cmd script improved (Grubinst on an NTFS formatted drive didn't always work + switch to root folder). Now moved to the \_ISO\docs\Make_E2B_USB_Drive folder.
• Change detection code of WINHELPER.USB file on 'Helper' Flash drive
• .isoWB file extension (Easy2Boot_v1.30b_TESTONLY.zip)

This version should be considered experimental and can be found via the Alternate Download link on the Easy2Boot.com download page here.

grub4dos 0.4.6a USB driver

Using the grub4dos USB driver helps in cases where some BIOSes contain bugs in their USB driver code, and as a result you may find that one or two 'special' systems will not boot to certain payload/ISO files unless you use the new grub4dos USB driver. See my previous blog for details.

The Main menu in E2B v1.30 now has a new menu entry which has two functions. When you first boot E2B, you will see an entry to 'Switch to grub4dos v0.4.6'. If you select this, E2B will reload E2B using grub4dos 0.4.6 (the file is in \_ISO\e2b\grub\grub_46a).

When the the Main menu reloads, you will now see a Main menu entry 'Install Grub4DOS USB Driver'. If you run this the internal grub4dos USB driver will run and will replace the BIOS USB driver. The Main menu will reload and the the 'Install Grub4DOS USB Driver' menu entry will no longer be present; thus if you don't see this menu entry, it means that the grub4dos USB drivers are already loaded.

If you don't want this new menu entry, simply delete the \_ISO\MAINMENU\ZGrub_USB_046.mnu file.

I have not replaced grub4dos v0.4.5 with grub4dos v0.4.6 because there are still a few problems with it. e.g. running usb --init to load the USB drivers twice under VBox/VMWare seems to cause it to hang; does not detect USB drives on Dell Inspiron 530, does not work very well on reboot - best if run from power on.

Note, when you load the grub4dos USB driver, you will see a brief message about how many USB drives were detected. If it does not detect your USB drive, then the BIOS will still be used to access the USB drive. If you have any systems which don't detect your USB drive (always test from power on, not a reboot), please report it to chenal here. Give give details of the grub4dos version you are using as 0.4.6a Date=20140306 Size=281,839 bytes and the make and model of your system and USB drive.

Buggy USB support in BIOSes

Most BIOSes can boot from a USB drive these days, but they don't always work well.

For instance, my Asus EeepC 904HA netbook has buggy USB boot code in the BIOS. If I boot to Easy2Boot from my 2TB USB hard disk, when CheckAccess.g4b runs (which checks access to the end of the last partition on the USB boot drive), it reports loads of read errors. This is because the USB code in the BIOS cannot access any sector past 137GB on the USB drive (although the BIOS can access all sectors on the internal hard disk).

For another example of buggy USB BIOS support, see here. In this case Leolo had a system with an AsRock P4i65G board which would not boot successfully to WinPE (Win7 install) from a USB drive - but only the AsRock system - all other systems booted fine.

In the past, one way of getting around this type of buggy BIOS was to use Plop! This loads it's own USB USB 2.0 driver and so does not use the BIOS USB driver. The problem with Plop! is that the driver is read-only. This is OK for primitive booting but not for boot managers which need read/write access to the USB drive (such as Easy2Boot).

In both these cases, using the new grub4dos v0.4.6a  fixes the problem. This version of grub4dos contains a USB 2.0 driver but it is not active by default. You need to issue a command to initialise the USB driver:

      usb --init

The USB driver in grub4dos 0.4.6a still has a few bugs, but the latest (developer) version is looking much better.

When it is robust enough, I will add it in to Easy2Boot. If the CheckAccess.g4b fails, E2B will ask if you want to switch to v0.4.6a. There will also be a variable in the MyE2B.cfg file to use the new version and USB driver, as well as a new Main menu entry (determined by a .mnu file which you can remove if you don't want it).

If you have a system with a 'bad' BIOS that boots to Easy2Boot but no further, you can try the new version with Easy2Boot as follows:

1. Extract the grldr file from here (this is a new, developer test version 5th March 2014 - check here for a later version) and overwrite the grldr file in the root of your E2B drive.

2. Edit the \menu.lst file in the root of the E2B drive and add

usb --init

as the first line, or try 

set /p ask=Use grub4dos USB driver (Y/N) : 

if /i "%ask%"=="Y" usb --init

if you want the option to install the USB drivers.

Easy2Boot downloads now at www.easy2boot.com

It seems DropBox got upset with me using my DropBox Public folder for others to download E2B+DPMS.zip and so they have suspended my Public folder as there was too much traffic!

So I have now moved the Downloads to the new www.easy2boot.com site.

New Easy2Boot website

I have started a new website dedicated to Easy2Boot at www.easy2boot.com

It should be faster to load and more attractive than the rmprepusb.com site.

Please give me your feedback (my web design skills are sadly lacking, as you may have noticed!)

cheers
Steve

Easy2Boot v1.29 now available

V1.29 changes are:

• New grldr grub4dos file - now shows the case of file names and folder names correctly on FAT32 volumes when in grub4dos console.
• New .isomemF01 and .isofira01 file extensions added (use .isomemF01 for ERD 2005 XP ISOs)
• Sample .mnu files added or modified (Porteus-v2.0-i486.mnu, FD0-konboot-v2.1.mnu, dft32_v416_b00_install.mnu, BIOS Detect and Run.mnu, Porteus-generic-persistent.mnu, Win7_8_Install_Auto64_32.mnu, RedHat.mnu, RedHat_NTFS.mnu, 64Bit_Auto_Detect_ISO.mnu, WinBuilderPE_ISO.mnu, ERD5_DPMS.mnu, ERD2005.mnu, ERD5_FIRA.mnu)
• "E2B TXT Maker" and "E2B MNU Maker" batch files available in \_ISO\docs\E2B Utilities folder. See blog post here for details.

Download from Tutorial 72a here.

Add Microsoft ERD Commander and MSDart Windows Recovery ISOs to Easy2Boot

Microsoft Diagnostics and Recovery Toolset (Microsoft DaRT) allows you diagnose and repair a Windows system that has problems starting or has other issues.

Using Easy2Boot you can even boot directly from an MSDaRT 5.0 XP PE based ISO and repair a SATA Windows XP hard disk even though the MSDaRT ISO does not contain any AHCI mass storage drivers!

You can also boot via UEFI too (see below for details).

DaRT is not available from Microsoft as an .iso file. Instead, you use a program that creates an ISO file, which is based on Windows Recovery Environment (WinRE) and a set of tools that DaRT provides. This boot media starts the Windows RE, from which you can start ERD Commander. ERD Commander provides a launch platform for the DaRT tools.

Note: MediCat contains Win10/8/7 MSDaRT ISOs, just copy it to your E2B USB drive (keep extension as .iso) ;-)

Typically, you use something like the ERD Commander Boot Media Wizard to create the ERD Commander ISO (although this can vary depending on the version you are trying to create). It is easiest to create the ISO on the same type of OS that it was designed for (e.g. you cannot create a Dart 5.0 ISO using a 64-bit Win7 system).

.isoPE01

DaRT 8.1 supports Windows 8.1 and Windows Server 2012 R2.
DaRT 8.0 supports Windows 8 and Windows Server 2012.
DaRT 7.0 supports Windows 7 and Windows Server 2008 R2.
DaRT 6.5 supports Windows 7 and Windows Server 2008 R2.
DaRT 6.0 supports Windows Vista and Windows Server 2008.

.isomemF01 or .isomemwinv
DaRT 5.0 supports Windows 2000, Windows XP, and Windows Server 2003.

ERD Commander 2005 supports Windows NT 4.0 (Service Pack 4 or later required), Windows 2000, Windows XP and Windows Server 2003. There is also an ERD Commander 2007.

Tip: If you download an .msi file which won't run under Win x64, try extracting the .ISO file from it using 7zip.

If your ISO is based on Vista, Win7 or Win8 PE (DaRT6 and above), then simply rename the .ISO file to .isope01 and copy it to a suitable folder (e.g. \_ISO\MAINMENU). You can also add a .txt file if you want to define the menu entry text and help text.


XP-based ISOs

If you have an XP-based ERD/MSDaRT ISO (DaRT5 or ERD Commander 2005), try using the file extension .isomemF01 or .isomemwinv.

To determine which E2B file extension works best, rename the .ISO files to .ISOask, and then you will be able to choose from a variety of different methods (see below).

For XP-based Recovery PE ISOs, you can also try using one of the .mnu files. In E2B \_ISO\docs\Sample menu files - there are three .mnu files you can try:

1. ERD5_FIRA.mnu    - this uses FiraDisk
2. ERD2005.mnu         - this uses WinVBlock to load the ISO
3. ERD5_DPMS.mnu   - this uses FiraDisk+WinVBlock+DPMS (recommended)

Typically, #1 or #2 will work for most systems. However, if you try to boot an XP PE ISO which does not contain AHCI mass storage drivers, once the Windows Recovery OS has booted, it will not be able to see your internal hard disks or may even BSOD.

To overcome this, use E2B+DPMS which has lots of internal mass storage drivers and use the ERD5_DPMS.mnu file (read the instructions inside it) . This mimics the same process as if you were booting from an XP Install ISO file and it will try to identify and load the correct mass-storage driver for the system. This will allow you to boot from an ERD v5 ISO on a SATA/RAID Windows XP 32-bit system and then repair it.

I would recommend that you use the ERD5_DPMS.mnu for any XP-based Recovery ISOs. You can modify the .mnu file as required.

# Use for MSDaRT 5 XP-based ISOs (use DPMS2 version of Easy2Boot for AHCI drivers)

# Make new empty folder, e.g. \_ISO\MAINMENU\MSDART5

# Copy this .mnu file and the single ERD or PE ISO file to the empty folder (one .iso file per folder)

title XP MSDart ERD (DPMS)\n If DPMS is not installed\n Remember to select an AHCI driver\n AND the FiraDisk driver (using F6).

set MFOLDER=$HOME$

pause --wait=3 Press ENTER 3 times to boot...

/%grub%/XPStep1.g4b

boot

Note: E2B v1.A7 and later versions support the .isoDPMS file extension so you don't need to use this .mnu file.

Note that Easy2Boot+DPMS only contains Windows XP 32-bit mass storage drivers (which may or may not work for Windows 2003). If your DaRT is XP 64-bit based, then DPMS will not work. You can still use F6 to load the FiraDisk 64-bit driver for IDE systems (note: the ISO file name must contain the characters '64' so that E2B will not automatically use 32-bit DPMS drivers).

UEFI booting

You can add any number of MSDaRT (or other) images to an E2B drive and boot from each image in both MBR\CSM mode and UEFI mode (if the image supports UEFI booting - so not XP or Vista 32-bit!). The process is as follows:

1. Run MakePartImage.cmd to make a FAT32 partition image file from your DaRT ISO (or from a working MSDaRT USB Flash drive if you have already made one). It is easiest to just drag-and-drop the ISO onto the Windows MPI_FAT32 Desktop shortcut.
2. Copy the .imgPTN file to your E2B USB drive (ensure it has the .imgPTN file extension or try .imgPTNLBAa for better 'bootability')
3. Run WinContig to make sure the file is contiguous (e.g. \Make_this_drive_contiguous.cmd)

For more details visit the www.easy2boot.com site and look for MakePartImage.

How to quickly make Easy2Boot .txt and .mnu files

The next version of E2B will be v1.29. and will include two script (.cmd) files in the \_ISO\docs\E2B Utilities folder:

E2B TXT Maker.cmd    - generates an Easy2Boot .txt file
E2B MNU Maker.cmd  - generates an Easy2Boot .mnu file
Make SubMenu.cmd     - generates a SubMenu .mnu file

New! E2B v1.78 now includes \_ISO\TXT_Maker.exe.

This may prove useful if you want to make .txt files or .mnu files for all your payload files.

The easiest way to use these files is to first copy them to your Windows Desktop.

Now to make a .txt file or .mnu file for any payload file on your E2B USB drive, simply Drag-and-Drop the payload file onto the E2B TXT Maker.cmd icon or the E2B MNU Maker.cmd icon on your Windows Desktop.

The script will then ask you what menu text and help text you want and then make a new .mnu or .txt file (in the same folder as the payload file) on your E2B USB drive.

You can specify a hotkey key by using ^^ before the keyname (^^ will be translated into ^ when the file is made).

Here are some screenshots of them in action:

E2B TXT Maker (red boxes show user entries)

E2B MNU Maker (no help text was entered in this example)

The E2B MNU Maker assumes that the payload (ISO) file will always be in the same folder as the .mnu file. You can edit the .mnu file after it has been generated. By default, the .mnu file simply runs QRUN and forces the file extension to be the same as it already is, using this line of grub4dos code:

      /%grub%/qrun.g4b force.iso $HOME$/rhel-server-5.7-i386-dvd.iso 

However, you can change this to force the file extension to be something different, e.g. to run a Microsoft Windows 7 Recovery ISO as if it had an .ISO01 file extension, use:

      /%grub%/qrun.g4b force.iso01 $HOME$/ERD6.iso

Or you can add in your own grub4dos menu.

N.B. For the MNU Maker script to work, the E2B payload file name must not contain spaces!


If you want to make a new SubMenu folder:
1. Create a new folder under \_ISO (spaces not allowed) - e.g.  \_ISO\RESCUE_WIN
2. Drag-and-drop the new \_ISO\RESCUE_WIN folder onto the Make SubMenu.cmd file and answer the questions.

This will make a new .mnu file in the \_ISO\MAINMENU folder for \_ISO\RESCUE_WIN.
Now put your ISO files, etc. in \_ISO\RESCUE_WIN and you will see the new menu entry in the Main Menu.