View Issue Details

IDProjectCategoryView StatusLast Update
0008441Kali LinuxGeneral Bugpublic2023-09-18 04:44
ReporterAkeo Assigned Toarnaudr  
Status resolvedResolutionfixed 
Product Version2023.3 
Summary0008441: GRUB Regression: Kali 2023.3 broke File System Transposition support

Either intentionally or unintentionally, sometimes between 2022.x and 2023.3, the Kali ISOs introduced a regression in that it is no longer possible to use File System Transposition to create a working UEFI bootable media.

For reference, per the GRUB mailing list, File System Transposition (FST) is [1]:

The ability to take the content of a UEFI bootable media and copy it, at the file system level, to a partition that was independently created and formatted by the user, while preserving the ability of the media to boot in UEFI mode.

As described in the 'Steps To Reproduce' below, it basically means that, instead of using dd to write the Kali ISOHybrid, one should be able to just format a flash based media to FAT32, extract the content of the Kali ISO there, and use that to boot a UEFI system.

With Kali 2022.x, this method used to work just fine.
With Kali 2023.3, this no longer works, and instead the user ends up with a GRUB error prompt.

[1][email protected]/msg34346.html

Steps To Reproduce

To replicate the issue, as well as understand how FST works, and validate that there exists a regression, you can perform the following:

  1. Partition a USB Flash Drive to GPT with a single generic data partition occupying all the drive (or at least more than the space needed to copy the ISO content)

  2. Format that partition to FAT32/vfat

  3. Extract the whole content of the kali-linux-2022.2-live-amd64.iso onto the FAT32 partition (while making sure to also copy the .disk/ directory, as it is what GRUB uses to locate the boot media)

  4. Plug that media onto any UEFI based x86_64 PC and validate that you can get both to the GRUB menu and then to the Kali live desktop

  5. Repeat steps 1-4 with kali-linux-2023.3-live-amd64.iso and validate that, this time, the boot process stops at a GRUB prompt and that the GRUB configuration files are inaccessible.

Additional Information

Considering that there is a non-negligible possibility that Kali maintainers may be tempted to dismiss this issue as a "Why should we care about FST? Users should only ever use dd to write a Kali ISOHybrid anway.", we feel we need to highlight the following with regards to the reason why treating ISOHybrid ISOs as glorified dd images is actually doing a disservice to people who are trying to install Linux on account that:

  1. Because Windows does not come with dd, Windows users are then being forced to install third-party utilities such as balenaEtcher or Rufus to write an ISOHybrid using dd, which, for a security focused distribution, is less than ideal. Part of the reason for FST is to allow the creation of media, on any base OS (Linux, Mac, Windows, *BSD) without the need to install any third party utilities.
  2. Writing an ISOHybrid in dd mode restricts the user's ability to edit config files. As a matter of fact, the reason we were notified about this regression is because an existing Kali user was using Rufus (which for all intent and purposes uses FST when the user elects to write an ISOHybrid image in "ISO Mode") so that they could modify their GRUB configuration to their liking.
  3. Writing an ISOHybrid in dd mode prevents the user from being able to copy additional files that may be critical for a successful installation, such as WiFi firmware blobs, etc. Nor everybody has two media at hand.
  4. A lot of Windows users are left under the impression that the media creation process did not work if it was created through dd, on account that Windows will no longer list any content on the device or make it look like it drastically reduced in size, leading them to not even attempt to try to boot the media. From my experience, this is actually a very widespread problem for first time Linux users coming from Windows.

There are also some advanced functionality that FST does provide, that DD mode cannot, such as the ability to install a Linux distro using a single media for both source and target (Which is something that is used for instance to install Debian using a single USB 3.0 drive on SBC based devices such as the Raspberry Pi [2]).

All in all then, we hope that one will understand how restricting ISOHybrid creation to dd copy only is doing a disservice to users, because it deprives them of the freedom to create an installation media in the conditions that may be much more appropriate to them.

Furthermore, it also needs to be stressed out that Debian, which appears to be what Kali is based on, does make a very deliberate effort to support FST, which is why we believe that this Kali issue is an involuntary regression, since following the Debian process should produce an ISO that can work both in DD and FST mode.

As a matter of fact, it can be determined that the issue with Kali is not due to a Debian regression, as one can run the latest Debian release and testing ISOs through the steps highlighted above, to validate that FST works just fine for the Debian ISOs. Thus, this issue appears to be restricted to Kali.

Now, while I did find the GRUB source repository for Kali (and validated that it appears to closely follow what Debian does [3]), I have been unable to find public data on how the Kali UEFI GRUB bootloaders are actually being built, so my guess is that the Kali build process has somehow been altered to not include some of the necessary GRUB modules that must be embedded in the GRUB EFI executable for FST to work. At the very least, these should include part_gpt, part_msdos and vfat. And to be comprehensive, one would also want to add ntfs and exfat. These needed to be embedded, rather than provided as separate modules to load, as they are required to access the partitions and file systems where additional GRUB modules can be loaded.

Thus I hope the Kali maintainers can fix FST support, as this is a feature that existing Kali users do rely to have the freedom to customise their boot media [4] in a manner that ISOHybrid-as-dd will never provide, and not see FST support through the narrow lens of "But restricting how users can create Kali bootmedia from ISOHybrid actually makes our job easier", which, if it needs to be reiterated, is something that the GRUB development team as well as the Debian maintainers have made a conscious effort to strive away from because they have come to the realisation that supporting both DD and FST on a equal footing does benefit all users.







2023-09-17 16:48

reporter   ~0018488

Could the priority of this bug please be raised? It is having a very negative impact for Kali users, especially ones trying to create bootable media with persistence.

For evidence of this, please see:

Considering that there is ample evidence that limiting Kali ISOHybrids to dd image writing only is having a detrimental effect for people who have come to rely on FST, at least some acknowledgement of the issue by Kali maintainers and an indication of the planned steps of actions would be welcome.

Thank you.



2023-09-18 04:28

manager   ~0018491

The issue is fixed in the latest weekly image. Please download it from, choose "Weekly Image". The version in the image filename should be "2023-W38" (or higher).

For more background: this is a regression between image 2023.2 and 2023.3, a bogus change in the build tool (live-build, that we get from Debian) broke something. It wasn't detected in Debian, as their live image is not exactly the same as Kali, and the change didn't trigger any issue for Debian images. It broke only the Kali live image, and only in the case where it's installed with Rufus.

We fixed it for Kali, and also we're also upstreaming the fix in Debian:

Thanks for reporting it, and thanks for the links above, I'll update it to ask users to use the latest weekly image.

Issue History

Date Modified Username Field Change
2023-08-30 18:58 Akeo New Issue
2023-09-17 16:48 Akeo Note Added: 0018488
2023-09-18 04:28 arnaudr Note Added: 0018491
2023-09-18 04:44 arnaudr Assigned To => arnaudr
2023-09-18 04:44 arnaudr Status new => resolved
2023-09-18 04:44 arnaudr Resolution open => fixed