View Issue Details

IDProjectCategoryView StatusLast Update
0006660Kali LinuxGeneral Bugpublic2020-12-01 10:48
ReporterLRitzdorf Assigned Tosbrun  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version2020.2 
Summary0006660: Updates prevented by apt hash mismatch (VirtualBox)
Description

In Kali 2020.2a, running in VirtualBox (installed via the official VB appliance, which was also verified by hash after download), running sudo apt update throws a hash mismatch error.

It appears that the update server tells Kali to expect a Packages.gz with a certain hash, but Kali calculates a different one. Kali being as security-conscious as it is, this blocks any kind of update from occurring.

Steps To Reproduce

Install Kali Linux in VirtualBox (currently 2020.2a and 6.1.12 r139181, respectively) via the OVA file available from the official Kali download page.

Start the VM, log in, and run sudo apt update.

Observe error message regarding hash mismatch.

Additional Information

apt error message:

Get:1 http://kali.download/kali kali-rolling InRelease [30.5 kB]
Get:2 http://kali.download/kali kali-rolling/main amd64 Packages [16.5 MB]                                                                                           
Err:2 http://kali.download/kali kali-rolling/main amd64 Packages                                                                                                     
  Hash Sum mismatch
  Hashes of expected file:
   - Filesize:16521503 [weak]
   - SHA256:274dc57d633c1759ed28e14c690fad2307290afcd4e47202ebc0760b647a3849
   - SHA1:a53a4cf5dbf3a3d7d82cc9b27e67a73133e349f6 [weak]
   - MD5Sum:ef2e2dd621d70046a1a65059af3b33a7 [weak]
  Hashes of received file:
   - SHA256:a0f93ca9e312fac80af6e41922b50cb80f4023e394c9d50602f57faf76f4de9f
   - SHA1:a9c20a9df910858db914cf3430414896fb3c6adf [weak]
   - MD5Sum:ef2e2dd621d70046a1a65059af3b33a7 [weak]
   - Filesize:16521503 [weak]
  Last modification reported: Mon, 17 Aug 2020 12:02:40 +0000
  Release file created at: Mon, 17 Aug 2020 12:03:35 +0000
Get:3 http://kali.download/kali kali-rolling/non-free amd64 Packages [197 kB]                                                                                        
Get:4 http://kali.download/kali kali-rolling/contrib amd64 Packages [102 kB]                                                                                         
Fetched 16.9 MB in 9s (1,889 kB/s)                                                                                                                                   
Reading package lists... Done
E: Failed to fetch http://kali.download/kali/dists/kali-rolling/main/binary-amd64/Packages.gz  Hash Sum mismatch
   Hashes of expected file:
    - Filesize:16521503 [weak]
    - SHA256:274dc57d633c1759ed28e14c690fad2307290afcd4e47202ebc0760b647a3849
    - SHA1:a53a4cf5dbf3a3d7d82cc9b27e67a73133e349f6 [weak]
    - MD5Sum:ef2e2dd621d70046a1a65059af3b33a7 [weak]
   Hashes of received file:
    - SHA256:a0f93ca9e312fac80af6e41922b50cb80f4023e394c9d50602f57faf76f4de9f
    - SHA1:a9c20a9df910858db914cf3430414896fb3c6adf [weak]
    - MD5Sum:ef2e2dd621d70046a1a65059af3b33a7 [weak]
    - Filesize:16521503 [weak]
   Last modification reported: Mon, 17 Aug 2020 12:02:40 +0000
   Release file created at: Mon, 17 Aug 2020 12:03:35 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Note that the weaker MD5 sums do indeed match, while both SHA-based sums do not.

Activities

iiamloz

iiamloz

2020-08-18 15:08

reporter   ~0013288

You most likely have a IDS or IPS system in place which may be affecting the hash values after it intercepts the traffic if you try this:

sudo nano /etc/apt/sources.list

change this line:
deb http://http.kali.org/kali kali-rolling main non-free contrib

to:
deb https://http.kali.org/kali kali-rolling main non-free contrib

LRitzdorf

LRitzdorf

2020-08-18 19:48

reporter   ~0013290

Okay, now it looks different. After changing to the HTTPS protocol as @iiamloz suggested, none of the sums match (not even MD5, which did previously), and the error persists.

It's worth noting that I am currently on a university network, although this issue first appeared on private, secure home WiFi.

New error message:

Get:1 https://kali.download/kali kali-rolling InRelease [30.5 kB]
Get:2 https://kali.download/kali kali-rolling/main amd64 Packages [16.5 MB]
Err:2 https://kali.download/kali kali-rolling/main amd64 Packages
Hash Sum mismatch
Hashes of expected file:

  • Filesize:16529193 [weak]
  • SHA256:1c1acd65e890e618abd7e9485c10e4dc3910ac4541eaf5257d43bd7cd4354c97
  • SHA1:776197031efaa718628377e7af681b7e8f775c71 [weak]
  • MD5Sum:3a74af9ae9acc09c0a3781c2be62e5eb [weak]
    Hashes of received file:
  • SHA256:4dc97b03880feff385f46b678dc77478c0e2f4c59419e1574ae01768cc2b6ab2
  • SHA1:072f2bc1f1db1276a9fc402e8220767dd2a4680f [weak]
  • MD5Sum:97e0eff3ff379af20c9c702f1e8c3d6c [weak]
  • Filesize:16529193 [weak]
    Last modification reported: Tue, 18 Aug 2020 18:04:27 +0000
    Release file created at: Tue, 18 Aug 2020 18:05:22 +0000
    Get:3 https://kali.download/kali kali-rolling/non-free amd64 Packages [197 kB]
    Get:4 https://kali.download/kali kali-rolling/contrib amd64 Packages [102 kB]
    Fetched 16.5 MB in 17s (961 kB/s)
    Reading package lists... Done
    E: Failed to fetch https://kali.download/kali/dists/kali-rolling/main/binary-amd64/Packages.gz Hash Sum mismatch
    Hashes of expected file:
    • Filesize:16529193 [weak]
    • SHA256:1c1acd65e890e618abd7e9485c10e4dc3910ac4541eaf5257d43bd7cd4354c97
    • SHA1:776197031efaa718628377e7af681b7e8f775c71 [weak]
    • MD5Sum:3a74af9ae9acc09c0a3781c2be62e5eb [weak]
      Hashes of received file:
    • SHA256:4dc97b03880feff385f46b678dc77478c0e2f4c59419e1574ae01768cc2b6ab2
    • SHA1:072f2bc1f1db1276a9fc402e8220767dd2a4680f [weak]
    • MD5Sum:97e0eff3ff379af20c9c702f1e8c3d6c [weak]
    • Filesize:16529193 [weak]
      Last modification reported: Tue, 18 Aug 2020 18:04:27 +0000
      Release file created at: Tue, 18 Aug 2020 18:05:22 +0000
      E: Some index files failed to download. They have been ignored, or old ones used instead.

Note that HTTPS is now being used, as expected.

Kunkaa

Kunkaa

2020-08-19 15:00

reporter   ~0013291

You can try this, it worked for me

Because apt use sha256 method from libgcrypto20, but optimized too much. We can deny this opt.
using configuration file /etc/gcrypt/hwf.deny.

$ sudo bash

mkdir /etc/gcrypt

echo all >> /etc/gcrypt/hwf.deny

Kunkaa

Kunkaa

2020-08-19 15:14

reporter   ~0013292

After the code just run apt-get update it should work

LRitzdorf

LRitzdorf

2020-08-19 16:32

reporter   ~0013293

Thank you, that did indeed fix the issue.

However, since the hash problem occurs immediately with a default VirtualBox install, shouldn't this fix be implemented by default?

Kunkaa

Kunkaa

2020-08-19 16:58

reporter   ~0013294

Oh it definetly should, took a while to find the solution.

LRitzdorf

LRitzdorf

2020-08-19 17:37

reporter   ~0013295

All right, another update on the issue. After your modifications (and while still using HTTPS), sudo apt update works properly. However, upon running sudo apt upgrade, further hash sum mismatches occur, on more than 50% of all downloaded ".deb" files. Removing the hardware optimization flag file does not resolve this.

steev

steev

2020-08-19 17:42

manager   ~0013296

What is your host system? Is there any antivirus running?

The workaround above disables hardware optimizations in gcrypt, nothing else.

LRitzdorf

LRitzdorf

2020-08-19 19:16

reporter   ~0013297

I'm on Windows 10, build 1909 (as up-to-date as I can be at the moment). I have Windows Defender enabled, and the free version of Malwarebytes installed. (Note that the latter cannot monitor proactively, however.)

As noted previously, I'm also on a university network for the moment, although this issue first appeared on a private, secure home network, with Kali 2020.1.

LRitzdorf

LRitzdorf

2020-08-21 04:04

reporter   ~0013300

A friend of mine just installed Kali in VirtualBox on a Macbook, and was able to run apt update and apt upgrade without issue, using default settings. Therefore, this issue appears to be restricted to Windows host systems.

rhertzog

rhertzog

2020-08-26 12:11

administrator   ~0013324

@Kunkaa Where/how did you find this solution?

It's either a bug if libgcrypt or (more likely) a bug in virtualbox badly virtualizing some hardware feature to compute checksums. It's nice to have a work around but this should really be fixed where it's broken, aka likely in virtualbox itself.

I have not found any related bug report in https://www.virtualbox.org/report/10

The closest I found is this one: https://www.virtualbox.org/ticket/19695
But here it's really the downloaded data that is corrupted, it's not only the checksum that differs. Still it would be nice to know if disabling HyperV and Container features helps.

One of those who can reproduce the issue should file a bug report there... see https://www.virtualbox.org/wiki/Bugtracker

LRitzdorf

LRitzdorf

2020-08-27 04:43

reporter   ~0013331

All right, I'll file a report with VirtualBox. Also, it would be nice to test another linux distro in a VM to verify this - I'll look around and see if I can find a Ubuntu appliance or something similar.

Relatedly, how does one close an issue here?

LRitzdorf

LRitzdorf

2020-09-25 04:14

reporter   ~0013486

After a rather long lapse, I have another update. There was an existing VirtualBox ticket for this (https://www.virtualbox.org/ticket/19695), and it appears the issue will be fixed when a VirtualBox version above r140284 is released.

I don't see a way to mark this issue as closed, so I'll just leave it for now -- if a moderator (or someone else) with this ability sees this, feel free to close this ticket.

Also, thank you to everyone who contributed above! As this is my first Kali bug report, I'm grateful for the assistance.

rhertzog

rhertzog

2020-09-25 06:26

administrator   ~0013487

We will keep this bug open until we import virtualbox 6.1.15 (once released).

Kunkaa

Kunkaa

2020-09-25 06:52

reporter   ~0013488

thumbs up

LRitzdorf

LRitzdorf

2020-10-24 22:26

reporter   ~0013589

I'm happy to announce that a post to a Kali Forum topic related to this issue has contributed to its resolution! (Original post https://forums.kali.org/showthread.php?48809-Kali-2020-2-OVA-(VirtualBox)-setup-errors&p=99519#post99519)

Solution (for Windows):
Open the "Turn Windows features on or off" item from the Start search bar and disable the "Windows Hypervisor Platform" entry.

After rebooting my actual computer twice, VirtualBox no longer showed the turtle variant of its virtual CPU icon (which it did previously, and began doing so near the time when this issue appeared for me), and apt update and apt upgrade worked as expected. Also, this change does not seem to have affected my WSL (version 1) installation of Kali Linux, and should thus be completely safe.

LRitzdorf

LRitzdorf

2020-10-28 06:43

reporter   ~0013596

Okay, yet another update. I just received an update to Windows 10 version 2004, which enables WSL 2. Unfortunately, WSL 2 requires that the "Virtual Machine Platform" Windows feature be enabled, which causes VirtualBox to return to its slow virtualization mode (with the turtle vCPU icon). For users who need both VirtualBox and WSL, I feel this issue would be best resolved with a VirtualBox setting to select the virtualization mode (if that's even possible).

sbrun

sbrun

2020-10-29 16:00

manager   ~0013601

virtualbox version 6.1.16 is now in kali-rolling

Issue History

Date Modified Username Field Change
2020-08-17 17:16 LRitzdorf New Issue
2020-08-18 15:08 iiamloz Note Added: 0013288
2020-08-18 19:48 LRitzdorf Note Added: 0013290
2020-08-19 15:00 Kunkaa Note Added: 0013291
2020-08-19 15:14 Kunkaa Note Added: 0013292
2020-08-19 16:32 LRitzdorf Note Added: 0013293
2020-08-19 16:58 Kunkaa Note Added: 0013294
2020-08-19 17:37 LRitzdorf Note Added: 0013295
2020-08-19 17:42 steev Note Added: 0013296
2020-08-19 19:16 LRitzdorf Note Added: 0013297
2020-08-21 04:04 LRitzdorf Note Added: 0013300
2020-08-26 12:11 rhertzog Note Added: 0013324
2020-08-27 04:43 LRitzdorf Note Added: 0013331
2020-09-25 04:14 LRitzdorf Note Added: 0013486
2020-09-25 06:26 rhertzog Note Added: 0013487
2020-09-25 06:52 Kunkaa Note Added: 0013488
2020-09-25 06:55 sbrun Assigned To => sbrun
2020-09-25 06:55 sbrun Status new => assigned
2020-10-24 22:26 LRitzdorf Note Added: 0013589
2020-10-28 06:43 LRitzdorf Note Added: 0013596
2020-10-29 16:00 sbrun Status assigned => resolved
2020-10-29 16:00 sbrun Resolution open => fixed
2020-10-29 16:00 sbrun Note Added: 0013601
2020-12-01 10:48 g0tmi1k Priority immediate => normal