Wednesday, June 30, 2021

Allowing Unsupported CPU’s on ESXI 6.7

 

Allowing Unsupported CPU’s on ESXI 6.7

Important Notes:

  • This doesn’t guarantee CPU’s to run with 6.7, only that they will pass the checks, and can install and start to load.

  • This does not apply to other devices that also may be Unsupported.

  • Everything done from here on out will involve modifying the ESXI binaries.

  • I will not provide the modified files.

  • Anything you see from here on is Unsupported and not fit for commercial purposes. Use at your own risk.

Further Considerations:

This workaround will need to be applied every upgrade/patch. Make sure that if you are going to keep the modified files make sure that it is only used for the build you originally modified. If you download a new ISO or Patch your system, go through the steps again with a fresh file related to the new build.


Requirements:

  • Multiple USB sticks (3 or more for ease of access)

  • Hex Editor (https://mh-nexus.de/en/hxd/ )

  • OS other than Windows (Live CD’s work fine https://www.ubuntu.com/ )

  • 7zip (https://www.7-zip.org/download.html )

  • ESXI 6.7 ISO

  • Rufus bootable USB

Additional but not necessary:

  • Disassembler (IDA 7.0 is free https://www.hex-rays.com/products/ida/support/download_freeware.shtml )

  • ESXI 6.5

Introduction

Now this whole process came about for me because I’ve never truly been a fan of the HTML/Flash GUI provided be VMware, mostly not containing the same features as the Thick client or just being unbearable slow to use.

6.5 Brought some much needed improvements, particularly the swap to HTML5, but it wasn’t an ideal platform for me, though with many features not available on the thick client any more migrating had to happen. So I was keen to see the improvements in 6.7 as this was one area that was focused heavily on.


Conquering the Problem

Now I myself had just received 2 Brand new servers’ days before the release and in realty it wasn’t a consideration for what CPU’s I needed. The ones I ended up with as you probably guessed weren’t supported and failed before the install.

Now doing some research on the web yielded minor success and gave some insight into the nature of the problem particularly the article be Reddit user,

blahcodyde

https://www.thehumblelab.com/vsphere-67-homelabs-unsupported-cpu/

He proceeds to use a manual update to bypass many installation CPU checks, and this allows some chips access.

This still doesn’t help me, who had pairs of Intel x5550’s.

I naively thought that perhaps there was merely a script that ran on install that could just be deleted. So I downloaded a copy of 6.7, opened Rufus and created a bootable USB. I opened the USB and started looking for anything obvious.

To my surprise there was indeed something quite obvious located within the upgrade folder, called precheck.py, hmmm.

Loading it up in notepad++, it was indeed a script designed to check the hardware and compare it against supported versions, this seemed promising. So of course looing through it I eventually find the section dedicated to CPU’s. It was a simple routine designed to check the CPU’s model information, particularly anything in the Intel Family of 6 or higher and had a model of higher then 36 was considered supported.

This was a good sign as this literally means that it’s not checking a big list of supported CPU’s just a reported number. I spent a bit of time messing with the numbers but didn’t get anywhere I even just out right deleted the entire folder and it still didn’t work, shame there must be another check elsewhere embedded in the binaries.

Now at this point I wasn’t sure how to proceed, going through the binaries could take quite some time, like a really long time. And I wasn’t quite keen on brute forcing the problem. So I compromised and brute forced the problem.

I decided that the easiest way to find this check was to simply replace files on my 6.7 install USB with files directly from the 6.5 install until I stopped seeing the CPU error message.

This could take some time as there were quite a few files there. I started by ignoring any and all obvious driver files. Then ignore any files related to making the USB bootable. This took he total files I needed to check to around 20 this isn’t too bad. So I started swapping files.

Success I managed to get a boot error related to missing drivers instead of CPU errors, excellent this is good news but not a guaranteed success. The culprit file is B.B00.

Now let’s look into this file opening it up in HxD and I see garbage, great. Not good news. Now having dealt with VIB files in the past and going through quite some trouble with driver injection I know literally most files used by ESXI are compressed or archive files. So I load up B.B00 file into 7zip and I check to see if it can recognise an archive, and it does it’s a gzip file. Cool.

Unzipping the B.B00 file gives an elf file vmkBoot.ELF32-psigned, now this is looking better. Loading up the elf file in HxD we see what is, useful data. Now I need to try and confirm this is indeed the correct file. Having recorded the error message that is first shown when loading 6.7 and do a couple of text searches and

We see the exact error message including pointers for the system to input your unsupported CPU. Now I need to find the check in the code, this is a more difficult as all code is now in assembly.

An easy way is to trace forward from when the program asks for the CPU information. Now you might ask well how the program gets the CPU info well there is a specific assembly instruction for it CPUID.

You can read more here https://c9x.me/x86/html/file_module_x86_id_45.html this site also provides the Assembly code 0F A2 which we can search for HxD will provide quite a few references to this op code and not all of them will be CPUID calls. Going back to the CPUID reference page we can read how CPUID works and find the one we want.

At this point switching to IDA 7.0 is more useful being a full disassembler it becomes considerable easier to read the values.

Now reading the CPUID info what we are looking for is the CPUID call that will provide the CPUs Version Information: Type, Family, Model, and Stepping ID, this is provided when the EAX register contains 1 just before it’s called. This is located at hex off set 5109h.

So now we need to find out where we compare this info to the compatible info. It would be handy if we knew what the lower CPU checking limit was as we wouldn’t need to dredge through too much code. Well remember that precheck file we found earlier. It gives us that code 36h (Don’t expect it to always be this easy, as they may remove that fileJ). Now knowing this code made searching significantly faster as I was able to use a few hex searches to narrow the exact code down and I came across the assembly call cmp ecx,byte +$00000036 also expressed in hex as 83 F9 36. This is the model check. There should only be one instance of this in the file but just to be safe it is at hex offset 124D5. Now we just need to make sure that the model of our own CPU is higher than this check. So you can either find the hex value of you CPU, and make sure that this number is one lower, or just set it to 0. For example 83 F9 17.

Great we made the change let’s test it. Save the file, use 7zip to compress it back into a GZIP file and overwrite the B.B00 file on your bootable USB. Lets load it up and try to install.

Success we’ve come to the install screen and can proceed to set options lets install. But wait It’s telling us the CPU isn’t compatible again. Just as we click install. Hmm. Its also a different message this time, but this one looks quite familiar, we’ve seen it before in fact it is the one that the precheck.py file spits out. Alright lets also modify that file and give its check a smaller number.

After doing this I tried to install again, again no luck the same message, so this file still does nothing. Write then it’s probably embedded in the binaries. Let’s start looking again.

I’m not exactly at a loss here as there was something to note within that python script and it was mentioned at the top. The script is based on weasel.

Looking over the files in the USB there is a peculiar one called Weaselin, let’s take a look. So we extract it out of the gzip archive. And we get weaselin-weasel.vtar. oh another Archive.

&zip can’t handle vtar files, and upon doing some research it appears that vtar is a slightly modified tar archive based on 4k page size rather than tar 256b, and the only tool I found that could extract it is within esxi itself. Though luckily tar files have no compression on them so hopefully this modified one doesn’t either. So opening it up and doing a search for the new error code results in success. In fact it takes us to a location that looks almost exactly like the precheck.py script.

Apart from the difficult to read text translation it shouldn’t take too long to find the relevant par to change. Editing the text field this time and not the hex values changing 36 to anything lower then your CPU’s Model number will work. I selected 17.

Now re zip the back into a gzip file and let’s try to install again.

Success we have the system installed.

Lets boot into our new install.

Failure? And with the first error message again. Alright let’s take a look at the install. I pull out the hard drive plug it into my windows pc and start browsing through the partitions. The first file I look at it the B.B00 file located on one of the partitions. And I notice that the file is different from the modified one I created? So this means that there is yet another copy of this file compressed somewhere on the install ISO. Well not to worry I replace with my modified file and plug the harddrive back into the system.

I get an error but this time it appears that there is a missing boot sector what? Sometime later and I’ve come across info that this happens only when you plug an installed copy of ESXI into windows. Apparently windows machine modifies the partition tables when it sees them.

So there isn’t any fix to this other than reinstalling esxi. Righto let’s move on, Reinstall 6.7 pull the drive boot up my Ubuntu live USB, insert ESXI harddrive, copy my modified B.B00 file across and overwrite the old one, plug the harddrive back in and boot.

Closing Thoughts

What a pain this turned out to be, I also tried to modify the Offline Package in an attempt to do a manual install. This however was a far worse option as the number of SHA values that needed to be change was significant and resulted in upwards of 15 changes needing to be made.

I would also recommend doing an upgrade from 6.5 instead of a fresh install just to include any drivers. Several brief warning/errors occurred during boot but were 2 quick to read appeared. I’m not sure on how important they were though I have yet to have a stability problem with the system.

I also never did figure out the purpose of the upgrade folder included on the 6.7 ISO. No matter whether changed removed or left it it never seemed to be used in any process. I suppose there really wasn’t any harm in just modifying it as well just in case.

I also had another idea of simply rewriting what the CPUID returned when called so that you could always return a valid CPU. This maybe the path in future if it becomes exceedingly difficult to find check value needed to change.

Installing ESXI on to a USB drive is handy for the last step, and it should still be possible to install on a RAID drive if you use a Ubuntu live CD on the target machine then copy the Modified B.B00 file over.

Summarized steps

  • Create Install USB with ESXI 6.7

  • Copy files B.B00 and Weaselin to desktop

  • Extract B.B00 and Weaselin.t00

  • Open vmkBoot.ELF32-psigned in HxD

  • Search for the Hex string 83 F9 36 Change to 83 F9 01

  • Save

  • Gzip back to an archive called B.B00

  • Open weaselin-weasel.vtar HxD

  • Search for family == 0x06 and model <= 0x36 and change 0x36 to 0x01

  • Save

  • Gzip back to weasilin.t00

  • Copy both files back on to the install USB

  • Use the usb and install ESXI onto target machine.

  • Once installed remove the harddrive plug it into an Ubuntu machine.

  • Find the drive that contains the B.B00 file overwrite it with your modified B.B00 file.

  • Remove the harddrive plug back into system and boot into ESXI.


From Original Post by DISCOVER TECH ADVENTURES

Some More Useful tips

Fixing “This host does not support virtualizing real mode” error

Now when VMware ESXi 7.0 is up and running and all looks good you still might run into problems, not unlikely this one (when trying to start a VM):

Failed to power on virtual machine Home-Assistant. This host does not support virtualizing real mode. The Intel "VMX Unrestricted Guest" feature is necessary to run this virtual machine on an Intel processor. Click here for more details.
Power On VM
Key
haTask-2-vim.VirtualMachine.powerOn-3629474806
Description
Power On this virtual machine
Virtual machine:

Home-Assistant
State
Failed - This host does not support virtualizing real mode. The Intel "VMX Unrestricted Guest" feature is necessary to run this virtual machine on an Intel processor.

Errors
- This host does not support virtualizing real mode. The Intel "VMX Unrestricted Guest" feature is necessary to run this virtual machine on an Intel processor.
- Module 'CPUIDEarly' power on failed.
- Failed to start the virtual machine.

This issue has to be solved for every current and future VM, but the process is easy and fast when you know how.

  • Click “Edit” on the VM
  • Select “VM Options”
  • Open “Advanced” and select “Edit Configuration…”
  • Click “Add parameter” and fill as follows:
    Key: monitor.allowLegacyCPU
    Value: true
  • Click “OK” and “Save”
  • The VM should now work, but be patient. It can take a very long time to boot.

I hope this is to good use. Running unsupported is however not recommended, but for uncritical servers why not.


From Original Post by Flemming's Blog

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)