I've just applied the latest version of this patch but it's still looking for aarch64/Makefile.inc both during a "make buildworld" and a "make" in usr.sbin/bhyve on my Ampere system:

bcran@gluon:~/src/freebsd/usr.sbin/bhyve % make
make: "/home/bcran/src/freebsd/usr.sbin/bhyve/Makefile" line 68: Could not find aarch64/Makefile.inc
make: Fatal errors encountered -- cannot continue
make: stopped in /home/bcran/src/freebsd/usr.sbin/bhyve

The change to usr.sbin/Makefile.aarch64 no longer applies following commit 773606fcdfae00a3f850bcd39969a63d9a8fb129.
I'm probably missing something since I'm just catching up with this work, but 'make buildworld' fails in usr.sbin/bhyve:

===> usr.sbin/bhyve (includes)
make[4]: "/home/bcran/src/usr.sbin/bhyve/Makefile" line 68: Could not find aarch64/Makefile.inc
make[4]: Fatal errors encountered -- cannot continue
make[4]: stopped in /home/bcran/src/usr.sbin/bhyve

In D35419#803280, @bapt wrote:

this will at best not work, it might even break some of the tools which are using MOVED

Change the MOVED entry and add a section to UPDATING.

Add ONLY_FOR_ARCH lines and revert i386 addition at top

Add separate qemu_x64 and qemu_i386 flavors.

I'm not sure how to handle the fact that qemu (OVMF) firmware can be built for both x86_64 and i386.
The existing port uses FLAVOR, but since sysutils/edk2 already uses FLAVOR for the platform, I'm not sure how selecting the architecture should be done.

Fix summary.

In D35321#800709, @gerald wrote:

Hi Rebecca, changes like this don't need review. It's best to commit them as quickly as possible since otherwise make index is broken.

Committed in https://cgit.freebsd.org/ports/commit/?id=2e6a6ccd15d1c2b83ea238325b9feddff4f8eaa7

Updated sysutils/bhyve-firmware to depend on edk2@bhyve

In D35053#799648, @gerald wrote:

Hi Rebecca, intuitively this look good and you being our resident expert - do you want to go ahead and commit this?

Urgh, I remembered just in time about sysutils/bhyve-firmware that will also need updated.

Added entry to MOVED.

Actually deleted sysutils/uefi-edk2-bhyve this time.

In D35148#797304, @manu wrote:

You probably need a CONFLICT too for this flavor this you install links at the place of the other port.

Deleted sysutils/uefi-edk2-bhyve and added an entry to MOVED.

Fixed Makefile

As I mentioned in the summary, the following code doesn't do what I want it to: it doesn't create /usr/local/share/uefi-firmware.

Keep USE_GCC

Could I get approval from a port committer for this please?

Keep PORTEPOCH at 2.

Is it valid to reset PORTEPOCH to 1 when updating the PORTVERSION?

Remove romfile_dup. Fix strdup/strsep.

Put allocations on one line.
Added {}.
Removed gpa variable, and used gpa_alloctop instead.

man page updates.

Fix romfile[,varfile] in man page.

Add back checks for var_size and total_size.

Fix tabs vs spaces.

Update code to work against FreeBSD-CURRENT.

I tested the code from https://github.com/FreeBSD-UPB/freebsd-src/tree/projects/bhyvearm64/ (as of 2021-05-29) on my SolidRun Honeycomb system and it fails to allocate resources in the gicv3 code when I run kldload vmm.

With examples like Apple's "goto fail" bug (https://www.imperialviolet.org/2014/02/22/applebug.html) I'm surprised style(9) only allows, and doesn't recommend or mandate braces around single-line statements.

I noticed this is against the svn repo:

Repository rS FreeBSD src repository - subversion"

You'll probably want to switch over to Git and rebase this patch.

Just a comment typo: "number of contained objecct handles" - should be "object".

@manu Sorry, could you re-approve the change please?

In D29254#656248, @manu wrote:

OK I see. Well for now just have a build time dep is better then.
Go ahead and commit this.

We'll also need to update the port revision.

In D29254#655711, @manu wrote:

Is the 4.8 version required for something ? Otherwise depending on lang/gcc would be better (not really sure if we have some way to do that).

@woodsb02 Could you review and approve this for me, please?

Now the firmware work has been committed, I'll switch back to working on this.

Committed as r565866.

In D27230#642824, @jhb wrote:

I'm not a ports committer so can't really review the ports part, but I think moving the non-CSM ROM to the newer one seems fine. I think grehan@ has also already signed off on this?

At this point, I'd like to get this committed as soon as possible so we can move away from gcc 4.8 and python 2.

Updated to code from today's master, 2021-02-14.

The patch no longer applies against 13-CURRENT: I needed to apply several lines manually.

This works for me. I was seeing an assert failure in pci_emul.c, but it disappeared for some reason.
One person on Slack said they'd like the BAR in a fixed (high) location, so I wouldn't be confident committing this without other people (e.g. @jhb and @grehan ) reviewing it too.

OVMF puts it at the next 16GB boundary, which might be better.

Also, /boot/boot1.efifat is no longer included in -CURRENT.

/boot/boot1.efifat is the 800 KB filesystem image. /boot/boot1.efi is the EFI bootloader application.

UEFI is configuring an address size less than that which the host CPU supports, because (cc @grehan) :

//
// As guest-physical memory size grows, the permanent PEI RAM requirements
// are dominated by the identity-mapping page tables built by the DXE IPL.
// The DXL IPL keys off of the physical address bits advertized in the CPU
// HOB. To conserve memory, we calculate the minimum address width here.

@c.koehne_beckhoff.com
I see what's going wrong, and have reproduced the issue on my Threadripper system.
EDK2 firmware configures the maximum address based on the size of guest memory, with a minimum of 64GB (36 bits).
But the PCI code in bhyve puts the BAR at (highest host CPU address / 4) - meaning that on my system it puts it at 64TB.

I successfully installed and ran Windows 10 with the new firmware today.
So I think the last remaining issue will be the MemAbove4G problem.

The patch no longer applies cleanly: I had to make some changes.
But after that, I was able to run Ubuntu 20.10 with gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration set to TRUE and FALSE with the following command line, which uses both rfb and passthru:

bhyve -AH -s 0:0,hostbridge -s 31:0,lpc -c 1 -m 16G -l bootrom,BHYVE_CODE.fd -l com1,stdio -s 3:0,ahci-hd,bhyve-ubuntu-20_10.img  -s 7,ahci-cd, -s 8,virtio-rnd -s 6,virtio-net,tap0 -s 9,fbuf,rfb=10.0.10.117:5900,w=1920,h=1200 -s 10,xhci,tablet -s 11,passthru,24/0/0 -S guest

In D27230#619596, @woodsb02 wrote:

Apologies for being absent recently - life took over... but I’ll have some time to look at this again between Xmas and New Year.

In D27230#616878, @bcran wrote:

I've had a report that Windows 10 crashes (bugchecks) with this firmware, so I need to do more testing (on FreeBSD 11, 12 and 13-CURRENT) before committing it.

Any progress fixing this? Would you like to commit the update using one revision older from upstream that didn’t have this issue?