Just to chime in with what @mav said. Once FreeBSD is merged with the soon to be OpenZFS repo, the missing features (rename -u, FreeBSD's ashift mechanism, etc) have been upstreamed and the port has been given a few months to soak - the current code in tree - based as it is on a near dead upstream - is going away.

LGTM

Sorry, I'll take a look at the drop logic / buffer size

fix mismerge

In D19886#435308, @ae wrote:

In D19886#431575, @hselasky wrote:

FYI: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237329

So, what you think about this patch? It just restores the behavior that was in stable/11. I used this patch with head/ for two weeks and seems there is no regression.

In D20109#434039, @hselasky wrote:

@mmacy: The epoch_call() is not covered by epoch_wait_preempt(). We simply see deferred epoch calls being executed after the the IFP has been destroyed. Typically the multicast destroy ones.

@hselasky this is completely contrary to the intent of epoch - what callbacks are you seeing executed after a destroy?

I feel like there's a better way to do this, but haven't come up with a solution yet.

In D19886#432900, @ryan_freqlabs.com wrote:

before running tests:

ether_multi    17     2K       -       17  16,32,64,128
   in_multi     2     1K       -        2  256
  in6_multi    15     2K       -       15  32,256

after:

ether_multi 32839  2449K       -    40807  16,32,64,128
   in_multi     2     1K       -     3538  256
  in6_multi  9270  2316K       -    18525  32,256

sigh

use right patch

• initialize media pointer for non miibus drivers
• add flag for miibus drivers to set pass own media

In D19886#431573, @pho wrote:

20190426 12:09:07 all (1/1): multicast2.sh
Apr 26 12:09:09 t2 mDNSResponder[14951]: mDNSResponder (Engineering Build) (Jan  5 2019 05:09:39) starting
Apr 26 12:09:09 t2 mDNSResponder[14951]: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
panic: Assertion ifma->ifma_ifp == NULL || ifma->ifma_ifp == ifp failed at ../../../net/if.c:3777
cpuid = 8
time = 1556273349
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00004ec870
vpanic() at vpanic+0x19d/frame 0xfffffe00004ec8c0
panic() at panic+0x43/frame 0xfffffe00004ec920
if_delmulti_locked() at if_delmulti_locked+0x1e7/frame 0xfffffe00004ec940
if_delmulti_ifma_flags() at if_delmulti_ifma_flags+0xbb/frame 0xfffffe00004ec990
inm_release_task() at inm_release_task+0x1ac/frame 0xfffffe00004ec9f0
gtaskqueue_run_locked() at gtaskqueue_run_locked+0xf9/frame 0xfffffe00004eca40
gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe00004eca70
fork_exit() at fork_exit+0x84/frame 0xfffffe00004ecab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004ecab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Details @ https://people.freebsd.org/~pho/stress/log/mmacy035.txt

Fix at least a half dozen different issues exposed by the test case in the summary.

Makes sense ... you mean call the ifdi_init method directly? If so, it looks like init_locked does a bunch of queue/iflib management ... is all of that unnecessary on reset?

In D19946#428673, @aryeeteygerald_rogers.com wrote:

the custom media would have to call the wrappers, especially if restart is necessary

In D19946#428687, @aryeeteygerald_rogers.com wrote:

As is, the patch works.

I would still suggest exposing the iflib_media_* because init_locked is unavailable in the case of reset and I think the locks in the wrapper are useful?

@aryeeteygerald_rogers.com can you test with this patch and let me know? It should be a no-op for current drivers so can probably go in quickly.

don't call ifmedia_init for user provided ifmedia

I can just skip the iflib_media_init if we can just trust that the phy is correctly inited. Those two routines are just wrappers to drive user callbacks:

static int
iflib_media_change(if_t ifp)
{
	if_ctx_t ctx = if_getsoftc(ifp);
	int err;

missed change

In D19622#428305, @adrian wrote:

So in a Previous Project a Long Long Time Ago we solved this by having the receive/send state being an ifindex into an array of "ifnet" pointers, and a gencount so you can see if it's stale. Then all the code had to handle that the interface ifindex was stale (ie, a NULL pointer versus a garbage pointer) and decide at each point how to make forward progress. In some cases it wasn't needed for forward progress - eg it was already on a transmit queue, so the fact the receive interface went away wasn't a huge deal. But sometimes it was - eg tunnel (l2tp in one case) went away.

I agree this isn't the best solution, but it beats the heck out of panicking. It's panicing because SOME (interesting) state is being dereferenced through rcvif at some very late stage point in the stack. Maybe finding/tidying those up would also help things but this at least gets the panic out of the way.

So without delving in to the specifics of this issue I'd like to point out a couple of things:

• When an interface is taken down, freeing all unprocessed inbound and outbound is standard protocol as we have no idea how long the device will be down for, if it's more than 500ms the packets are good as dead
• inbound mbufs have a non refcounted pointer to their ifp - this would be easy to fix, but extremely invasive

In D19886#427223, @hselasky wrote:

if ((error = in6_joingroup(ifp, &in6, NULL, &in6m, 0)) != 0) {
        in6_leavegroup(im6o->im6o_membership[0], NULL);
        free(im6o->im6o_membership, M_CARP);
        break;
}
in6m_acquire(in6m);

Cool. +1

• fix case where failure path could lead to ref leak

All changes have gone in and been MFC’d through smaller reviews.

FWIW ZFS works fine on the POWER9 branch. However, there are a few things needed to make it play nice with HPT. Have you tracked them down?

• update to fix LINT

In D19274#413106, @hselasky wrote:

If you grep for llseek you'll see it is used a couple of places in sys/ofed .

@hselasky ping

@hselasky ping

• incorporate feedback printing file/line on failure

• incorporate feedback