@mmel I also find it odd that you were the first to suggest an enum.

That's a misunderstanding. It seems I was too brief and probably inaccurate. I apologize for that.
I always meant to say that INTR_TYPE should not be a #define with bitfield-like values, but with enum-like values.
Given my aversion to having enum in interfaces, it didn't occur to me at all that someone might take that as a recommendation to use enum. My bad, I'll try not to repeat it next time.

+1 for deduplication

In D47279#1078241, @jrtc27 wrote:

In D47279#1078240, @mmel wrote:

In D47279#1078231, @jrtc27 wrote:

In D47279#1078226, @mmel wrote:

And we also don't compile the kernel with -fno-short-enums. So the actual behavior/variant depends only on the compiler (for which both variants are equally valid).
What happens when someone uses this enum as a member of KABI compliant structure?

They get the same as any other enum in a structure? There are plenty already out there, and unless you start to reach 128 enum values there is no risk of the type changing by adding one.

???? The enum (with less than 128 values) size changes with the default compiler settings.

Well, yours is sparse. In the case of intr_root_type it's not, it's required to be contiguous from 0, so it must hit 128 distinct root types (SCHAR_MAX + 1) to potentially change.

No, sizeof(enum) with < 128 values is 4 (with -fno-short-enums) or 1 (with -fshort-enums).

In D47279#1078231, @jrtc27 wrote:

In D47279#1078226, @mmel wrote:

And we also don't compile the kernel with -fno-short-enums. So the actual behavior/variant depends only on the compiler (for which both variants are equally valid).
What happens when someone uses this enum as a member of KABI compliant structure?

They get the same as any other enum in a structure? There are plenty already out there, and unless you start to reach 128 enum values there is no risk of the type changing by adding one.

And we also don't compile the kernel with -fno-short-enums. So the actual behavior/variant depends only on the compiler (for which both variants are equally valid).
What happens when someone uses this enum as a member of KABI compliant structure?

This code

ohh, sorry, taking back this part. I noticed u_register_t in the original commit and missed the subsequent change.
But my complaint about enum) being a low-level interface remains. if I'm not mistaken, the enum size is not fixed which is not a good feature for the interface.
You cannot assume sizeof(MyEnum) == sizeof(int) in general. see

imho the original commit is wrong and should be reverted. Enum is clearly not the right type of variable for a low-level interface, especially if it should be used in assembler. A direct consequence of this problem is that the patch uses two different types (enum and u_register_t) for the same variable.

imho, arm32 has slightly more correct code/condition, see|: https://cgit.freebsd.org/src/tree/sys/arm/arm/trap-v6.c#n403

While it probably does not directly apply to this case, we should be very careful with similar constructions.
Clang is slightly schizophrenic and declare minimal common list of extension if -march is passed to the compiler, but maximal list of extensions (including optional ones) if -mcpu is passed to the compiler.
See https://github.com/llvm/llvm-project/issues/90365.
To my eyes this is a bug, nothing else.

Adapted to kib's objections.

I've written this several times in other reviews, I think. Initialization of PICs on secondary cores must be coordinated with the parent PIC and must follow the PIC hierarchy. Imho, the only way to do this correctly is implement this in pic_init_secondary() for each PIC driver.

Hmm, shouldn't the -masm-syntax-unified be added to gcc CFLAGS? (just guessing, untested)

I think it's the wrong way. The initiation of child PICs is the job of the parent PIC and must be done in right order and in coordination with the parent PIC. Please see D43452 for more details.

However, these new IPIs still generate interrupts/exceptions - so some form of more or less abstract PIC is still needed. Of course, if these instructions have evolved into a fixed part of the CPU (that is, the IPI will be generated by the exact instruction and raise an exact exception), we can hardcode it and omit it from INTRNQ...

Thanks fixing my bugs..

Maximum PLL lock time is 20 microsecond. Alternatively, we may use the 'us' to avoid UTF-8 in source files.

I'm afraid that using virtual time outside the hypervisor environment is somewhat problematic, or creates a new (not always fulfilled) dependency on firmware. The virtual timer uses the value stored in CNTVOFF as an offset to generate the timer value. CNTVOFF is stored in a bank with an undefined reset value, so it *MUST* be initialized, otherwise we can end up with a situation where each core will have a different, non-zero value - so each core will have a different time. So the actual code expects someone (firmware, hypervisor) to initialize CNTVOFF on all cores - which is not always true (or better said, it is uncommon in the ARMv7 world and guaranteed in ARM64 ).

This broke (at least) armv7 cross-buildworld on amd64.

I certainly don't want to block your work. I just don't want to introduce new KPIs that are problematic ( at least for me) for further expansion , nothing else.

I see. So if I assume that the full implementation should have multiple roots (per type). My main objection is about using bitfield for interrupt type. Is misleading, not much compatible with multiple root implementation and it allows only single bit value almost everywhere.

The problem is that the FIQ tree may have a different topology than a normal interrupts. So instead of passing flags to the root handler, we should create a new root for the FIQ tree (or for any other type of interrupt), including a new intr_pic_claim_root_<foo>() (or we can convert intr_irq_root_dev and irq_root_* to arrays or so...).

I strongly disagree, it's a clear step backwards. Intrng was designed with strong layer separation and you are trying to break it in a similar way to x86.
The consumer interface for interrupts is in the kern_intr.c file and no one else is allowed to use it. The provider interface is intr_isrc_*. All other functions should (and must) remain private or unpublished outside of intrng.

More than enough, thanks. And sorry for starting this in the wrong review.

I see. So in this case can we enable swp emulation using sysctl, disabled by default? I'm worried that this may cause bad behavior for ports build for arm32 - the port may detect swp during build. A bit paranoid, I know.

Hmm, the SWP/SWPB instruction is obsolete in ARMv7 ISA (in favor of atomic operations). It not supported/implemented on ARMv7 SMP cpus and arm32 kernel doesn't support it at all. IMHO we should be consistent at least between arm32 and compat32 on aarch64.

fyi, original idea was provide (not-preferred) way how to (limited) boot FreeBSD on a board with a problematic u-boot that might be locked or need lots of GPL or Linux specific utilities. It was easy for arm64 (where any u-boot can handle a booti image that is generic and has an easy format), but it's hard for armv7, where bootm needs an external program and the resulting image is not generic - it has a built-in load and boot address.
from my current point of view something like user submake stubs will be better (in simple words - be may include and run user definable makefile fragment) -> so system make can try to run this fragment for postprocessing

I like that a lot, I'd do it anyway, thanks. .

IMHO NEW_PCIB is an essential feature for all arm/arm64 FDT based boards. Or more generally, it is essential for all boards where the bootloader/firmware does not fully initialize all PCI hardware, i.e. all systems without ACPI.

I don't think this feature is appropriate for a generic layer. Practically no physical clock can implement it. A PLL can typically produce trillions of discrete frequencies, which makes it practically impossible to enumerate.

LGTM

Thanks .

Isn't that a bit (I don't want to say too much) excessive? All that's needed is to teach sysctl about skipping deleted (empty) intrnames, and to add topology sx lock to deliver a consistent view into user space (plus an allocator for free intrnames item in isrc->isrc_index). I'm almost certain that this big unnecessary change is uncommittable.

I see a misunderstanding here.
INTR_SOLO have nothing to do with shared interrupts. It's a fast path for cascading interrupts, and instead of removing it, we should extend it to support pass-through interrupts, polish the API for it, and put it in the regular build. Both of these are common in the arm/aarch64 world, and we lose performance significantly in these cases (and the big question is whether this should help us for LPI interrupts as well).

I don't have a single problem with intr_irq_root_dev partitioning. I feel that this implementation (priority based selection) is probably not viable in the long term and it seems to be RISC-V specific (i.e. not suitable for a platform independent framework). But decoupling the IPI controller from the root controller will allow us to do it all in a platform-specific "virtual" layer, so I don't went block you. But I have a slight problem with keeping the fallback IPI functionality in the root interrupt controller. We actually only have a few root controllers in the codebase, so adding a one-line intr_ipi_pic_register() function to all of them is not a big deal, which an reduce future bloating of spaghettis like functionally. What do you think?

Perfect, many thanks for cooperation.

I'm so sorry, but NACK.
This is the wrong way. A thermal sensor is not a thermal zone - > one sensor can be part of multiple zones, and one zone can consist of multiple sensors. The zone is a property of the board/case, while the sensor is (mostly) a property of the SOC. We should not mix them.
Moreover an in this case, since the sensors are located on the SoC, there is no reason why we should not hardcode its name (and number of sensors) directly in the driver.

That's fine with me, thank you.

Only minor nit.
Is necessary to shadow HWPARAMS? Shadowing the HW registry is always a pain and a source of confusion and errors. IMHO it should only be used as a last resort...

Thanks