usbdump: Remove unit filters which shadow more specific unit.endpoint/.endpoint filters, extend parsing for filters
Details
- Reviewers
imp • hselasky pauamma_gundo.com - Group Reviewers
manpages
See current behavior when trying to filter out an input endpoint of a device with multiple (in my case, a Logitech G502 mouse with buttons bound to Scan_Previous_Track/Scan_Next_Track). As-is, an invocation such as -d ugen3.2 -f 2.130 will not alter output, since -d will add a unit filter for 2.-1 which will match all packets and cause the -f 2.130 filter to have no effect. With these changes, the 2.-1 filter is automatically removed so the -f 2.130 filter functions properly.
Diff Detail
- Repository
- rS FreeBSD src repository - subversion
- Lint
Lint Passed - Unit
No Test Coverage - Build Status
Buildable 45980 Build 42868: arc lint + arc unit
Event Timeline
I'll be adding Hans-Peter to the review. He can confirm your logic.
Having a more detailed description of what you are changing and why would be good in the summary. I'm sure that hps (hans peter) will want to know. What you said to me on discord would be a good start.
Describing the end effect is a good start, but the details from discord will help others understand why these changes were made.
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 226 | So is this an off-by-one error that's made several times? You are jumping to a jeq statement which is confusing to me. | |
| 229 | This is a size error that you are fixing, yes? | |
| 232 | This is likely a question for hps, since this is a non-nop change. I'll add him to the review. | |
| 920–921 | this likely should be a separate commit since it fixes a different problem. | |
Split into seperate commits.
- usbdump: Fix -f option without a unit parsing wrong, and allow specifying unit/endpoint in hex
- usbdump: Fix filters being OR'd instead of AND'd
Oops, yeah, I got a bit caught up in following the template that I wasn't sure what all to include.
Originally I had mentioned endianess confusion, but that was actually just me being confused and can be ignored.
The exact changes are:
- Fix generated BPF program accepting packets which match any filter (instead of all filters combined, considering the default behavior already accepts all packets I don't see any point in a way to accept more than every one).
- Fix parsing for -f inserting incorrect filters when a unit isn't specified. (-f .82 would make the first strol return 0, making it act the same as -f 0.82)
- Allow specifying values for filters in hex (endpoints are already displayed as hex, so I figure it would be best to allow copy-pasting output to write a filter instead of having to convert back to decimal for the -f argument)
As for the actual BPF, it boils down to a bit of bad logic as far as I can tell. The original logic is similar to
// for unit+endpoint filters
if (packet->unit == unit) {
if (packet->endpoint == endpoint) {
return packet;
}
else {
goto next_filter;
}
}
else {
goto next_filter;
}
// for individual unit/endpoint filters
if (packet->field == field) {
return packet;
}
else {
goto next_filter;
}
last_filter:
return 0;Which isn't totally incorrect. However, any "partial" unit/endpoint filter that matches any field of a given packet all will skip over all future filters and accept the packet immediately.
Additionally, when invoking with a device that already includes a unit (such as -d ugen3.2) a default unit filter for unit 2 is inserted, meaning that every packet will be matched before and user-provided filters will even be reached.
Instead of giving every filter the chance to accept a packet, these changes make it so individual filters can only deny a packet, and only packets which "survive" each filter in sequence will be accepted at the end of the program.
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 226 | Not exactly, the original logic jumps over a return entire_packet statement when the field doesn't match, but falls through and returns the entire packet if the field(s) do match. I've found it useful to think of the jump offsets as "skip N instructions", so the original skips ld, jeq, ret and the revised version just skips ld, jeq and ends up at a ret. Of course, this alone would mean that we would be accepting packets the moment a field doesn't match, and would only reject packets which did match every filter, so the ret inside of each filter needs to be switched to a return 0 and the "fallthrough return" at the end of the program needs to be switched to return entire_packet The functional difference is that the revised version "bails out" as soon as possible when it hits a filter that doesn't match and only accepts the packet when all filters didn't return, while the original does the same but for filters that did match and only denies packets which didn't match any filters. | |
| 229 | False alarm, ties into the endianness problem below. | |
| 232 | Sorry, was just a bit of confusion on my end. I'm not 100% sure why comparing this field as big-endian works when it has to be converted from little-endian inside of print_apacket, but the code works without these changes (and the size error mentioned above was just a symptom of "fixing" this non-issue) | |
OR-ing the "-f" endpoint filters is the expected behaviour. They should not be ANDed from what I can see. It's if (A | B | C) capture packet.
Regarding the strtol() to accept hex values, that is probably OK.
--HPS
Do you have an example using usbdump in user-space, where your fixes make a difference? It would help understand what is wrong!
Hmm, looking back the same "bug" is present in match_filter and dates all the way back to the initial commit for the utility, so you're right about it being intentional. I had always been invoking with the full -d argument which implicitly added a X.-1 filter for -d ugenY.X, which was why I was so convinced it was wrong (as far as I could at the time, no matter what was entered for -f it had no effect).
(edit: just noticed you were the original author! Sorry about my phrasing there, I didn't mean to sound so skeptical, I was just hoping to show my thought process. I'm also not sure how I've marked more comments as done, I'm going to go to sleep before I end up hitting any wrong buttons)
Considering this "problem" has been around for so long and nobody else has seen it as such, I think it would probably be best for me to come back with another version of my changes which allow for both flavors of logic for filters, since there's 10 years of proof that the original was correct (and that "fixing" this could actually break things).
As for my actual usage, I have a Logitech G502 mouse with some extra buttons bound to Scan_Previous_Track/Scan_Next_Track, where the all regular "mouse" events happen on endpoint 0x81, but the media key events happen on endpoint 0x82. I was trying to debug uhidd dropping events so I decided to use usbdump as my ground truth. However, the default filter from -d alongside extra filters being OR'd ended up meaning that no combination of arguments would give me the output I was after. I've just now figured out that invoking with -d ugen3 -f 2.130 does actually get me the output I need (although in an a slightly obscure way, so I'll still see about allowing for more flexible filters).
I've thought about this more, and I think that the real problem is that OR-ing a unit and unit.endpoint filter for the same unit isn't intuitive, since the lone unit filter will always match and make the unit.endpoint filter pointless. I'm not sure if it is fair to expect someone writing a filter to know this and avoid unit.-1 filters when using unit.endpoint or -1.endpoint filters though (and the implicit unit filter for -d ugenX.Y muddies the waters a bit more).
So, I've come up with two potential solutions:
- Make it more clear that this is the behavior on the man page ("for dummies" kind of stuff, aka "for me")
- Add a way to "correct" invocations with both unit and unit.endpoint filters to only actually apply the unit.endpoint filters (and warn for redundant unit.endpoint filters)
Of course, solution 2 would include solution 1 since a new argument would have to be documented and explained.
I'll be updating the revision to implement number 2 in a moment since it seems like the optimal solution (and shouldn't break any existing uses unless they parse verbose output poorly). I haven't touched the man page yet though, since I'm not 100% sure if a code change is the best fit for a mostly human problem, and don't want to get invested in documenting unless the code changes will stick.
Don't AND filters, but do try to correct/warn when filters might end up having unexpected results.
Hi,
Thanks for your detailed explanation. Now I see what was your problem!
Is there any reason we shouldn't prune by default?
--HPS
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 311 | Missing: free(puf) Here. | |
My only worry with pruning by default is breaking any existing uses. Despite how unlikely it is that someone built a script/automation around usbdump output with conflicting filters, there's a chance that their filters were wrong but seemed to work thanks to shadowing.
I'll start on updating the man page now (and will fix that leak). For the time being I'll keep things -p agnostic so we can still remove it if desired.
I think that will be OK. As long as it says in dmesg what is wrong, it will be good.
And if we do this change in FreeBSD-14 and newer, that is also an option for pruning by default.
Alright, sounds good to me. This definitely isn't an urgent change so I'll switch to pruning by default and get the man page updated, then this should be good to go.
I totally forgot that I need to update the revision when I make a new commit, sorry about the delay!
The man page changes didn't end up being too intrusive thanks to pruning by default, since I didn't have to explain how to avoid ending up with a set of filters that needs to be pruned. I also got that tiny leak fixed, and updated -h output, which I had forgotten before.
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 282 | Can you name this function "check_filters()" ? | |
| 288 | This check can just be removed. | |
| 292 | I would do like this to reduce the indention level: if (puf->unit == -1 || puf->endpoint != -1) continue; | |
| 295 | Don't try to match against ourself: if (puf == puf2) continue; | |
| 301 | "without -p" -> "with -p" ??? | |
| 301 | I wonder of %d is more common for integers than %i | |
| 825 | Wording??? | |
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 301 | I repurposed -p to disable pruning. Since we had decided to prune by default, I thought it would be best to include an easy way to switch back to the old behavior. | |
Since the behavior is changing from never pruning (until 13) to pruning by default with an option to return to earlier behavior in 14 , the manual page needs a COMPATIBILITY section.
Of the options marked "also work with" in the DESCRIPTION, which does it make sense to use alone, and which would likely be used in groups? If there's enough of the latter; I think they should be grouped in the synopsis, maybe by splitting the synopsis line.
| usr.sbin/usbdump/usbdump.8 | ||
|---|---|---|
| 72 | "data part" as opposed to which other part(s)? | |
| 107 | "shadow"? Do you mean "overlap with"? | |
| 108–109 | Maybe something like "Without this option, if unit and endpoint filters overlap, the unit filters are ignored and only the endpoint filters are used to limit output", if that's what you mean. | |
| 138 | I don't understand how USB works, but given this and the mention of control endpoints 0 and 128 under -f, I assume the latter aren't the only endpoints? | |
| 151–154 | I would just refer readers to -p in DESCRIPTION and focus on making that clearer (see above comment). | |
I've added the compatibility section, and I realized that almost all of the options work together, so I changed it to only mention options which aren't compatible instead.
| usr.sbin/usbdump/usbdump.8 | ||
|---|---|---|
| 72 | Good catch, I didn't notice how rough that sounds without any context. | |
| 107 | That's a much better phrase for it than I came up with, thanks! | |
| 138 | Yes, I tried to keep the pattern of 2/3/4 from the previous examples to show that they are just placeholders. | |
| usr.sbin/usbdump/usbdump.8 | ||
|---|---|---|
| 147 | In the third option should it be "-f .3" ? | |
| usr.sbin/usbdump/usbdump.c | ||
|---|---|---|
| 913–917 | This parsing also allows you to specify -f -1.3, so should negative filt_unit lead to some usage()? | |
Don't allow invalid values for -f, fix typo, replace "shadow" with "overlap" in usage
| usr.sbin/usbdump/usbdump.8 | ||
|---|---|---|
| 147 | It should be 4 for all of them, I just fat-fingered the wrong number in the next sentence. | |