You can use the existing ext4gd fields but for the others we need to have extra care.

I think/suspect the 64BIT flag needs a 64 bit system and basically breaks the on-disk compatibility: I recall it was causing issues with some bootloaders.

To be able to test the patch for real you need a huge (>16T) filesystem. IMHO enabling the feature for smaller filesystems i nonsense, but it is something that is not under our control.

In D11530#238803, @pfg wrote:

You can use the existing ext4gd fields but for the others we need to have extra care.

I think/suspect the 64BIT flag needs a 64 bit system and basically breaks the on-disk compatibility: I recall it was causing issues with some bootloaders.

The feature, just called 64bit, but, as I told, it is does not related to base types sizes, it is only about increasing size of struct ext2_gd, to add additional fields. Could you please describe about bootloadres? Do you mean, that FreeBSD ext2fs uses to mount boot volume somewhere?

To be able to test the patch for real you need a huge (>16T) filesystem. IMHO enabling the feature for smaller filesystems i nonsense, but it is something that is not under our control.

Here, you are completely right, only one way to test blocks allocation outside of UINT_MAX range is to use large volume, filled with real data, I am thinking about renting FreeBSD server somewhere, because I do not have access to large drives with TB sizes.

In D11530#238821, @thisisadrgreenthumb_gmail.com wrote:

In D11530#238803, @pfg wrote:

You can use the existing ext4gd fields but for the others we need to have extra care.

I think/suspect the 64BIT flag needs a 64 bit system and basically breaks the on-disk compatibility: I recall it was causing issues with some bootloaders.

The feature, just called 64bit, but, as I told, it is does not related to base types sizes, it is only about increasing size of struct ext2_gd, to add additional fields. Could you please describe about bootloadres? Do you mean, that FreeBSD ext2fs uses to mount boot volume somewhere?

Not a problem on FreeBSD since we don't regularly boot from ext2. The problem for external bootloaders are hinted here:

https://mail.gnome.org/archives/commits-list/2016-June/msg01079.html

In D11530#238821, @thisisadrgreenthumb_gmail.com wrote:

In D11530#238803, @pfg wrote:

You can use the existing ext4gd fields but for the others we need to have extra care.

I think/suspect the 64BIT flag needs a 64 bit system and basically breaks the on-disk compatibility: I recall it was causing issues with some bootloaders.

The feature, just called 64bit, but, as I told, it is does not related to base types sizes, it is only about increasing size of struct ext2_gd, to add additional fields. Could you please describe about bootloadres? Do you mean, that FreeBSD ext2fs uses to mount boot volume somewhere?

To be able to test the patch for real you need a huge (>16T) filesystem. IMHO enabling the feature for smaller filesystems i nonsense, but it is something that is not under our control.

Here, you are completely right, only one way to test blocks allocation outside of UINT_MAX range is to use large volume, filled with real data, I am thinking about renting FreeBSD server somewhere, because I do not have access to large drives with TB sizes.

I don't have such big disks either: perhaps with a cloud provider (Amazon or Azure), or I think ZFS a zvol with compression could be made to emulate those.

To be honest, the interesting test is to try such big disks in a 32bit OS but those are not very common in cloud environments.

Ha...! The zfs with compressed zvol is good idea.

In D11530#238833, @thisisadrgreenthumb_gmail.com wrote:

Ha...! The zfs with compressed zvol is good idea.

Oops, yes ... I meant zvol. This kind of testing was the main reason why they were created.

ping @kevlo: any comments?

Sorry for long delay from my side, but, seems like, this review revision should be frozen. It contain some "fundamental" problems, which can not be resolved directly. I found it, when started to play with large zvols. So, I need some time, and I can not estimate it for now.

In D11530#245279, @thisisadrgreenthumb_gmail.com wrote:

Sorry for long delay from my side, but, seems like, this review revision should be frozen. It contain some "fundamental" problems, which can not be resolved directly. I found it, when started to play with large zvols. So, I need some time, and I can not estimate it for now.

I am very happy that you are using zvols to test this. Take your time.

What is the status .. is this ready or just an update for the record?

I can confirm that it is ready to commit to head with long MFC for additional testing.
I made normal tests using fsx/fstorture + tests for modes when blocks are allocated out of UINT_MAX range.
More detailed information could be found in second test scripts attachment.
The reason why it was done in two phases (I mean a bunch of time between first and second review revisions),
because this feature heavily related with extents.
There were two ways to integrate extents and 64bit:

• RO extents, then RO 64bit, then RW extents, then RW 64bit.
• RO extents, then RW extetns, then RW 64bit.

So, the second way was chosen, because I decided, that it is more difficult to test RO feature implementation, than RW.
The next required feature is metadata_csum. When it will be integrated, it will be possible to switch to meta_bg/flex_bg features set.

In D11530#284091, @fsu wrote:

I can confirm that it is ready to commit to head with long MFC for additional testing.
I made normal tests using fsx/fstorture + tests for modes when blocks are allocated out of UINT_MAX range.
More detailed information could be found in second test scripts attachment.
The reason why it was done in two phases (I mean a bunch of time between first and second review revisions),
because this feature heavily related with extents.
There were two ways to integrate extents and 64bit:

• RO extents, then RO 64bit, then RW extents, then RW 64bit.
• RO extents, then RW extetns, then RW 64bit.

So, the second way was chosen, because I decided, that it is more difficult to test RO feature implementation, than RW.
The next required feature is metadata_csum. When it will be integrated, it will be possible to switch to meta_bg/flex_bg features set.

What I find confusing is that there is "force_64bit_alloc", labelled for testing/debugging only and a couple of TODO things that give the impresion that the implementation is still not ready.

What I find confusing is that there is "force_64bit_alloc", labelled for testing/debugging only and a couple of TODO things that give the impresion that the implementation is still not ready.

Yep, seems like I should ask the suggestion here...
Let me explain. This is hack to allow to allocate blocks outside of UINT_MAX range, which was used for testing purposes..
I mean, it is too difficult to test 16TB volume in case of ENOSPC without it.
So, I want to store it in svn history, to explain, how it was tested.
My plan was to commit main patch and then remove this hack. Or it is too dirty behavior to store stuff like this in the history?

In D11530#284219, @fsu wrote:

What I find confusing is that there is "force_64bit_alloc", labelled for testing/debugging only and a couple of TODO things that give the impresion that the implementation is still not ready.

Yep, seems like I should ask the suggestion here...
Let me explain. This is hack to allow to allocate blocks outside of UINT_MAX range, which was used for testing purposes..
I mean, it is too difficult to test 16TB volume in case of ENOSPC without it.
So, I want to store it in svn history, to explain, how it was tested.
My plan was to commit main patch and then remove this hack. Or it is too dirty behavior to store stuff like this in the history?

svn history is not the right place to put this hack ...
If you are not likely to use these ever again you can obviate them.
If you want to keep some of that around, more elegant options are:

1. A straight comment.
2. the code already has some

#ifdef EXT2FS_DEBUG
#endif

3. both of the above.