summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-09-23btrfs/287: filter snapshot IDs to avoid failures when using some featuresv2023.09.24Filipe Manana
When running btrfs/287 with features that create extra trees or don't the need to create some trees, such as when using the free space tree (default for several btrfs-progs releases now) versus when not using it (by passing -R ^free-space-tree in MKFS_OPTIONS), the test can fail because the IDs for the two snapshots it creates changes, and the golden output is requiring the numeric IDs of the snapshots. For example, when disabling the free space tree, the test fails like this: $ MKFS_OPTIONS="-R ^free-space-tree" ./check btrfs/287 FSTYP -- btrfs PLATFORM -- Linux/x86_64 debian0 6.6.0-rc2-btrfs-next-138+ #1 SMP PREEMPT_DYNAMIC Thu Sep 21 17:58:48 WEST 2023 MKFS_OPTIONS -- -R ^free-space-tree /dev/sdc MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1 btrfs/287 1s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad) --- tests/btrfs/287.out 2023-09-22 12:39:43.060761389 +0100 +++ /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad 2023-09-22 12:40:54.238849251 +0100 @@ -44,52 +44,52 @@ Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap1' Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap2' resolve first extent: -inode 257 offset 16777216 root 257 -inode 257 offset 8388608 root 257 -inode 257 offset 16777216 root 256 -inode 257 offset 8388608 root 256 ... (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/287.out /home/fdmanana/git/hub/xfstests/results//btrfs/287.out.bad' to see the entire diff) HINT: You _MAY_ be missing kernel fix: 0cad8f14d70c btrfs: fix backref walking not returning all inode refs Ran: btrfs/287 Failures: btrfs/287 Failed 1 of 1 tests So add a filter to logical reserve calls to replace snapshot root IDs with a logical name (snap1 and snap2). Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23btrfs: use full subcommand name at _btrfs_get_subvolid()Filipe Manana
Avoid using the shortcut "sub" for the "subvolume" command, as this is the standard practice because such shortcuts are not guaranteed to exist in every btrfs-progs release (they may come and go). Also make the variables local. Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23btrfs/259: fix output's wrong wordNaohiro Aota
It prints "File extent layout before defrag" for the both outputs, but the latter one should be "after defrag". Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23generic: test new directory entries are returned after rewinding directoryFilipe Manana
Test that if names are added to a directory after an opendir(3) call and before a rewinddir(3) call, future readdir(3) calls will return the names. This is mandated by POSIX: https://pubs.opengroup.org/onlinepubs/007904875/functions/rewinddir.html This exercises a regression in btrfs which is fixed by a kernel patch that has the following subject: ""btrfs: refresh dir last index during a rewinddir(3) call"" Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23btrfs/239: call fsync to create tree-log dedicated block group for zoned modeNaohiro Aota
Running btrfs/239 on a zoned device often fails with the following error. btrfs/239 5s ... - output mismatch (see /host/btrfs/239.out.bad) --- tests/btrfs/239.out 2023-09-21 16:56:37.735204924 +0900 +++ /host/btrfs/239.out.bad 2023-09-21 18:22:45.401433408 +0900 @@ -1,4 +1,6 @@ QA output created by 239 +/testdir/dira still exists +/dira does not exists File SCRATCH_MNT/testdir/file1 data: 0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab * ... This happens because "testdir" and "dira" are not logged on the first fsync (fsync $SCRATCH_MNT/testdir), but are written as a full commit. That prevents updating the log on "mv" time, leaving them pre-mv state. The full commit is induced by the creation of a new block group. On the zoned mode, we use a dedicated block group for tree-log. That block group is created on-demand or assigned to a metadata block group if there is none. On the first mount of a file system, we need to create one because there is only one metadata block group available for the regular metadata. That creation of a new block group forces tree-log to be a full commit on that transaction, which prevents logging "testdir" and "dira". Fix the issue by calling fsync before the first "sync", which creates the dedicated block group and let the files be properly logged. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23btrfs: add missing _fixed_by_kernel_commit for a few testsAnand Jain
A few tests were still using the older style of mentioning the fix in the comment section. This patch migrates them to using _fixed_by_kernel_commit. Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-23overlay: add test for rename of lower symlink with NOATIME attrAmir Goldstein
Test for a regression in copy up of symlink that has the S_NOATIME inode flag. This is a regression from v5.15 reported by Ruiwen Zhao: https://lore.kernel.org/linux-unionfs/CAKd=y5Hpg7J2gxrFT02F94o=FM9QvGp=kcH1Grctx8HzFYvpiA@mail.gmail.com/ In the bug report, the symlink has the S_NOATIME inode flag because it is on an NFS/FUSE filesystem that sets S_NOATIME for all inodes. The reproducer uses another technique to create a symlink with S_NOATIME inode flag by using chattr +A inheritance on filesystems that inherit chattr flags to symlinks. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20btrfs: add missing commit ids for a few tests using _fixed_by_kernel_commitFilipe Manana
The tests btrfs/288, btrfs/289 and btrfs/300 are using the "xxxx..." stub for commit ids, as when they were submitted/merged the corresponding btrfs patches were not yet in Linus' tree. So replace the stubs with the commit ids. Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20btrfs/076: use _fixed_by_kernel_commit to tell the fixing kernel commitNaohiro Aota
The fix commit is written in the comment without a commit hash. Use _fixed_by_kernel_commit command to describe it. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20btrfs/076: support smaller extent size limitNaohiro Aota
Running btrfs/076 on a zoned null_blk device will fail with the following error. - output mismatch (see /host/results/btrfs/076.out.bad) --- tests/btrfs/076.out 2021-02-05 01:44:20.000000000 +0000 +++ /host/results/btrfs/076.out.bad 2023-09-15 01:49:36.000000000 +0000 @@ -1,3 +1,3 @@ QA output created by 076 -80 -80 +83 +83 ... This is because the default value of zone_append_max_bytes is 127.5 KB which is smaller than BTRFS_MAX_UNCOMPRESSED (128K). So, the extent size is limited to 126976 (= ROUND_DOWN(127.5K, 4096)), which makes the number of extents larger, and fails the test. Instead of hard-coding the number of extents, we can calculate it using the max extent size of an extent. It is limited by either BTRFS_MAX_UNCOMPRESSED or zone_append_max_bytes. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20fstests: btrfs add more tests into the scrub groupAnand Jain
I wanted to verify tests using the command "btrfs scrub start" and found that there are many more test cases using "btrfs scrub start" than what is listed in the group.list file. So, get them to the scrub group. Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20common/rc: make _get_max_file_size find file size on mount pointAndrey Albershteyn
Currently, _get_max_file_size finds max file size on $TEST_DIR. The tests/generic/692 uses this function to detect file size and then tries to create a file on $SCRATCH_MNT. This works fine when test and scratch filesystems have the same block size. However, it will fail if they differ. Make _get_max_file_size accept mount point on which to detect max file size. Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20tools/mvtest: ensure testcase is executable (755)Shiyang Ruan
Some test cases lack executable permission ('x'). Before running each test case, `./check` checks and grants them 'x' permission. However, this always leads to a dirty git repo. And the absence of 'x' permission in test cases is often overlooked during reviews. Since maintainers use mvtest to assign new case, add this change for convenience of maintainers. Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20fstests: btrfs/185 update for single device pseudo device-scanAnand Jain
As we are obliterating the need for the device scan for the single device, which will return success if the basic superblock verification passes, even for the duplicate device of the mounted filesystem, drop the check for the return code in this testcase and continue to verify if the device path of the mounted filesystem remains unaltered after the scan. Also, if the test fails, it leaves the local non-standard mount point remained mounted, leading to further test cases failing. Call unmount in _cleanup(). Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20overlay: add test for persistent unique fsidAmir Goldstein
Test overlayfs fsid behavior with new mount options uuid=null/on that were introduced in kernel v6.6: - Test inherited upper fs fsid with mount option uuid=off/null - Test uuid=null behavior for existing overlayfs by default - Test persistent unique fsid with mount option uuid=on - Test uuid=on behavior for new overlayfs by default Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20xfs/270: actually test log recovery with unknown rocompat featuresDarrick J. Wong
Make sure that log recovery will not succeed if there are unknown rocompat features in the superblock and the log is dirty. Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-20xfs/270: actually test file readabilityDarrick J. Wong
Make sure we can actually read files off the ro mounted filesystem that has an unknown rocompat feature set. Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-16fstests: btrfs/261 fix failure if /var/lib/btrfs isn't writableAnand Jain
We don't need scrub status; it is okay to ignore the warnings due to the readonly /var/lib/btrfs if any. Redirect stderr to seqres.full. We check the scrub return status. +WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.42fad803-d505-48f4-a04d-612dbf8bd724: Read-only file system. Progress cannot be queried +WARNING: failed to write the progress status file: Read-only file system. Status recording disabled Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-16fstests: use btrfs check repair for repairing btrfs filesystemsAnand Jain
There are two repair functions: _repair_scratch_fs() and _repair_test_fs(). As the names suggest, these functions are designed to repair the filesystems SCRATCH_DEV and TEST_DEV, respectively. However, these functions never called proper comamnd for the filesystem type btrfs. This patch fixes it. Thx. Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02xfs/559: adapt to kernels that use large folios for writesv2023.09.03Darrick J. Wong
The write invalidation code in iomap can only be triggered for writes that span multiple folios. If the kernel reports a huge page size, scale up the write size. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02common: rename get_page_size to _get_page_sizeDarrick J. Wong
This function does not follow the naming convention that common helpers must start with an underscore. Fix this. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02common: split _get_hugepagesize into detection and actual queryDarrick J. Wong
This helper has two parts -- querying the value, and _notrun'ing the test if huge pages aren't turned on. Break these into the usual _require_hugepages and _get_hugepagesize predicates so that we can adapt xfs/559 to large folios being used for writes. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02btrfs/282: skip test if /var/lib/btrfs isnt writableDarrick J. Wong
I run fstests in a readonly container, and accidentally uninstalled the btrfsprogs package. When I did, this test started faililng: --- btrfs/282.out +++ btrfs/282.out.bad @@ -1,3 +1,7 @@ QA output created by 282 wrote 2147483648/2147483648 bytes at offset 0 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +WARNING: cannot create scrub data file, mkdir /var/lib/btrfs failed: Read-only file system. Status recording disabled +WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.3e1cf8c6-8f8f-4b51-982c-d6783b8b8825: No such file or directory. Progress cannot be queried +WARNING: cannot create scrub data file, mkdir /var/lib/btrfs failed: Read-only file system. Status recording disabled +WARNING: failed to open the progress status socket at /var/lib/btrfs/scrub.progress.3e1cf8c6-8f8f-4b51-982c-d6783b8b8825: No such file or directory. Progress cannot be queried Skip the test if /var/lib/btrfs isn't writable, or if /var/lib isn't writable, which means we cannot create /var/lib/btrfs. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/187: don't run this test on NFSJeff Layton
This test is unreliable on NFS. It fails consistently when run vs. a server exporting btrfs, but passes when the server exports xfs. Since we don't have any sort of attribute that we can require to test this, just skip this one on NFS. Also, subsume the check for btrfs into the _supported_fs check, and add a comment for it. Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/357: don't run this test on NFSJeff Layton
NFS doesn't keep track of whether a file is reflinked or not, so it doesn't prevent this behavior. It shouldn't be a problem for NFS anyway, so just skip this test there. Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/294: don't run this test on NFSJeff Layton
When creating a new dentry (of any type), NFS will optimize away any on-the-wire lookups prior to the create since that means an extra round trip to the server. Because of that, it consistently fails this test. Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/*: add a check for security attrsJeff Layton
There are several generic tests that require "setcap", but don't check whether the underlying fs supports security attrs. Add the appropriate checks. Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/578: add a check to ensure that fiemap is supportedJeff Layton
This test requires FIEMAP support. Suggested-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02common/attr: fix the _require_acl testJeff Layton
_require_acl tests whether you're able to fetch the ACL from a file using chacl, and then tests for an -EOPNOTSUPP error return. Unfortunately, filesystems that don't support them (like NFSv4) just return -ENODATA when someone calls getxattr for the POSIX ACL, so the test doesn't work. Fix the test to have chacl set an ACL on the file instead, which should reliably fail on filesystems that don't support them. Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/61[67]: support SOAK_DURATIONDarrick J. Wong
Now that I've finally gotten liburing installed on my test machine, I can actually test io_uring. Adapt these two tests to support SOAK_DURATION so I can add it to that too. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/650: race mount and unmount with cpu hotplug tooDarrick J. Wong
Ritesh Harjani reported that mount and unmount can race with the xfs cpu hotplug notifier hooks and crash the kernel, which isfixed by: https://lore.kernel.org/linux-xfs/ZO6J4W9msOixUk05@dread.disaster.area/T/#t Extend this test to include that. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/650: add SOAK_DURATION controlsDarrick J. Wong
Make this test controllable via SOAK_DURATION, for anyone who wants to perform a long soak test of filesystem vs. cpu hotplug. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02btrfs/237: kick reclaim process with a small filesystemNaohiro Aota
Since commit 3687fcb0752a ("btrfs: zoned: make auto-reclaim less aggressive"), the reclaim process won't run unless the 75% (by default) of the filesystem volume is allocated as block groups. As a result, btrfs/237 won't success when it is run with a large volume. To run the reclaim process, we need to either fill the FS to the desired level, or make a small FS so that the test write can go over the level. Since the current test code expects the FS has only one data block group, filling the FS is both cumbersome and need effort to rewrite the test code. So, we take the latter method. We create a small (16 * zone size) FS. The size is chosen to hold a minimal FS with DUP metadata setup. However, creating a small FS is not enough. With SINGLE metadata setup, we allocate 3 zones (one for each DATA, METADATA and SYSTEM), which is less than 75% of 16 zones. We can tweak the threshold to 51% on regular btrfs kernel config (!CONFIG_BTRFS_DEBUG), but that is still not enough to start the reclaim process. So, this test requires CONFIG_BTRFS_DEBUG to set the threshold to 1%. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02fstests: generic/352 should accomodate other pwrite behaviorsBill O'Donnell
xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite. Modify test to limit pwrite and reflink to a single block. Signed-off-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02fstests: test fix for an agbno overflow in __xfs_getfsmap_datadevDarrick J. Wong
Dave Chinner reported that xfs/273 fails if the AG size happens to be an exact power of two. I traced this to an agbno integer overflow when the current GETFSMAP call is a continuation of a previous GETFSMAP call, and the last record returned was non-shareable space at the end of an AG. This is the regression test for that bug. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02generic/551: bail out test if aio-dio-write-verify failedNaohiro Aota
When the AIO program failed, it is better to bail out the test to keep the failed state intact. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02aio-dio-write-verify: print more info on the error caseNaohiro Aota
When short read or corruption happened, it is difficult to locate which IO event failed. Print the address to make it identifiable. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-09-02aio-dio-write-verify: check for the IO errorsNaohiro Aota
The async write IOs can return some errors, which may lead to a short read or corruption in io_verify() stage. Catch an error early to identify the root cause easily. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-25generic/471: Remove this broken casev2023.08.27Yang Xu
I remember this case fails on last year becuase of kernel commit cae2de69 ("iomap: Add async buffered write support") kernel commit 1aa91d9 ("xfs: Add async buffered write support"). as below: pwrite: Resource temporarily unavailable wrote 8388608/8388608 bytes at offset 0 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -RWF_NOWAIT time is within limits. +pwrite: Resource temporarily unavailable +(standard_in) 1: syntax error +RWF_NOWAIT took seconds So For async buffered write requests, the request will return -EAGAIN if the ilock cannot be obtained immediately. Here also a discussion[1] that seems generic/471 has been broken. Now, I met this problem in my linux distribution, then I found the above discussion. IMO, remove this case is ok and then we can avoid to meet this false report again. [Additional information from Dave Chinner] We changed how timestamps are updated so that they are aware of IOCB_NOWAIT. If the IOCB_NOWIAT DIO write now needs to update the inode timestamps, it will return -EAGAIN instead of doing potentially blocking operations that require IO to complete (i.e. taking a transaction reservation). Hence the first time we go to do a DIO read an inode, it's going to do an atime update, which now occurrs from an IOCB_NOWAIT context and we return -EAGAIN.... Yes, we added non-blocking timestamp updates as part of the async buffered write support, but this was a general XFS IO path change of behaviour to address a potential blocking point in *all* IOCB_NOWAIT reads and writes, buffered or direct. The test is not validating that RWF_NOWAIT is behaving correctly - it just was a simple operation that kinda exercised RWF_NOWAIT semantics when we had no other way to test this code. It has outlived it's original purpose, so it should be removed... [1]https://lore.kernel.org/linux-xfs/b2865bd6-2346-8f4d-168b-17f06bbedbed@kernel.dk/ Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Jens Axboe <axboe@kernel.dk> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-25fstests: fsstress: wait interrupted aio to finishQu Wenruo
[BUG] There is a very low chance to hit data csum mismatch (caught by scrub) during test case btrfs/06[234567]. After some extra digging, it turns out that plain fsstress itself is enough to cause the problem: ``` workload() { mkfs.btrfs -f -m single -d single --csum sha256 $dev1 > /dev/null mount $dev1 $mnt #$fsstress -p 10 -n 1000 -w -d $mnt umount $mnt btrfs check --check-data-csum $dev1 || fail } runtime=1024 for (( i = 0; i < $runtime; i++ )); do echo "=== $i / $runtime ===" workload done ``` Inside a VM which has only 6 cores, above script can trigger with 1/20 possibility. [CAUSE] Locally I got a much smaller workload to reproduce: $fsstress -p 7 -n 50 -s 1691396493 -w -d $mnt -v > /tmp/fsstress With extra kernel trace_prinkt() on the buffered/direct writes. It turns out that the following direct write is always the cause: btrfs_do_write_iter: r/i=5/283 buffered fileoff=708608(709121) len=12288(7712) btrfs_do_write_iter: r/i=5/283 direct fileoff=8192(8192) len=73728(73728) <<<<< btrfs_do_write_iter: r/i=5/283 direct fileoff=589824(589824) len=16384(16384) With the involved byte number, it's easy to pin down the fsstress opeartion: 0/31: writev d0/f3[285 2 0 0 296 1457078] [709121,8,964] 0 0/32: chown d0/f2 308134/1763236 0 0/33: do_aio_rw - xfsctl(XFS_IOC_DIOINFO) d0/f2[285 2 308134 1763236 320 1457078] return 25, fallback to stat() 0/33: awrite - io_getevents failed -4 <<<< 0/34: dwrite - xfsctl(XFS_IOC_DIOINFO) d0/f3[285 2 308134 1763236 320 1457078] return 25, fallback to stat() Note the 0/33, when the data csum mismatch triggered, it always fail with -4 (-EINTR). It looks like with lucky enough concurrency, we can get to the following situation inside fsstress: Process A | Process B -----------------------------------+--------------------------------------- do_aio_rw() | |- io_sumit(); | |- io_get_events(); | | Returned -EINTR, but IO hasn't | | finished. | `- free(buf); | malloc(); | Got the same memory of @buf from | thread A. | Modify the memory | Now the buffer is changed while | still under IO This is the typical buffer modification during direct IO, which is going to cause csum mismatch for btrfs, and btrfs properly detects it. This is the direct cause of the problem. The root cause is that, io_uring would use signals to handle submission/completion of IOs. Thus io_uring operations would interrupt AIO operations, thus causing the above problem. [FIX] To fix the problem, we can just retry io_getevents() so that we can properly wait for the IO. This prevents us to modify the IO buffer before writeback really finishes. With this fixes, I can no longer reproduce the data corruption. Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-25btrfs/004: use shuf to shuffle the file linesNaohiro Aota
The "sort -R" is slower than "shuf" even with the full output because "sort -R" actually sort them to group the identical keys. $ time bash -c "seq 1000000 | shuf >/dev/null" bash -c "seq 1000000 | shuf >/dev/null" 0.18s user 0.03s system 104% cpu 0.196 total $ time bash -c "seq 1000000 | sort -R >/dev/null" bash -c "seq 1000000 | sort -R >/dev/null" 19.61s user 0.03s system 99% cpu 19.739 total Since the "find"'s outputs never be identical, we can just use "shuf" to optimize the selection. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-25fstests/btrfs: use _random_file() helperNaohiro Aota
Use _random_file() helper to choose a random file in a directory. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-25common/rc: introduce _random_file() helperNaohiro Aota
Currently, we use "ls ... | sort -R | head -n1" (or tail) to choose a random file in a directory.It sorts the files with "ls", sort it randomly and pick the first line, which wastes the "ls" sort. Also, using "sort -R | head -n1" is inefficient. For example, in a directory with 1000000 files, it takes more than 15 seconds to pick a file. $ time bash -c "ls -U | sort -R | head -n 1 >/dev/null" bash -c "ls -U | sort -R | head -n 1 >/dev/null" 15.38s user 0.14s system 99% cpu 15.536 total $ time bash -c "ls -U | shuf -n 1 >/dev/null" bash -c "ls -U | shuf -n 1 >/dev/null" 0.30s user 0.12s system 138% cpu 0.306 total So, we should just use "ls -U" and "shuf -n 1" to choose a random file. Introduce _random_file() helper to do it properly. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19fstests: Verify dir permissions when creating a stub subvolumeLee Trager
btrfs supports creating nesting subvolumes however snapshots are not recurive. When a snapshot is taken of a volume which contains a subvolume the subvolume is replaced with a stub subvolume which has the same name and uses inode number 2. This test validates that the stub volume copies permissions of the original volume. Signed-off-by: Lee Trager <lee@trager.us> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19btrfs/220: do not run async discard test on zoned deviceNaohiro Aota
The mount option "discard=async" is not meant to be used on the zoned mode. Skip it from the test. Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19common/rc: drop 'fsck -f' parameter from _repair_test_fsDavid Disseldorp
The '-f' parameter is fsck.ext# specific, where it's documented to: Force checking even if filesystem is marked clean _repair_test_fs() is only called on _check_test_fs() failure, so dropping the parameter should be possible without changing ext# behaviour. Doing so fixes _repair_test_fs() on exfat, where fsck.exfat doesn't support '-f'. Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19generic/{175,297,298}: fix use of uninitialized varAmir Goldstein
The truncate command in those tests use an uninitialized variable i. in kdevops, i must contain some leftover, so we get errors like: /data/fstests-install/xfstests/tests/generic/298: line 45: /dev/loop12): syntax error: operand expected (error token is "/dev/loop12)") Apparently, noone including the author of the tests knows why this truncate command is in the test, so remove the wrong truncate command. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19check: fix parsing expunge file with commentsAmir Goldstein
commit 60054d51 ("check: fix excluded tests are only expunged in the first iteration") change to use exclude_tests array instead of file. The check if a test is in expunge file was using grep -q $TEST_ID FILE so it was checking if the test was a non-exact match to one of the lines, for a common example: "generic/001 # exclude this test" would be a match to test generic/001. The commit regressed this example, because the new code checks for exact match of [ "generic/001" == "generic/001 " ]. Change the code to match a regular expression to deal with this case and any other suffix correctly. NOTE that the original code would have matched test generic/100 with lines like "generic/1000" when we get to 4 digit seqnum, so the regular expression does an exact match to the first word of the line. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19fsx: tidy options usage and formatShiyang Ruan
1. Add missing options and wrap the cli example line. 2. Cleanup and also add missing "-K" operation for options description part. Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
2023-08-19t_ofd_locks: fix sem initialization sequenceStas Sergeev
The locker was waiting for sem_otime on sem0 to became non-zero after incrementing sem0 himself. So sem_otime was never 0 at the time of checking it, so the check was redundant/wrong. This patch: - moves the increment of sem1 to the lock-tester site - lock-setter waits for that sem1 event, for which this patch replaces the wait loop on sem_otime with GETVAL loop, adding a small sleep - increment of sem0 to 2 moved past that sem1 event. That sem0 event is currently not used/waited. This guarantees that the lock-setter is working only after lock-getter is fully initialized. CC: fstests@vger.kernel.org CC: Murphy Zhou <xzhou@redhat.com> CC: Jeff Layton <jlayton@kernel.org> CC: Zorro Lang <zlang@redhat.com> Signed-off-by: Stas Sergeev <stsp2@yandex.ru> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-16 02:55:32 +0000