diff options
| author | Hao Xu <haoxu@linux.alibaba.com> | 2020-09-29 20:00:45 +0800 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2020-09-29 07:54:00 -0600 |
| commit | c8d317aa1887b40b188ec3aaa6e9e524333caed1 (patch) | |
| tree | ae6e8870004473d388c5f3275ad7888b6f74ec01 | |
| parent | fad8e0de4426a776c9bcb060555e7c09e2d08db6 (diff) | |
io_uring: fix async buffered reads when readahead is disabledio_uring-5.9-2020-10-02
The async buffered reads feature is not working when readahead is
turned off. There are two things to concern:
- when doing retry in io_read, not only the IOCB_WAITQ flag but also
the IOCB_NOWAIT flag is still set, which makes it goes to would_block
phase in generic_file_buffered_read() and then return -EAGAIN. After
that, the io-wq thread work is queued, and later doing the async
reads in the old way.
- even if we remove IOCB_NOWAIT when doing retry, the feature is still
not running properly, since in generic_file_buffered_read() it goes to
lock_page_killable() after calling mapping->a_ops->readpage() to do
IO, and thus causing process to sleep.
Fixes: 1a0a7853b901 ("mm: support async buffered reads in generic_file_buffered_read()")
Fixes: 3b2a4439e0ae ("io_uring: get rid of kiocb_wait_page_queue_init()")
Signed-off-by: Hao Xu <haoxu@linux.alibaba.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
| -rw-r--r-- | fs/io_uring.c | 2 | ||||
| -rw-r--r-- | mm/filemap.c | 6 |
2 files changed, 7 insertions, 1 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 556e4a2ead07..b89cc87830b1 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3049,6 +3049,7 @@ static int io_async_buf_func(struct wait_queue_entry *wait, unsigned mode, if (!wake_page_match(wpq, key)) return 0; + req->rw.kiocb.ki_flags &= ~IOCB_WAITQ; list_del_init(&wait->entry); init_task_work(&req->task_work, io_req_task_submit); @@ -3106,6 +3107,7 @@ static bool io_rw_should_retry(struct io_kiocb *req) wait->wait.flags = 0; INIT_LIST_HEAD(&wait->wait.entry); kiocb->ki_flags |= IOCB_WAITQ; + kiocb->ki_flags &= ~IOCB_NOWAIT; kiocb->ki_waitq = wait; io_get_req_task(req); diff --git a/mm/filemap.c b/mm/filemap.c index 1aaea26556cc..ea383478fc22 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2267,7 +2267,11 @@ readpage: } if (!PageUptodate(page)) { - error = lock_page_killable(page); + if (iocb->ki_flags & IOCB_WAITQ) + error = lock_page_async(page, iocb->ki_waitq); + else + error = lock_page_killable(page); + if (unlikely(error)) goto readpage_error; if (!PageUptodate(page)) { |