Option to use O_DIRECT #29

New Issue

Closed

opened 2022-05-29 18:24:34 +00:00 by a-j-wood · 8 comments

a-j-wood commented

2022-05-29 18:24:34 +00:00

(Migrated from github.com)

Provide an option to use the O_DIRECT flag to bypass cache.

👍 1

thesamesam commented

2023-07-28 12:32:16 +00:00

I think this one can be closed now as of ae81f83e9b.

I think this one can be closed now as of https://github.com/a-j-wood/pv/commit/ae81f83e9bef98cd03a5776f3534930fda564c13.

a-j-wood commented

2023-07-28 20:28:56 +00:00

(Migrated from github.com)

That's true - thanks.

cyckax commented

2023-08-07 04:54:06 +00:00

(Migrated from github.com)

Hi @thesamesam ,
when I used option --direct-io and the command like :
cat data4002.tar.gz | pv --rate-limit=10485760 --buffer-size=4194304 --direct-io --sync >>/tdsqlbackup/testpv/3333.tar.gz
the /tdsqlbackup/ is an nfs device, mounted like :172.21.16.6:/tdsqlbackup /tdsqlbackup nfs rw,bg,nfsvers=3,tcp,hard,timeo=600,rsize=4194304,wsize=4194304,_netdev 0 0
I found "--direct-io" option can not bypass os page cache ,

Is there anywhere i missed ?

Hi @thesamesam , when I used option --direct-io and the command like : cat data4002.tar.gz | pv --rate-limit=10485760 --buffer-size=4194304 --direct-io --sync >>/tdsqlbackup/testpv/3333.tar.gz the /tdsqlbackup/ is an nfs device, mounted like :172.21.16.6:/tdsqlbackup /tdsqlbackup nfs rw,bg,nfsvers=3,tcp,hard,timeo=600,rsize=4194304,wsize=4194304,_netdev 0 0 I found "--direct-io" option can not bypass os page cache , <img width="1459" alt="image" src="https://github.com/a-j-wood/pv/assets/9485649/bb1e632a-0759-4ecc-b5c5-ba9e80488295"> Is there anywhere i missed ?

a-j-wood commented

2023-08-07 21:11:31 +00:00

(Migrated from github.com)

Hello

I'm not sure how much effect "--direct-io" will have over NFS. The manual page for open(2) says that O_DIRECT will only bypass the page cache on the client, not the server, for example.

Hello I'm not sure how much effect "`--direct-io`" will have over NFS. The manual page for **open(2)** says that *O_DIRECT* will only bypass the page cache on the client, not the server, for example.

cyckax commented

2023-08-08 09:13:08 +00:00

(Migrated from github.com)

> Hello
>
> I'm not sure how much effect "--direct-io" will have over NFS. The manual page for open(2) says that O_DIRECT will only bypass the page cache on the client, not the server, for example.

yep I know. the cache I showed before is produced on client side ,not on nfs server side.

- > Hello > > I'm not sure how much effect "`--direct-io`" will have over NFS. The manual page for **open(2)** says that _O_DIRECT_ will only bypass the page cache on the client, not the server, for example. yep I know. the cache I showed before is produced on client side ,not on nfs server side.

a-j-wood commented

2023-08-08 11:23:31 +00:00

(Migrated from github.com)

Can you please use "strace" on the command you're trying, and see if you can determine whether it's calling fcntl() to set O_DIRECT on the relevant file descriptors?

If it's not, that may be a bug in pv; if it is, then there's not much we can do since O_DIRECT is having no effect, which is up to the OS.

OK Can you please use "`strace`" on the command you're trying, and see if you can determine whether it's calling `fcntl()` to set *O_DIRECT* on the relevant file descriptors? If it's not, that may be a bug in pv; if it is, then there's not much we can do since *O_DIRECT* is having no effect, which is up to the OS.

cyckax commented

2023-08-10 04:25:50 +00:00

(Migrated from github.com)

Hi.
I capture the strace, It seemed that fcntl(1, F_SETFL, O_WRONLY|O_APPEND|O_DIRECT|O_LARGEFILE) = -1 EINVAL (Invalid argument), O_APPEND can not support this. pv has no bug at this situation~

Hi. I capture the strace, It seemed that fcntl(1, F_SETFL, O_WRONLY|O_APPEND|O_DIRECT|O_LARGEFILE) = -1 EINVAL (Invalid argument), O_APPEND can not support this. pv has no bug at this situation~

a-j-wood commented

2023-08-11 21:36:45 +00:00

(Migrated from github.com)

Thank you for the investigation and confirmation.