linux-kernel - Re: recommended programming practices for writing (was Linux 2.6.29)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20090326234030.4bce8801@werewolf.home>
Date:	Thu, 26 Mar 2009 23:40:30 +0100
From:	"J.A. Magallón" <jamagallon@....com>
To:	LKML <linux-kernel@...r.kernel.org>
Subject: Re: recommended programming practices for writing (was Linux
 2.6.29)

On Thu, 26 Mar 2009 15:03:28 -0400, Adam Turk <bofh1234@...mail.com> wrote:

> 
> I have been reading the Linux 2.6.29 thread with interest.  I have written several (10 or so) C programs that write large amounts of data (between 1 and 2 GB file sizes are common).  A snippet of code looks like this:
> 
> if((fptr = fopen(outfilename,"w")) == NULL) {
>   printf("File %s could not be created\n", outfilename);
> }
> else {
>   fprintf(fptr,"%s\n",datablock);
>   while(!writeouput(datablock,amount,tax)) {
>     getnext(dtablock)
>     fprintf(fptr,"%s\n",datablock);
>   }
>   fclose(fptr);
> }
> 
> I learned C about 15 years ago and there was no mention of a fsync.  My C book doesn't mention fsync either.  Granted I have written only 25-30 applications in the last 15 years or so so I am not an expert C programmer. 
> 
> From what Linus posted about git and checking the return from fclose I think I going to start doing that.  I also think I am going to start checking the return from fprintf and maybe write to a /tmp/file and then rename it.
> 
> So is there a C fsync that I should add before my fclose?
> What is the proper way to write to files?
> 
> Is what I am thinking about doing something that would be good or is it just a lack of my understanding the problem?
> 

You _really_ should not worry bout that. You write a file the way C or Fortran
or ADA tells you to do so. I think no language standard worries about when
the file really is dumped to disk from the kernel side. Even fflush() just
guarantees that the _high level_ buffer in the C library is sent to the kernel
routines to write to the device. But none gives you a standard function to
get the pages really witten to disk, or wait even till the hardware queues
have really sent the data to the platter. You will never know that.

Assume this world is imperfect. Your computer can always crash. I think just
routines like link/unlink/rename could be near to metal. And with journaled
filesystems, even that.

>From what I have understood reading LKML these years, if you do something like

    fwrite(one megabyte)
    fsync()
    pause()
    fwrite(other megabyte)

and unplug the cord while in the pause, nobody guarantees you that the first
megabyte is on the disk. Live with it.

PD: I'm not saying this is good, I think it's just the way it is.

-- 
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
Mandriva Linux release 2009.1 (Cooker) for x86_64
Linux 2.6.28.2-desktop-1mnb (gcc 4.3.2 (GCC) #1 Wed Jan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/