[lug] named pipe missunderstanding

[Anthony Foiani]

Jason Davis <jdavis at openactive.org> writes:

> I think I have a missunderstanding about named pipes. Im using them
> for IPC and Im losing data.

Your update says that you get an inconsistent number of *reads*.

Do you get an inconsistent number of *bytes*, though?  Named pipes do
not packetize their content.

  $ ls -al fedora11-i386-dvd.img 
  -rw-rw-r--. 1 tony tony 3683844096 Oct 15  2009 fedora11-i386-dvd.img

  $ mkfifo my-pipe

  $ dd if=my-pipe of=tmp.img bs=1M &
  [1] 2412
  $ dd if=fedora11-i386-dvd.img of=my-pipe bs=1M
  3513+1 records in
  3513+1 records out
  3683844096 bytes (3.7 GB) copied, 227.972 s, 16.2 MB/s
  3513+1 records in
  3513+1 records out
  3683844096 bytes (3.7 GB) copied, 227.966 s, 16.2 MB/s
  [1]+  Done                    dd if=my-pipe of=tmp.img bs=1M

  $ md5sum fedora11-i386-dvd.img tmp.img 
  8bb458918512bcd41a834ca05b92d252  fedora11-i386-dvd.img
  8bb458918512bcd41a834ca05b92d252  tmp.img

If the reading process happens to loop while the writing process is
*not* writing to the pipe, the reader will see EOF.

In terminal 1:

  $ cat *.pdf > ../all.pdf

  $ for i in *.pdf ; do dd if="$i" of=my-pipe bs=16k ; done
  1+1 records in
  1+1 records out
  19070 bytes (19 kB) copied, 0.000147709 s, 129 MB/s
  425+1 records in
  425+1 records out
  6976935 bytes (7.0 MB) copied, 0.128906 s, 54.1 MB/s
  253+1 records in
  253+1 records out
  4153413 bytes (4.2 MB) copied, 0.0641728 s, 64.7 MB/s
  253+1 records in
  253+1 records out
  4153413 bytes (4.2 MB) copied, 0.0684014 s, 60.7 MB/s
  8+1 records in
  8+1 records out
  135683 bytes (136 kB) copied, 0.00947213 s, 14.3 MB/s
  33+1 records in
  33+1 records out
  556324 bytes (556 kB) copied, 0.00314683 s, 177 MB/s

In terminal 2:

  $ dd if=my-pipe of=out1.pdf bs=1M
  12+7 records in
  12+7 records out
  15438514 bytes (15 MB) copied, 0.297442 s, 51.9 MB/s
  $ dd if=my-pipe of=out2.pdf bs=1M
  0+3 records in
  0+3 records out
  1593046 bytes (1.6 MB) copied, 0.0785324 s, 20.3 MB/s
  $ dd if=my-pipe of=out3.pdf bs=16k
  12+1 records in
  12+1 records out
  201281 bytes (201 kB) copied, 0.0184194 s, 10.9 MB/s
  $ dd if=my-pipe of=out4.pdf bs=1M
  1+2 records in
  1+2 records out
  1282032 bytes (1.3 MB) copied, 0.0298385 s, 43.0 MB/s

But I'm pretty sure that all the data will eventually be sent.  Using
the same command in terminal 1 to send each file individually, I can
use 'cat' to capture them in terminal 2:

   $ rm ../all2.pdf

   $ while true ; do cat my-pipe >> ../all2.pdf ; done
   ^C

   $ ls -al ../all*
   -rw-rw-r--. 1 tony tony 60324399 Feb  1 16:08 ../all2.pdf
   -rw-rw-r--. 1 tony tony 60324399 Feb  1 15:47 ../all.pdf

   $ md5sum ../all*
   38257d8db7a14d2d3ba69ad145697ee8  ../all2.pdf
   38257d8db7a14d2d3ba69ad145697ee8  ../all.pdf

I did have some issues using 'dd' to *receive* the data; I'm not sure
what was going on there.

HTH,
t.