[lug] q: multithreading on smp weirdness

[Zan Lynx]

On Mon, 2004-11-29 at 19:45 -0700, karheng at softhome.net wrote:
> greetings. 
> 
> i've got a test app with 2 threads processing
> several chunks of data. thread 1 fetches chunk 1
> & processes it. once done, it's relayed to thread 2.
> then thread 1 fetches & processes chunk 2, while
> thread 2 receives & processes chunk 1. this goes
> on until all chunks are processed by both threads. 
> 
> my problem is that if the chunk size is made small
> enough, the elapse time suddenly almost doubles...
> and i need these chunks to be small eventually. 
> 
> more specifically:
> (chunk size * iteration count always == 6000)
> single threaded:
> chunk size of 6000 unit, 1 iteration, 12+ secs elapse time
> multi-threaded:
> chunk size of 12 unit, 500 iterations, 6+ secs elapse time
> chunk size of 10 unit, 600 iterations, 6+ secs elapse time
> chunk size of 8 unit, 750 iterations, 6++ secs elapse time
> chunk size of 6 unit, 1000 iterations, 6++ secs elapse time most of the time
> chunk size of 5 unit, 1200 iterations, 6+++ secs elapse time most of the 
> time, 11+ secs at other times.
> chunk size of 4 unit, 1500 iterations, 11+ secs elapse time
> chunk size of 3 unit, 2000 iterations, 11++ secs elapse time
> chunk size of 2 unit, 3000 iterations, 11+++ secs elapse time
> chunk size of 1 unit, 6000 iterations, 12+ secs elapse time
> (note that in above, elapse times of 7, 8, 9 & 10 secs are
> mostly not present, just 1 or 2 sporadic ones). 
> 
> i ran this on a 4 CPU itanium linux server. 
> 
> this problem doesn't appear when i ran a port
> of this app on a windows hyperthreading machine. 

An exotic guess: maybe 4 units is just under the size of a cache line?
Then thread 1's 4 units might be sharing a cache line with thread 2's 4
units.  Memory accesses will cause the cache line to bounce from cpu 1
to cpu 2 and back.
-- 
Zan Lynx <zlynx at acm.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20041130/9c1980f3/attachment.pgp>