multithreading - OmniThreadLibrary memory leak (consumption) on pipeline running from another thread -

i'm running pipeline (thread's pipeline omnithreadlibrary) thread , got memory leak or rather memory consumption. when application close it's ok , there no memory leak report (reportmemoryleaksonshutdown := true;).

here example: click button 10 times , test app ~600 mb of memory. windows 7 x64, delphi xe6, latest omni source.

it's bug? or need use code?

uses   otlparallel,   otlcommon;  procedure tform75.button1click(sender: tobject); begin   // run empty pipeline threads   parallel.&for(1, 100).execute(     procedure(value: integer)     var       pipe: iomnipipeline;     begin       pipe := parallel.pipeline         .stage(procedure(const input: tomnivalue; var output: tomnivalue) begin end)         .run;       pipe.cancel;       pipe.waitfor(100000);       pipe := nil;     end   ); end; 

edit 1: tested code processexplorer , found threads count @ runtime constant, handles count grown. if i'm insert application.processmessages; @ end of "for loop" (after pipe's code) test app running good, handles closing , memory consumption constant. don't know why.

how many threads create ?

check in sysinternals process explorer example. or in delphi ide (view -> debug windows -> threads)

i think because block each for-worker wuite long waitfor application creates many worker threads every button click, , when click 10 times consequently creates 10 times many threads.

and yes, in general-purpose operating systems windows threads expensive! google "windows thread memory footprint" - , multiply number of threads created 10 parallel-for loop spawn.

• https://msdn.microsoft.com/en-us/library/windows/desktop/ms686774.aspx
• https://blogs.technet.microsoft.com/markrussinovich/2009/07/05/pushing-the-limits-of-windows-processes-and-threads/

this fact reason making highly parallel server applications special approaches done create light-eight application-level threads , bypass os threads, name few

• make special language spawn dozens of thousands of cooperative thread , cross-thread enforce memory safety strict language rules: https://www.erlang.org/docs
• make library, cannot enforce regulations @ least can demand programmer follow them voluntarily: https://en.wikipedia.org/wiki/actor_model
• fibers: no-protection threads within threads: what difference between thread , fiber?

however otl being generic library generic threads imposes little restrictions relies on os-provided native threads, , heavy expensive in both cpu time needed create/release windows threads (mitigated thread pools concept) , memory footprint needed maintain each windows threads os (which unavoidable , see manifestation).

of course later, when loops worked through, threads getting closed , released, memory used maintain them. no memory leak indeed, once wait enough threads terminated - are, temporarily allocated memory used workplaces.

upd. how check hypothesis? easiest way change how many threads spawned every instance of for-loop (by every button click).

see .numtasks options of parallel.for object:

http://otl.17slon.com/book/chap04.html#leanpub-auto-iomniparallelsimpleloop-interface

by default every button click should spawn 1 thread every cpu core. can enforce own size of thread pool. add .numtasks(1) call , check memory consumption, check .numtasks(10) , again. if memory consumption grow approximately tenfold after - it.