Programmable arguments in perl pipes -


i'm gradually working way perl learning curve (with contributors helpful site), struggling how approach particular issue.

i'm building perl utility utilises 3 (c++) third party programmes. these run: $file_list | b -args | c $file_out

where process reads multiple files, process b modifies each individual file , process c collects input files in pipe , produces single output file, null input file signifying end of input stream.

the input files large(ish) @ around 100mb , around 10 in number. processes cpu intensive , whole process need applied thousands of groups of files each day, simple solution of reading , writing intermediate files disk inefficient. in addition, process above part of processing sequence, input files in memory , output file needs in memory further processing.

there number of solutions documented , have prototype version utilising ipc::open3(). far, good. :)

however - when piping each file process through process b i need modify arguments in process b each input file without interrupting forward flow process c. come unstuck , looking suggestions.

as further background:

  1. running in ubuntu 16.04 lts (currently within virtual box)and perl v5.22.1
  2. the programme run on (and within) single machine 1 user (me !), i.e. no external network communication or multi user or public requirement - simplicity of programming preferred on strong security.
  3. since process must run repeatedly without interruption, robust/reliable i/o handling required.
  4. i have access source code of each process, modified (although i'd prefer not to).

my apologies lack of "code date", thought question more 1 of "how approach this?" rather "how code work?".

any pointers or appreciated.

you need fourth program (call d) determines arguments b should , executes b arguments , d's stdin , stdout connected b's stdin , stdout. can replace b d in pipeline.

what language use d you.


Comments

Popular posts from this blog

angular - Is it possible to get native element for formControl? -

unity3d - Rotate an object to face an opposite direction -

javascript - Why jQuery Select box change event is now working? -