Menu
  • HOME
  • TAGS

Batch processing in jdbc gateway

jdbc,batch-processing,spring-integration

First of all since version 4.0.4 Spring Integration's <splitter> supports Iterator as payload to avoid memory overhead. We have a test-case for the JDBC which shows that behaviour. But as you see it is based on the Spring Integration Java DSL and Java 8 Lamdas. (Yes, it can be done...

Move a range of jobs to another queue with qmove

bash,batch-processing,hpc,torque

If the job ids are in sequential order, you could use Bash's brace extension. For example: $ echo {0..9} 0 1 2 3 4 5 6 7 8 9 Transferred to moving all jobs ranging from 1000 to 2000, the qmove command would be: qmove newQueue {1000..2000} This might even...

How to evaluate a variable with two words in a batch file?

batch-file,cmd,batch-processing

if syntax is if somestring operator anotherstring dosomething. The most common operator is ==, and to dosomething then two strings must exactly match. To group strings containing separators such as spaces, "enclose the strings in double-quotes" Hence if "%alt%" equ "ex" goto ex should work for you (it's unclear whether...

Meta tables for H2 db in Spring Batch

java,spring,batch-processing,spring-batch,h2

A couple things: That Exception isn't from not being able to connect to the database, but because the tables haven't been created (that Exception indicates that you can connect to the database). I don't see you running the initialization scripts. You're overdoing all the wiring. Based on what you're doing,...

Batch script to archive subfolder contents using 7zip with FOR /R

batch-file,batch-processing,7zip

You can do this from the command line with no batch file needed: FOR /F "usebackq tokens=* delims=" %A IN (`DIR "C:\Logs\*.log" /B /S`) DO "C:\Path\To\7za.exe" a "%~dpnA.zip" "%~fA" & DEL "%~fA" To use in a batch file, just replace each % with %%....

Mac Terminal batch file rename: flip string and digits

regex,osx,terminal,batch-processing

This will do, or come quite close to doing, what you are asking for. The technique I am using is called Bash Parameter Substitution and it is documented and described very well here. #!/bin/bash for file in *.pdf do echo DEBUG: Processing file $file f=${file%.*} # strip extension from right...

XMLStarlet namespace issue/can't extract node value from XML

xml,batch-processing,xmlstarlet

The -N x=http... options must go first just after the sel. Outside of the FOR construct, this works XMLSTARLET.EXE sel -N x=http://ns.adobe.com/air/application/17.0 -t -c //x:versionNumber myApp-app.xml but it gives <versionNumber xmlns="http://ns.adobe.com/air/application/17.0">1.0.0</versionNumber>. I think you were after just 1.0.0, so you should change the -c (--copy-of) to -v (--value-of). XMLSTARLET.EXE sel...

how to call all the files inside a folder using batchfile

windows,batch-file,cmd,batch-processing,saxon

In your XSLT you can use collection('file:///C:/testXsl/In/?select=*.xml') to read in all .xml files in that folder. See http://saxonica.com/documentation9.5/sourcedocs/collections.html for details on that syntax. That way a single stylesheet can process a collection of documents.

Custom operations using Stream API

java,java-8,batch-processing,java-stream

As mentioned in the comments by @MarkoTopolnik map is the solution.

Superpose two sets of images

imagemagick,batch-processing

Assuming your environment is Mac OSX or Linux or Unix... Then you could this: for img in A/* ; do \ convert \ A/${img} \ B/$(basename ${img}) \ -gravity center \ -compose blend \ -composite \ -alpha set \ composed.png \ done Now you have to understand: the description of...

How to move folders if they contain a certain file?

batch-file,for-loop,batch-processing

for /r "e:\videos\catalogar\" %%F in (serie.txt) do ( executes a directory-scan, but does not use the filemask. It will only use the filemask if the filemask contains * or ?. Personally, I believe that's a bug (OK if the filemask is '.' but should-match-mask-regardless otherwise) and perhaps asking Microsoft to...

What is Spark Job ?

apache-spark,batch-processing,job-scheduling

Well, terminology can always be difficult since it depends on context. In many cases, you can be used to "submit a job to a cluster", which for spark would be to submit a driver program. That said, Spark has his own definition for "job", directly from the glossary: Job A...

How do i handle ERRORLEVEL 9 in cmd

batch-file,batch-processing

If you read the description of if command (available via if /?), you will realize that if errorlevel number command will execute the command when the errorlevel is greater or equal than the number. If you want to execute the command when the number is precisely a certain value, use...

Multiple Threads wait for batch operation

java,multithreading,concurrency,batch-processing

This could be accomplished by using a CyclicBarrier. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. Create a barrier that all task can access. The arguments are the amount of tasks you want to wait for when calling await()...

Setting EXIT_MESSAGE in batch_job_execution

spring-batch,batch-processing

The way to manipulate the exit status of a job (aka the Job's ExitStatus) is via a JobExecutionLisener. The way you're attempting to manipulate it is using a copy of the real thing. We do that so that rollback can be implemented cleanly. You can read more about the JobExecutionListener...

Expanding variables in a Windows batch file

batch-file,command-line-arguments,batch-processing,command-prompt

Wrong quotation in set commands. Use set "variable=value" syntax as follows: set "PDF1=G:\my pdfs\file 1.pdf" set "PDF2=G:\my pdfs\file 2.pdf" start "" "C:\PDF Viewer\PDFXCview.exe" /A "page=1&zoom=33.3" "%PDF1%" /A "page=4&zoom=55.5" "%PDF2%" ...

R Batch mode - Can't execute script

r,csv,batch-processing

In your example you are trying to look for csv file in the cd c:\Program Files\R\R-3.1.3\bin\x64. You need to execute R.exe or even better Rscript.exe in the directory where myData.csv is located. R exe dir is one thing. R script dir is another. csv dir is another. Use absolute paths...

Passing sgm files in which the filenames contain spaces and comma using batch script

windows,batch-file,batch-processing

Just one extract from the code. The same problem happens in all the file %OMNIDIR%\Perl\bin\perl.exe srt_tables.pl %1.sgm for %%F in (%1.tables) do ( if %%~zF equ 0 goto NOTABLE ) %1 contains a quoted file name with extension, so what the parser interprets is c:\somewhere\Perl\bin\perl.exe srt_tables.pl "Medco Health .... 2002.sgm".sgm...

Batch move files based on part of file name - Windows 8

batch-file,batch-processing

Test this batch file in a folder of sample files: call it movepdz.bat so the name doesn't clash with another executable. @echo off for %%a in (*.pdz) do ( echo processing "%%a" for /f "tokens=1,2 delims=-" %%b in ("%%~nxa") do ( md "%%b-%%c" 2>nul move "%%a" "%%b-%%c" >nul ) )...

Scheduling inside step using Spring-Batch

spring,batch-processing,spring-batch

Polling the DB for a specific result sounds like a situation where you need a scheduler. Spring-batch assumes the scheduling of the job is done from outside it's scope. You can use @Scheduled spring annotation if you want to keep all inside spring configuration or use an external tool like...

Checking for Not Listening Ports from Batch File

windows,batch-file,batch-processing

netstat -an | findstr /r /c:"^ [TU][CD]P *[^ ]*:%oPort% " >nul && goto :getPort There is no need for the for command. Just test if the value can be found. If it can be found ask for another port....

bugfixer.bat batch file ask end user which option they want to run

batch-file,command-line,batch-processing

Next script logic could become helpful: @echo OFF >NUL setlocal enableextensions :TOP echo( set "USERSPEC=" set /P "USERSPEC=Which bug fixer do you need to run? A=all B=some C=repair? " if /I "%USERSPEC%"=="A" goto :FIRST if /I "%USERSPEC%"=="B" goto :NEXT if /I "%USERSPEC%"=="C" goto :LAST echo NO CHECK CHOOSEN, BATCH ENDS...

Batch file on Mac that makes a change to a bunch of files

osx,unix,csv,batch-processing

Assuming changes.tab is has tab separated values like your example above (and not comma separated as you state), this should get you started: #!/bin/sh cat changes.tab | while read line; do TARGETDIR="`echo "$line" | awk -F"\t" '{ print $1 }'`" COL2="`echo "$line" | awk -F"\t" '{ print $2 }'`" COL3="`echo...

How to implement batch processing in Grails

grails,gorm,batch-processing

This is quite different from the original question, so if I understand it properly, you need to load different information into tables accessed through different dataSources. You have multiple dataSources defined in your project. And you want to use groovy.sql.Sql with these dataSources. All your dataSources are available in your...

square connect api batch processing

api,batch-processing,square,square-connect

From the Square docs: Note the following when using the Submit Batch endpoint: You cannot include more than 30 requests in a single batch. Recursive requests to the Submit Batch endpoint are not allowed (i.e., none of the requests included in a batch can itself be a request to this...

copy each line from 2 files to 1 file Windows batch

windows,for-loop,concatenation,batch-processing,findstr

@echo off setlocal enableDelayedExpansion set "file1=path_to\date.txt" set "file2=path_to\id.txt" set "out=path_to\output.txt" for /f %%N in ('type "%file1%"^|find /c /v ""') do set "cnt=%%N" >"%out%" 9<"%file1%" <"%file2%" ( for /l %%N in (1 1 %cnt%) do ( set "ln1=" set "ln2=" <&9 set /p "ln1=" set /p "ln2=" echo !ln1! is the...

Settings for the Batch import tool

neo4j,batch-processing

The new import tool writes continuously and concurrently to disk, so a really fast disk is making a big difference. It also manages memory completely on its own there is no configuration needed. If you can provide the data fast enough it can saturate an SSD-RAID writing 1GB/s....

Configure settings when importing a large dataset using Neo4j Batch-importer

neo4j,out-of-memory,batch-processing

You should try the new import tool that comes with Neo4j 2.2.0-M03 It uses less memory, scales much better across CPUs. If you want to go with my batch-importer: Usually it imports 1M nodes /s and about 100k to 500k rels / second. How much heap do you use? use...

SAS : How to ignore errors?

sas,batch-processing

I'd be careful doing this in production, but it appears that you can get back into normal processing mode by putting this line after every one of your data steps and procs that are prone to error. It basically undoes the usual mode that SAS switches into when it encounters...

Extract a number from string in batch script

regex,windows,batch-file,batch-processing

Try this DO SET "driveIndex=%%a" Your line ... do set driveIndex=%%a & goto ... is interpreted as set driveIndex=%%a<space>& goto ..., this is, where the additional space in \Device\Harddisk1 \Partition3 comes from. Of course you could write: ... do set driveIndex=%%a& goto ... but the better syntax is: ... do...

How do I remove trailing \ from %~p in a For command?

cmd,batch-processing

You can assign the directory name (with the trailing backslash) to a normal variable, and then echo it with substring variable expansion of 0,-1 to exclude the final character, all in a command list in the for-loop body: for /r %%i in (dir) do @(set d=!server!%%~pi& echo !d:~0,-1!) >>!filename! Here's...

SQL Server Using TableDiff on large tables

sql-server,sql-server-2008-r2,batch-processing

The performance issue is because the remote queries pull every record from each place to do the comparison to generate the output. Indexes can help slightly to make the pull a little faster from each location, but it's not likely to be significant. An incremental approach is definitely better. I...

R distinguishing between batch and interactive mode

r,batch-processing

You can use the interactive function. For example, executing this from a terminal Rscript -e 'cat(interactive())' returned FALSE for me, while executing interactive() from my RStudio session returned TRUE....

SAXON Error- How to ignore/skip it?

javascript,xslt,batch-processing,saxon

As the error message says, it is an error reported by the underlying XML parser that Saxon uses to parse the markup of the document you are providing to it. If that is not well-formed XML then any XML parser will reject it. Saxon offers you the choice to use...

Handling error messages with files run automatically with a batch script

batch-file,error-handling,batch-processing,errorlevel

Firstly, if all you want to do is conditional execution based on the zero or non-zero exit code of "PROGRAM", use && and || to designate code blocks for success or fail like this: for /f "delims=_" %%J IN ('forfiles /p "%%F" /m *.ext/c "cmd /c echo @path"') DO (...

Spring Batch Add Custom Fields

spring,spring-batch,batch-processing

Yes , all the task that you have reported can be done by spring batch - For the Reader you may use - multi Resource Item Reader with your wild card name matching your - file names . To validate the rows from file you can use item processor and...

Use call inside a forfiles

batch-file,batch-processing

Just use a for-loop: for /r "%myPath%" %%a in (*) do call:LaunchRun "%%~a" "%%~nxa" ...

Spring batch FlatFileItemWriter writing model containing a property of List type

spring,spring-batch,batch-processing

Write a custom LineAggregator implementation if standard ones don't fits your need

Batch: create a folder based on part of filename and then copy the file into it

batch-file,batch-processing

You are really close. You are just missing a delimiter in your FOR statement (if you omit the delims parameter it will use a space as the default) to break the name apart by an underscore. Then you need to "reassemble" the file name back in your MOVE statement by...

Automated httr authentication with twitteR , provide response to interactive prompt in “batch” mode

r,twitter-oauth,batch-processing

You can try setting the httr_oauth_cache option to TRUE: options(httr_oauth_cache=T) The twitteR package uses the httr package, on the Token manual page for that package they give tips about caching: OAuth tokens are cached on disk in a file called .httr-oauth saved in the current working directory. Caching is enabled...

Jenkins can't use net start ?

jenkins,windows-services,batch-processing

Okay i found out what was wrong: For some reason jenkins didnt have my net.exe on the path So what worked: pathtoNet.exe start servicename...

REG command after SET is “unknown” in DOS/Windows batch file (.BAT)

windows,batch-file,cmd,registry,batch-processing

path is a logical name, but it's not a good name to use as it is assigned by Windows. path is a semicolon-separated list of the directories that Windows uses to find programs. When you change it, Windows can no longer find reg.exe since reg.exe is not in mypath. Simply...

Java 8 Stream with batch processing

java,stream,java-8,batch-processing

You could do it with jOOλ, a library that extends Java 8 streams for single-threaded, sequential stream use-cases: Seq.seq(lazyFileStream) // Seq<String> .zipWithIndex() // Seq<Tuple2<String, Long>> .groupBy(tuple -> tuple.v2 / 500) // Map<Long, List<String>> .forEach((index, batch) -> { process(batch); }); Behind the scenes, zipWithIndex() is just: static <T> Seq<Tuple2<T, Long>> zipWithIndex(Stream<T>...

remove blank lines from multiple files within a specific directory

batch-file,batch-processing

I've decided to include all the explanations as comments. There are some ways of doing it without a rename/move operation, but are not as reliable as this. Anyway, at the end, files will have the same name but no empty lines. @echo off setlocal enableextensions disabledelayedexpansion rem There are some...

BigQuery - same query works when submitted from UI and reports SQL syntax error from batch

batch-processing,google-bigquery

The difference between the succeeding and the failing query appears to be that you are setting flattenResults=false when you run the query in batch. This mode has slightly different behavior with JOINs that can cause subtle issues like this one. From BigQuery docs on JOINs: BigQuery executes multiple JOIN operations...

What is the benefit of the in-memory processing engines with a huge amount of data? [closed]

hadoop,apache-spark,bigdata,batch-processing

I find Spark's advantages over Hadoop's MapReduce are more than just in-memory computation engine even input from disk. As far as I can concern, there are at least two major advancements: Spark's DAG execution engine over MapReduce's two phase execution Thread level parallel execution over Process level parallel execution To...

How to send a filename through batch & accept it in Java

java,batch-file,batch-processing

You know that every main method is declared like this: public static void main(String[] args) {...} That args array represents command line arguments. That is, parameters you wrote on the java command line in your cmd window or your batch. So for example, if you have a line in your...

Overcoming the Midnight Barrier in Batch Script

performance,batch-file,time,timer,batch-processing

if %elapsed% lss 0 set /a elapsed +=8640000 Where 8640000 is the number of hundredths-of-seconds in 24 hours. Naturally, if you're using a 12-hour clock (you don't say) you would need to use a different constant - and the critical time becomes 1am/1pm...

Windows batch to add prefix to file names, why adds twice?

windows,file,batch-file,batch-processing,prefix

/f removes the issue of recapturing an existing file: FOR /f "delims=" %%F IN ('DIR /a-d /b *.pdf') DO (RENAME "%%F" "hello%%F") ...

Changing image names programmatically to match their paths

windows,powershell,batch-file,command-line,batch-processing

A PowerShell answer. It is not an efficient use of the language but I wanted to present something that was easy to read. Change the $rootPath to the path where your Root Folder from your question. $rootPath = "f:\temp" Get-ChildItem -Path $rootPath -Filter *.gif -Recurse | ForEach-Object{ $newnamePrefix = $_.DirectoryName...

Best approach using Spring batch to process big file

java,spring,batch-processing,spring-batch,spring-batch-admin

If you have a large file, I'd recommend storing it to disk unless there is a good reason not to. By saving the file to disk it allows you to restart the job without the need to re-download the file if an error occurs. With regards to the Tasklet vs...

org.springframework.dao.DataIntegrityViolationException Caused by: java.sql.BatchUpdateException: Data truncation

java,batch-processing

You are trying to insert (probably) a String with a length that exceeds the corresponding columns capacity. Either log the information about the user that you try to insert, or validate that all the data to be input is in correspondance with the size of the columns in the database....

insert to different tables batche in one loop

java,multithreading,jdbc,batch-processing

You could create an another PreparedStatement for the INSERT INTO USERSETTINGS and call addBatch / executeBatch for that one within the same loop. If this is an offline / one-time operation, you might have more alternatives (vendor-specific loader tools), but this depends on your RDBMS....

How to set a sqlcmd output to a batch variable?

sql-server,batch-file,for-loop,batch-processing,sqlcmd

Try turning on NOCOUNT: for /f %%a in ('sqlcmd -S <SERVER> -d <DATABASE> -Q "SET NOCOUNT ON; select max(Column1)+1 from Table1"') do set ColumnVar=%%a echo %ColumnVar% pause ...

Rename files after a list of names

windows,batch-file,batch-processing

I think I found what you need in this post: How to batch rename files in a directory from a list of names in a text file I've tailored one of their solutions to try to answer your question... @ECHO OFF SETLOCAL ENABLEDELAYEDEXPANSION rem Load the list of new filenames...

Streaming libpq results from PQexec

c,postgresql,batch-processing,postgresql-9.3,libpq

With a recent enough libpq (libpq-5.5 shipping with PostgreSQL 9.2), you may call PQsetSingleRowMode as described in Retrieving Query Results Row-By-Row. PQexec will have to be replaced by the asynchronous PQsendQuery. Otherwise a technique that works with all versions is to open a cursor at the SQL level (see DECLARE...CURSOR)...

Will batch script wait while it is executing NET STOP?

batch-file,batch-processing

NET STOP is synchronous. This means that script will wait for it to finish. Further you don't even need to check the service with SC. NET STOP has the following return codes: 0 = Success 1 = Not Supported 2 = Access Denied 3 = Dependent Services Running 4 =...

Batch: Rename extension and increment with letter

batch-file,batch-processing

@echo off setlocal EnableDelayedExpansion rem Create nextLetter array used to increment letters set "letter=z" for %%a in (a b c d e f g h i j k l m n o p q r s t u v w x y z) do ( set "nextLetter[!letter!]=%%a" set "letter=%%a" )...

Batch script not running

windows,batch-file,architecture,batch-processing

The batch script appears to work fine. It will not echo if your machine is not Windows 7 though. Try the following: @echo off set os_ver="unknown!" set os_bits="unknown!" ver | findstr "5\.1" > nul if %ERRORLEVEL% EQU 0 ( set os_ver="xp" ) ver | findstr "6\.1" > nul if %ERRORLEVEL%...

Batch convert DDS file to DDS without mipmaps

bash,scripting,batch-processing,gimp,dds-format

You should use the 'define' dds:mipmaps if you don't want to keep the mipmaps. Setting it to zero will disable the writing of mipmaps. convert input.dds -define dds:mipmaps=0 output.dds You can find a list of all dds defines here: http://www.imagemagick.org/script/formats.php....

How to store a specific result of where.exe in a variable for multiple results

batch-file,batch-processing

"I need the n-th" is problematic. Better filter to your needs. Examples: You know, that you are searching explicite for imageMagick: for /f "delims=" %%a in ('where.exe convert^|find "ImageMagick"') do @set progpath="%%a" or you know, you don't want to get the windows-built-in command, but search for any additional command: for...

How to schedule Batch Execution Service of Azure Machine Learning in Azure Scheduler

azure,machine-learning,batch-processing,azure-scheduler,azure-machine-learning

You would use Azure Data Factory instead of the scheduler. This would allow you to schedule the BES call into the future while identifying where the result file will end up. There are lots of examples online on how to do that....

Make batch code to connect to server and stop services

batch-file,batch-processing

try this : @echo off setlocal enabledelayedexpansion for %%a (SRV-01,SRV-02,SRV-03) do ( set srv= set srv=%%a net use \!srv! /USER:domain\user password sc \\!srv! stop'service-name' ) exit SRV-01/02/03 etc.. can be also IP address or just by name....

Broken results on batch convert with ImageMagick command line on Linux

php,linux,shell,imagemagick,batch-processing

I'm not sure if this is an answer. For now it is pure speculation. So here goes... By setting the limits to a 0 value, you are basically telling ImageMagick: "Your resources are not limited at all. You do not need to care for any limits." What if didn't set...

Need a BAT file to copy & rename all files in specific tree

windows,batch-file,cmd,batch-processing

@ECHO OFF SETLOCAL ENABLEDELAYEDEXPANSION SET "sourcedir=u:\Users\myname\Desktop\maindir" FOR /r "%sourcedir%" %%a IN (*.full.jpg) DO ( FOR %%b IN ("%%~dpna") DO ECHO(COPY "%%a" "%%~dpnb.jpg" ) GOTO :EOF The inner for examines the drive-path-name only of the complete filename in %%a (ie. it drops the .jpg) and delivers the drive-path-name of that name...

Convert a directory of Kodak PhotoCD images into TIFF format

linux,imagemagick,batch-processing,image-conversion

This worked perfectly for me on a Bash command line: for file in *.pcd;\ do convert $file[6] -colorspace RGB "`basename $file .pcd`.tiff";\ done (This can all be typed onto a single line without the backslashes, but I've used the Bash backslash escape to break this long line into three, to...

How to exit a for /f loop in a batch ?

batch-file,batch-processing

You need to make two changes. Change the goto :eof two lines below your for line to goto :step2. Add goto :EOF as the last line of your :displayinfo method. This will cause the method to return to the for loop. It should look like this. @echo off setlocal enabledelayedexpansion...

Merging CSV files with partly same filename

csv,batch-file,merge,batch-processing

You could simply iterate a static list of property names in a FOR loop. Note that multiple word names should be enclosed within quotes: @echo off for %%A in ( level pressure volume "multiword name" etc. ) do copy /b %%A??.csv %%a-day.csv /y Or you could dynamically discover the names...

How to set a variable in these batch loops?

batch-file,for-loop,variable-scope,batch-processing,findstr

for %%J in ( ... ) do ( .... %%J is visible here, inside the do clause .... ) <- here %%J goes out of scope So, you can include your second (third : %%F?) for loop inside the do clause of the first one for %%J in ("%%F\*.ext") do...

PowerShell command - what does this output?

powershell,batch-processing

Taken from https://technet.microsoft.com/en-us/library/hh849937.aspx Tee-Object uses Unicode encoding when it writes to files. As a result, the output might not be formatted properly in files with a different encoding. To specify the encoding, use the Out-File cmdlet. The command Select-String is found in PowerShell....

Click and load multiple links with different GET parameters (which auto submits)

javascript,ajax,batch-processing,auto

This should do the trick with jQuery, but you need to write the values directly to the database in upload.php not auto submit a form in between... function batchUpload(){ $.ajax({ type: "GET", url: "upload.php?id=123&name=abc" }); $.ajax({ type: "GET", url: "upload.php?id=456&name=efg" }); $.ajax({ type: "GET", url: "upload.php?id=789&name=hi" }); } ...

Batch script to create large text file

batch-file,batch-processing

Rather than open, append, close, open, append, close, open, append, close, etc. for 23616999 iterations, it'll greatly improve the efficiency if you open the file one time, perform the entire loop sequence with the file open, then close it after finished. Change your syntax thusly: rem; @cls @echo off setlocal...

Batch - Output all possible combinations of strings in two files

batch-file,combinations,permutation,batch-processing

Now Tested: @echo off set "file1=fileA.txt" set "file2=fileB.txt" set "file3=fileC.txt" break>"%file3%" for /f "useback tokens=* delims=" %%# in ("%file1%") do ( for /f "useback tokens=* delims=" %%$ in ("%file2%") do ( echo %%#%%$>>"%file3%" ) ) ...

Domain Driven Design and batch processing

php,doctrine2,domain-driven-design,batch-processing

In my opinion you should consider the batch operation as part of your domain not just like some "trivial" operation aside. Write down the requirements and you will see that it requires some domain modelling too. Eg. you need to store basic data about every batch run (type, when, how...

Mule ESB - design a multi file processing flow when files are dependent on each other

java,mule,batch-processing,esb

This has been asked on StackOverflow several times, with different wording. Usually the answer is to have a file inbound-endpoint pick one of the many files and then pick the other files down in the flow with a requester. See: https://github.com/mulesoft/mule-module-requester In your case, the main file would be available...

date validation in a text file using vba macros

vba,excel-vba,batch-processing

My first thought went to Regular Expressions (RegEx) as a possible solution for this problem since extracting the dates could be otherwise problematic. There's some great info on how to use RegEx in Excel here: How to use Regular Expressions (Regex) in Microsoft Excel both in-cell and loops I slapped...

Command to delete files all files NOT like a wildcard

windows,command-line,batch-processing,delete-file

From the command line: for /f "eol=: delims=" %F in ('dir /b /a-d *.pdf ^| findstr /biv doc') do @del "%F" The DIR command lists all .pdf files. The result is piped to FINDSTR which discards files that begin with DOC (case insensitive). The result is processed by FOR /F,...

How to correctly quote a command with parameters in a Windows batch file

batch-file,command-line-arguments,batch-processing,command-prompt

When using start, specify the /A command and parameters for each file to be opened: start "" "C:\PDF Viewer\PDFXCview.exe" /A "page=1&zoom=33.3" "G:\mypdfs\file1.pdf" /A "page=4&zoom=55.5" "G:\my pdfs\file2.pdf" ...

Grails DuplicateKeyException / NonUniqueObjectException when batch loading within Async Promise

grails,asynchronous,gorm,promise,batch-processing

The solution was to do as suggested by @JoshuaMoore and to use new sessions. Additionally, there was a reference to a domain object that was being referred to from outside one transaction that was then not having merge() called on it within a new session, thereby causing the error. i.e....

Batch File: Read path from text file and store every instance as new variable?

batch-file,batch-processing

@ECHO OFF SETLOCAL :: remove variables starting $ FOR /F "delims==" %%a In ('set $ 2^>Nul') DO SET "%%a=" FOR /f "tokens=1*" %%a IN (q27630202.txt) DO ( IF "%%~b" neq "" SET "$%%~a=%%~b" ) SET $ GOTO :EOF I used a file named q27630202.txt containing your data for my testing....

How to extract a string from the first line of a file using batch?

windows,batch-file,cmd,batch-processing

I suspect you have problems with it because it has Unix line endings instead of DOS style line endings. This appears to work, but it's a bit brittle - tied to the position of the contents of the example you linked: for /f "usebackq delims== tokens=28" %%a in (`findstr /n...

SPM in matlab: how to call matlab function in Batch Editor

matlab,batch-processing

Enter the function in single quotes as follows: 'sqrt' I just tried it and it worked. Your batchfile should end up looking like this: matlabbatch{1}.cfg_basicio.run_ops.call_matlab.inputs{1}.evaluated = 25; matlabbatch{1}.cfg_basicio.run_ops.call_matlab.outputs{1}.strtype.r = true; matlabbatch{1}.cfg_basicio.run_ops.call_matlab.fun = 'sqrt'; ...

Adding “with ur” or any other prefix in query being generated by JdbcPagingItemReader

spring,jdbc,db2,batch-processing,spring-batch

JdbcPagingItemReader allow you to set a PagingQueryProvider; maybe using a custom one - instead of using the one generated by SqlPagingQueryProviderFactoryBean - can solve your problem

My Concerns about Spring-Batch that you cant actually multi-thread/read in chunks while reading items

spring,batch-processing,spring-batch,spring-batch-admin

From past discussions, the CSV reader may have serious performance issues. You might be better served by writing a reader using another CSV parser. Depending on your validation data, you might create a job scoped filter bean that wraps a Map that can be either preloaded very quickly or lazy...

PHP - how to resize directory full of images if under a certion width x height [closed]

php,image,batch-processing

Use opendir() and readdir () to read all the images in the directory. Check the image width and height using getimagesize(). Resize it via Imagick::resizeImage. ...

Doctrine EntityManager clear method in nested entities

php,symfony2,doctrine2,batch-processing

Soon You will hit memory limit. The idea of flush() in batches is ok, but You need to clear() EntityManager at the end of each batch to release used memory. But in Your case, You will have ORMInvalidArgumentException (a.k.a. "new entity foung through relationship") if You will call $child->setParent($parent); after...

Batch modify text files

regex,markdown,batch-processing

You could use sed -i 's/^# \(.*\)$/---\ntitle: \"\1\"\n---/' *. The * means apply to all files in the current directory. Alternatively search for files using Unix find. An example usage: find . -type f -name "*.java" -exec sed 's/^# \(.*\)$/---\ntitle: \"\1\"\n---/' "{}" > "{}.bak" \; -exec mv "{}.bak" "{}" \;...

python corenlp batch parse

python,batch-processing,stanford-nlp

It was my mistake. I missed "raw_output" in parameter passing of batch_parse. So, it should be like this: for value in batch_parse(raw_text_directory, corenlp_dir,raw_output=True): print value ...

Unable to fetch logs from multiple servers using PsLogList

batch-file,batch-processing

(for %%a in ( Server1 Server2 ) do ( psloglist - \\%%a -h 4 system -i 7036 -o "Service Control Manager" )) > c:\temp\logs.txt Iterate over the list of servers...

GPG: How to sign with multiple signatures with different passphrases?

encryption,batch-processing,gnupg,pgp

You have different options. Completely remove the passwords, since they're stored somewhere anyway. Use the same password (as you already discovered). Use the gpg-agent and preset the passphrase. I'm unsure whether this is GnuPG 2-only (usually installed as gpg2, maybe to be installed from a gnupg2 package). Presetting the passphrase...