I found that my linux workstation with 12 CPUs had almost stopped to work after I executed a shell script (tcsh) having a for-loop where more than hundreds of loops are executed simultaneously by adding '&' at the end of the command. Is there any way to control the number or executing time for background processes in the for-loop using tcsh?
How to control multithreaded background jobs in for loop in shell script
184 views Asked by Yong Wook Shin At
1
There are 1 answers
Related Questions in MULTITHREADING
- How can I outsource worker processes within a for loop?
- OpenMP & oneTbb difference
- Receiving Notifications for Individual Task Completion OmniThreadLibrary Parallel.ForEach
- C++ error: no matching member function for call to 'enqueue' futures.emplace_back(TP.enqueue(sum_plus_one, x, &M));
- How can I create a thread in Haskell that will restart if it gets killed due to any reason?
- Qt: running callback in the main thread from the worker thread
- Using `static` on a AVX2 counter function increases performance ~10x in MT environment without any change in Compiler optimizations
- Heap sort with multithreading
- windows multithreading CreateMutex
- The problem of "fine-grained locks and two-phase locking algorithm"
- OpenMP multi-threading not working if OpenMPI set to use one or two MPI processor
- WPF Windows Initializing is locking the separated thread in .Net 8
- TCP Client Losing Connection When Writing Data
- vc++ thread constructor throwing compiler error c2672
- ASP.NET Core 6 Web API : best way to pause before resending email
Related Questions in SHELL
- macOS - Most secure way of a GUI SUDO_ASKPASS
- When does Bash read heredocs?
- Why `set -o pipefail` gives different output even though the pipe is not failing
- Run multiple shell scripts in Dockerfile
- Alias does not take effect when I use Vim to execute external commands
- why variable substitution is so different?
- Error: fish: ${ is not a valid variable in fish
- Custom Bash functions & custom statements - Need some advice
- unexpected operator == in square brackets when trying to use gum lib
- Delete first three lines containing a certain word
- Keep the log for the last 14 days
- Iterate over items in one array and groups of items in second array
- Keep multi-version of a static-lib like what we do for shared-libs
- How to write function in bash for reuse shell commands inside using osascript?
- Why is it that when I pass certain directory names to `ls`, sometimes it does not list their contents?
Related Questions in FOR-LOOP
- Setting the counter (j) for (inner for loop)
- Set req query output to a variable
- Variable doesn't get updated when I run a loop
- Print DT datatables in for loop rendering to html
- How to list several items in the dialog box for execution?
- How to iteratively create matrices/vectors from columns/unique row values of dataframe, and pass them to subsequent code?
- Loop through multiple levels of a variable in R
- iterating through raster bands to perform calculation
- In pairs for loop not looping inside of another in pairs loop
- Going back to an earlier index in list iteration
- filter() function not working within the for loop
- Launch jobs in cache in a loop in bash script
- I can print listurl when print(list_url) is in the if loop, but I can't print it when it's outside the for loop
- Printing prime number
- .NET 9 Loop optimizations, is it more than remove the zero extension?
Related Questions in BACKGROUND
- Problem with Android App background service stoping
- flutter stripe_android:verifyReleaseResources'. > A failure occurred while executing com.android.build.gradle.tasks > Android resource linking failed
- How to Properly Interrupt the Download of a Background Image Using JavaScript
- Maintaining Background Consistency as Button Count Decreases
- Issue with simple java app that removes background from one image and overlays it onto another image and downloads the processed image
- Is there a way to gather the SSIDs of the connected networks in the background?
- Unity 2D - animated background
- CSS Animating Button Background - how to make this background move horizontally not vertically
- Change background on Tab react native app
- How can I edit ggMarginal background to match the plot background?
- How to structure a SwiftUI Push To Talk App backend?
- MPAndroidChart - Fill gradient horizontally
- Chrome Extension Highlight Feature Not Applying Background Color
- CSS image in div ignoring padding
- Under the swiftUI framework, the timer cannot continue to count in the background
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
GNU Parallel is made for this kind of situations.
GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to.
If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:
GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:
Installation
If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:
For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README
Learn more
See more examples: http://www.gnu.org/software/parallel/man.html
Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html
Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel