I would like to use a bash script and qsub to run 30-40 python programs at once. Each python program reads and searches through the same set of files (~400 total) for a set of sequences. Is there a problem where multiple python programs could be trying to read from the same file? If so, what are the consequences?
Can multiple qsub submissions read the same group of files?
105 Views Asked by Patrickc01 At
1
There are 1 best solutions below
Related Questions in QSUB
- How does PBS_NODEFILE work in pbs?
- c language - Submit a qsub job programmatically
- qsub: get last job id submitted
- how to run PBS (portable batch system) with multiple user environment?
- How do I schedule a job on multiple nodes with qsub Univa 8.1.7?
- SGE Cluster qsub email notifications not working
- qsub: not sharing nodes with other users
- qsub for a Python script
- SGE - QSUB fails to submit jobs in -sync mode
- Wait for all jobs of a user to finish before submitting subsequent jobs to a PBS cluster
- pass variable script argument to another script then qsub to program
- bash script PBS_ARRAYID variable argument does not qsub to job
- Capture job id of a job submitted by qsub
- Running script on my local computer when jobs submitted by qsub on a server finish
- Why do my results become un-reproducible when run from qsub?
Related Questions in SAMETIME
- how to connect 2 flutter app into a single drf api
- set wget to download 10 file at same time
- Serial Port Read and Write with Python in same time
- Two Lslidar(Leishen) connected with One Embedded Board(Jetson Xavier)`s possible?
- Lotus Sametime Embedded - Window Format
- UWP - can't interact with localhost IBM Sametime
- My Threading Chat Server isn't work in parallel . Python 2.7
- Creating common database connection for sevaral forms using qt c++
- Sametime install on ubuntu giving error
- javafx - multiple scenes in same stage
- Save image on Clipboard, and can paste to textview on other application
- OpenCV: Detect face using Haar Cascades and record video at the same time
- Can multiple qsub submissions read the same group of files?
- Sametime plugin development on mac
- In Domino, can we find out if a particular person in names field is online on sametime by using JAVA or Lotusscript?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
There are no problems with multiple jobs reading from the same files that are imposed by Torque. (I think that statement is likely to be true for any resource manager / scheduler.)
The main issue I can imagine would be your file system's performance and whether or not it can keep up with potentially concurrent accesses to those files.