Search Results

Search found 47692 results on 1908 pages for 'system io'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 122/1908 | < Previous Page | 118 119 120 121 122 123 124 125 126 127 128 129  | Next Page >

  • Efficient file buffering & scanning methods for large files in python

    - by eblume
    The description of the problem I am having is a bit complicated, and I will err on the side of providing more complete information. For the impatient, here is the briefest way I can summarize it: What is the fastest (least execution time) way to split a text file in to ALL (overlapping) substrings of size N (bound N, eg 36) while throwing out newline characters. I am writing a module which parses files in the FASTA ascii-based genome format. These files comprise what is known as the 'hg18' human reference genome, which you can download from the UCSC genome browser (go slugs!) if you like. As you will notice, the genome files are composed of chr[1..22].fa and chr[XY].fa, as well as a set of other small files which are not used in this module. Several modules already exist for parsing FASTA files, such as BioPython's SeqIO. (Sorry, I'd post a link, but I don't have the points to do so yet.) Unfortunately, every module I've been able to find doesn't do the specific operation I am trying to do. My module needs to split the genome data ('CAGTACGTCAGACTATACGGAGCTA' could be a line, for instance) in to every single overlapping N-length substring. Let me give an example using a very small file (the actual chromosome files are between 355 and 20 million characters long) and N=8 import cStringIO example_file = cStringIO.StringIO("""\ header CAGTcag TFgcACF """) for read in parse(example_file): ... print read ... CAGTCAGTF AGTCAGTFG GTCAGTFGC TCAGTFGCA CAGTFGCAC AGTFGCACF The function that I found had the absolute best performance from the methods I could think of is this: def parse(file): size = 8 # of course in my code this is a function argument file.readline() # skip past the header buffer = '' for line in file: buffer += line.rstrip().upper() while len(buffer) = size: yield buffer[:size] buffer = buffer[1:] This works, but unfortunately it still takes about 1.5 hours (see note below) to parse the human genome this way. Perhaps this is the very best I am going to see with this method (a complete code refactor might be in order, but I'd like to avoid it as this approach has some very specific advantages in other areas of the code), but I thought I would turn this over to the community. Thanks! Note, this time includes a lot of extra calculation, such as computing the opposing strand read and doing hashtable lookups on a hash of approximately 5G in size. Post-answer conclusion: It turns out that using fileobj.read() and then manipulating the resulting string (string.replace(), etc.) took relatively little time and memory compared to the remainder of the program, and so I used that approach. Thanks everyone!

    Read the article

  • fprintf() within a subprogram

    - by sergio
    Im stuck when trying to write to my file within my subprogram. void new_page(float *a, float *b, float *c, int *d){ fprintf(results,"\nPage Totals: %f\t%f\t%f\t%d", *a,*b,*c,*d); } I get a warning saying "Warning: incompatible implicit declaration of built-in function 'fprinf' [enabled by default]" "error: 'results' undeclared (first use in this function)" in main fprintf works fine, its just when it comes to the subprogram/function it wont work. from my understanding it thinks that results is undeclared, so do i have to pass the name or location of the file to make it work?

    Read the article

  • How can I exclude words with apostrophes when reading into a table of strings?

    - by rearden
    ifstream fin; string temp; fin.open("engldict.txt"); if(fin.is_open()) { bool apos = false; while(!fin.eof()) { getline(fin, temp, '\n'); if(temp.length() > 2 && temp.length() < 7) { for(unsigned int i = 0; i < temp.length(); i++) { if(temp.c_str()[i] == '\'') apos = true; } if(!apos) dictionary.insert(temp); } } } This code gives me a runtime error: Unhandled exception at 0x00A50606 in Word Jumble.exe: 0xC0000005: Access violation reading location 0x00000014. and throws me a break point at: size_type size() const _NOEXCEPT { // return length of sequence return (this->_Mysize); } within the xstring header. This exception is thrown no matter what character I use, so long as it is present within the words I am reading in. I am aware that it is probably a super simple fix, but I just really need another set of eyes to see it. Thanks in advance.

    Read the article

  • Modifying File while in use using Java

    - by Marquinio
    Hi all, I have this recurrent Java JAR program tasks that tries to modify a file every 60seconds. Problem is that if user is viewing the file than Java program will not be able to modify the file. I get the typical IOException. Anyone knows if there is a way in Java to modify a file currently in use? Or anyone knows what would be the best way to solve this problem? I was thinking of using the File canRead(), canWrite() methods to check if file is in use. If file is in use then I'm thinking of making a backup copy of data that could not be written. Then after 60 seconds add some logic to check if backup file is empty or not. If backup file is not empty then add its contents to main file. If empty then just add new data to main file. Of course, the first thing I will always do is check if file is in use. Thanks for all your ideas.

    Read the article

  • Java I/O: How to append to an already existing text file.

    - by Joe
    Hi I am having no problem writing to or appending to a file, the only problem is that as soon as I quit the program and then run it again, it creates a new file overwriting my original file. This is a problem, as I am using the text file to keep a running tally. Is there a way to get an already created text file as an object and then append to it? Thanks in advance.

    Read the article

  • iphone file download not working

    - by Anonymous
    Hi, In my app I 'm first connecting to a web service, which in return sends a url for a file. I use the url to download the file and then display it on the new view. I get the correct URL but not able to download file from that location. I have another test app which will download file from the same location and it works like a charm. following is my code for webservice-file download. This is a snippet of the code where i 'm parsing the web service xml and then pass the result to NSData for file download. Any suggestions where am i going wrong -- I 'm referring to the following tutorials. Web Service PDF Viewer if ([elementName isEqualToString:@"PRHPdfResultsResult"]) { NSLog(soapResults); UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Report downloaded from:" message:soapResults delegate:self cancelButtonTitle:@"OK" otherButtonTitles:nil]; NSData *pdfData = [[NSData alloc] initWithContentsOfURL:[NSURL URLWithString:soapResults]]; //Store the Data locally as PDF File NSString *resourceDocPath = [[NSString alloc] initWithString:[[[[NSBundle mainBundle] resourcePath] stringByDeletingLastPathComponent] stringByAppendingPathComponent:@"Documents"]]; NSString *filePath = [resourceDocPath stringByAppendingPathComponent:@"myPDF.pdf"]; [pdfData writeToFile:filePath atomically:YES]; [alert show]; [alert release]; [soapResults setString:@""]; elementFound = FALSE; }

    Read the article

  • Homemade fstat to get file size, always return 0 length.

    - by Fred
    Hello, I am trying to use my own function to get the file size from a file. I'll use this to allocate memory for a data structure to hold the information on the file. The file size function looks like this: long fileSize(FILE *fp){ long start; fflush(fp); rewind(fp); start = ftell(fp); return (fseek(fp, 0L, SEEK_END) - start); } Any ideas what I'm doing wrong here?

    Read the article

  • Parsing CSV File to MySQL DB in PHP

    - by Austin
    I have a some 350-lined CSV File with all sorts of vendors that fall into Clothes, Tools, Entertainment, etc.. categories. Using the following code I have been able to print out my CSV File. <?php $fp = fopen('promo_catalog_expanded.csv', 'r'); echo '<tr><td>'; echo implode('</td><td>', fgetcsv($fp, 4096, ',')); echo '</td></tr>'; while(!feof($fp)) { list($cat, $var, $name, $var2, $web, $var3, $phone,$var4, $kw,$var5, $desc) = fgetcsv($fp, 4096); echo '<tr><td>'; echo $cat. '</td><td>' . $name . '</td><td><a href="http://www.' . $web .'" target="_blank">' .$web.'</a></td><td>'.$phone.'</td><td>'.$kw.'</td><td>'.$desc.'</td>' ; echo '</td></tr>'; } fclose($file_handle); show_source(__FILE__); ?> First thing you will probably notice is the extraneous vars within the list(). this is because of how the excel spreadsheet/csv file: Category,,Company Name,,Website,,Phone,,Keywords,,Description ,,,,,,,,,, Clothes,,4imprint,,4imprint.com,,877-466-7746,,"polos, jackets, coats, workwear, sweatshirts, hoodies, long sleeve, pullovers, t-shirts, tees, tshirts,",,An embroidery and apparel company based in Wisconsin. ,,Apollo Embroidery,,apolloemb.com,,1-800-982-2146,,"hats, caps, headwear, bags, totes, backpacks, blankets, embroidery",,An embroidery sales company based in California. One thing to note is that the last line starts with two commas as it is also listed within "Clothes" category. My concern is that I am going about the CSV output wrong. Should I be using a foreach loop instead of this list way? Should I first get rid of any unnecessary blank columns? Please advise any flaws you may find, improvements I can use so I can be ready to import this data to a MySQL DB.

    Read the article

  • Why Doesn't This Java Code Skip Lines with #?

    - by Nathan
    I'm trying to allow an external .txt file that is read by a Java script be able to have some comments in the beginning of the file so others can easily edit it and add more to it. But if the file contains # (the sign designated for a line that is a comment) it just returns the error that there is a "Format Error in file" (the IOException - so it is getting past that first "IF"...) Can someone help? Here's the portion of the code that deals with commenting lines out of the .txt file being called earlier in the script: while ((line = br.readLine()) != null) { line = line.trim(); if (line.length() < 1 || line.charAt(0) == '#') { // ignore comments continue; } final String[] parts = line.split("="); if (parts.length != 2) { throw new IOException("Format error in file " + JLanguageTool.getDataBroker().getFromRulesDirAsUrl(getFileName()) + ", line: " + line); }

    Read the article

  • Can't access my files in ASP.NET web site

    - by jumbojs
    I'm having a very difficult time. I am running windows 2008 server, I have an Able Commerce site using ASP.NET with C#. I'm writing an automated task that will ftp some xml files down into a local directory on our web server and then the program parses the xml file and saves information to our database. The problem, once I save the files to our local directory, my program has no access to the files. The NETWORK SERVICE user permissions isn't being inherited by the xml files so my program can't do anything with them. I can manually change the permissions, but this wouldn't be automated and won't work. How can I get this to work? help please, it's very frustrating.

    Read the article

  • Check whether a folder is a local or a network resource in .NET

    - by rwmnau
    Is there a quick way to check whether a path I have is on a local disk or somewhere on the network? I can't just check to see if it's a drive letter vs. UNC, because that would incorrectly identify mapped drives as local. I assumed it would be a boolean in the DirectoryInfo object, but it appears that it's not. I've found classic VB code to do this check (through an API), but nothing for .NET so far.

    Read the article

  • Improve performance writing 10 million records to text file using windows service

    - by user1039583
    I'm fetching more than 10 millions of records from database and writing to a text file. It takes hours of time to complete this operation. Is there any option to use TPL features here? It would be great if someone could get me started implementing this with the TPL. using (FileStream fStream = new FileStream("d:\\file.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite)) { BufferedStream bStream = new BufferedStream(fStream); TextWriter writer = new StreamWriter(bStream); for (int i = 0; i < 100000000; i++) { writer.WriteLine(i); } bStream.Flush(); writer.Flush(); // empty buffer; fStream.Flush(); }

    Read the article

  • Python: How to write data in file in specific format?

    - by sasha
    i have an array called MAC1_Val: MAC1_Val array([ 1.00000000e+00, -1.00000000e+01, -2.06306600e+02, 2.22635749e+02, 1.00000000e+00, 1.00000000e+01, 1.00000000e+01, -2.06306600e+02, 2.22635749e+02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00, -1.08892735e+01, 1.88607749e+01, 1.03153300e+01, -1.78666757e+01, 3.33333333e-07, -3.33333333e-07, -4.21637021e-05, 4.21637021e-05, 9.98844400e-01, -1.73973001e-03, 1.20938900e-03, 1.87742948e-03, -3.33333333e-03, 6.66666667e-03, -3.33333333e-03, -2.64911064e-01, -2.60959501e+01, 2.81614422e+01, 3.33333333e-03, -6.66666667e-03, 3.33333333e-03, 0.00000000e+00, 0.00000000e+00]) and i want to write in file (.txt) values in specific format like this: 1.000000e+00 -1.000000e+01 -2.063066e+02 2.226357e+02 1.000000e+00 1.000000e+01 ....... note that are 6 digits behind floating point any suggestions how to do this? thanks in advance!

    Read the article

  • Is there a concise way to create an InputSupplier for an InputStream in Google Guava?

    - by Fabian Steeg
    There are a few factory methods in Google Guava to create InputSuppliers, e.g. from a byte[]: ByteStreams.newInputStreamSupplier(bytes); Or from a File: Files.newInputStreamSupplier(file); Is there a similar way to to create an InputSupplier for a given InputStream? That is, a way that's more concise than an anonymous class: new InputSupplier<InputStream>() { public InputStream getInput() throws IOException { return inputStream; } }; Background: I'd like to use InputStreams with e.g. Files.copy(...) or ByteStreams.equal(...).

    Read the article

  • unix shell, redirect output but keep on stdin

    - by Mike
    I'd like to have a shell script redirect stdout of a child process in the following manner Redirect stdout to a file Display the output of the process in real time I know I could do something like #!/bin/sh ./child > file cat file But that would not display stdout in real time. For instance, if the child was #!/bin/sh echo 1 sleep 1 echo 2 The user would see "1" and "2" printed at the same time

    Read the article

  • How can I do block-oriented disk I/O with Java? Or similar for a B+ tree

    - by Sanoj
    I would like to implement an B+ tree in Java and try to optimize it for disk based I/O. Is there an API for accessing individual disk blocks from Java? or is there an API that can do similar block-oriented access that fits my purpose? I would like to create something like Tokyo Cabinet in 100% Java. Is there anyone that knows what Java only databases like JavaDB is using in the back-end for this? I know that there are probably other languages than Java that can do this better, but I do this in a learning purpose only.

    Read the article

  • Android: How do you delete a folder starts with period

    - by jclova
    I am trying to delete a folder in sdcard. I can delete normal directories, but a directory that starts with period cannot be deleted. (ex. ".helloDir") if (dir.isDirectory()) { String[] children = dir.list(); for (int i = 0; i < children.length; i++) { new File(dir, children[i]).delete(); } } children is null if dir starts with period (ex. ".helloDir").

    Read the article

  • writing to an ioport resulting in segfaults...

    - by Sniperchild
    I'm writing for an atmel at91sam9260 arm 9 cored single board computer [glomation gesbc9260] Using request_mem_region(0xFFFFFC00,0x100,"name"); //port range runs from fc00 to fcff that works fine and shows up in /proc/iomem then i try to write to the last bit of the port at fc20 with writel(0x1, 0xFFFFFC20); and i segfault...specifically "unable to handle kernel paging request at virtual address fffffc20. I'm of the mind that i'm not allocating the right memory space... any helpful insight would be great...

    Read the article

  • How to copy files without slowing down my app?

    - by Kevin Gebhardt
    I have a bunch of little files in my assets which need to be copied to the SD-card on the first start of my App. The copy code i got from here placed in an IntentService works like a charm. However, when I start to copy many litte files, the whole app gets increddible slow (I'm not really sure why by the way), which is a really bad experience for the user on first start. As I realised other apps running normal in that time, I tried to start a child process for the service, which didn't work, as I can't acess my assets from another process as far as I understood. Has anybody out there an idea how a) to copy the files without blocking my app b) to get through to my assets from a private process (process=":myOtherProcess" in Manifest) or c) solve the problem in a complete different way Edit: To make this clearer: The copying allready takes place in a seperate thread (started automaticaly by IntentService). The problem is not to separate the task of copying but that the copying in a dedicated thread somehow affects the rest of the app (e.g. blocking to many app-specific resources?) but not other apps (so it's not blocking the whole CPU or someting) Edit2: Problem solved, it turns out, there wasn't really a problem. See my answer below.

    Read the article

  • Writing a plist

    - by iOS-Newbie
    I am trying to test out writing a dictionary to a plist. The following code does not report any errors, but I cannot find any trace of the file that I supposed wrote. Here is the code snippet: NSDictionary *myDictionary = [NSDictionary dictionaryWithObjectsAndKeys: @"First letter of the alphabet", @"A", @"Second letter of the alphabet", @"B", @"Third letter of the alphabet", @"C", nil ]; I can see the dictionary contents displayed properly with either method calls: NSLog(@"Here is my partial dictionary %@", myDictionary); for (NSString *key in myDictionary) NSLog(@"here it is again %@ %@", key, [myDictionary objectForKey:key]); The following code displays the "succeeded" message when the program is run repeatedly if ([myDictionary writeToFile: @"myDictionary" atomically:YES ] == NO) NSLog(@"write to file failed"); else NSLog(@"write to file succeeded"); even when changing the atomically: argument to NO to not write a temporary file. However, when I search my current directory, or even my entire Mac, I cannot find any file called "myDictionary.plist" or any file with the string "myDictionary". Isn't the path variable "@myDictionary" supposed to represent the file at the current directory, i.e. where the class executable resides?

    Read the article

  • How to delete duplicate/aggregate rows faster in a file using Java (no DB)

    - by S. Singh
    I have a 2GB big text file, it has 5 columns delimited by tab. A row will be called duplicate only if 4 out of 5 columns matches. Right now, I am doing dduping by first loading each coloumn in separate List , then iterating through lists, deleting the duplicate rows as it encountered and aggregating. The problem: it is taking more than 20 hours to process one file. I have 25 such files to process. Can anyone please share their experience, how they would go about doing such dduping? This dduping will be a throw away code. So, I was looking for some quick/dirty solution, to get job done as soon as possible. Here is my pseudo code (roughly) Iterate over the rows i=current_row_no. Iterate over the row no. i+1 to last_row if(col1 matches //find duplicate && col2 matches && col3 matches && col4 matches) { col5List.set(i,get col5); //aggregate } Duplicate example A and B will be duplicate A=(1,1,1,1,1), B=(1,1,1,1,2), C=(2,1,1,1,1) and output would be A=(1,1,1,1,1+2) C=(2,1,1,1,1) [notice that B has been kicked out]

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 118 119 120 121 122 123 124 125 126 127 128 129  | Next Page >