Search Results

Search found 7651 results on 307 pages for 'execution plan'.

Page 264/307 | < Previous Page | 260 261 262 263 264 265 266 267 268 269 270 271  | Next Page >

  • Batch Inserts And Prepared Query Error

    - by ircmaxell
    Ok, so I need to populate a MS Access database table with results from a MySQL query. That's not hard at all. I've got the program written to where it copies a template .mdb file to a temp name and opens it via odbc. No problem so far. I've noticed that Access does not support batch inserting (VALUES (foo, bar), (second, query), (third query)). So that means I need to execute one query per row (there are potentially hundreds of thousands of rows). Initial performance tests show a rate of around 900 inserts/sec into Access. With our largest data sets, that could mean execution times of minutes (Which isn't the end of the world, but obviously the faster the better). So, I tried testing a prepared statement. But I keep getting an error (Warning: odbc_execute() [function.odbc-execute]: SQL error: [Microsoft][ODBC Microsoft Access Driver]COUNT field incorrect , SQL state 07001 in SQLExecute in D:\....php on line 30). Here's the code I'm using (Line 30 is odbc_execute): $sql = 'INSERT INTO table ([field0], [field1], [field2], [field3], [field4], [field5]) VALUES (?, ?, ?, ?, ?, ?)'; $stmt = odbc_prepare($conn, $sql); for ($i = 200001; $i < 300001; $i++) { $a = array($i, "Field1 $", "Field2 $i", "Field3 $i", "Field4 $i", $i); odbc_execute($stmt, $a); } So my question is two fold. First, is there any idea on why I'm getting that error (I've checked, and the number in the array matches the field list which matches the number of parameter ? markers)? And second, should I even bother with this or just use the straight INSERT statements? Like I said, time isn't critical, but if it's possible, I'd like to get that time as low as possible (Then again, I may be limited by disk throughput, since 900 operations/sec is high already)... Thanks

    Read the article

  • Optimizing near-duplicate value search

    - by GApple
    I'm trying to find near duplicate values in a set of fields in order to allow an administrator to clean them up. There are two criteria that I am matching on One string is wholly contained within the other, and is at least 1/4 of its length The strings have an edit distance less than 5% of the total length of the two strings The Pseudo-PHP code: foreach($values as $value){ foreach($values as $match){ if( ( $value['length'] < $match['length'] && $value['length'] * 4 > $match['length'] && stripos($match['value'], $value['value']) !== false ) || ( $match['length'] < $value['length'] && $match['length'] * 4 > $value['length'] && stripos($value['value'], $match['value']) !== false ) || ( abs($value['length'] - $match['length']) * 20 < ($value['length'] + $match['length']) && 0 < ($match['changes'] = levenshtein($value['value'], $match['value'])) && $match['changes'] * 20 <= ($value['length'] + $match['length']) ) ){ $matches[] = &$match; } } } I've tried to reduce calls to the comparatively expensive stripos and levenshtein functions where possible, which has reduced the execution time quite a bit. However, as an O(n^2) operation this just doesn't scale to the larger sets of values and it seems that a significant amount of the processing time is spent simply iterating through the arrays. Some properties of a few sets of values being operated on Total | Strings | # of matches per string | | Strings | With Matches | Average | Median | Max | Time (s) | --------+--------------+---------+--------+------+----------+ 844 | 413 | 1.8 | 1 | 58 | 140 | 593 | 156 | 1.2 | 1 | 5 | 62 | 272 | 168 | 3.2 | 2 | 26 | 10 | 157 | 47 | 1.5 | 1 | 4 | 3.2 | 106 | 48 | 1.8 | 1 | 8 | 1.3 | 62 | 47 | 2.9 | 2 | 16 | 0.4 | Are there any other things I can do to reduce the time to check criteria, and more importantly are there any ways for me to reduce the number of criteria checks required (for example, by pre-processing the input values), since there is such low selectivity?

    Read the article

  • Boost shared_ptr use_count function

    - by photo_tom
    My application problem is the following - I have a large structure foo. Because these are large and for memory management reasons, we do not wish to delete them when processing on the data is complete. We are storing them in std::vector<boost::shared_ptr<foo>>. My question is related to knowing when all processing is complete. First decision is that we do not want any of the other application code to mark a complete flag in the structure because there are multiple execution paths in the program and we cannot predict which one is the last. So in our implementation, once processing is complete, we delete all copies of boost::shared_ptr<foo>> except for the one in the vector. This will drop the reference counter in the shared_ptr to 1. Is it practical to use shared_ptr.use_count() to see if it is equal to 1 to know when all other parts of my app are done with the data. One additional reason I'm asking the question is that the boost documentation on the shared pointer shared_ptr recommends not using "use_count" for production code.

    Read the article

  • PHP Array to CSV

    - by JohnnyFaldo
    I'm trying to convert an array of products into a CSV file, but it doesn't seem to be going to plan. The CSV file is one long line, here is my code: for($i=0;$i<count($prods);$i++) { $sql = "SELECT * FROM products WHERE id = '".$prods[$i]."'"; $result = $mysqli->query($sql); $info = $result->fetch_array(); } $header = ''; for($i=0;$i<count($info);$i++) { $row = $info[$i]; $line = ''; for($b=0;$b<count($row);$b++) { $value = $row[$b]; if ( ( !isset( $value ) ) || ( $value == "" ) ) { $value = "\t"; } else { $value = str_replace( '"' , '""' , $value ); $value = '"' . $value . '"' . "\t"; } $line .= $value; } $data .= trim( $line ) . "\n"; } $data = str_replace( "\r" , "" , $data ); if ( $data == "" ) { $data = "\n(0) Records Found!\n"; } header("Content-type: application/octet-stream"); header("Content-Disposition: attachment; filename=your_desired_name.xls"); header("Pragma: no-cache"); header("Expires: 0"); print "$data"; Also, the header doesn't force a download. I've been copy and pasting the output and saving as .csv

    Read the article

  • C# Dictionary Loop Enhancment

    - by Toto
    Hi, I have a dictionary with around 1 milions items. I am constantly looping throw the dictionnary : public void DoAllJobs() { foreach (KeyValuePair<uint, BusinessObject> p in _dictionnary) { if(p.Value.MustDoJob) p.Value.DoJob(); } } The execution is a bit long, around 600 ms, I would like to deacrese it. Here is the contraints : MustDoJob values mostly stay the same beetween two calls to DoAllJobs() 60-70% of the MustDoJob values == false From time to times MustDoJob change for 200 000 pairs. Some p.Value.DoJob() can not be computed at the same time (COM object call) Here, I do not need the key part of the _dictionnary objet but I really do need it somewhere else I wanted to do the following : Parallelizes but I am not sure is going to be effective due to 4. Sorts the dictionnary since 1. and 2. (and stop want I find the first MustDoJob == false) but I am wondering what 3. would result in I did not implement any of the previous ideas since it could be a lot of job and I would like to investigate others options before. So...any ideas ?

    Read the article

  • Looking forward to a programming future but confused where to start.

    - by Kraivyne
    Hi there, I am very new to this site and to programming. I started doing some basic programming with python a few weeks ago and recently, messing around with Java basics. My main problem is that I am completely overwhelmed and haven't got the slightest clue where I should be starting. I want to learn programming because I really enjoy doing it, the simple applications that I have managed to conjure up put a smile on my face. My plan is to eventually (by eventually I'm talking about 6 years+) go into games programming. I have been informed that C++ is the best way to go about this but haven't got the slightest clue what book/sight is optimal for someone who is still learning the very basics. These are my questions: I have been to the Definitive C++ Book Guide but am still unsure which book is best to start of with. Should I stick with Python or Java instead of moving on to C++? Is there any advice you would give to a beginner programmer? Thanks again for all your help.

    Read the article

  • NSView only redraws on breakpoint

    - by Jacopo
    I have a custom view inside a NSPopover. It should change according to user input and it does the first time the user interact with it but it fails to redraw the following times. I have tried to put an NSLog inside the -drawRect: method and it doesn't get called during normal execution. When I try to debug and put a breakpoint inside the method it gets called normally and the app works as it should. I explicitly call the view -setNeedsDisplay: method every time I need it to redraw. I don't understand why it should make a difference. Here is the code that update the status of the view. These methods are part of the NSTextField delegate method -textDidChange: and I checked that these get called every time the user type something in the textfield associated with popover. [tokenCloud tokensToHighlight:[NSArray arrayWithObject:completeSuggestionString]]; tokenCloud.tokens = filteredTokens; [tokenCloud setNeedsDisplay:YES]; The views is a series of recessed button. The first line update the status of all the buttons in the popover and the second add or delete buttons. They both work properly because the first time they are called the view is update properly. I have also checked that both the status of the buttons in tokenCloud and its property tokens are updated correctly. The problem is that the NSView subclass, tokenCloud, doesn't redraw so the changes are not reflected in the UI the second time. Here is the draw method of the view: - (void)drawRect:(NSRect)rect { [self recalculateButtonLocations]; NSLog(@"Redrawn"); } Again this method gets called normally every time I update the view if I place a breakpoint in [self recalculateButtonLocations];. If instead I let the app run normally nothing gets logged in the console the second time I update the view. Same thing if I include the NSLog in the recalculateButtonLocations method, nothing gets logged the second time meaning that the method is not called.

    Read the article

  • Telling me my stored procedure isn't declared

    - by Scott
    Here is where the error is occuring in the stack: public static IKSList<DataParameter> Search(int categoryID, int departmentID, string title) { Database db = new Database(DatabaseConfig.CommonConnString, DatabaseConfig.CommonSchemaOwner, "pkg_data_params_new", "spdata_params_search"); db.AddParameter("category_id", categoryID); db.AddParameter("department_id", departmentID); db.AddParameter("title", title, title.Length); DataView temp = db.Execute_DataView(); IKSList<DataParameter> dps = new IKSList<DataParameter>(); foreach (DataRow dr in temp.Table.Rows) { DataParameter dp = new DataParameter(); dp.Load(dr); dps.Add(dp); } return dps; } And here is the error text: ORA-06550: line 1, column 38: PLS-00302: component 'SPDATA_PARAMS_SEARCH' must be declared ORA-06550: line 1, column 7: PL/SQL: Statement ignored Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Data.OracleClient.OracleException: ORA-06550: line 1, column 38: PLS-00302: component 'SPDATA_PARAMS_SEARCH' must be declared ORA-06550: line 1, column 7: PL/SQL: Statement ignored Source Error: Line 161: db.AddParameter("title", title, title.Length); Line 162: Line 163: DataView temp = db.Execute_DataView(); Line 164: Line 165: IKSList dps = new IKSList(); My webconfig is pointing to the correct place and everything so idk where this is coming from.

    Read the article

  • Writing JavaScript from a Custom Control

    - by coffeeaddict
    I'm new to writing custom controls. I have MyCustomControl.cs and in my Render method I want to render out about 50 lines of JavaScript. What's the best way to do this, use the writer? protected override void Render(HtmlTextWriter writer) { writer.write(@"<script type....rest of opening tag here"); writer.Write(@" function decode(s) { return s.replace(/&amp;/g, ""&"") .replace(/&quot;/g, '""') .replace(/&#039;/g, ""'"") .replace(/&lt;/g, ""<"") .replace(/&gt;/g, "">""); };" ); I plan on having around 6 more writer.Write to write out some more sections here. Is that the best approach to actually perform the writing of JavaScript in this manor? or should I use ClientScript.RegisterClientScriptBlock? So what's the best practice or common way people are writing javascript from a custom control? (I'm not talking about a user control here!!, custom control/Class!) I also want to keep any indentation for readability once it's spit out/rendered on the client when viewing source.

    Read the article

  • Should I be worried if I don't get any internships by 3rd year's end?

    - by karamba
    I am in some mediocre college in some corner of India. Am about to complete 3rd year C.S. in a month and a half. I have no idea how to go about "finding an internship" as everyone seems to put it. Looking at online advice, I find that a primary way is to "use your contacts". I am sad to say that I don't have many friends(those I have, I am trying to get help from them for all it's worth), and my family can't help me as they have no idea about the software industry. My college has no official facility for aiding students in this, and the few faculty members who had contacts in "whatever" part of the industry have favoured some students that they have personally come to know. (Though I hear that the "internships" they got involve them stocking equipment in some small companies.... still it's something?) I'm getting nervous. I am considering just spending the coming summer refining my skills in C++ and begin learning MySQL and C#, both of which I have zero experience in. Maybe work on my own project... like a library management system. Relative to those in my college, I think I am among the best programmers there, but that isn't saying much as a lot of students can barely write basic code. I have experience in teaching myself C++, and DirectX9 having created a Tetris clone, some basic 3D apps (bouncing balls), and a basic console-based, text-file-database-using library management system (which I plan to improve this summer). Is it alright if I spend my summer so? Will I be able to get a job later on? I know I have to improve my social skills to get anywhere in life, and I will try, but say I am stuck like this till 4th year's end... will such self studying, online learning help me in landing a decent job? Perhaps after I have learned a bit more, joining some open source project?

    Read the article

  • What is a good RPC model for building a AJAX web app using PHP & JS?

    - by user366152
    I'm new to writing AJAX applications. I plan on using jQuery on the client side while PHP on the server side. I want to use something like XML-RPC to simplify my effort in calling server-side code. Ideally, I wouldn't care whether the transport layer uses XML or JSON or a format more optimized for the wire. If I was writing a console app I'd use some tool to generate function stubs which I would then implement on the RPC server while the client would natively call into those stubs. This provides a clean separation. Is there something similar available in the AJAX world? While on this topic, how would I proceed with session management? I would want it to be as transparent as possible. For example, if I try to hit an RPC end-point which needs a valid session, it should reject the request if the client doesn't pass a valid session cookie. This would really ease my application development. I'd then have to simply handle the frontend using native JS functions. While on the backend, I can simply implement the RPC functions. BTW I dont wish to use Google Web Toolkit. My app wont be extremely heavy on AJAX.

    Read the article

  • C# / this. and List<T>

    - by user3533030
    I am having trouble understanding how to initialize a List of objects and use it with methods inside of a class. I understand the mechanics of the List, but not how to initialize it inside a method and use it later. For example, I want to have a class that creates the List when it is constructed. Then, I want to use a method of that class to add elements to the list. The elements in the list are objects defined by the SolidWorks API. So, to construct the List, I used... public class ExportPoints : Exporter { public List<SldWorks.SketchPoint> listOfSketchPoints; public ExportPoints(SldWorks.SldWorks swApp, string nameSuffix) : base(swApp, nameSuffix) { List<SldWorks.SketchPoint> listOfSketchPoints = new List<SldWorks.SketchPoint>(); } public void createListOfFreePoints() { try { [imagine more code here] this.listOfSketchPoints.Add(pointTest); } catch (Exception e) { Debug.Print(e.ToString()); return; } } This fails during execution as if the listOfSketchPoints was never initialized as a List. So, I tried a hack and this worked: public ExportPoints(SldWorks.SldWorks swApp, string nameSuffix) : base(swApp, nameSuffix) { List<SldWorks.SketchPoint> listOfSketchPoints = new List<SldWorks.SketchPoint>(); this.listOfSketchPoints = listOfSketchPoints; } This approach creates the behavior that I want. However, it seems that I lack some understanding as to why this is necessary. Shouldn't it be possible to "initialize" a List that is a property of your object with a constructor? Why would you need to create the list, then assign the pointer of that new List to your property?

    Read the article

  • Is MySQL caching occurring, how to fix it?

    - by rlb.usa
    I think that MySQL or ASP.NET is caching my queries. I edited my MySQL sproc to remove some parameters but it keeps saying that those parameters are missing. Here is what happens: ASP.NET app calls a MySQL stored procedure. Everything works perfect. I delete some parameters from the sproc and ASP.NET parameter list accordingly. All parameters exactly match in case and order from the new ASP.NET and MySQL sproc code Upon execution, it fails, saying : System.ArgumentException: Parameter 'deleted_parameter_foo_bar' not found in the collection. at MySql.Data.MySqlClient.MySqlParameterCollection ... I delete the sproc from the database, restart my browser, and reexecute the ASP.NET page. It says the same error, that the parameter is missing - but the sproc itself doesn't exist anymore. ( I know 100% that I am editing/deleting from the right database. ) How do I fix this or make it work again; I want it to use my new sproc instead of the old one ? _o

    Read the article

  • How would I sort files to directories based on filenames?

    - by gnomed
    I have a huge number of files to sort all named in some terrible convention. Here are some examples: (4)_mr__mcloughlin____.txt 12__sir_john_farr____.txt (b)mr__chope____.txt dame_elaine_kellett-bowman____.txt dr__blackburn______.txt These names are supposed to be a different person (speaker) each. Someone in another IT department produced these from a ton of XML files using some script but the naming is unfathomably stupid as you can see. I need to sort literally tens of thousands of these files with multiple files of text for each person; each with something stupid making the filename different, be it more underscores or some random number. They need to be sorted by speaker. This would be easier with a script to do most of the work then I could just go back and merge folders that should be under the same name or whatever. There are a number of ways I was thinking about doing this. parse the names from each file and sort them into folders for each unique name. get a list of all the unique names from the filenames, then look through this simplified list of unique names for similar ones and ask me whether they are the same, and once it has determined this it will sort them all accordingly. I plan on using Perl, but I can try a new language if it's worth it. I'm not sure how to go about reading in each filename in a directory one at a time into a string for parsing into an actual name. I'm not completely sure how to parse with regex in perl either, but that might be googleable. For the sorting, I was just gonna use the shell command: `cp filename.txt /example/destination/filename.txt` but just cause that's all I know so it's easiest. I dont even have a pseudocode idea of what im going to do either so if someone knows the best sequence of actions, im all ears. I guess I am looking for a lot of help, I am open to any suggestions. Many many many thanks to anyone who can help. B.

    Read the article

  • php parsing csv with ftell

    - by Robert82
    I have a 500mb csv file with over 500,000 lines, each with 80 fields. I am using fget to process the file line by line. $col1 = array(); while (($row = fgetcsv($handle, 1000, ",")) !== FALSE) { $col1[] = $row[0]; } Because of an execution time limit on the PHP file by my hosting provider (120 seconds), I can't process the whole file in one run. I tried using ftell() and fseek() to remember the last position for restart. The trouble is, sometimes the ftell() position is in the middle of a row, and resuming means missing the first half of the row. Is there an elegant way to know the last line successfully processed, and resume from the one after it? I realize I can do a simple counter, and then loop through to that point again, but that would produce diminishing returns on the rows I can process towards the end of the file. Is there something like ftell() and fseek() that would work in my case? Or a way to limit ftell() to return the pointer for the end of the previous line?

    Read the article

  • How best to present a security vulnerability to a web development team in your own company?

    - by BigCoEmployee
    Imagine the following scenario: You work at Big Co. and your coworkers down the hall are on the web development team for Big Co's public blog system, which a lot of Big Co employees and some public people use. The blog system allows any HTML and JavaScript, and you've been told that it was a choice (not by accident) but you aren't sure if they realize the implications of this. So you want to convince them that this is a bad idea. You write some demonstration code and plant a XSS script in your own blog, and then write some blog posts. Soon after, the head blog admin (down the hall) visits your blog post and the XSS sends his cookies to you. You copy them into your browser and you are now logged in as him. Okay, now you're logged in as him... And you start realizing that it maybe wasn't such a good idea to go ahead and 'hack' the blog system. But you are a good guy! You don't touch his account after logging into it, and you definitely don't plan on publicizing this weakness; you just maybe want to show them that the public is able to do this, so that they can fix it before someone malicious realizes the same thing! What is the best course of action from here?

    Read the article

  • Smarter println that shows the depth in the stack

    - by Hectoret
    I am using System.out.println in my code to track the execution of a program and get some useful output. This creates results like this in the console: Main function. Program starts. Method getArea. Getting values Method getSide. Side is 6 Method getArea. First value is 6 Method getSide. Side is 8 Method getArea. Second value is 8 Method getArea. Area is 48 Main function. The final area is 48 I would like to create tha method, which adds a space in front of the output every time the code goes deeper in the method call stack. For example, the same code but instead of using System.out.println, now with Misc.smartPrintln: Main function. Program starts. Method getArea. Getting values Method getSide. Side is 6 Method getArea. First value is 6 Method getSide. Side is 8 Method getArea. Second value is 8 Method getArea. Area is 48 Main function. The final area is 48 The method would have this definition: public static void smartPrintln(String string); I don't know how to implement this functionality. Any ideas how to solve this? And, could the use of a logger offer this functionality?

    Read the article

  • javascript error handling

    - by pankaj
    I have a javascript function for checking errors which I am calling on OnClicentClick event of a button. Once it catch a error I want to stop execution of the click event. But in my case it always it always executes the onclick event. Following is my function: function DisplayError() { if (document.getElementById('<%=txtPassword.ClientID %>').value.length < 6 || document.getElementById('<%=txtPassword.ClientID %>').value.length > 12) { document.getElementById('<%=lblError.ClientID %>').innerText = "Password length must be between 6 to 12 characters"; return false; } var str = <%=PhoneNumber()%>; if(str.length <10) { alert('<%=phoneNum%>'.length); document.getElementById('<%=lblError.ClientID %>').innerText = "Phone Number not in correct format"; return false; } } button html code: <asp:Button runat="server" Text="Submit" ID="btnSubmit" ValidationGroup="submit" onclick="btnSubmit_Click" OnClientClick="DisplayError()"/> It should not execute the button click event once it satisfies any of the IF condition in the javascript function.

    Read the article

  • SQL Server 2005 FREETEXT() Perfomance Issue

    - by Zenon
    I have a query with about 6-7 joined tables and a FREETEXT() predicate on 6 columns of the base table in the where. Now, this query worked fine (in under 2 seconds) for the last year and practically remained unchanged (i tried old versions and the problem persists) So today, all of a sudden, the same query takes around 1-1.5 minutes. After checking the Execution Plan in SQL Server 2005, rebuilding the FULLTEXT Index of that table, reorganising the FULLTEXT index, creating the index from scratch, restarting the SQL Server Service, restarting the whole server I don't know what else to try. I temporarily switched the query to use LIKE instead until i figure this out (which takes about 6 seconds now). When I look at the query in the query performance analyser, when I compare the ´FREETEXT´query with the ´LIKE´ query, the former has 350 times as many reads (4921261 vs. 13943) and 20 times (38937 vs. 1938) the CPU usage of the latter. So it really is the ´FREETEXT´predicate that causes it to be so slow. Has anyone got any ideas on what the reason might be? Or further tests I could do?

    Read the article

  • Can anyone explain this impossible bit of PHP logic?

    - by user268208
    I'm attempting to debug a simple PHP script. Essentially, there's a variable which is defined with: $variable = ($_GET['variable'] == 'true') ? TRUE : FALSE; Then, in the view file, the following code is meant to display a box if $variable == TRUE: <? if ($variable == true) { ?> <p class="box">You have imported a new plan.</p> <? } ?> Now, even when that $variable, as shown by var_dump($variable); == FALSE, that HTML is printed between the if { } tags. To me, this defies logic. I simply can't figure out this problem out. Furthermore, this code works fine on many PHP4 and PHP5 installations except for one particular server running PHP5.2. Any possible suggestions? Leads? I'm pulling out my hair trying to figure this one out. Thank you.

    Read the article

  • two view controllers and reusability with delegate

    - by netcharmer
    Newbie question about design patterns in objC. I'm writing a functionality for my iphone app which I plan to use in other apps too. The functionality is written over two classes - Viewcontroller1 and Viewcontroller2. Viewcontroller1 is the root view of a navigation controller and it can push Viewcontroller2. Rest of the app will use only ViewController1 and will never access Viewcontroller2 directly. However, triggered by user events, Viewcontroller2 has to send a message to the rest of the app. My question is what is the best way of achieving it? Currently, I use two level of delegation to send the message out from Viewcontroller2. First send it to Viewcontroller1 and then let Viewcontroller1 send it to rest of the app or the application delegate. So my code looks like - //Viewcontroller1.h @protocol bellDelegate -(int)bellRang:(int)size; @end @interface Viewcontroller1 : UITableViewController <dummydelegate> { id <bellDelegate> delegate; @end //Viewcontroller1.m @implementation Viewcontroller1 -(void)viewDidLoad { //some stuff here Viewcontroller2 *vc2 = [[Viewcontroller2 alloc] init]; vc2.delegate = self; [self.navigationController pushViewController:vc2 animated:YES]; } -(int)dummyBell:(int)size { return([self.delegate bellRang:size]); } //Viewcontroller2.h @protocol dummyDelegate -(int)dummyBell:(int)size; @end @interface Viewcontroller2 : UITableViewController { id <dummyDelegate> delegate; @end //Viewcontroller2.m @implementation Viewcontroller2 -(int)eventFoo:(int)size { rval = [self.delegate dummyBell:size]; } @end

    Read the article

  • Python2.7: How can I speed up this bit of code (loop/lists/tuple optimization)?

    - by user89
    I repeat the following idiom again and again. I read from a large file (sometimes, up to 1.2 million records!) and store the output into an SQLite databse. Putting stuff into the SQLite DB seems to be fairly fast. def readerFunction(recordSize, recordFormat, connection, outputDirectory, outputFile, numObjects): insertString = "insert into NODE_DISP_INFO(node, analysis, timeStep, H1_translation, H2_translation, V_translation, H1_rotation, H2_rotation, V_rotation) values (?, ?, ?, ?, ?, ?, ?, ?, ?)" analysisNumber = int(outputPath[-3:]) outputFileObject = open(os.path.join(outputDirectory, outputFile), "rb") outputFileObject, numberOfRecordsInFileObject = determineNumberOfRecordsInFileObjectGivenRecordSize(recordSize, outputFileObject) numberOfRecordsPerObject = (numberOfRecordsInFileObject//numberOfObjects) loop1StartTime = time.time() for i in range(numberOfRecordsPerObject ): processedRecords = [] loop2StartTime = time.time() for j in range(numberOfObjects): fout = outputFileObject .read(recordSize) processedRecords.append(tuple([j+1, analysisNumber, i] + [x for x in list(struct.unpack(recordFormat, fout))])) loop2EndTime = time.time() print "Time taken to finish loop2: {}".format(loop2EndTime-loop2StartTime) dbInsertStartTime = time.time() connection.executemany(insertString, processedRecords) dbInsertEndTime = time.time() loop1EndTime = time.time() print "Time taken to finish loop1: {}".format(loop1EndTime-loop1StartTime) outputFileObject.close() print "Finished reading output file for analysis {}...".format(analysisNumber) When I run the code, it seems that "loop 2" and "inserting into the database" is where most execution time is spent. Average "loop 2" time is 0.003s, but it is run up to 50,000 times, in some analyses. The time spent putting stuff into the database is about the same: 0.004s. Currently, I am inserting into the database every time after loop2 finishes so that I don't have to deal with running out RAM. What could I do to speed up "loop 2"?

    Read the article

  • Avoiding repetition with libraries that use a setup + execute model

    - by lijie
    Some libraries offer the ability to separate setup and execution, esp if the setup portion has undesirable characteristics such as unbounded latency. If the program needs to have this reflected in its structure, then it is natural to have: void setupXXX(...); // which calls the setup stuff void doXXX(...); // which calls the execute stuff The problem with this is that the structure of setupXXX and doXXX is going to be quite similar (at least textually -- control flow will prob be more complex in doXXX). Wondering if there are any ways to avoid this. Example: Let's say we're doing signal processing: filtering with a known kernel in the frequency domain. so, setupXXX and doXXX would probably be something like... void doFilter(FilterStuff *c) { for (int i = 0; i < c->N; ++i) { doFFT(c->x[i], c->fft_forward_setup, c->tmp); doMultiplyVector(c->tmp, c->filter); doFFT(c->tmp, c->fft_inverse_setup, c->x[i]); } } void setupFilter(FilterStuff *c) { setupFFT(..., &(c->fft_forward_setup)); // assign the kernel to c->filter ... setupFFT(..., &(c->fft_inverse_setup)); }

    Read the article

  • php:: apply backticks to first word in sentence

    - by Hailwood
    Hi guys, basically what I am trying to do is, I have an array that looks something like this: array( array( 'select' =>'first string', 'escape' => true ), array( 'select' =>'second', 'escape' => true ), array( 'select' =>'the third string', 'escape' => true ), array( 'select' =>'fourth string', 'escape' => false ), ) I am looping over it and I want to end up with this output array( array( 'select' =>'`first` string', 'escape' => true ), array( 'select' =>'`second`', 'escape' => true ), array( 'select' =>'`the` third string', 'escape' => true ), array( 'select' =>'fourth string', 'escape' => false ), ) so basic rules are backticks are only applied if escape is true backticks are only applied to the first word in a sentence if there is only one word backticks are applied to the word My plan was to use if($item['escape']) { $pos = (strpos($item['select'], ' ') === false ? strlen($item['select']) : strpos($item['select'], ' ')); $item['select'] = '`' . substr($item['select'], 0, $pos) . '`' . substr($item['select'], $pos, strlen($item['select'])); } but the $item['select'] = line seems rather long winded, is there a better way to write it?

    Read the article

  • Variable Scoping in a method and its persistence in C++

    - by de costo
    Consider the following public method that adds an integer variable to a vector of ints(private member) in a class in C++. KoolMethod() { int x; x = 10; KoolList.Add(x); } Vector<int>KoolList; But is this a valid addition to a vector ??? Upon calling the method, it creates a local variable. The scope of this local variable ends the moment the execution control leaves the method. And since this local variable is allocated on a stack(on the method call), any member of KoolList points to an invalid memory location in deallocated stack which may or may not contain the expected value of x. Is this an accurate description of above mechanism ?? Is there a need for creating an int in heap storage using "new" operator everytime a value needs to be added to the vector like described below ????: KoolMethod() { int *x = new int(); *x = 10; KoolList.Add(x); } Vector<int*>KoolList;

    Read the article

< Previous Page | 260 261 262 263 264 265 266 267 268 269 270 271  | Next Page >