Search Results

Search found 10910 results on 437 pages for 'john speed'.

Page 392/437 | < Previous Page | 388 389 390 391 392 393 394 395 396 397 398 399  | Next Page >

  • Testing performance of queries in mysl

    - by Unreason
    I am trying to setup a script that would test performance of queries on a development mysql server. Here are more details: I have root access I am the only user accessing the server Mostly interested in InnoDB performance The queries I am optimizing are mostly search queries (SELECT ... LIKE '%xy%') What I want to do is to create reliable testing environment for measuring the speed of a single query, free from dependencies on other variables. Till now I have been using SQL_NO_CACHE, but sometimes the results of such tests also show caching behaviour - taking much longer to execute on the first run and taking less time on subsequent runs. If someone can explain this behaviour in full detail I might stick to using SQL_NO_CACHE; I do believe that it might be due to file system cache and/or caching of indexes used to execute the query, as this post explains. It is not clear to me when Buffer Pool and Key Buffer get invalidated or how they might interfere with testing. So, short of restarting mysql server, how would you recommend to setup an environment that would be reliable in determining if one query performs better then the other?

    Read the article

  • jQuery Cycle : Multiple Instances, detect end of each instance

    - by guylabbe.ca
    I am using multiple instances of Cycle for a portfolio; Only one is shown and the others are hidden. Each Cycle is a project with multiple images. The only thing I can't figure out is how to detect the end of the « current » Cycle instance. with after: function, it is triggered for all instances at once, impossible to get « local » events. Here is my code : $('.sldr_closeup').each(function() { var currId = $(this).attr('id'); window['num_'+currId] = 0; $(this).cycle({ timeout:0, speed:500, fx:'fade', next:'.nextimages', prev:'.previmages', fit:1, nowrap:1, autstop:0, after : function(c,n,o,f) { var currId = $(this).parent('div').attr('id'); (f) ? window['num_'+currId] ++ : window['num_'+currId] --; if ((o.slideCount == window['num_'+currId] )) { alert(currId); $('.nextimages').stop().fadeTo(400,0,function(){ if(currId != $('.projectList .projet').last().find('a').attr('rel')) $('.nextproject').stop().fadeTo(400,1); $(this).hide(); }); } } }).cycle('pause'); });

    Read the article

  • Can Haskell's Parsec library be used to implement a recursive descent parser with backup?

    - by Thor Thurn
    I've been considering using Haskell's Parsec parsing library to parse a subset of Java as a recursive descent parser as an alternative to more traditional parser-generator solutions like Happy. Parsec seems very easy to use, and parse speed is definitely not a factor for me. I'm wondering, though, if it's possible to implement "backup" with Parsec, a technique which finds the correct production to use by trying each one in turn. For a simple example, consider the very start of the JLS Java grammar: Literal: IntegerLiteral FloatingPointLiteral I'd like a way to not have to figure out how I should order these two rules to get the parse to succeed. As it stands, a naive implementation like this: literal = do { x <- try (do { v <- integer; return (IntLiteral v)}) <|> (do { v <- float; return (FPLiteral v)}); return(Literal x) } Will not work... inputs like "15.2" will cause the integer parser to succeed first, and then the whole thing will choke on the "." symbol. In this case, of course, it's obvious that you can solve the problem by re-ordering the two productions. In the general case, though, finding things like this is going to be a nightmare, and it's very likely that I'll miss some cases. Ideally, I'd like a way to have Parsec figure out stuff like this for me. Is this possible, or am I simply trying to do too much with the library? The Parsec documentation claims that it can "parse context-sensitive, infinite look-ahead grammars", so it seems like something like I should be able to do something here.

    Read the article

  • What is SSIS order of data transformation component method calls

    - by Ron Ruble
    I am working on a custom data transformation component. I'm using NUnit and NMock2 to test as I code. Testing and getting the custom UI and other features right is a huge pain, in part because I can't find any documentation about the order in which SSIS invokes methods on the component at design time as well as runtime. I can correct the issues readily enough, but it's tedious and time consuming to unregister the old version, register the new version, fire up the test ssis package, try to display the UI, get an obscure error message, backtrace it, modify the component and continue. One of the big issues involves the UI component needing access to the componentmetadata and buffermanager properties of the component at design time, and what I need to provide for to support properties that won't be initialized until after the user enters them in the UI. I can work through it; but if someone knows of some docs or tips that would speed me up, I'd greatly appreciate it. The samples I've found havn't been much use; they seem to be directed to showing off cool stuff (Twitter, weather.com) rather than actual work. Thanks in advance.

    Read the article

  • Adding time to a timer/counter

    - by BoneStarr
    I've looked all over the web and everyone can teach you how to make a timer for your game or a countdown, but I can't seem to find out how to add time to an already counting timer. So here is my counter class: package { import flash.display.MovieClip; import flash.display.Stage; import flash.text.TextField; import flash.events.Event; import flash.utils.Timer; import flash.events.TimerEvent; public class Score extends MovieClip { public var second:Number = 0; public var timer:Timer = new Timer(100); private var stageRef:Stage; public function Score(stageRef:Stage) { x = 560.95; y = 31.35; this.stageRef = stageRef; timer.addEventListener(TimerEvent.TIMER, scoreTimer); timer.start(); } public function scoreTimer(evt:TimerEvent):void { second += 1; scoreDisplay.text = String("Score: " +second); } That works without any issues or problems and just keeps counting upwards at a speed of 100ms, what I want to know is how to add say 30 seconds if something happens in my game, say you kill an enemy for example. Please help!

    Read the article

  • jQuery XML loading and then innerfade effect

    - by Ryan Max
    Hello, I think I can explain myself without code, so for brevity's sake here we go: I am using jquery to pull data from an xml and put it into a ul on the page with each xml entry as a li. This is working great! However, what I am trying to do afterwards is use the innerfade plugin to make a simple animation between each of the li's. It's not working though, as it is still just loading the static list with each item visible (whereas if innerfade was working it would only display the first....then fade into the second, etc) It's not an innerfade problem however, because if I add the list in manually to the page (not injecting it with jquery) then the innerfade works fine. I'm relatively new to DOM scripting, so I think I am missing something here. I'm not quite sure how jQuery sequences everything, and I'm having trouble phrasing my question in a search engine friendly manner so here I am. Is it possible to have jquery pull the data from xml, then inject it into the page, then have innerfade work it's magic? Or am I thinking about this the wrong way? xml code: $.ajax({ type: "GET", url: "xml/playlist.xml", dataType: "xml", success: function(xml) { $(xml).find('song').each(function(){ var name = $(this).attr('title'); var date = $(this).attr('artist'); var message = $(this).attr('path'); $('<li></li>').html('<span id="an_name">'+name+'</span><span id="an_date">'+date+'</span><span id="an_message">'+message+'</span>').appendTo('#anniversary'); }); } }); innerfade code: <script type="text/javascript"> jQuery.noConflict(); jQuery(document).ready( function(){ jQuery('#anniversary').innerfade({ speed: 1000, timeout: 5000, type: 'sequence', }); });

    Read the article

  • Speedup writing C programs using a subset of the Python syntax

    - by psihodelia
    I am constantly trying to optimize my time. Writing a C code takes a lot of time and requires much more keyboard touches than say writing a Python program. However, in order to speed up the time required to create a C program, one can automatize many things. I'd like to write my programs using smth. like Python but with C semantics. It means, all keywords are C keywords, but syntax is optimized. For example, this C code: #include "dsplib.h" #include "coeffs.h" #define MODULENAME "dsplib" #define NUM_SAMPLES 320 typedef float t_Vec; typedef struct s_Inter { char *pc_Name; struct s_Inter *px_Next; }t_Inter; typedef struct s_DspLibControl { t_Vec f_Y; }t_DspLibControl; void v_DspLibName(void) { printf("Module: %s", MODULENAME); printf("\n"); } int v_DspLibInitInterControl(t_DspLibControl *px_Con) { int y; px_Con->f_Y = 0.0; for(int i=0;i<10;i++) { y += i * i; } return y; } in optimized pythonized version can look like: include dsplib, coeffs define MODULENAME="dsplib", NUM_SAMPLES=320 typedef float t_Vec typedef struct s_Inter: char *pc_Name struct s_Inter *px_Next t_Inter typedef struct s_DspLibControl: t_Vec f_Y t_DspLibControl v_DspLibName(): printf("Module: %s", MODULENAME); printf("\n") int v_DspLibInitInterControl(t_DspLibControl *px_Con): int y px_Con->f_Y = 0.0 for int i=0;i<10;i++: y += i * i return y My question is: Do you know any VIM script, which allows to translate an original pythonized C code into a standard C code? For example, one is writing a C code but uses pythonized syntax, once she decides to translate pythonized blocks into standard C, she selects such blocks and press some key. And she doesn't save such pythonized code of course, VIM translates it into standard C.

    Read the article

  • How to Manage CSS Explosion

    - by Jason
    I have been heavily relying on CSS for a website that I am working on (currently, everything is done as property values within each tag on the website and I'm trying to get away from that to make updates significantly easier). The problem I am running into, is I'm starting to get a bit of "CSS explosion" going on. It is becoming difficult for me to decide how to best organize and abstract data within the CSS file. For example: I am using a large number of div tags within the website (previously it was completely tables based). So I'm starting to get a lot of CSS that looks like this... div.title { background-color: Blue; color: White; text-align: center; } div.footer { /* Stuff Here */ } div.body { /* Stuff Here */ } etc. It's not too bad yet, but since I am learning here, I was wondering if recommendations could be made on how best to organize the various parts of a CSS file. What I don't want to get to is where I have a separate CSS attribute for every single thing on my website (which I have seen happen), and I always want the CSS file to be fairly intuitive. (P.S. I do realize this is a very generic, high-level question. My ultimate goal is to make it easy to use the CSS files and demonstrate their power to increase the speed of web development so other individuals that may work on this site in the future will also get into the practice of using them rather than hard-coding values everywhere.)

    Read the article

  • What are some typing patterns using a standard QWERTY keyboard that work well for you as a programme

    - by OrbMan
    After hunting and pecking for about 35 years, I have decided to learn to type. I am learning QWERTY and have learned about 2/3 of the letters so far. While learning, I have noticed how asymmeterical the keyboard is, which really bothers me. (I will probably switch to a symmetrical keyboard eventually, but for now am trying to do everything as standard and "correct" as possible.) Although I am not there yet in my lessons, it seems that many of the keys I am going to use as a C# web developer are supposed to be typed by the pinky of my right hand. Are there any typing patterns you have developed that are more ergonomic (or faster) when typing large volumes of code rife with braces, colons, semi-colons and quotes? Or, should I just accept the fact that every other key is going to be hit with my right pinky? It is not that speed is such a huge concern, as much as that it seems so inefficient to rely on one finger so much... As an example, some of the conventions I use as a hunt and pecker, like typing open and close braces right away with my index and middle finger, and then hitting the left arrow key to fill in the inner content, don't seem to work as well with just a pinky. What are some typing patterns using a standard QWERTY keyboard that work really well for you as a programmer? Update: US layout and I use home row Update 2: Despite my best efforts to the contrary, people are interpreting this questionas "how do I learn to type" or "what keyboard should I use". Take it as a given, that I will learn to type, and that I will be doing so on a standard QWERTY layout keyboard, not DVORAK. I am interested in aquiring a skill that will be useful wherever I go.

    Read the article

  • How to access C arrays from assembler for Windows x64?

    - by 0xdword32
    I've written an assembler function to speed up a few things for image processing (images are created with CreateDIBSection). For Win32 the assembler code works without problems, but for Win64 I get a crash as soon as I try to access my array data. I put the relevant info in a struct and my assembler function gets a pointer to this struct. The struct pointer is put into ebx/rbx and with indexing I read the data from the struct. Any idea what I am doing wrong? I use nasm together with Visual Studio 2008 and for Win64 I set "default rel". C++ code: struct myData { tUInt32 ulParam1; void* pData; }; CallMyAssemblerFunction(&myData); Assembler Code: Win32: ... push ebp; mov ebp,esp mov ebx, [ebp + 8]; pointer to our struct mov eax, [ebx]; ulParam1 mov esi, [ebx + 4]; pData, 4 byte pointer movd xmm0, [esi]; ... Win64: ... mov rbx, rcx; pointer to our struct mov eax, [rbx]; ulParam1 mov rsi, [rbx + 4]; pData, 8 byte pointer movd xmm0, [rsi]; CRASH! ...

    Read the article

  • Android - I can't make a widget clickable to launch an intent

    - by Daniele
    Hi all. I am new to Android development. I have developed a very simple widget that was meant to interact with the user via an ImageButton. What I am trying to do now is as follows. When a user taps the button (after adding the widget to their home screen), I want the phone to dial a certain telephone number. A sort of speed dial for your home screen. Unfortunately when I tap the button nothing happens. This is the body of my SpeedDialAppWidgetProvider.onUpdate method: Log.d("", "beginning of onUpdate"); final int N = appWidgetIds.length; for (int i=0; i<N; i++) { int appWidgetId = appWidgetIds[i]; Log.d("", "dealing with appWidgetId: " + appWidgetId); // Create an Intent to launch ExampleActivity Intent dialIntent = new Intent(Intent.ACTION_CALL, Uri.parse("tel:1234567")); PendingIntent pendingIntent = PendingIntent.getActivity(context, 0, dialIntent, 0); Log.d("", "pendingIntent classname " + pendingIntent.getClass().getName()); // Get the layout for the App Widget and attach an on-click listener to the button RemoteViews remoteViews = new RemoteViews(context.getPackageName(), R.layout.speed_dial_appwidget); remoteViews.setOnClickPendingIntent(R.id.dial_icon, pendingIntent); Log.d("", "remoteViews classname " + remoteViews.getClass().getName()); // Tell the AppWidgetManager to perform an update on the current App Widget appWidgetManager.updateAppWidget(appWidgetId, remoteViews); Log.d("", "end of onUpdate"); I can see the method is called and the result of the logging makes sense. The speed_dial_appwidget.xml file is like this: <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" androidrientation="vertical" android:layout_width="wrap_content" android:layout_height="wrap_content" > <ImageButton id="@+id/dial_icon" android:src="@drawable/speed_dial" android:layout_width="wrap_content" android:layout_height="wrap_content"/> </LinearLayout> Can you please help me with this? Thanks in advance, Dan

    Read the article

  • fastest SCM tool available for Embedded software development

    - by wrapperm
    Hi All, In my company, presently we are using Rational clearcase as the Software Configuration Management tool for our Embedded software development. The software is basically for Automobiles, to be specific for Engines (I dont think these information really matters). But I find Clearcase to be very slow is performing any the activities (accesing files, branching and labelling), in addition to which there are various other limitations. We have recently decided to research on some free & open source, distributed version control system which could be able to handle our large projects with speed and efficiency. This tool should be a full-fledged repository with complete history and full revision tracking capabilities, not dependent on network access or a central server. Branching and merging are fast and easy to do. It should have multisite development facility. With these above mentioned requirement, we have come up with some of the tools that are presently available in the market: GIT, Mercurial, Bazaar, Subversion, CVS, Perforce, and Visual SourceSafe. I need everybody's help in finding me an approrpiate SCM tool for me which meets the above mentioned requirements. Thanking you in Advance, Rahamath.

    Read the article

  • .net real time stream processing - needed huge and fast RAM buffer

    - by mack369
    The application I'm developing communicates with an digital audio device, which is capable of sending 24 different voice streams at the same time. The device is connected via USB, using FTDI device (serial port emulator) and D2XX Drivers (basic COM driver is to slow to handle transfer of 4.5Mbit). Basically the application consist of 3 threads: Main thread - GUI, control, ect. Bus reader - in this thread data is continuously read from the device and saved to a file buffer (there is no logic in this thread) Data interpreter - this thread reads the data from file buffer, converts to samples, does simple sample processing and saves the samples to separate wav files. The reason why I used file buffer is that I wanted to be sure that I won't loose any samples. The application doesn't use recording all the time, so I've chosen this solution because it was safe. The application works fine, except that buffered wave file generator is pretty slow. For 24 parallel records of 1 minute, it takes about 4 minutes to complete the recording. I'm pretty sure that eliminating the use of hard drive in this process will increase the speed much. The second problem is that the file buffer is really heavy for long records and I can't clean this up until the end of data processing (it would slow down the process even more). For RAM buffer I need at lest 1GB to make it work properly. What is the best way to allocate such a big amount of memory in .NET? I'm going to use this memory in 2 threads so a fast synchronization mechanism needed. I'm thinking about a cycle buffer: one big array, the Bus Reader saves the data, the Data Interpreter reads it. What do you think about it? [edit] Now for buffering I'm using classes BinaryReader and BinaryWriter based on a file.

    Read the article

  • Reading same file from multiple threads in C#

    - by Gustavo Rubio
    Hi. I was googling for some advise about this and I found some links. The most obvious was this one but in the end what im wondering is how well my code is implemented. I have basically two classes. One is the Converter and the other is ConverterThread I create an instance of this Converter class that has a property ThreadNumber that tells me how many threads should be run at the same time (this is read from user) since this application will be used on multi-cpu systems (physically, like 8 cpu) so it is suppossed that this will speed up the import The Converter instance reads a file that can range from 100mb to 800mb and each line of this file is a tab-delimitted value record that is imported to another destination like a database. The ConverterThread class simply runs inside the thread (new Thread(ConverterThread.StartThread)) and has event notification so when its work is done it can notify the Converter class and then I can sum up the progress for all these threads and notify the user (in the GUI for example) about how many of these records have been imported and how many bytes have been read. It seems, however that I'm having some trouble because I get random errors about the file not being able to be read or that the sum of the progress (percentage) went above 100% which is not possible and I think that happens because threads are not being well managed and probably the information returned by the event is malformed (since it "travels" from one thread to another) Do you have any advise on better practices of implementation of threads so I can accomplish this? Thanks in advance.

    Read the article

  • Error Converting PIL B&W images to Numpy Arrays

    - by Elliot
    I am getting weird errors when I try to convert a black and white PIL image to a numpy array. An example of the code I am working with is below. if image.mode != '1': image = image.convert('1') #convert to B&W data = np.array(image) #convert data to a numpy array n_lines = data.shape[0] #number of raster passes line_range = range(data.shape[1]) for l in range(n_lines): # process one horizontal line of the image line = data[l] for n in line_range: if line[n] == 1: write_line_to(xl, z+scale*n, speed) #conversion to other program code elif line[n] == 0: run_to(xl, z+scale*n) #conversion to other program code I have tried this using both array and asarray for the conversion, and gotten different errors. If I use array, then the data I get out is nothing like what I put in. It looks like several very shrunken partial images side by side, with the remainder of the image space filled in in black. If I use asarray, then the entirety of python crashes during the raster step (on a random line). If I work with a greyscale image ('L'), then neither of these errors occurs for either array or asarray. Does anyone know what I am doing wrong? Is there something odd about the way PIL encodes B&W images, or something special I need to pass numpy to make it convert properly?

    Read the article

  • How can I initialize a QTMovie object with certain attributes using writable data?

    - by c-had
    I'm trying to create an empty QTMovie object that I can add segments to, and then play. This is easy to do with something like: movie = [[QTMovie alloc] initToWritableData:[NSMutableData dataWithCapacity:1048576] error:&error]; I can then use -insertSegmentOfMovie to insert segments from other movies into this one so I can play it back. The problem is that I also need to set a certain attribute when creating the QTMovie object. In particular, I need to set the QTMovieRateChangesPreservePitchAttribute attribute, so that I can alter playback speed during playback without changing pitch. This attribute cannot be written after the movie is initialized. So, I can create the QTMovie object like this: movie = [[QTMovie alloc] initWithAttributes:[NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], QTMovieRateChangesPreservePitchAttribute, nil] error:&error]; Unfortunately, this is not editable. I've tried setting the QTMovieEditableAttribute as well on creation, but it does not help. I still get an exception when I try to insert anything into this movie. I presume this is because there is no writable file or data reference associated with the QTMovie. Any ideas on how to solve this?

    Read the article

  • linux new/delete, malloc/free large memory blocks

    - by brian_mk
    Hi folks, We have a linux system (kubuntu 7.10) that runs a number of CORBA Server processes. The server software uses glibc libraries for memory allocation. The linux PC has 4G physical memory. Swap is disabled for speed reasons. Upon receiving a request to process data, one of the server processes allocates a large data buffer (using the standard C++ operator 'new'). The buffer size varies depening upon a number of parameters but is typically around 1.2G Bytes. It can be up to about 1.9G Bytes. When the request has completed, the buffer is released using 'delete'. This works fine for several consecutive requests that allocate buffers of the same size or if the request allocates a smaller size than the previous. The memory appears to be free'd ok - otherwise buffer allocation attempts would eventually fail after just a couple of requests. In any case, we can see the buffer memory being allocated and freed for each request using tools such as KSysGuard etc. The problem arises when a request requires a buffer larger than the previous. In this case, operator 'new' throws an exception. It's as if the memory that has been free'd from the first allocation cannot be re-allocated even though there is sufficient free physical memory available. If I kill and restart the server process after the first operation, then the second request for a larger buffer size succeeds. i.e. killing the process appears to fully release the freed memory back to the system. Can anyone offer an explanation as to what might be going on here? Could it be some kind of fragmentation or mapping table size issue? I am thinking of replacing new/delete with malloc/free and use mallopt to tune the way the memory is being released to the system. BTW - I'm not sure if it's relevant to our problem, but the server uses Pthreads that get created and destroyed on each processing request. Cheers, Brian.

    Read the article

  • Render an SSRS report with a Map as an image map without actually having a ReportViewer on the page

    - by Erica Merchant
    I have a report that has a Map with spatial data. Clicking an object on that map sends you to other pages on the site. I have tried a few different ways of displaying the report: If I put a ReportViewer on the actual page, the page sometimes takes 10+ seconds to load, but the report viewer creates a fully operable image map. If I create a ReportViewer in the code behind, I can use the Render method to to format the report as HTML4.0 and get the streamids from which I can extract the image (painfully). This is pretty fast (1-2 seconds), but only gives me an image, no image map. I can get a very similar functionality as the above example by using rendering extensions on the report URL to create an image and then set an image's source to this url. This is the fastest method, but still does not create an image map. So is there a way to create an image map from the report without having to use the ReportViewer? Or a way to substantially speed up the Report Viewer?

    Read the article

  • making check boxes clickable once in javascript?

    - by OVERTONE
    Sorry but im an absolute noob with javascript. Ive made a form for a simple quiz but cant figure out how to make radio's only click once. I can select two or three buttons as my answer. i want to change this. <form name = "Beginners Quiz"> <p>Film speed refers to:</p> <p><input type="radio" name="Answer 1" id="Answer1" value = "a" onclick = "recordAnswer(1,this.value"/>How long it takes to develop film. <br/> <p><input type="radio" name="Answer 2" id="Answer2" value = "b" onclick = "recordAnswer(1,this.value"/>How fast film moves through film-transport system. <br/> <p><input type="radio" name="Answer 3" id="Answer3" value = "c" onclick = "recordAnswer(1,this.value"/> How sensitive the film is to light. <br/> <p><input type="radio" name="Answer 4" id="Answer4" value = "d" onclick = "recordAnswer(1,this.value"/> None of these makes sense. <br/> ive been rooting around w3shcools tutorials to no avail. can someone shed some light?

    Read the article

  • Parse numbers in singe textbox for query

    - by Joshua Slocum
    I’ve built a webform in Visual Web Developer Express 2008 to help me with my work. I use a webform to run query requests that are emailed to me. The inputs are in this format 12312 12312 12312 12312 12312 12312 12312 12312 I enter the first number in a textbox and the second number in another textbox and click a button that runs a query and returns the results in a gridview(single row). string strConn, strSQL; strConn = AppConfig.Connection strSQL = 'select fields from table where FirstNum=:FirstNum and SecondNum=:SecondNum'; using (OracleConnection cn = new OracleConnection(strConn)) { OracleCommand cmd = new OracleCommand(strSQL, cn); cmd.Parameters.AddWithValue(":FirstNum", txtFirstNum.Text); cmd.Parameters.AddWithValue(":SeconNum", txtSecondNum.Text); cn.Open(); using (OracleDataReader rdr = cmd.ExecuteReader()) { dgResults.DataSource = rdr; dgResults.DataBind(); } cn.Close(); } I had an idea to help me speed up my work. I’d like to be able to past both numbers in a single textbox ( like this 12312 12312 ) and have the code parse out the nubmers for the query. Or even better would be to past all of them in a multiline textbox like this 12312 12312 12312 12312 12312 12312 12312 12312 And have them all parsed and the query run for each line and the results all output to one gridview. I’m just not sure how to approach this. Any suggestions would be appreciated. Thank you.

    Read the article

  • Generate and merge data with python multiprocessing

    - by Bobby
    I have a list of starting data. I want to apply a function to the starting data that creates a few pieces of new data for each element in the starting data. Some pieces of the new data are the same and I want to remove them. The sequential version is essentially: def create_new_data_for(datum): """make a list of new data from some old datum""" return [datum.modified_copy(k) for k in datum.k_list] data = [some list of data] #some data to start with #generate a list of new data from the old data, we'll reduce it next newdata = [] for d in data: newdata.extend(create_new_data_for(d)) #now reduce the data under ".matches(other)" reduced = [] for d in newdata: for seen in reduced: if d.matches(seen): break #so we haven't seen anything like d yet seen.append(d) #now reduced is finished and is what we want! I want to speed this up with multiprocessing. I was thinking that I could use a multiprocessing.Queue for the generation. Each process would just put the stuff it creates on, and when the processes are reducing the data, they can just get the data from the Queue. But I'm not sure how to have the different process loop over reduced and modify it without any race conditions or other issues. What is the best way to do this safely? or is there a different way to accomplish this goal better?

    Read the article

  • How toget a list of "fastest miles" from a set of GPS Points

    - by santiagobasulto
    I'm trying to solve a weird problem. Maybe you guys know of some algorithm that takes care of this. I have data for a cargo freight truck and want to extract some data. Suppose I've got a list of sorted points that I get from the GPS. That's the route for that truck: [ { "lng": "-111.5373066", "lat": "40.7231711", "time": "1970-01-01T00:00:04Z", "elev": "1942.1789265256325" }, { "lng": "-111.5372056", "lat": "40.7228762", "time": "1970-01-01T00:00:07Z", "elev": "1942.109892409177" } ] Now, what I want to get is a list of the "fastest miles". I'll do an example: Given the points: A, B, C, D, E, F the distance from point A to point B is 1 mile, and the cargo took 10:32 minutes. From point B to point D i've got other mile, and the cargo took 10 minutes, etc. So, i need a list sorted by time. Similar to: B -> D: 10 A -> B: 10:32 D -> F: 11:02 Do you know any efficient algorithm that let me calculate that? Thank you all. PS: I'm using Python. EDIT: I've got the distance. I know how to calculate it and there are plenty of posts to do that. What I need is an algorithm to tokenize by mile and get speed from that. Having a distance function is not helpful enough: results = {} for point in points: aux_points = points.takeWhile(point>n) #This doesn't exist, just trying to be simple for aux_point in aux_points: d = distance(point, aux_point) if d == 1_MILE: time_elapsed = time(point, aux_point) results[time_elapsed] = (point, aux_point) I'm still doing some pretty inefficient calculations.

    Read the article

  • Save memory in Python. How to iterate over the lines and save them efficiently with a 2million line

    - by skyl
    I have a tab-separated data file with a little over 2 million lines and 19 columns. You can find it, in US.zip: http://download.geonames.org/export/dump/. I started to run the following but with for l in f.readlines(). I understand that just iterating over the file is supposed to be more efficient so I'm posting that below. Still, with this small optimization, I'm using 10% of my memory on the process and have only done about 3% of the records. It looks like, at this pace, it will run out of memory like it did before. Also, the function I have is very slow. Is there anything obvious I can do to speed it up? Would it help to del the objects with each pass of the for loop? def run(): from geonames.models import POI f = file('data/US.txt') for l in f: li = l.split('\t') try: p = POI() p.geonameid = li[0] p.name = li[1] p.asciiname = li[2] p.alternatenames = li[3] p.point = "POINT(%s %s)" % (li[5], li[4]) p.feature_class = li[6] p.feature_code = li[7] p.country_code = li[8] p.ccs2 = li[9] p.admin1_code = li[10] p.admin2_code = li[11] p.admin3_code = li[12] p.admin4_code = li[13] p.population = li[14] p.elevation = li[15] p.gtopo30 = li[16] p.timezone = li[17] p.modification_date = li[18] p.save() except IndexError: pass if __name__ == "__main__": run()

    Read the article

  • delete row from result set in web sql with javascript

    - by Kaijin
    I understand that the result set from web sql isn't quite an array, more of an object? I'm cycling through a result set and to speed things up I'd like to remove a row once it's been found. I've tried "delete" and "splice", the former does nothing and the latter throws an error. Here's a piece of what I'm trying to do, notice the delete on line 18: function selectFromReverse(reverseRay,suggRay){ var reverseString = reverseRay.toString(); db.transaction(function (tx) { tx.executeSql('SELECT votecount, comboid FROM counterCombos WHERE comboid IN ('+reverseString+') AND votecount>0', [], function(tx, results){ processSelectFromReverse(results,suggRay); }); }, function(){onError}); } function processSelectFromReverse(results,suggRay){ var i = suggRay.length; while(i--){ var j = results.rows.length; while(j--){ console.log('searching'); var found = 0; if(suggRay[i].reverse == results.rows.item(j).comboid){ delete results.rows.item(j); console.log('found'); found++; break; } } if(found == 0){ console.log('lost'); } } }

    Read the article

  • Any high-level languages that can use c libraries?

    - by Isaiah
    I know this question could be in vain, but it's just out of curiosity, and I'm still much a newb^^ Anyways I've been loving python for some time while learning it. My problem is obviously speed issues. I'd like to get into indie game creation, and for the short future, 2d and pygame will work. But I'd eventually like to branch into the 3d area, and python is really too slow to make anything 3d and professional. So I'm wondering if there has ever been work to create a high-level language able to import and use c libraries? I've looked at Genie and it seems to be able to use certain libraries, but I'm not sure to what extent. Will I be able to use it for openGL programing, or in a c game engine? I do know some lisp and enjoy it a lot, but there aren't a great many libraries out there for it. Which leads to the problem: I can't stand C syntax, but C has libraries galore that I could need! And game engines like irrlicht. Is there any language that can be used in place of C around C? Thanks so much guys

    Read the article

< Previous Page | 388 389 390 391 392 393 394 395 396 397 398 399  | Next Page >